Compose Rényi bounds at matching orders and convert once

For finite order alpha greater than one, Rényi divergence compares two output distributions using a logarithmic moment of their density ratio. The denominator distribution must dominate the numerator for the finite-density expression; otherwise the divergence can be infinite. An RDP guarantee bounds this divergence for every ordered neighboring pair. Alpha is an analysis parameter, not a batch size, probability or privacy unit. Keep each order’s rho value paired with that order throughout accounting.

For equal-covariance Gaussian outputs, divergence is exactly alpha times the squared difference of means divided by twice the noise variance. A query with Euclidean sensitivity at most Delta and Gaussian coordinate SD s therefore has the displayed worst-case RDP bound. Setting s=sigma Delta makes the bound alpha/(2sigma²). Here that is the unsampled mechanism with q=1. A Poisson-sampled Gaussian has a mixture distribution and requires its sampled analysis; do not attach a q² multiplier and call the result exact.

For T adaptive releases, each conditional mechanism must satisfy its per-step RDP bound for every allowed prior output history. Then add the rho values at the same alpha. Do not combine a rho at order two with another at order eight and label their sum as one valid order. Changing noise or sampling rates across steps is allowed when each step has the corresponding valid bound. Account for every relevant release, including private tuning or evaluation mechanisms that are not simply another training step.

For any delta strictly between zero and one, the basic conversion adds log(1/delta)/(alpha−1) to the composed rho. Taking the minimum over a finite set of valid orders gives a valid epsilon parameter, though not necessarily the tightest possible one. This is a bound, not the mechanism’s exact smallest epsilon. Tighter conversion theorems and numerical accountants can return different valid values. Preserve theorem and implementation versions rather than treating two nonidentical bounds as evidence that one run changed its underlying randomness.

Store the adjacency, sensitivity convention, noise schedule, sampling law, q values, logical step count, order grid, target delta and accountant version. A sigma defined relative to C cannot be passed unchanged as sensitivity-relative noise under replacement when the relevant bound is 2C. A resumed run needs prior accountant state. Opacus v1.5.4 implements sampled-Gaussian RDP and a specific conversion; the elementary conversion used here is deliberately stated separately rather than claimed to reproduce every library result.

The function lab illustrates adding public cost terms only. Reset and set both a=0.5 and x=2; predict y=2. Separately Reset and set both a=1 and x=2; predict y=4, the sum of two identical first-run values. This quadratic has no Rényi divergence, sampling or privacy semantics. It cannot compute rho or epsilon. Use it only to check ordinary addition and scaling, while the worked example supplies the explicit Gaussian theorem and analysis domains required for an actual bound.