Use sampling amplification only for the sampler actually implemented
Distinguish independent inclusion, fixed-size sampling and visible participation.
On this page
The question for today
What you’ll understand
- Compute independent-inclusion count moments without confusing them with a privacy budget.
- Apply a stated Poisson amplification theorem under matching add/remove adjacency.
- Identify sampler, transcript and stopping changes that need a different analysis.
Sampling can improve a differential-privacy bound because a protected contribution is not used in every invocation. That improvement is a theorem about a particular randomized mechanism and observable transcript, not a reward attached to any small batch. Independent inclusion, fixed-size sampling without replacement and shuffled epochs have different laws. The accountant’s sampling assumptions must match the protected unit and the implementation that produced the released model.
Poisson inclusion is not a fixed-size batch
Poisson subsampling independently includes each eligible contribution with probability q. Conditional on a public population of N eligible units, the count is binomial with mean Nq and variance Nq(1−q). In contrast, choosing exactly b units without replacement has deterministic size b and dependent inclusion indicators. Both can have the same marginal inclusion rate b/N, but their joint distributions differ. Sampling an entire shuffled epoch introduces another dependence pattern across updates that cannot be silently replaced by independent Poisson draws.
For add/remove record protection, the sampler operates on records; for user-level protection, it must select the user contribution or use a different proven analysis. Repeated records from one person do not become independent protected users merely because they are independently sampled. Keep population size, eligibility filtering and participation observations in the threat model. Publishing selected identities, private counts or data-dependent eligibility can change the transcript beyond what a hidden-subsample theorem covers.
State the theorem and its adjacency scope
For an add/remove (epsilon₀,delta₀)-DP base mechanism, applying it to an independently Poisson-sampled input with inclusion q gives the displayed valid amplified parameters when the released output is covered by that subsampling theorem. The formula bounds a privacy guarantee; it does not assert that every mechanism’s smallest epsilon equals it. The base mechanism must also be defined for possible empty samples. Fixed-size sampling has its own matching theorems and adjacency conditions, rather than inheriting this result from an equal expected batch size.
Sampled-Gaussian RDP accounting can be much tighter than repeatedly converting a base mechanism to an epsilon-delta pair and applying this elementary theorem. Those calculations depend on Gaussian noise, sensitivity convention, sampling law and numerical domains. Do not substitute a small-q asymptotic expression as an exact finite-parameter accountant. Opacus v1.5.4 exposes Poisson sampling and an accounting hook with a sample rate; disabling its sampler does not prove that an unchanged accountant matches a replacement loader.
Audit the implemented schedule
Record q, the eligibility universe, replacement policy and every logical noised update. Empty samples, dropped batches, adaptive stopping and retries can change the schedule or expose additional information. A data-dependent decision to skip a difficult batch is not equivalent to independent sampling. Parameters chosen adaptively from already private outputs can be handled by appropriate composition, but private raw-data decisions require their own treatment. Keep sampler configuration and accountant state together in checkpoints so a resumed job does not silently restart its privacy history.
Reset the probability lab and set both p=0.1 and n=20. Predict count mean two, variance 1.8 and zero-count probability 0.9²⁰≈0.1216. Play highlights exact binomial outcomes rather than sampling a private batch. This matches the count law of twenty independent inclusion opportunities only. It contains no base DP mechanism, Gaussian noise or accountant and cannot compute amplification for an application. Compare that variable count with a hypothetical fixed batch of two, whose count variance would be zero.
Work it through
Equal expected size does not identify the sampler
For public N=100 eligible units, independently include each with q=0.1. Separately suppose a matching add/remove base mechanism has pure epsilon₀=log 2 and the Poisson amplification theorem applies to its entire released output. Compare batch-count moments and the theorem’s amplified epsilon bound.
Independent inclusion gives expected batch count 100×0.1=10 and variance 100×0.1×0.9=9. Choosing exactly ten units without replacement would have the same expected count but zero count variance.
Because exp(epsilon₀)=2, the valid amplified epsilon is log(1+0.1×(2−1))=log 1.1≈0.09531. Delta remains zero under the stated pure base guarantee.
Simply multiplying q by epsilon₀ gives approximately 0.06931, which is smaller than this theorem’s bound and is not its exact formula. This comparison does not show the mechanism’s optimal epsilon; it shows why an unjustified approximation cannot replace the declared accounting result.
The sampling law and privacy theorem are separate pieces of evidence. A count of ten or a marginal rate 0.1 alone does not establish which amplification result applies.
The living notebook
Explore the model
Interactive lab · probability
Independent inclusion produces a variable batch count
Reset → set p=0.1 and n=20 → predict mean 2, variance 1.8 and P(K=0)≈0.1216. Playback highlights exact outcomes rather than selecting private records.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forThis binomial count model has no DP mechanism. It cannot calculate epsilon, replace an accountant or justify a different sampling law.
Pause & reason
Check your understanding
Your turn
Try it yourself
With public N=200 and Poisson inclusion q=0.05, find count mean and variance. Under the same pure base epsilon₀=log 2 and valid theorem scope, compute the amplified epsilon parameter.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use the independent-inclusion moments and log(1+q) for this particular base epsilon.
Compare with the solution
Mean count is ten and variance is 200×0.05×0.95=9.5. The valid amplified epsilon is log 1.05≈0.04879 with delta zero. A fixed batch of ten has zero count variance and requires a theorem matching its own sampler and adjacency, rather than importing this calculation solely from the shared mean.
Keep the idea
Take it with you
- Sampling amplification is a theorem about a sampler, unit, adjacency and transcript.
- Poisson and fixed-size batches can have equal means but different laws.
- Small-rate approximations are not exact sampled-Gaussian accounting.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.