advanced28 min studyLesson 634 in Mathematics

Chernoff bounds in combinatorial models

Use exponential moments to control many simultaneous deviations while preserving independence assumptions.

On this page

What you’ll understand

  • Derive a multiplicative tail bound from a Bernoulli moment-generating function.
  • Translate integer thresholds into the correct relative deviation.
  • Combine marginal concentration with a union bound without assuming events are independent.

A variance estimate can be adequate for one count and too weak for thousands of constraints. Chernoff bounds trade stronger assumptions for exponential tail decay. Their power comes from applying Markov’s inequality to an exponential rather than to the original variable. In a combinatorial application, the essential check is which summands really are independent; separate graph degrees may interact even when the edges inside one degree count are independent.

Build the exponential estimate from the model

Let X be the sum of independent Bernoulli variables, with possibly different success probabilities and positive total mean mu. Independence factors the exponential moment into a product. The elementary inequality one plus u is at most the exponential of u bounds each factor. Exponential Markov then introduces a positive parameter lambda, which can be optimized after the threshold is chosen. Merely knowing the mean and variance is insufficient to justify this factorization or the resulting exponential decay.

The optimized upper-tail expression involves the entropy-like function h(δ)=(1+δ)log⁡(1+δ)−δh(\delta)=(1+\delta)\log(1+\delta)-\delta. A convenient weaker bound replaces h by δ2/(2+δ)\delta^2/(2+\delta) for nonnegative delta. Its constants are less sharp but its arithmetic is transparent. At a finite integer threshold r above mu, use delta equal to r divided by mu minus one. Rounding r in the wrong direction can turn an estimate for a smaller event into an unsupported bound for the event actually requested.

EeλX≤exp⁡ ⁣(μ(eλ−1)),P(X≥(1+δ)μ)≤exp⁡ ⁣(−μδ22+δ),δ≥0\mathbb E e^{\lambda X}\le\exp\!\left(\mu(e^\lambda-1)\right),\qquad \mathbb P(X\ge(1+\delta)\mu)\le\exp\!\left(-\frac{\mu\delta^2}{2+\delta}\right),\quad\delta\ge0
The variables are independent Bernoulli summands, lambda is real, and the upper-tail formula assumes a positive mean.↔ Scroll wide equations horizontally.

Lower tails and simultaneous requirements

For a lower tail, use a negative exponential parameter, or equivalently a positive parameter on minus X. The standard multiplicative lower bound applies for delta between zero and one. It is not obtained by simply substituting a negative delta into a memorized upper-tail denominator. These bounds remain useful for nonidentical Bernoulli probabilities because the derivation depends on their sum, not on equality of their individual probabilities. When the mean is zero, the Bernoulli sum is zero almost surely and needs separate treatment.

Suppose a construction has M constraints, each with failure probability at most q. The probability of any failure is at most Mq, regardless of dependencies between the constraint events. Thus exponentially small marginal tails can withstand a polynomial number of checks. For example, if every relevant mean exceeds a suitable constant times log M, fixed relative deviations can often be excluded simultaneously. This requires a uniform lower bound on those means and a count of all constraints; checking only one representative constraint is not a simultaneous theorem.

P(X≤(1−δ)μ)≤e−μδ2/2,0≤δ≤1,P ⁣(⋃j=1MBj)≤∑j=1MP(Bj)\mathbb P(X\le(1-\delta)\mu)\le e^{-\mu\delta^2/2},\quad0\le\delta\le1,\qquad \mathbb P\!\left(\bigcup_{j=1}^{M}B_j\right)\le\sum_{j=1}^{M}\mathbb P(B_j)
The tail bound uses independent Bernoulli summands within X; the union bound places no independence requirement on the events B_j.↔ Scroll wide equations horizontally.

Choose concentration at the level where independence lives

A vertex degree in G(n,p) is a sum of independent edge indicators, so a Chernoff estimate applies directly to that degree. Degree counts at two different vertices share an edge and are not independent. Nevertheless a union bound can combine their individual estimates. By contrast, the triangle count is a sum of overlapping indicators, and applying the independent-summand formula to that sum is generally unjustified. One must expose independent edges through a different theorem or explicitly account for triangle dependencies.

A bound larger than one should be capped at one, and a small graph may give a weak result even when the asymptotic method is effective. Compare the proven upper bound with an exact binomial tail when available; their difference is the cost of a convenient inequality, not a contradiction. The distribution instrument displays a fully independent model with exact masses, which makes this gap inspectable. It should not be interpreted as empirical evidence for an arbitrary dependent combinatorial count.

Two tails from eighty independent choices

Let X be binomial with eighty trials and success probability 1/4. Bound P(X≥30) and P(X≤10) using the stated convenient Chernoff forms.

  1. The mean is 80/4=2080/4=20. The threshold 30 is 1.5 times the mean, so the upper-tail delta is 1/2.

  2. The upper bound is exp⁡(−20(1/2)2/(2+1/2))=e−2≈0.135335\exp(-20(1/2)^2/(2+1/2))=e^{-2}\approx0.135335.

  3. The threshold 10 is half the mean, so the lower-tail delta is also 1/2. Its distinct formula gives exp⁡(−20(1/2)2/2)=e−5/2≈0.082085\exp(-20(1/2)^2/2)=e^{-5/2}\approx0.082085.

The result

The upper-tail probability is at most 0.135335 and the lower-tail probability is at most 0.082085; these are bounds on different events, not equal tail probabilities.

Explore the model

Interactive lab

Interactive lab · sampling

An exact distribution beneath an exponential envelope

Pause and Reset, then set p = 0.25 and n = 80. Predict sample-mean expectation 0.25 and variance 0.25·0.75/80 = 0.00234375. The event K≥30 begins at mean 0.375; inspect its small exact binomial bars and compare with the proven bound e⁻². Reset and set both p = 0.25 and n = 40 to predict a wider sample-mean distribution. Playback highlights exact outcomes, not independent reruns, and the normal overlay is an approximation.

An exact distribution beneath an exponential envelopeExact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Current readings: Mean sample proportion: 0.5; Sample-mean variance: 0.0125; Uniform CDF error bound: ≤ 0.1062. All parameters are adjustable with the labeled controls below.Exact distribution of the mean of 20 independent Bernoulli variables with success probability 0.5; playback highlights outcome 0.00.250.50.75100.0580.1150.1730.23Sample mean · k / nProbability massNormal bin masses are an approximationk = 0 · exact probability = 9.54e-7
Exact binomial massNormal interval massHighlighted outcome / mean
Mean sample proportion
0.5
Sample-mean variance
0.0125
Uniform CDF error bound
≤ 0.1062
t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forExact bars, an approximate normal comparison, and a rigorous Chernoff upper bound answer distinct questions.

Check your understanding

Why may one union-bound Chernoff estimates for all degrees in G(n,p)?

Try it yourself

For X binomial with one hundred trials and p=0.3, bound P(X≥40). If forty such counts each obey that bound, does the union bound certify that none reaches forty with positive probability?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compute the relative upper deviation from mean thirty, and compare forty times the bound with one.

Compare with the solution

The mean is thirty and delta is 40/30−1=1/340/30-1=1/3. The exponent is −30(1/9)/(7/3)=−10/7-30(1/9)/(7/3)=-10/7, giving an upper bound approximately 0.239651. Summing forty such bounds gives approximately 9.58604, which is capped at one. Consequently this calculation does not give a positive lower bound for avoiding all forty failures. It does not establish impossibility either. A sharper estimate, larger means at fixed relative error, fewer constraints, or structural information about their interaction could produce a useful simultaneous conclusion.

Take it with you

  • Exponential-moment factorization needs justified independence of summands.
  • Translate each upper or lower threshold into its own relative deviation.
  • Dependent constraint events can be combined by a union bound after their marginal tails are established.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.