Bernstein, Bennett, and subexponential tails

Let independent centered variables X_i have finite variances, satisfy X_i≤b almost surely for one deterministic b>0, and let v be their positive total variance. Bennett bounds the upper tail of their sum by exp(−v h(bt/v)/b²), where h(u)=(1+u)log(1+u)−u. A lower bound on X_i is not needed for this upper-tail theorem. To apply it to the lower tail, bound −X_i from above instead. A two-sided absolute bound follows with a common b only if both directions are controlled.

The proof bounds each centered mgf by variance times the exponential remainder e^(λb)−1−λb. Optimizing over positive λ produces Bennett's h. The elementary lower bound h(u)≥u²/(2(1+u/3)) gives Bernstein's denominator v+bt/3. Thus one third is a specified constant, not an interchangeable convention. The zero-variance case is deterministic and handled separately. If the variables were not centered initially, the almost-sure bound must be checked after centering, rather than applied blindly to their raw maximum.

Here a centered variable is called (ν²,β)-subexponential in the mgf sense if log E exp(λX)≤ν²λ²/2 for every |λ|<1/β, with positive ν and β. This is a local quadratic bound with no singular denominator. Other texts use a subgamma bound or an Orlicz norm and convert parameters with additional constants; they are equivalent classes but not identical numerical definitions. Independence adds ν² parameters for a sum and uses the largest β as a valid common domain scale.

Optimizing within that finite interval gives Gaussian-shaped decay when t≤ν²/β and exponential-shaped decay beyond it. At the boundary, use limits from inside the domain. One convenient two-sided consequence is twice exp of minus one half times the minimum of t²/ν² and t/β. This is a conservative piecewise simplification, not Bennett's exact rate function. The word subexponential in this concentration convention also differs from the heavy-tail theory convention involving convolution tails; it does not describe that class here.

For a sum of rare Bernoulli successes, centering leaves interval width one, but total variance is np(1−p), much smaller than n/4 when p is small. Bernstein can then improve substantially on Hoeffding without assuming a normal approximation. It still remains an upper bound and can exceed the exact binomial probability by a noticeable factor. Choosing the tight upper centered bound 1−p can improve a directed result further, while b=1 is a simple safe common bound for both centered directions.

The small-deviation regime is controlled mainly by v, while a very large upward deviation feels the bound b. Neither parameter should be estimated from the same data and substituted without a theorem that accounts for its randomness. Empirical Bernstein inequalities exist, but they include additional assumptions and correction terms. Likewise unconditional variance bounds alone do not extend this independent-sum result to an adaptive process; predictable conditional variances are the relevant quantities in a martingale counterpart.