Exponential moments and the Chernoff method

For a real random variable S and λ positive, the event S≥a implies exp(λS)≥exp(λa). Markov's inequality therefore gives a bound whenever the positive exponential moment is finite. Define the cumulant generating function K(λ)=log E exp(λS), allowing value infinity outside its domain. Minimizing K(λ)−λa over positive admissible parameters improves the estimate. The endpoint λ=0 always yields the trivial bound one, so an infimum can include it by continuity when the right-hand domain is nonempty.

The lower event S≤a instead uses λ negative; multiplying an inequality by λ reverses its direction before exponentiation. Equivalently apply the positive-parameter upper-tail method to −S. Finite positive moments alone say nothing comparable about a heavy lower tail. Centering is not needed for the basic Markov step, but concentration around the mean requires subtracting that mean and changes the cumulant by a linear term. A two-sided statement generally combines two directed bounds through a union bound.

For independent variables, the exponential of their sum is a product whose expectation factors. Their cumulants therefore add on the common finite domain. This turns a difficult distribution of a sum into individual moment calculations. Identical distribution is convenient but unnecessary. Dependence breaks this product identity: knowing every marginal mgf does not determine the sum's mgf. Conditional mgf bounds can replace independence for a martingale, but they require information at each step of the filtration rather than just unconditional moments.

For a Bernoulli variable of success probability p, the mgf is 1−p+p exp(λ), finite for every real λ. A binomial sum of n independent copies therefore has cumulant n log(1−p+p exp(λ)). At an upper threshold nq with p<q<1, differentiation gives the positive optimizer log(q(1−p)/(p(1−q))). Substitution yields the binary relative entropy exponent. The threshold need not be an integer; the event still means the integer count is at least its ceiling, and rounding the threshold upward can improve the bound.

An exponential variable of rate one has mgf 1/(1−λ) only for λ<1. Treating its algebraic expression as valid for larger λ would produce nonsensical bounds and can even introduce the logarithm of a negative number. More generally a stationary point found by differentiating a formal expression is a candidate only inside the finite-moment domain. If the infimum lies at an open boundary, take a limit from inside that domain rather than asserting an unattained minimizing parameter exists.

A Chernoff bound is an upper bound, not an approximation with a guaranteed small relative error. It can capture the correct exponential scale while missing a substantial prefactor. Exact finite distributions provide a useful audit: the bound must dominate the event probability, but equality is not expected. For very small exponents it can remain close to one and convey little information. A result above one should simply be capped at one; that numerical weakness does not invalidate the underlying inequality.