Characteristic functions and convolution
Encode probability laws with bounded complex exponentials and turn independent sums into products.
On this page
The question for today
What you’ll understand
- Derive the characteristic function of a finite discrete law.
- Explain why independence turns convolution into multiplication.
- Apply the continuity theorem with its continuity-at-zero requirement.
Adding independent random variables combines many possible pairs of outcomes. Directly listing those pairs works for a small finite example, but quickly becomes awkward. A characteristic function packages every possible value into an average complex rotation. Its usefulness comes from two facts: the rotation always has magnitude one, and the rotation associated with a sum factors into a product. This gives a transform that exists even for heavy-tailed laws without a finite mean, while still retaining enough information to determine the entire distribution.
A transform that always exists
For a real random variable X, define its characteristic function at a real frequency t as the expectation of exp(itX). The real part averages cosines and the imaginary part averages sines. Absolute integrability is automatic because the complex exponential has magnitude one. Thus the transform is defined at every real t, equals one at zero, and has absolute value at most one. This differs from a moment-generating function, which may diverge at every nonzero real argument.
Dominated convergence proves continuity: when the frequency changes by a small amount, the integrand converges pointwise and remains uniformly bounded. In fact the transform is uniformly continuous. Translation and scaling are especially transparent. Adding a constant rotates the transform by a deterministic phase, while multiplying X by a constant rescales the frequency. These rules concern the law of a variable and do not require a probability density.
Independent sums and convolution
If X and Y are independent, exp(itX) and exp(itY) are independent bounded variables. Their product is exp(it(X+Y)), so its expectation factors. At the measure level, the law of the sum is the convolution of the two laws: integrate the indicator that x plus y belongs to the desired event against the product measure. The transform changes this two-variable integration into an ordinary product of functions.
Independence is the hypothesis that licenses the product rule. Knowing the two marginal laws alone does not determine the sum when dependence is unknown. For instance, two symmetric signs can be equal or opposite; their individual transforms agree in both constructions, but their sums have different laws. If the second absolute moment exists, differentiating under the expectation gives the first two derivatives at zero and recovers the mean and second moment. Without the relevant integrability, formal differentiation is unjustified.
From algebra back to a probability limit
A probability law is uniquely determined by its characteristic function. Lévy’s continuity theorem makes this uniqueness useful for limits: if characteristic functions converge pointwise to a function continuous at zero, that limiting function is itself a characteristic function and the associated laws converge weakly. If the candidate is already known to be the transform of a probability law, continuity at zero is automatic. Pointwise convergence without that condition can describe probability mass escaping to infinity.
For a Bernoulli variable with success probability p, the transform is one minus p plus p exp(it). The transform of n independent copies is its nth power; expanding the polynomial recovers the binomial coefficients and probabilities. This is a useful two-way check: convolution predicts a sum law, while polynomial multiplication checks its normalization and coefficients. It also prepares the central limit theorem, where centering and scaling force the transform to be inspected near frequency zero.
Work it through
Recover a sum law from its polynomial
Three independent indicators each have success probability 0.4. Compute their sum distribution using the characteristic function, then check its mean and variance.
One indicator has transform 0.6 + 0.4 exp(it). Independence makes the sum transform the cube of this expression. Introduce z = exp(it) to keep the expansion readable.
Expand (0.6 + 0.4z)³ = 0.216 + 0.432z + 0.288z² + 0.064z³. The coefficients are the probabilities of zero, one, two, and three successes. Their sum is one.
Compute the first moment from the coefficients: 0.432 + 2(0.288) + 3(0.064) = 1.2. The second moment is 0.432 + 4(0.288) + 9(0.064) = 2.16, so the variance is 2.16 − 1.44 = 0.72.
The sum is binomial with three trials and p = 0.4. Its probability of exactly two successes is 0.288, its mean is 1.2, and its variance is 0.72.
The living notebook
Explore the model
Interactive lab · probability
Read convolution as binomial coefficients
Set p = 0.40 and n = 3. Predict the four bar heights from the cubic expansion before inspecting them. Then keep p fixed and raise n to 6: predict that the transform would be the square of the three-trial transform.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forThe lab shows exact binomial probabilities, mean, and variance. It does not plot a complex characteristic function, perform arbitrary convolutions, or sample experimental outcomes.
Pause & reason
Check your understanding
Your turn
Try it yourself
Independent variables X and Y have Poisson laws with means two and three. Their characteristic functions are exp(2(exp(it) − 1)) and exp(3(exp(it) − 1)). Identify the law of X + Y and explain which step would fail if independence were removed.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Multiply the transforms, combine their exponents, and compare with the given Poisson transform formula.
Compare with the solution
Independence gives the product transform exp(5(exp(it) − 1)). This is the characteristic function of a Poisson law with mean five, and uniqueness therefore identifies the sum distribution. Without independence, the expectation of exp(itX)exp(itY) need not equal the product of the two expectations. The marginal Poisson laws alone would then be insufficient to determine the law of the sum.
Keep the idea
Take it with you
- Characteristic functions exist for every real probability law because the integrand is bounded.
- Convolution becomes multiplication when the summed variables are independent.
- A continuity condition at frequency zero turns transform limits into weak convergence.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.