greatitself.
Continue
advanced28 min studyLesson 196 in Mathematics

Modes of probabilistic convergence

Distinguish pathwise, probabilistic, moment, and distributional limits, and test the implications using rare events.

Before you begin
On this page

The question for today

What you’ll understand

  • State precisely what almost sure, probability, and Lp convergence control.
  • Use moment inequalities to justify implications between convergence modes.
  • Construct rare-event examples that separate modes instead of relying on a hierarchy alone.

A sequence of random variables can settle down in several different senses. Following one outcome through all experiments asks a different question from bounding the fraction of bad outcomes at one experiment size. Averaging the size of an error asks something stronger again when rare errors can be enormous. Start every limit argument by naming the quantity that converges and the probability space on which the variables live. A distributional statement can compare separate experiments without any chosen coupling; a pathwise statement cannot.

Four questions hidden inside the word limit

Almost sure convergence says that, outside one event of probability zero, the numerical sequence at each outcome converges. Convergence in probability instead fixes a positive error tolerance and asks that the probability of exceeding it vanish. The exceptional outcomes may change with the index. For random variables in Lp on a common probability space, with p at least one, Lp convergence asks that the expected pth power of the error vanish. It therefore measures magnitude as well as frequency.

Convergence in distribution concerns only the laws: expectations of every bounded continuous test function approach the corresponding limit expectation. It does not compare two variables at the same outcome. For example, a symmetric random sign and its negative have identical laws, despite differing by two everywhere. Alternating between these two coupled variables preserves the distribution at every index while preventing convergence in probability to the original sign.

Xn→PX  ⟺  ∀ε>0: P(∣Xn−X∣>ε)⟶0,Xn→LpX  ⟺  E∣Xn−X∣p⟶0.X_n\xrightarrow{\mathbb P}X\iff\forall\varepsilon>0:\ \mathbb P(|X_n-X|>\varepsilon)\longrightarrow0,\qquad X_n\xrightarrow{L^p}X\iff\mathbb E|X_n-X|^p\longrightarrow0.
Both notions use a common probability space, and Lp convergence is stated for variables in Lp. The tolerance is fixed before taking the limit.↔ Scroll wide equations horizontally.

Prove the arrows and inspect their hypotheses

Markov’s inequality applied to the pth power of the error proves that Lp convergence implies convergence in probability. Almost sure convergence also implies convergence in probability: the error indicators for a fixed tolerance converge to zero almost surely and are bounded by one, so dominated convergence applies. Convergence in probability implies convergence in distribution. The latter implication can be proved by extracting an almost surely convergent subsubsequence from any subsequence and applying bounded continuous test functions.

The converse arrows generally fail. Nevertheless, convergence in probability to a constant is equivalent to convergence in distribution to that constant. Also, if a sequence converges in probability, it has a subsequence converging almost surely to the same limit: choose successive indices whose failure probabilities at tolerance powers of one half form a summable sequence, then use Borel–Cantelli. A subsequence is the conclusion; this construction does not upgrade the entire sequence to almost sure convergence.

P(∣Xn−X∣>ε)≤E∣Xn−X∣pεp.\mathbb P(|X_n-X|>\varepsilon)\leq\frac{\mathbb E|X_n-X|^p}{\varepsilon^p}.
A vanishing error moment forces the error probability to vanish, but the reverse needs additional control of large values.↔ Scroll wide equations horizontally.

Rare errors explain the missing arrows

Take a single uniform random variable U on the unit interval and let the error be n whenever U is at most one over n squared, and zero otherwise. The error eventually vanishes at every positive U, so convergence is almost sure. Its first absolute moment vanishes, but its second moment remains one. Increasing a spike while shrinking its support can therefore preserve a moment even while every typical path becomes exactly zero.

For a different failure, let independent indicator variables have success probabilities one over n, beginning at n equals two. Each indicator tends to zero in probability and in every finite Lp norm, because its pth power is itself. Yet the probabilities sum to infinity, and independence lets the second Borel–Cantelli lemma conclude that successes occur infinitely often almost surely. The indicators do not converge almost surely to zero. Independence is essential for that particular use of the second lemma.

Xn=n1{U≤n−2}:E∣Xn∣=n−1,E∣Xn∣2=1.X_n=n\mathbf1_{\{U\leq n^{-2}\}}:\qquad \mathbb E|X_n|=n^{-1},\qquad \mathbb E|X_n|^2=1.
The same coupled sequence converges almost surely and in L1, but not in L2.↔ Scroll wide equations horizontally.

Work it through

Classify a shrinking spike

Let U be uniform on (0,1), and define Xn to be n when U is at most 1/n² and zero otherwise. Classify almost sure, probability, L1, and L2 convergence to zero.

  1. For each fixed U greater than zero, choose an index beyond 1/√U. All later indicators vanish. This proves almost sure convergence on the entire sample space (0,1).

  2. For any fixed positive tolerance and all sufficiently large n, the only error event is the interval of length 1/n². Its probability tends to zero, independently confirming convergence in probability.

  3. Multiply each spike size by its event probability: the first absolute moment is n/n² = 1/n, whereas the second moment is n²/n² = 1. Only the first vanishes.

The result

The sequence converges to zero almost surely, in probability, and in L1, while failing to converge in L2. No contradiction occurs: the second moment detects larger errors more strongly.

The living notebook

Explore the model

Interactive lab

Interactive lab · probability

Separate count spread from average error

Set p to 0.50, then compare n = 5 and n = 20. Predict the displayed count variance before moving the control. Divide it by n² to obtain the variance of the sample proportion: it falls from 0.05 to 0.0125 although the count variance rises.

Separate count spread from average errorThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forThe bars are exact binomial laws for finite n, and playback only highlights outcomes. They illustrate moment concentration of proportions, not sample paths, almost sure convergence, or rare-event counterexamples.

Pause & reason

Check your understanding

Independent indicators Xn satisfy P(Xn = 1) = 1/n for n ≥ 2. Which statement is correct?

Your turn

Try it yourself

Let independent events An have probabilities 1/n for n ≥ 2, and let Yn = 1 on An and zero otherwise. Decide whether Yn converges to zero in L3 and almost surely. Explain why the two answers can differ.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

The third absolute power of an indicator equals the indicator. For the pathwise claim, check both hypotheses of the second Borel–Cantelli lemma.

Compare with the solution

The third error moment is P(An) = 1/n, which tends to zero; hence L3 convergence holds. The events are independent and their probabilities have a divergent sum, so An occurs infinitely often with probability one. Along those outcomes, Yn keeps returning to one and cannot converge to zero. Moment convergence controls the distribution of each separate error, whereas almost sure convergence controls the entire infinite sequence at a fixed outcome.

Keep the idea

Take it with you

  • A convergence claim must identify whether it compares outcomes, error moments, or only laws.
  • Almost sure and Lp convergence imply convergence in probability through different arguments.
  • Shrinking probabilities do not by themselves control large moments or infinitely many failures.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.