advanced32 min studyLesson 709 in Mathematics

Bootstrap consistency and counterexamples

Compare conditional laws and expose a persistent resampling atom.

Before you begin

On this page

The question for today

What you’ll understand

Express bootstrap consistency as conditional-law approximation in probability.
Separate a valid bootstrap distribution from its Monte Carlo approximation.
Prove an empirical-bootstrap failure for an endpoint estimator.

The empirical bootstrap treats the observed empirical distribution as a population and draws a new sample from it with replacement. It can approximate an estimator's sampling error without explicitly fitting a parametric family. However, consistency of the original estimator does not guarantee consistency of the resampling distribution. The theorem must compare the right centered and scaled conditional laws. A sample mean supplies a regular success case, while a uniform endpoint supplies an exact failure mechanism that persists no matter how many resampling replicates are computed.

The probability law being approximated is conditional

Given the data, draw independent bootstrap observations from the empirical measure and recompute the statistic. The bootstrap distribution is conditional on the original sample and is itself random across original samples. For a normalization chosen for the statistic, consistency means this conditional law approaches the original statistic's limiting error law in probability. One can express this using bounded-Lipschitz distance; for a continuous scalar limiting distribution, appropriate distribution-function convergence yields a uniform distribution-function conclusion.

The number of bootstrap replicates controls numerical approximation to that conditional law. Sending the replicate count to infinity for a fixed data set recovers the exact bootstrap distribution, which may still be statistically wrong. Increasing the original sample size addresses a different question. Centering also matters: the bootstrap statistic is usually compared with its original fitted target, not with the unknown population target. Using the wrong center can obscure a failure even when all the resampling computations are internally accurate.

d_{\mathrm{BL}}\!\left(\mathcal L^*\{r_n(T_n^*-T_n)\},\mathcal L(T)\right)\xrightarrow{P}0,\qquad r_n(T_n-\theta)\Rightarrow T.

The star denotes the law conditional on the observed data; the normalization must match the statistic.↔ Scroll wide equations horizontally.

Smooth maps inherit a suitable bootstrap limit

For an iid sample mean with finite covariance, the conditional bootstrap central limit theorem gives the same centered normal limit as the original mean. Smooth finite-dimensional transformations can then inherit validity through a conditional delta method. For empirical-distribution functionals, one needs a suitable conditional empirical-process limit and Hadamard differentiability in the relevant tangent space. Merely having a named statistic or a pointwise derivative does not establish those requirements. Studentization also needs a consistent, well-behaved variance estimate.

In the Bernoulli case, conditional on the observed success fraction, the bootstrap success count has an exact binomial law with that fraction as its success probability. This bounded triangular-array setting makes the plug-in idea explicit. If the true probability is fixed and interior, the observed fraction converges to it, and the conditional normal limit has the correct limiting variance. At a degenerate endpoint, or for a different nonsmooth statistic, one must recheck the argument rather than transferring the mean theorem by analogy.

K^*\mid X_1,\ldots,X_n\sim\operatorname{Binomial}(n,\widehat p),\qquad \operatorname{Var}^*(\widehat p^*)=\frac{\widehat p(1-\widehat p)}{n}.

For binary data, the exact conditional bootstrap mean law is a binomial plug-in law.↔ Scroll wide equations horizontally.

An empirical distribution cannot invent a larger maximum

Suppose observations are uniform between zero and an unknown positive endpoint. Their maximum is consistent, but its informative error has sample-size scaling and an exponential limit. An empirical resample can never exceed the observed maximum. More decisively, it includes that maximum with a probability tending to a positive constant. Consequently its centered maximum has an atom at zero that the continuous limiting endpoint error does not possess. This discrete defect is structural, rather than Monte Carlo noise.

Some modified resampling schemes can handle particular nonregular problems, but their sample sizes, smoothing, model assumptions, or alternative centering require separate justification. No single modification follows just from diagnosing the atom. The practical lesson is to examine a statistic's local map and rate before trusting an automated interval. A bootstrap can faithfully reproduce a fitted empirical population while failing to reproduce the statistical feature, such as an unseen support endpoint, that determines the original estimator's limiting error.

Work it through

A uniform maximum retains a spurious bootstrap atom

Let $X_i$ be independent uniform observations on $[0,2]$ , and write $M_n=\max_iX_i$ . Compare the limit of $n(2-M_n)/2$ with the empirical-bootstrap quantity $n(M_n-M_n^*)/M_n$ . Calculate its conditional atom at zero for $n=10$ .

For fixed nonnegative $t$ and sufficiently large $n$ , the original upper-tail probability is $P\{n(2-M_n)/2>t\}=(1-t/n)^n\to e^{-t}$ . Thus the endpoint error converges to a continuous unit-exponential law.
The observed maximum is unique with probability one. A bootstrap maximum equals it exactly when at least one of the $n$ resampled indices chooses that observation. The conditional probability is $1-(1-1/n)^n$ .
At $n=10$ , this atom is $1-0.9^{10}=0.6513215599$ . As sample size grows it tends to $1-e^{-1}\approx0.632121$ , whereas the limiting exponential distribution assigns probability zero to the origin.

The result

The resampling error cannot converge to the required continuous exponential law because its nonvanishing atom remains at zero. More bootstrap replicates estimate that incorrect atom more accurately.

The living notebook

Explore the model

Interactive lab

Interactive lab · sampling

Inspect an exact binary bootstrap distribution

For a sample of $80$ with $28$ successes, Pause/Reset and set BOTH controls to $p=0.35$ , $n=80$ . Predict mean $0.35$ and variance $0.00284375$ . This is the exact conditional bootstrap proportion law. For a fresh comparison Pause/Reset and set BOTH to $p=0.50$ , $n=80$ : variance is $0.003125$ . Changes retain the paused clock.

Exact binomial massNormal interval massHighlighted outcome / mean

Mean sample proportion: 0.5
Sample-mean variance: 0.0125
Uniform CDF error bound: ≤ 0.1062

Success probability p0.5Independent samples n20

t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forThe bars are deterministic exact laws, not generated resamples. There is no replicate-count control, and the binary mean display does not repair the endpoint counterexample.

Pause & reason

Check your understanding

Interactive checks need JavaScript. You can still read the explanation.

Read the checkpoint explanation

The maximum is consistent, but its empirical-bootstrap error has a persistent atom absent from the exponential limit. Replicate count addresses simulation error, while this discrepancy belongs to the exact conditional law.

Your turn

Try it yourself

A binary data set contains $28$ successes in $80$ observations. Find the conditional empirical-bootstrap proportion variance, its standard deviation, and the variance of its root-sample-size centered error. What does increasing the replicate count change?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Conditional on the data, each resampled observation is Bernoulli with success probability equal to the observed fraction.

Compare with the solution

The plug-in fraction is $\widehat p=28/80=0.35$ . The conditional variance is $0.35(0.65)/80=0.00284375$ and its square root is approximately $0.0533268$ . The variance of $\sqrt{80}(\widehat p^*-0.35)$ is $0.2275$ . Increasing the replicate count improves the Monte Carlo approximation to this same conditional law; it changes neither its exact variance nor the original sample's information. For fixed interior true probability, a conditional CLT supplies the corresponding consistency argument.

Keep the idea

Take it with you

Bootstrap consistency concerns a centered, scaled conditional distribution.
A valid conditional-process theorem and smooth map can transfer regular limits.
More replicates cannot correct a structural discrepancy in the exact bootstrap law.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.