advanced32 min studyLesson 707 in Mathematics

Asymptotic efficiency and convolution bounds

State what regularity rules out before comparing an estimator with information.

On this page

What you’ll understand

  • Define local regularity with centering at the moving target.
  • Interpret an information-bound Gaussian component plus independent extra noise.
  • Separate asymptotic limiting variance bounds from finite-sample claims.

An estimator can look extraordinarily accurate at a single parameter point by sacrificing stability nearby. Efficiency theory therefore needs a class of estimators whose local behavior is regular. Within a finite-dimensional LAN model, the convolution theorem says that a regular estimator's limiting error contains an information-determined Gaussian component and possibly independent additional noise. This is a statement about local distributions, not a universal variance bound for every biased finite-sample procedure. We develop the scalar-target formulation and calculate a deliberately inefficient regular estimator.

Regularity uses a moving center

Let the target be a differentiable scalar function of a finite-dimensional parameter. An estimator is regular at the reference point if, under every fixed local displacement, its root-sample-size error centered at the displaced target converges to the same limiting law. Centering at the moving target is essential. An estimator's distribution under the reference law alone cannot establish this property. The definition makes performance stable across alternatives that remain statistically comparable at the LAN scale.

For an interior dominated iid model with differentiability in quadratic mean, nonsingular information, and the usual LAN structure, the finite-dimensional convolution theorem constrains any such regular estimator with a limiting distribution. Its limit is the convolution of the efficient centered Gaussian with another probability law. The two components can be realized independently. If the limiting law has finite second moment, its variance is at least the efficient Gaussian variance. Convergence of actual scaled finite-sample variances requires extra uniform integrability; weak convergence alone does not give that conclusion.

L=N ⁣(0,ψ˙θ0I−1ψ˙θ0T)∗M.L=N\!\left(0,\dot\psi_{\theta_0}I^{-1}\dot\psi_{\theta_0}^{\mathsf T}\right)*M.
A regular limiting error contains the efficient Gaussian component, with independent additional noise represented by M.↔ Scroll wide equations horizontally.

An influence function reveals the extra noise

For a regular asymptotically linear estimator, an observation-level influence function has mean zero and finite second moment. Its covariance with the model score equals the derivative of the target. The efficient influence function is the target derivative times inverse information times the score. Subtracting it from any other valid influence function leaves a residual orthogonal to the score. Orthogonality splits the variance into efficient variance and a nonnegative remainder; the joint central limit theorem makes the corresponding Gaussian components independent.

This argument applies within its stated asymptotically linear class, whereas the convolution theorem also covers regular limits that are not introduced through an explicit influence expansion. Equality in the variance bound means no additional limiting variance, provided those variances exist. It does not by itself compare finite-sample bias, numerical cost, robustness to model misspecification, or higher-order coverage. Choosing an estimator still involves those questions, but an efficiency claim should identify the model and the target whose information bound is being attained.

ϕeff=ψ˙θ0I−1ℓ˙θ0,E(ϕℓ˙θ0T)=ψ˙θ0,Var⁡(ϕ)=ψ˙θ0I−1ψ˙θ0T+Var⁡(ϕ−ϕeff).\phi_{\mathrm{eff}}=\dot\psi_{\theta_0}I^{-1}\dot\ell_{\theta_0},\quad E(\phi\dot\ell_{\theta_0}^{\mathsf T})=\dot\psi_{\theta_0},\quad \operatorname{Var}(\phi)=\dot\psi_{\theta_0}I^{-1}\dot\psi_{\theta_0}^{\mathsf T}+\operatorname{Var}(\phi-\phi_{\mathrm{eff}}).
The score-orthogonal residual accounts for excess asymptotic variance.↔ Scroll wide equations horizontally.

Why exceptional pointwise performance is allowed

In a normal location model, a Hodges-type estimator sets the sample mean to zero whenever its magnitude is at most the inverse fourth root of sample size, and otherwise uses the sample mean. At the zero parameter, its root-sample-size error converges to zero. Yet under a local mean equal to a fixed nonzero displacement divided by the square root of sample size, it still selects zero with probability tending to one, so the error centered at the local target tends to the negative displacement.

The limit therefore depends on the local displacement and violates regularity. This does not refute the convolution bound; it explains why that condition is present. A pointwise variance comparison at zero would hide the unstable neighboring behavior. When assessing a claimed improvement over information, inspect the target's differentiability, the local centering, the estimator's regularity, and the existence of the relevant moments. A single favorable simulation at one parameter cannot settle these structural requirements.

Efficient estimation of a squared normal mean

For independent N(θ,4)N(\theta,4) observations, estimate ψ(θ)=θ2\psi(\theta)=\theta^2 at θ0=3\theta_0=3. Find the efficient influence function and root-sample-size variance, then give a standard-error approximation at n=400n=400.

  1. The score is (X−3)/4(X-3)/4, information is I=1/4I=1/4, and the target derivative at this point is ψ˙=6\dot\psi=6. The target is locally nonflat because the reference mean is nonzero.

  2. The efficient influence function is 6⋅4⋅(X−3)/4=6(X−3)6\cdot4\cdot(X-3)/4=6(X-3). Its variance is 36⋅4=14436\cdot4=144, equal to ψ˙2/I\dot\psi^2/I.

  3. The squared sample mean has this first-order influence by the delta method, and its bias is negligible at root-sample-size scale. The standard-error approximation is 144/400=0.6\sqrt{144/400}=0.6.

The result

The squared sample mean attains the local bound at θ0=3\theta_0=3. At zero, the derivative vanishes and a separate second-order analysis is needed.

Explore the model

Interactive lab

Interactive lab · sampling

Observe the cost of using only half the data

Pause/Reset and set BOTH controls to p=0.40p=0.40, n=50n=50: predict variance 0.00480.0048. For a fresh comparison Pause/Reset and set BOTH to p=0.40p=0.40, n=100n=100: predict 0.00240.0024. Compare the exact fraction distributions. Control changes pause and retain elapsed time; Reset clears the clock.

Observe the cost of using only half the dataExact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Current readings: Mean sample proportion: 0.5; Sample-mean variance: 0.0125; Uniform CDF error bound: ≤ 0.1062. All parameters are adjustable with the labeled controls below.Exact distribution of the mean of 20 independent Bernoulli variables with success probability 0.5; playback highlights outcome 0.00.250.50.75100.0580.1150.1730.23Sample mean · k / nProbability massNormal bin masses are an approximationk = 0 · exact probability = 9.54e-7
Exact binomial massNormal interval massHighlighted outcome / mean
Mean sample proportion
0.5
Sample-mean variance
0.0125
Uniform CDF error bound
≤ 0.1062
t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forHalving sample size doubles this estimator variance. The instrument illustrates a data-use cost, rather than proving a convolution theorem.

Check your understanding

What does a finite-variance convolution bound imply for a regular estimator in the stated LAN model?

Try it yourself

For even nn, split independent N(θ,4)N(\theta,4) observations into equal halves with means AnA_n and BnB_n. Estimate θ\theta using only Tn=AnT_n=A_n. Decompose its root-sample-size error into a full-data efficient component and an independent residual. Compare variances.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use An=(An+Bn)/2+(An−Bn)/2A_n=(A_n+B_n)/2+(A_n-B_n)/2 and calculate the covariance of the two terms.

Compare with the solution

The full-data mean is (An+Bn)/2(A_n+B_n)/2 and the residual is (An−Bn)/2(A_n-B_n)/2. Each half-mean has variance 8/n8/n, and the halves are independent. Both displayed terms have variance 4/n4/n and covariance zero. They are jointly normal, so they are independent. Thus n(Tn−θ)\sqrt n(T_n-\theta) has variance 88, decomposed into efficient variance 44 and residual variance 44. The half-data estimator is regular but twice as variable at the limiting scale.

Take it with you

  • Regularity compares locally shifted laws after centering at the shifted target.
  • The efficient Gaussian component gives a limiting-distribution information bound.
  • Pointwise superefficiency and finite-sample variance claims need separate analysis.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.