advanced35 min studyLesson 710 in Mathematics

Semiparametric influence functions and efficient scores

Use tangent-space projection to separate target information from nuisance variation.

Before you begin

On this page

The question for today

What you’ll understand

Define pathwise differentiability using scores of regular submodels.
Relate a canonical gradient to nuisance-orthogonal efficient scores.
Calculate a missing-data efficiency variance and a finite-dimensional projection.

For iid observations, a semiparametric model combines a finite-dimensional target with an unrestricted or high-dimensional nuisance component. The geometry now lives in a space of square-integrable mean-zero functions of an observation. Scores describe directions in which the data law can change, and influence functions represent how the target changes along those directions. Projection identifies the variation that is useful for the target after nuisance variation has been removed. This language connects efficient estimation with conditional expectation, but it does not guarantee that every target admits regular root-sample-size inference.

A derivative represented in a Hilbert space

At a fixed distribution, collect scores of differentiable regular parametric submodels and take the closed linear span in the mean-zero square-integrable function space. This is the tangent space. A scalar target is pathwise differentiable when its derivative along every such submodel can be represented by the inner product with a single mean-zero finite-variance function. An influence function satisfying that identity need not be unique outside the tangent space; its projection onto the tangent space is the unique canonical gradient.

The canonical gradient has the smallest variance among these derivative representatives, because any other representative differs by a function orthogonal to the tangent space. Directions that leave the target unchanged form the nuisance tangent space, and every valid representative is orthogonal to those directions. If no suitable square-integrable derivative representative exists, the usual regular asymptotically linear root-sample-size framework may be unavailable. Introducing a symbol called an influence function cannot substitute for verifying the derivative identity and its integrability.

\left.\frac{d}{dt}\psi(P_t)\right|_{t=0}=E_P(\phi s),\quad E_P\phi=0,\quad E_P\phi^2<\infty,\qquad \phi_{\mathrm{can}}=\Pi_{\mathcal T}\phi.

Scores s describe regular submodel directions, and projection selects the minimum-variance derivative representative.↔ Scroll wide equations horizontally.

Efficient scores remove nuisance information

When a scalar target coordinate has a parametric score and nuisance directions are specified, subtract the score's orthogonal projection onto the closed nuisance tangent space. The residual is the efficient score. If its variance is finite and strictly positive, dividing it by that variance gives the efficient influence function in the regular identifiable setting. The reciprocal efficient information is then the regular asymptotic variance bound. A zero residual means the proposed target direction is not separated from nuisance variation by this argument.

An estimator attains the bound only after an actual asymptotic expansion is established with that influence function and a negligible remainder. In modern procedures, nuisance functions are estimated from data, so one must control both stochastic dependence and bias from those estimates. Sample splitting or cross-fitting can ease empirical-process conditions, but it does not eliminate bias by itself. Appropriate norm convergence, positivity, and sufficiently small product remainders remain essential. Rate statements belong to a particular estimator and model, rather than to projection geometry alone.

S_{\mathrm{eff}}=S_\psi-\Pi_{\mathcal T_\eta}S_\psi,\qquad I_{\mathrm{eff}}=E S_{\mathrm{eff}}^2,\qquad \phi_{\mathrm{eff}}=I_{\mathrm{eff}}^{-1}S_{\mathrm{eff}}.

This scalar normalization assumes a regular target coordinate and positive efficient information.↔ Scroll wide equations horizontally.

Missing outcomes reveal both nuisance roles

Let an observed covariate always be available, while a response is seen only when a response indicator equals one. Assume the indicator is conditionally independent of the outcome given the covariate, and its conditional probability is bounded away from zero. Then the population mean is identified by averaging the conditional outcome regression. In the unrestricted observed-data model with this missing-at-random interpretation and finite variance, its efficient influence combines a regression fluctuation with an inverse-response-probability weighted residual.

The residual has conditional mean zero, so its cross term with the regression fluctuation vanishes. Its conditional second moment is the outcome variance divided by the response probability. This shows precisely why sparse observation inflates uncertainty and why an accurate outcome regression can use information in covariates. Without missing-at-random identification or positivity, this formula need not identify the desired mean or have finite variance. The numerical example treats the nuisance functions as population quantities to calculate the bound, rather than claiming that an arbitrary fitted procedure attains it.

Work it through

An efficiency bound with missing responses

Let $W$ be binary with equal probabilities. Assume missing at random, response probabilities $\pi(0)=0.5$ , $\pi(1)=0.8$ , outcome means $m(0)=1$ , $m(1)=4$ , and conditional variances $v(0)=4$ , $v(1)=9$ . Find the mean and efficient influence variance for observations $(W,R,RY)$ .

The target mean is $\psi=E m(W)=2.5$ . In this observed-data model the efficient influence is $\phi=m(W)-\psi+R\{Y-m(W)\}/\pi(W)$ . The product involving an unobserved outcome is interpreted as zero when $R=0$ .
Conditional independence and the regression identity give conditional mean zero for the weighted residual. Its conditional variance is $v(W)/\pi(W)$ , and its covariance with $m(W)-\psi$ is zero.
The regression variance is $2.25$ . The average residual variance is $\tfrac12\{4/0.5+9/0.8\}=9.625$ , so the efficient root-sample-size variance is $11.875=95/8$ .

The result

With fully observed outcomes the variance would be $2.25+(4+9)/2=8.75$ . Missingness raises the information bound here; positivity and finite conditional variances keep it finite.

The living notebook

Explore the model

Interactive lab

Interactive lab · vectors

Project away a nuisance direction

Pause/Reset and set BOTH controls to angle $45$ degrees, magnitude $1$ . Predict horizontal and vertical components $1/\sqrt2$ . Regard the horizontal axis as nuisance: subtracting its projection leaves squared residual length $1/2$ . For a fresh comparison Pause/Reset and set BOTH to angle $90$ degrees, magnitude $1$ : residual squared length is $1$ . Changes retain the paused clock.

Vectorx / y projectionsTrace marker

x component: 1.393
y component: 0.975
Magnitude: 1.7

Angle35°Magnitude1.7

t = 0 s

Playback traces a fixed vector and its projections. Angle and magnitude set the geometry; the moving marker does not change them. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forEuclidean projection illustrates score orthogonality. The instrument displays two components, not a fitted missing-data estimator or an infinite-dimensional tangent space.

Pause & reason

Check your understanding

Interactive checks need JavaScript. You can still read the explanation.

Read the checkpoint explanation

Cross-fitting separates nuisance training from evaluation and can simplify stochastic control. It does not by itself identify a target, enforce positivity, or make nuisance-product bias negligible.

Your turn

Try it yourself

In the normal family $N(\mu,v)$ , treat $\mu$ as the target and the unknown positive variance $v$ as nuisance. Find the efficient score and influence for the mean. Give the resulting standard-error approximation when $v=9$ and $n=225$ .

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Calculate the covariance of the mean and variance scores using the vanishing third centered normal moment.

Compare with the solution

The mean score is $S_\mu=(X-\mu)/v$ and the variance score is $S_v=\{(X-\mu)^2-v\}/(2v^2)$ . Their inner product is zero because the first and third centered normal moments vanish. Thus nuisance projection does not change the mean score: $S_{\mathrm{eff}}=S_\mu$ , $I_{\mathrm{eff}}=1/v$ , and $\phi_{\mathrm{eff}}=X-\mu$ . The efficient root-sample-size variance is $v$ , giving standard error $\sqrt{9/225}=0.2$ . This finite-dimensional nuisance example illustrates the same projection principle used for closed infinite-dimensional tangent spaces.

Keep the idea

Take it with you

A canonical gradient represents path derivatives within the tangent space.
Efficient scores remove nuisance projections before information is inverted.
Efficient estimation requires an influence expansion with controlled nuisance remainders.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.