The delta method and functional delta method

Suppose the root-sample-size error of a vector estimator converges weakly to a tight random vector. If the transformation is differentiable at the true parameter, its linear approximation carries this limit to the derivative applied to that vector. For a normal input limit, the transformed covariance is obtained by multiplying on the left and right by the derivative and its transpose. The covariance formula describes the limiting scaled error; dividing it by sample size gives a large-sample variance approximation.

The derivative is evaluated at the population parameter in the theorem. A practical standard error may substitute a consistent estimator if the derivative and variance formula are continuous nearby. Transformations with singularities require checking that the true parameter lies away from them. For instance, log odds is smooth for an interior probability, but its derivative diverges at zero and one. A formula derived for a fixed interior probability does not automatically remain valid for probabilities approaching a boundary with sample size.

For a map between normed spaces, Hadamard differentiability tangentially to a set of directions asks that difference quotients converge along every admissible sequence of directions approaching a tangent direction. The derivative must be continuous and linear on the tangent space. If the scaled input error converges weakly to a tight limit supported on that space, the functional delta method applies. All perturbed inputs must remain in the map's domain; arbitrary signed perturbations of a distribution are not automatically distributions.

For a fixed quantile level strictly between zero and one, assume the distribution is differentiable at its unique target quantile with a positive derivative there. The inverse-distribution functional has derivative given below, along directions continuous at the quantile, using the usual uniform topology for distribution perturbations. Empirical-distribution fluctuations converge to a Brownian bridge, whose variance at that location is the level times one minus the level. A process theorem for many quantiles needs stronger uniform density conditions; the single-quantile calculation does not assert those.

A zero derivative yields a zero first-order limit rather than a useful normal approximation. Consider squaring an estimator whose true parameter is zero and whose root-sample-size limit is centered normal with variance $\sigma^2$. Multiplying the squared estimator by sample size then gives a scaled chi-square limit with scale $\sigma^2$. This follows directly from the continuous mapping theorem applied to the original root-sample-size error. It requires a different normalization from the first-order delta method.

Likewise, the existence of directional derivatives one direction at a time does not establish the functional theorem. The directions themselves fluctuate with the sample, so stability under changing directions matters. Maxima, kinks, and active inequality constraints can have nonlinear directional derivatives and nonnormal limits. They need a suitable directional theory or a direct calculation. Before using an automatic standard-error recipe, identify the input space, topology, limiting process, derivative, and any loss of first-order information.