Z-estimators and estimating equations

Assume iid observations, and let the parameter and estimating equation have the same finite dimension. Assume the estimator is consistent, the population map vanishes at the target, and its derivative there is nonsingular. Require the empirical residual at the fitted parameter to be negligible at root-sample-size scale. Finally, suppose the empirical process changes negligibly, at that same scale, when its argument moves from the target to the estimator. This last stochastic equicontinuity condition is not implied merely by differentiability of the population expectation.

For a smooth equation, a uniform law for the sample Jacobian can provide the required local control. For nonsmooth equations, empirical-process arguments may provide it instead. Add and subtract the population map at the fitted parameter, expand that population map, and isolate the empirical fluctuation at the fixed target. A central limit theorem for this fluctuation then supplies the distribution. This separation makes it clear which part is deterministic calculus and which part controls the random choice of argument.

Write the equation as an empirical average of a mean-zero vector function at the target. Its derivative matrix describes how the expected equation responds to a small parameter displacement. Moving the empirical fluctuation to the other side gives a minus sign in the expansion. The corresponding influence function is the negative inverse derivative applied to the observation's estimating function. Covariances therefore have the same sandwich form as for M-estimators, although the estimating function need not be a gradient.

A sample quantile illustrates why exact roots are unnecessary. Its indicator-based equation can jump over zero, but choosing a usual order-statistic quantile leaves a residual of order the inverse sample size when observations are distinct. That is negligible at root-sample-size scale. Positive density supplies the population derivative, while local empirical-process control handles the indicator jumps. A zero density or an atom at the target breaks this particular argument and can change both the rate and the limiting law.

An equation can be nearly zero across a wide neighborhood when its derivative is small, or can have several roots with different scientific meanings. Residual accuracy therefore needs identification and local stability before it becomes parameter accuracy. A numerical solver's stopping threshold must also be compared with the statistical scale: a fixed residual tolerance eventually overwhelms a shrinking sampling error. There is no universal conversion from a reported residual to a confidence interval without examining the local derivative.

When the derivative vanishes at the target, the inverse-Jacobian expansion cannot be used. A cubic equation driven by a sample mean supplies a simple alternative rate in the practice problem. More broadly, numerical and statistical convergence are different questions: bisection can reliably find a root of the equation supplied to it, while the statistical behavior of that root depends on the population geometry and observation law. Keeping those two questions distinct prevents an accurate computation from being mistaken for a valid normal approximation.