Leverage, influence, and residual diagnostics

In a full-rank fixed design, the diagonal entry $h_{ii}$ of the hat matrix measures how strongly response $Y_i$ contributes to its own fitted value. It depends on the predictor rows, not on the responses. Because H is an orthogonal projection, its diagonal entries lie between zero and one and sum to its rank p. Thus p/n is the average leverage, although a multiple of that average is only a screening guideline and not a universal statistical threshold.

For an intercept and a single nonconstant predictor, leverage is one over n plus the squared distance from the predictor mean divided by the total centered predictor sum of squares. Remote predictor values therefore tend to have high leverage. Under spherical errors the residual variance at observation i is $\sigma^2(1-h_{ii})$. A high-leverage point can pull the fit toward itself and thereby hide a large response discrepancy behind a relatively small raw residual. Standardizing residuals accounts for that geometry.

An influential observation materially changes a fitted quantity when it is removed or perturbed. Cook's distance summarizes the squared change in all fitted values after deleting one observation, scaled by p times the full-model residual variance estimate. In a full-rank least-squares fit with positive residual variance and $h_{ii}<1$, it can be calculated from the full residual and leverage. Its formula makes the interaction explicit: a large residual and a leverage close to one can produce a substantial change.

Influence is a sensitivity description, not proof that the observation is wrong. A rare but valid operating condition may be exactly where predictions matter most. Check provenance and units, compare the fit with and without the point, and report the consequences for the target question. Automatic deletion based on a diagnostic threshold can remove scientifically meaningful extremes and make reported uncertainty conditional on an unacknowledged selection procedure. If deleting a point destroys rank, ordinary deletion formulas need special treatment rather than division by zero.

A curved residual pattern can suggest missing mean structure; a changing spread can suggest heteroskedasticity; a run-order pattern can suggest dependence or drift. A normal-quantile plot examines compatibility with a Gaussian shape but cannot establish independence or the correctness of the mean. Ordinary fitted residuals are already correlated through I−H, even when the underlying errors are independent. Interpreting them as a fresh iid sample ignores the fitting operation that created them.

A useful diagnostic workflow links each plot to an assumption and an action. Revisit the mean specification when curvature is plausible, inspect collection order when clusters appear, and evaluate robust or covariance-aware uncertainty when variance patterns persist. A visually featureless plot can reflect low power or limited predictor coverage, so it does not verify the model. Diagnostics are strongest when combined with knowledge of how observations were collected and with sensitivity analyses for the actual predictions or contrasts being used.