Generalized least squares

Assume a fixed full-column-rank design, mean-zero errors, and covariance $\sigma^2\Omega$, where Ω is known, symmetric, positive definite, and does not depend on the response. A whitening matrix W satisfies $W\Omega W^{\mathsf T}=I$. Multiplying both the response and the design by W produces a transformed mean model with spherical covariance. Ordinary least squares on that transformed model therefore has the Gauss–Markov property in the transformed coordinates. Positive definiteness makes whitening invertible and preserves design rank.

The transformed squared residual length equals a weighted quadratic form in the original residual. Taking its derivative gives the GLS normal equations and coefficient formula. A Cholesky factorization can implement whitening without explicitly computing a matrix inverse. The transformation does not require Gaussian errors for unbiasedness or BLUE efficiency. If the errors are additionally multivariate Gaussian and Ω is genuinely known, the usual exact Gaussian inference can be applied after whitening with the appropriate residual degrees of freedom.

If errors are uncorrelated and have unequal known variances, inverse covariance is diagonal. Weighted least squares then gives an observation weight proportional to inverse variance, not inverse standard deviation. Averaging independent estimates of one common mean with those weights gives more influence to more precise measurements. Multiplying all weights by the same positive constant leaves the coefficient estimate unchanged, although variance formulas must keep track of the corresponding scale convention.

Correlated noise cannot generally be handled by merely attaching a separate weight to each observation. Off-diagonal covariance says which combinations share information. Two highly positively correlated measurements can contribute much less new information than two independent ones with the same marginal variance. Conversely, negative correlation can improve the precision of an average. The covariance model should arise from the sampling design, physical measurement process, or a supported statistical structure, rather than weights chosen solely to produce a preferred fitted coefficient.

In practice Ω is often unknown. Feasible GLS estimates a covariance structure, then substitutes it into the GLS calculation. The resulting weights depend on data, so the estimator is no longer necessarily a fixed linear function of Y. Its finite-sample unbiasedness and known-covariance BLUE proof do not transfer verbatim. Consistency and asymptotic inference require assumptions about the covariance estimator, mean model, sampling regime, and dependence; an arbitrary iterative reweighting rule is not automatically valid.

Another strategy retains ordinary least-squares coefficients and adjusts their estimated covariance using a heteroskedasticity- or dependence-robust method. That addresses uncertainty under specified large-sample conditions without claiming GLS efficiency. Neither strategy repairs a mean model with omitted structure or a nonzero conditional error mean. Keep the tasks distinct: choosing the mean, modeling covariance, calculating coefficients, and justifying uncertainty. A sophisticated weighting matrix cannot compensate for a scientific interpretation that the observed design does not support.