The Gauss–Markov theorem and efficiency

Take $Y=X\beta+\varepsilon$ with a fixed design of full column rank. Assume $E(\varepsilon)=0$ and $\operatorname{Cov}(\varepsilon)=\sigma^2 I$ for a finite positive variance. The covariance condition combines common marginal variance, called homoskedasticity, with zero covariance between different errors. Independence is sufficient for the latter but stronger than necessary. Gaussianity is not an assumption of this version of the theorem. With random predictors these statements should be made conditional on the realized design.

Write the least-squares estimator as $AY$, where $A=(X^{\mathsf T}X)^{-1}X^{\mathsf T}$. Because $AX=I$, its expectation is the target coefficient vector for every β. Its covariance is the common error variance times the inverse Gram matrix. These are different conclusions: the conditional mean assumption establishes unbiasedness, while the covariance assumption supplies this particular variance formula. Correlated or unequal-variance errors need not destroy unbiasedness, but they invalidate the spherical formula and its optimality comparison.

Any competing estimator linear in the response has form $BY$, with B fixed once the design is fixed. Unbiasedness for every β forces $BX=I$. Let $D=B-A$. Then $DX=0$: the competitor differs only by a linear combination of response directions that contain no modeled mean. The covariance cross terms vanish because A is built from the columns of X and D annihilates those columns. What remains is an added positive-semidefinite matrix.

For a coefficient contrast c, multiply that covariance difference on both sides by c. The result is $\sigma^2\|D^{\mathsf T}c\|^2$, which cannot be negative. This proves the ordering simultaneously for every contrast rather than merely comparing the diagonal entries. Equality for a particular contrast means the competitor agrees with least squares on that contrast. Equality of the entire covariance matrix, with positive σ², forces D to vanish and therefore the competing linear estimator to be identical.

The acronym BLUE means best linear unbiased estimator: linear in observed responses, unbiased for all allowed coefficient vectors, and best by variance. It does not compare biased procedures such as ridge regression. A biased estimator can reduce mean squared error by accepting a controlled displacement of its expectation. Nor does the theorem compare arbitrary nonlinear estimators under this minimal moment model. Stronger distributional assumptions can support additional results, but those require their own arguments and loss functions.

The theorem also says nothing about whether a scientific mean model is correct, whether a coefficient represents a causal intervention, or whether deployment predictors resemble training predictors. In a rank-deficient design, the corresponding statement applies to estimable linear functions rather than to arbitrary coefficient coordinates. If the error covariance is a known positive-definite matrix instead of a multiple of identity, whitening leads to the generalized least-squares extension. The model determines what efficiency means; an acronym cannot substitute for checking that model.