Estimability and rank-deficient designs

Work in the fixed-design mean model $E(Y)=X\beta$, without yet specifying error variances or distributions. If $v$ is in the nullspace of $X$, then $\beta$ and $\beta+v$ produce exactly the same mean vector. No information about that mean can distinguish the two parameter descriptions. A proposed quantity $c^{\mathsf T}\beta$ is therefore identifiable from the mean only if $c^{\mathsf T}v=0$ for every such invisible change.

The orthogonal complement of the nullspace is the row space. Consequently the same condition says that $c$ must lie in the column space of $X^{\mathsf T}$. Equivalently, there is a vector $a$ with $X^{\mathsf T}a=c$. Then $a^{\mathsf T}Y$ has expectation $c^{\mathsf T}\beta$ for every coefficient vector. This is linear estimability: it describes existence of an unbiased linear estimator, not whether its variance is small or whether a particular data realization is convincing.

Consider a group-means parameterization with an intercept and an indicator for every group. Adding a constant to the intercept while subtracting it from every group coefficient leaves all fitted means unchanged. Dropping one indicator, setting one group coefficient to zero, or imposing a sum-to-zero constraint can choose coordinates. These conventions do not supply extra observations. A group mean or a difference of group means can be estimable even though an isolated coefficient in the redundant parameterization is not.

A Moore–Penrose pseudoinverse chooses the coefficient vector of smallest Euclidean norm among least-squares solutions. That is a mathematical convention that depends on parameter scaling; it is not evidence that the true coefficients have that norm. All least-squares solutions differ by nullspace vectors, so an estimable combination has the same numerical value for every solution. A nonestimable combination can change while the fitted responses and the residual sum of squares remain exactly fixed.

At a new predictor row $x_*^{\mathsf T}$, the proposed mean is $x_*^{\mathsf T}\beta$. It is estimable only when $x_*$ belongs to the row space of the observed design. This is a stronger issue than ordinary extrapolation uncertainty: a new combination of perfectly aliased factors can be unidentified even if each factor level appeared in the data. Printing a prediction after silently choosing one coefficient convention can hide that ambiguity rather than resolve it.

Exact deficiency and near deficiency deserve different responses. Exact redundancy makes some combinations unidentified. Nearly dependent columns can leave every coefficient identifiable but make certain estimates highly variable and numerically sensitive. Singular values, scientific design knowledge, and the precise prediction target help distinguish the cases. With spherical error covariance, residual degrees of freedom are $n-r$, where $r$ is the design rank; counting all printed coefficients would incorrectly count redundant directions as information consumed.