Support recovery and irrepresentability

Let $G=X^\top X/n$, let $z=X^\top\varepsilon/n$, and assume the active principal block $G_{SS}$ is invertible. If a candidate has the true signed support with sign vector $v$, the active optimality equations determine its values explicitly. Set all inactive coordinates to zero and solve the displayed linear equation. This is a diagnostic construction using the unknown true support, not an implementable selection algorithm. It allows the proof to test whether a correctly signed solution can exist.

The candidate is valid only if every active entry has the proposed sign and every inactive residual score has absolute value at most the penalty. Strict inequality on inactive scores, together with invertibility of the active block, is sufficient for uniqueness. The sign requirement depends on the smallest true coefficient as well as the inverse active block and noise. Large signals can survive shrinkage; coefficients below the threshold scale can disappear even when the design is orthogonal and there is no observation noise.

Substituting the active candidate into an inactive score gives the expression in the second display. In the noiseless case it reduces to the penalty times $G_{S^cS}G_{SS}^{-1}v$. A signed irrepresentability condition bounds the maximum absolute coordinate of this vector strictly below one, leaving a margin for noise. The stronger induced row-sum norm condition bounds it uniformly over all possible sign vectors. These signed and uniform conditions should not be treated as identical statements.

A positive minimum eigenvalue for the active block alone does not control the inactive vector. Several moderately correlated active variables can collectively represent an inactive variable well enough to push its score past the permitted threshold. Conversely, failure of a strict margin is not a blanket impossibility theorem for every target, noise realization, or estimator. A boundary equality can allow a Lasso solution but without the same robust strict-dual certificate. Exact claims must specify the sign pattern, signal size, and estimator under discussion.

A sufficient active-sign condition is that the maximum absolute coordinate of $G_{SS}^{-1}(z_S-\lambda v)$ be smaller than the minimum nonzero true coefficient magnitude. This is a finite deterministic comparison for the realized noise. High-probability selection theorems combine it with a noise bound and a design margin, then specify how the minimum signal scales with sample size, sparsity, and dimension. Removing the signal assumption from such a theorem changes its meaning: arbitrarily weak coefficients cannot generally be selected reliably.

Selection instability can coexist with stable prediction when correlated columns have similar effects on responses. Reporting one selected column as the unique scientific explanation requires more information than a penalized fit provides. After selection, ordinary fixed-model confidence intervals also require separate justification because the selected model depends on the same responses. This lesson’s KKT construction certifies a particular selection claim; it neither repairs post-selection coverage nor converts a predictive tuning criterion into a support-recovery theorem.