Spiked models and principal components

Take independent real Gaussian observations with known mean zero and covariance $\Sigma_p=I_p+\theta vv^\top$, where $v$ is a deterministic unit vector and $\theta>0$ is fixed. The population top eigenvalue is $1+\theta$, its eigenvector is $v$ up to sign, and the other eigenvalues are one. Thus the population eigengap is theta. Estimate the covariance by the uncentered second-moment matrix divided by sample size, and let its largest eigenvalue and a unit top eigenvector define the ordinary sample principal component.

For the asymptotic formulas here, let both dimension and sample size tend to infinity with their ratio tending to a fixed $\gamma\in(0,1)$. This restriction matches the cited real-Gaussian theorem and avoids silently importing a different aspect-ratio convention. The spike strength is the excess above the unit noise eigenvalue, not the full population eigenvalue. A theorem stated with threshold $1+\sqrt\gamma$ for the population eigenvalue is therefore equivalent to threshold $\sqrt\gamma$ for this excess parameter.

In this regime, the top sample eigenvalue converges to the upper noise edge when the spike is at or below the threshold. Above it, the eigenvalue separates and converges to the outlier expression in the display. The corresponding squared overlap with the true direction tends to zero below or at threshold and to the positive displayed fraction above threshold. These are almost-sure limits in the stated Gaussian spiked model, not finite-sample identities for every data matrix.

For any fixed supercritical spike and positive aspect ratio, the limiting overlap remains below one. The direction is informative but inconsistent in Euclidean angle, and the top sample eigenvalue is inflated relative to the population eigenvalue. Calling a component detected does not mean its direction or variance is estimated consistently. Eigenvectors also have an arbitrary sign, so squared overlap or an angle insensitive to sign is the appropriate comparison. Comparing signed coordinate vectors without resolving that ambiguity can create a spurious apparent error.

In a regime with genuinely small operator covariance error relative to the population gap, a Davis–Kahan-type bound controls the sine of the leading angle by twice that error divided by the gap. This finite deterministic perturbation statement explains ordinary low-dimensional consistency. When dimension remains proportional to sample size, a generic covariance deviation need not shrink enough for that argument to yield a vanishing angle. The spiked-model limit describes more sharply what persists rather than merely declaring a perturbation bound inconclusive.

If the spike direction is itself sparse, one can restrict the variance maximization to sparse unit vectors. That changes the parameter class and can improve statistical sample requirements, but the resulting nonconvex search can be computationally difficult. Neither the ordinary-PCA threshold above nor its exact overlap formula automatically transfers to sparse PCA, unknown noise covariance, multiple interacting spikes, or non-Gaussian observations. Any extension needs its own assumptions and theorem. This distinction separates a clear baseline model from a universal claim about all dimension-reduction methods.