Concentration and nonasymptotic norm bounds

Let $A$ be an $M$-by-$k$ real matrix with independent standard normal entries and assume $M\geq k$. Its largest and smallest singular values are one-Lipschitz functions of all entries in the Frobenius metric. Gaussian concentration therefore controls deviations from their expectations. Gordon-type comparison estimates locate those expectations relative to the sum and difference of the square roots of the two dimensions. Combining the statements gives the simultaneous bound in the display for every nonnegative deviation parameter.

The factor two in the failure probability covers the two tails being used. To request failure probability delta, choose the deviation parameter as the square root of twice the logarithm of two divided by delta. This is a finite probability statement about the specified independent Gaussian model, with no limit in dimension required. It does not hold with these exact numerical constants for every dependent or heavy-tailed matrix merely because each entry has variance one.

If rows are observations, divide the data matrix by the square root of sample size $M$. The associated feature covariance is $A^\top A/M$, whose eigenvalues are the squared singular values of that normalized data matrix. Thus the same finite event controls the entire covariance spectrum. Square the lower endpoint only if it is nonnegative; squaring a negative lower bound would reverse its meaning and falsely claim a positive minimum eigenvalue. With a negative endpoint the valid spectral floor is simply zero.

For a fixed aspect ratio $k/M$ below one and a deviation parameter small relative to the square root of sample size, these endpoints approach one plus or minus the square root of that ratio. Their squares match the Marchenko–Pastur support edges. This comparison connects a finite confidence event to an asymptotic law without identifying the two statements. The finite bound is typically more conservative, especially when a very small failure probability is requested or the sample is close to square.

For subgaussian entries or isotropic subgaussian rows, one can often control the operator norm by testing bilinear or quadratic forms on a finite net of unit vectors. The net has exponentially many points in dimension, so concentration must overcome a geometric entropy cost. Constants then depend on subgaussian norm bounds and on the precise independence model. This explains why a theorem should include its tail parameter and row or entry assumptions instead of advertising a dimension scale alone.

A lower singular-value guarantee is usually more delicate than an upper norm bound: preventing every direction from collapsing is stronger than preventing any direction from stretching too much. Even a square Gaussian matrix is invertible almost surely, although the displayed difference-of-square-roots lower bound cannot certify a positive minimum there. Algebraic almost-sure invertibility, quantitative conditioning, and the probability of a numerical stability event are distinct claims. A vacuous inequality identifies the limitation of that estimate, not a contradiction in the matrix model.