Quotient manifolds and gauge redundancy

A smooth free and proper Lie-group action on a smooth manifold gives a smooth quotient with a submersion from the total space. Free means that only the identity fixes each point; properness prevents pathological orbit accumulation and supplies the needed separation properties. Without these hypotheses, a quotient may have singular strata or other structures requiring a different analysis. One cannot apply smooth tangent and Hessian formulas across rank-changing orbits merely because an algebraic equivalence relation remains meaningful.

The vertical tangent space consists of velocities along an equivalence orbit. A chosen invariant metric defines the horizontal space as its orthogonal complement. The quotient differential maps horizontal vectors isomorphically to quotient tangents, which are represented by these lifts. An invariant objective is constant vertically, and its total-space metric gradient is horizontal. A retraction or transport used upstairs must also be equivariant, or otherwise induce a well-defined quotient operation; an arbitrary representative-dependent update need not produce a unique next quotient point.

Write a rank-r matrix as UVᵀ, with both U and V having full column rank. For invertible Q, the factors UQ and VQ⁻ᵀ give the same product. On this full-rank factor space the action is free and proper and produces the usual smooth fixed-rank quotient. At rank loss those assumptions change, so the same fixed-rank formulas cannot simply be extended through the singular boundary. The gauge group is general linear rather than merely orthogonal, allowing substantial rescalings of representative coordinates.

The ordinary sum of Euclidean squared factor norms is not invariant under this action. An adapted metric weights U-direction pairings by VᵀV and V-direction pairings by UᵀU, as displayed. Full column rank makes these weights positive definite, and inserting the transformed factors and tangent vectors proves invariance by cyclicity of trace. This is one valid choice, not the unique quotient geometry. Choosing a metric is separate from identifying the physical matrix product and must precede formulas for horizontal projection and gradient size.

An implementation may rescale or rebalance factors after an update to improve numerical conditioning. If that operation changes only the gauge, the represented matrix is unchanged, but tangent directions, cached gradients and quasi-Newton information must transform accordingly. Comparing raw factor differences can misreport physical progress or stagnation. A quotient stationarity test uses the invariant metric and horizontal gradient, while a residual in the represented matrix can provide a complementary observable. Neither should be confused with a representative’s arbitrary scale.

Gauge redundancy also explains singular Hessians of factorized objectives at fitted solutions. A path along an equivalence orbit leaves the objective exactly constant, giving zero physical motion even though the factor velocity is nonzero. Passing to the quotient removes these vertical directions. Genuine ill-conditioning can remain, especially near rank loss or a small spectral gap, so quotienting is not a blanket cure for every optimization difficulty. The research question is which geometry captures the model’s symmetries while yielding practical, stable computations under the relevant regularity assumptions.