Riemannian gradients and metric choice

At x, the differential dfₓ assigns a directional derivative to each tangent η. A Riemannian metric is a smoothly varying positive-definite inner product on tangent spaces. The Riemannian gradient is the unique tangent vector satisfying gₓ(grad f,η)=dfₓ[η] for all η. Positive definiteness ensures uniqueness. In a coordinate frame with metric matrix G, its coordinate vector is G⁻¹ times the differential coefficients, not necessarily the ordinary list of coordinate partial derivatives. The frame and metric must both be specified.

The negative gradient gives the largest first-order decrease among tangent vectors of fixed metric norm, after normalization when it is nonzero. Its derivative of the objective is minus its squared metric norm. A different metric therefore changes which ambient direction qualifies as steepest while leaving the scalar differential along each fixed tangent direction unchanged. Critical points, where the differential vanishes, are metric independent, although gradient norms and rate constants are not. This distinction is central when comparing a geometric algorithm with an apparently different preconditioned method.

For an embedded manifold with the metric induced by its ambient Euclidean space, an ambient gradient can be orthogonally projected onto the tangent space. The normal part does not contribute to directional derivatives along tangents. This yields the intrinsic gradient of the restricted objective, independent of how the smooth objective was extended off the manifold. If the tangent metric changes, projecting the same ambient gradient with the old Euclidean projector generally no longer satisfies the required duality identity.

For a sphere with a metric induced by a constant ambient positive-definite matrix M, solve the duality equation together with xᵀgrad f=0. The answer involves M⁻¹ and a weighted normal correction. Preconditioning first and then using an unmodified Euclidean projection can be wrong, because the weighted normal direction is M⁻¹x rather than x. When M preserves the current tangent space the expression simplifies, but that is a special alignment condition rather than a general rule for weighted manifold gradients.

A metric can incorporate problem scales, approximations to curvature or quotient symmetry, acting as geometric preconditioning. To define a Riemannian metric it must be smooth and strictly positive on every nonzero tangent vector; a singular approximate Hessian is not automatically an admissible metric. A practical implementation must also solve the metric-duality system efficiently. A metric that makes a formal condition number attractive but requires an expensive dense inverse may lose the intended computational benefit.

On Stiefel manifolds, the induced Frobenius metric and the canonical metric differ in how they weight rotations within the current frame relative to changes of subspace. Their gradient formulas should not be interchanged. On a quotient, the metric must respect the equivalence action before it descends consistently. These are separate constraints from choosing a retraction, which only preserves the local first-order update structure. Rates or comparisons between methods must state the metric in which gradients, smoothness constants and trust-region radii are measured.