Riemannian Hessians and second-order conditions

For a twice continuously differentiable objective, the Riemannian Hessian maps a tangent η to the covariant derivative of grad f in direction η. The Levi–Civita connection makes its associated bilinear form symmetric. On a Euclidean embedded submanifold with induced metric, covariant differentiation projects the ambient derivative of a tangent field back onto the tangent space. This formula does not apply unchanged to a different metric: both the gradient and its compatible connection can change. The Hessian is intrinsic even when computed through an ambient extension.

On a sphere, differentiate grad f=Pₓ∇f̄ and then project. The derivative of Pₓ contributes a scalar normal-gradient correction multiplying η. Omitting it gives the wrong quadratic form unless that scalar vanishes at the point. More general embedded manifolds have an analogous Weingarten or second-fundamental-form contribution. The ambient extension can change its separate gradient and Hessian pieces, but the combined tangent expression for the restricted objective remains the same. This is a useful consistency check when an objective has several algebraically different extensions.

For a smooth curve c starting at x with velocity η, the second derivative of f(c(t)) is the Hessian quadratic form plus the gradient paired with the curve’s intrinsic acceleration. The radial curve of a retraction therefore has a pullback Hessian equal to the intrinsic Hessian at zero if the retraction is second order. At a critical point the acceleration term vanishes for every retraction, because the gradient is zero. Away from critical points, mere first-order retraction agreement is insufficient to identify second derivatives.

This distinction matters in numerical Hessian checks. A second-order Taylor error test along a first-order retraction may fail even when the covariant Hessian implementation is correct. Conversely, a finite-difference approximation of an ambient gradient that ignores transport and projector derivatives may appear plausible on one direction while missing the intrinsic correction. A sound check specifies the metric, retraction order and tangent direction, then compares the correct pullback expansion. Symmetry of the Hessian model should be tested in the metric used by the optimization subproblem.

At a local minimum in a manifold without boundary, first-order necessity gives zero gradient and second-order necessity gives a positive-semidefinite Hessian on the entire tangent space. A positive-definite Hessian at a critical point is sufficient for a strict local minimum in finite dimensions. Semidefiniteness alone is not sufficient, and positive definiteness is not necessary: a flat fourth-order minimum can have zero Hessian. A negative Hessian direction at a critical point certifies a saddle or local maximum behavior rather than a second-order minimum.

For quadratic eigenvector objectives on spheres, the tangent Hessian at an eigenvector consists of differences between other eigenvalues and the selected one. Eigenvalue ordering therefore determines local minima, maxima and saddles even when every ambient eigenvalue is positive. Repeated eigenvalues can create zero-curvature directions and nonisolated solution sets. Algorithms seeking approximate second-order criticality must control both gradient norm and the smallest tangent-Hessian eigenvalue; a small gradient alone cannot distinguish a desired minimum from a saddle with an escape direction.