First- and second-order optimality

Fix a candidate and restrict the objective to a straight line through it. If the candidate is a local minimum in an open domain, the scalar restriction has a minimum at line parameter zero for every direction. Its first derivative must vanish. Since that derivative is the inner product of the gradient with the chosen direction, the gradient must be the zero vector. This argument establishes necessity; it does not reverse itself simply because solving the gradient equation was difficult.

When the gradient vanishes, the second-order Taylor term becomes the first available local signal. A direction of negative quadratic curvature produces a lower value for sufficiently small nonzero movements, so a local minimum must have a positive-semidefinite Hessian. Both signs of the line parameter are allowed in this unconstrained argument. At a feasible boundary, or on a curved constraint manifold, one must instead identify feasible tangent and critical directions before using an analogous curvature test.

If the gradient is zero and the Hessian is positive definite, its smallest eigenvalue is positive. Continuity and the Taylor remainder then make the positive quadratic term dominate all smaller-order errors in a sufficiently small neighborhood. The result is a strict local minimum with quadratic growth. This is a local statement: the same objective can contain a deeper well outside that neighborhood, and inspecting the Hessian only at the candidate cannot exclude it.

Positive semidefiniteness without a strict gap is inconclusive. The functions $x^4$, $-x^4$, and $x^3$ all have zero first and second derivatives at zero, yet respectively give a strict minimum, a strict maximum, and neither. Higher-order terms matter precisely in directions where the quadratic model is flat. Conversely, a negative Hessian eigenvalue is a decisive rejection of local minimality, even when other directions have positive curvature.

In computation, gradients and eigenvalues are rarely exactly zero. Report the residual norm, the estimated smallest curvature, the norm convention, and the tolerances. A numerically positive eigenvalue smaller than the uncertainty of the derivative evaluation is weak evidence. Directional derivative checks and Hessian-vector products help investigate an implementation, but a few sampled directions cannot prove that a large matrix has no negative eigenvalue. Exact symbolic examples are useful training because they expose what each test can and cannot conclude.

Convexity supplies a separate global bridge: for a differentiable convex objective on a convex domain, a zero gradient gives a global minimizer. Nonlinear optimization usually lacks that bridge. In this module, a solver's convergence from a broad set of initial points will mean convergence toward an appropriate stationarity condition, when the algorithm's hypotheses hold. It will not silently mean that the best possible value over the whole domain has been located.