Descent directions and Wolfe line search

At a point with nonzero gradient g, a vector p is a descent direction when its inner product with g is strictly negative. Differentiability then makes the objective decrease for all sufficiently small positive step lengths. A positive-definite scaling matrix B gives the useful choice p solving Bp=−g: multiplying by p shows that the directional derivative is minus a positive quadratic form. An indefinite Newton Hessian need not have that property and must be checked or modified.

Write the objective along the ray as a scalar function phi. Armijo sufficient decrease asks the actual value to beat a fixed fraction of the tangent prediction. This excludes long trials whose improvement is inadequate, but arbitrarily small trials usually satisfy it too. Pure acceptance by decrease therefore does not address short-step stagnation. The additional curvature test asks whether the downhill slope has relaxed enough to justify stopping the search along this direction.

The weak Wolfe curvature condition requires the new directional derivative to be at least a fraction of the initial negative derivative. A slope that remains almost as negative as before indicates that the step was too short. Strong Wolfe instead bounds the absolute new slope, also rejecting a trial that overshoots into a very steep uphill region. Choose constants strictly ordered between zero and one. Search algorithms generally bracket and refine a step interval; simply shrinking forever cannot guarantee the curvature requirement.

For a continuously differentiable function bounded below along the entire search ray, with a genuine descent direction and admissible constants, Wolfe steps exist. This existence result concerns an ideal search and an unbounded positive ray, not a fixed finite list of trials or a bounded feasible segment. In constrained problems one must respect admissible step lengths, and in noisy simulations a derivative comparison may be dominated by evaluation error. Those changes require safeguards beyond the deterministic statement.

A standard Zoutendijk argument assumes the objective is bounded below, the gradient is Lipschitz on a neighborhood containing the relevant level set and search segments, every direction is descending, and Wolfe steps are accepted. Summing decrease bounds controls the sum of squared gradient norms weighted by the squared cosine of the angle to negative gradient. If those angles stay uniformly away from ninety degrees, the gradient norms tend to zero. Direction quality is part of the theorem, not a side effect of the line search.

This global convergence terminology means a stationarity guarantee under the stated assumptions without requiring an initial point already inside a local Newton neighborhood. It does not mean global minimization of a nonconvex function, nor does it guarantee that the iterates converge to a single point. Keep logs of trial evaluations, accepted step lengths, and gradient norms. A decreasing objective alone can conceal poor directions, derivative errors, or progress that has become negligible relative to the problem's scale.