Smoothness and strong convexity

In Euclidean geometry, an $L$-smooth differentiable function has a gradient whose change is at most $L$ times the change in position. This is stronger than having derivatives everywhere and stronger than simply having a continuous gradient. The bound must hold on the region under discussion. A polynomial can be smooth on every bounded interval without admitting one finite global gradient-Lipschitz constant on the whole real line.

Integrating the gradient along a segment gives the descent lemma: the tangent approximation can underestimate the function by at most a quadratic correction. This upper model explains a safe gradient step. If curvature is bounded above by $L$, choosing a step no larger than $1/L$ makes the model decrease. The statement uses an exact gradient and assumes the segment remains in the domain; noisy estimates and constraints require additional treatment.

A differentiable function is $\mu$-strongly convex when every tangent plane can be raised by a quadratic bowl of curvature $\mu>0$ and still remain below the function. At an unconstrained minimizer the gradient vanishes, so the objective gap bounds squared distance. This is a quantitative version of uniqueness: a point far away must pay at least a specified increase in value. Strict convexity alone supplies no such uniform quadratic constant.

For twice continuously differentiable functions on an open convex domain, Hessian eigenvalues between $\mu$ and $L$ imply these lower and upper bounds. The ratio $\kappa=L/\mu$ is a condition number for this geometry. Rescaling coordinates can change it, which is why feature scaling and preconditioning affect optimization. A small objective value is not automatically a small parameter error when the smallest curvature is tiny.

For a quadratic with positive definite Hessian, resolve the error into orthonormal eigenvectors. A gradient step multiplies each component by one minus the step size times that eigenvalue. A single shared step must accommodate the steepest direction, leaving flat directions to contract slowly. For globally $L$-smooth, $\mu$-strongly convex functions, the step $1/L$ gives the familiar objective-gap bound with factor $1-\mu/L$ per iteration.

The lab has only unit curvature, so both constants equal one. Use it to verify the multiplier in a single direction: with step 0.25, predict a factor 0.75 in position and 0.5625 in objective value each update. It cannot vary a Hessian spectrum or demonstrate ill-conditioning directly. The two-dimensional calculation below supplies that missing comparison and shows why a successful one-dimensional experiment does not establish a general iteration count.