Mathematics / advanced
Optimization beyond a gradient step
Reason about convexity, constraints, duality, stochastic updates, and nonsmooth objectives with explicit guarantees and limits.
Your route through mathematics
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Optimization beyond a gradient step
Reason about convexity, constraints, duality, stochastic updates, and nonsmooth objectives with explicit guarantees and limits.
- 01
Convex sets and functions
Use line segments and supporting planes to distinguish a globally tractable problem from a merely smooth one.
24 min - 02
Smoothness and strong convexity
Bracket curvature from above and below to predict stable step sizes, conditioning, and convergence rates.
26 min - 03
Gradient descent and line search
Derive descent from a local model, test trial steps, and distinguish progress in values from a certificate of optimality.
25 min - 04
Newton and quasi-Newton methods
Use local curvature to choose a direction, and identify when damping or curvature approximations are necessary.
28 min - 05
Lagrangian duality
Construct lower bounds using constraint prices and distinguish weak duality from conditions that close the gap.
29 min - 06
Karush–Kuhn–Tucker conditions
Read constrained optimality as feasibility, balanced forces, and complementary activity, with explicit qualification assumptions.
28 min - 07
Proximal and projected methods
Handle hard feasible sets and nonsmooth penalties by solving a local regularized subproblem instead of differentiating a kink.
29 min - 08
Stochastic gradient methods
Separate conditional unbiasedness, variance, and step-size schedules when replacing an expensive gradient with a random estimate.
28 min - 09
Mirror descent and geometry
Replace Euclidean distance by a chosen convex geometry and derive multiplicative updates on a probability simplex.
30 min - 10
Nonconvex optimization and saddles
Separate small gradients, local minima, strict saddles, and global solutions when convex certificates are unavailable.
30 min