The stochastic maximum principle

For the running-plus-terminal minimizing objective, define H as running cost plus drift paired with p plus diffusion paired with q. The adjoint solves a backward stochastic differential equation with drift minus the state derivative of H and terminal value equal to the terminal-cost gradient. The martingale coefficient q is part of the unknown adapted solution. Simply reversing a deterministic adjoint ODE in time would generally use future noise and would not preserve adaptedness.

Assume the state coefficients and costs have sufficient state and action derivatives, the first-variation state equations are well posed, and the state, adjoint, and product-integral terms have the moments needed for integration by parts and expectation. These hypotheses justify dualizing the cost variation. Bounded derivative conditions are one convenient sufficient framework; quadratic examples can instead use their explicit moment estimates. Existence of a formal terminal equation alone is not a complete maximum-principle theorem.

For a convex action set and valid convex control variations, an optimal action satisfies the first-order variational inequality involving the action derivative of H. If the minimizer is interior, this derivative vanishes. This is a necessary stationarity condition, not a guarantee of a global minimum. When diffusion depends on the action, its derivative contributes through q. Nonconvex action sets with spike variations can require second-order adjoint terms, beyond this first-order convex-control formulation.

A useful sufficient theorem assumes convex terminal cost and joint convexity of H in state and action along the candidate adjoint, together with admissibility and the integration-by-parts moment conditions. If the candidate globally minimizes H in the action, the convex inequalities bound the cost difference below by zero. Without convexity, stationary trajectories can be locally poor or globally suboptimal. The distinction is already visible in a scalar quartic cost with a stationary point that is actually a local maximum.

For scalar drift u, constant diffusion, effort $u^2/2$, and terminal cost $X_T^2/2$, the Hamiltonian action minimizer is $u=-p$. Its state derivative is zero. Try the adapted adjoint $p_t=P(t)X_t$ with terminal P equal to one. Itô's formula shows that its drift vanishes exactly when $P^{\prime}=P^2$, and its martingale coefficient is $q_t=P(t)\sigma$. The terminal adjoint condition and the state feedback then agree simultaneously.

The positive solution $P(t)=1/(1+T-t)$ gives finite-horizon linear feedback and finite second moments. The Hamiltonian is jointly convex in state and action because it is independent of state and quadratic in action, and the terminal cost is convex. Hence the stated sufficient conditions certify this candidate. It also agrees with the smooth HJB solution, but that agreement is a feature we have checked in this regular example, not an assumption that every maximum-principle trajectory comes from a differentiable value.