Verification theorems and feedback controls

Suppose v is continuous up to the terminal time, has one time derivative and two spatial derivatives before it, satisfies the minimizing HJB equation, and matches the terminal cost. For each admissible u, the HJB infimum implies $v_t+\mathcal L^uv+\ell^u\ge0$. Apply Itô's formula to v along the controlled state and add the accumulated running cost. The resulting process has nonnegative drift, giving a submartingale comparison after the stochastic integral is justified.

Taking expectations and using terminal matching gives $v(t,x)\le J(t,x;u)$, hence a lower bound on the minimizing value. This sign is easy to reverse when switching from reward maximization to cost minimization. The comparison should hold for all controls in the declared class, not only for the proposed optimizer. Otherwise it proves a local comparison or a restricted policy result, rather than a certificate for the full value.

If the stochastic integral has finite expected integrated squared integrand, it is a true martingale and its expectation vanishes directly. More generally, stop the state on increasing bounded regions, apply the identity to the localized process, and justify passage to the horizon using uniform integrability or a suitable domination and growth estimate. A local martingale need not have zero expectation at an unrestricted horizon. Polynomial growth of v therefore needs matching state moments and derivative estimates.

To achieve equality, select an action that attains the HJB infimum measurably and require the resulting closed-loop SDE to possess an admissible solution. Compact actions and continuity can support a measurable selector, but they do not automatically make a discontinuous selected feedback globally Lipschitz. One may need a separate existence theorem or only epsilon-optimal controls. Once an admissible selector exists and the martingale conditions hold, its HJB drift vanishes and the bound becomes equality.

Consider drift controls in the interval from minus one to one, unit additive diffusion, terminal cost equal to the terminal state, and running effort $u^2/2$. The candidate $v(t,x)=x-(T-t)/2$ is affine in state. Its HJB minimizer is the constant minus one. The diffusion term vanishes because the second spatial derivative is zero, while the stochastic integral is simply a Brownian increment and is square integrable on the finite horizon.

The nonnegative drift becomes half the square of u plus one. Integrating yields an exact difference between any admissible policy cost and the candidate. Bounded controls give the needed state moments even though the linear terminal cost can be negative. The square identifies the optimizer up to product-time-probability null sets and also distinguishes genuine optimality from a policy whose residual merely looks small at one state or one instant.