Risk-sensitive and robust stochastic control

For a total random cost C and positive parameter theta, define its entropic cost as the logarithm of the exponential moment divided by theta. Minimize this quantity over admissible policies with finite required exponential moments. A finite ordinary expectation does not imply a finite exponential moment. If exponential moments exist in a neighborhood of zero, a cumulant expansion gives mean cost plus half theta times variance to first order. Positive theta is risk averse for this cost convention.

Negative theta instead favors dispersion in cost in that local expansion, but multiplying by its negative reciprocal reverses optimization order when transforming to an exponential-value problem. We derive the PDE only for positive theta here. Even there, an unrestricted exponential quadratic Gaussian cost can become infinite for some parameter values or horizons. The moment domain is part of the problem, rather than a technicality that can be ignored after obtaining a formal differential equation.

For the finite-horizon diffusion model with positive theta, appropriate smoothness, and finite exponential values, the logarithmic transform of the multiplicative dynamic-programming equation gives the usual generator and running cost plus a positive squared-gradient term. The coefficient is theta over two times the squared diffusion-weighted value gradient. Thus simply adding a variance penalty to the running cost is generally not the same dynamic problem. The continuation gradient determines the local risk adjustment.

Algebraically, that term is the supremum over drift distortions of a linear gradient pairing minus a quadratic distortion penalty. To interpret it as a change of probability law, the distorted SDE must be well posed and its stochastic exponential must be a true martingale, with the required relative-entropy and cost integrability. The maximizing distortion must belong to the admitted class. A pointwise completed square proves the Hamiltonian identity, but does not by itself prove equality of two globally formulated control problems.

A robust model must specify whether each player uses open-loop controls, feedback, or nonanticipating strategies, and how policy pieces can be joined. Under a suitable compact-action, regular-coefficient formulation with dynamic programming and viscosity comparison, equality of lower and upper Hamiltonians is the Isaacs condition that identifies the two game values. It is not valid to exchange an infimum and a supremum merely because both symbols appear in a Bellman equation. Without Isaacs, upper and lower values can differ.

On a finite outcome space with positive reference probabilities, the entropy variational identity is exact for every finite cost vector. The adversarial distribution is proportional to the reference probabilities multiplied by exponentiated costs. This finite identity needs no diffusion theorem and makes the penalty interpretation concrete. Extending it dynamically introduces the filtration, density-martingale, integrability, and strategy issues just listed. Keeping those scopes separate avoids promoting a useful algebraic relationship into an unjustified universal robustness claim.