research36 min studyLesson 685 in Mathematics

Linear–quadratic–Gaussian control

Separate estimation and control under explicit Gaussian observation assumptions and solve a conditional quadratic decision.

Before you begin

On this page

The question for today

What you’ll understand

State the linear dynamics, quadratic cost, and Gaussian observation setting for separation.
Distinguish the control Riccati equation from the filtering covariance equation.
Compute a Gaussian posterior and minimize its conditional expected cost.

Under partial observation, an action may use measurements but not the hidden state itself. In the classical linear–quadratic–Gaussian setting, the optimal controller first estimates the state and then applies the full-information quadratic feedback to that estimate. This separation depends on a specific model: linear dynamics and observations, quadratic cost, Gaussian uncertainty, and an unconstrained risk-neutral objective. A one-step calculation makes the role of posterior variance visible without pretending that an arbitrary state estimator has the same property.

The observation filtration determines admissibility

Consider a linear state equation with deterministic matrices, additive Brownian process noise, and a linear observation equation with additive observation noise. Assume a Gaussian initial state independent of the independent noise drivers, and a uniformly positive observation-noise covariance. Controls are progressively measurable in the observation filtration, with finite expected integrated squared norm. Quadratic state and terminal weights are positive semidefinite, while the action weight is uniformly positive definite.

For a finite horizon with bounded coefficients, these assumptions give the standard unconstrained LQG problem. The conditional state law is Gaussian and its mean is computed by a Kalman filter. The covariance evolves independently of the observed measurement values and, in this classical model, independently of the chosen control. This independence prevents the controller from manipulating its future information quality through its action. Nonlinear observation models or action-dependent sensing can introduce that additional effect and break this simple separation.

dX=(AX+Bu)\,dt+G\,dW,\qquad dY=CX\,dt+D\,dZ,\qquad u_t\text{ uses only observations up to }t.

The state and observation noises are independent in the version treated here; correlated noises require modified filtering equations.↔ Scroll wide equations horizontally.

Estimation and feedback solve different Riccati equations

The full-information quadratic controller uses a backward Riccati equation with terminal weight S and gain $R^{-1}B^{\mathsf T}P$ . Under partial observation, replace the hidden state in that feedback by its conditional mean. The filter gain instead comes from the estimation-error covariance, which solves a forward equation with its initial prior covariance. These two matrices have different meanings and different boundary data; exchanging them would produce neither the optimal estimator nor the optimal controller.

The separation theorem identifies this combined controller as optimal within the observation-adapted admissible class. It does not make the partial-information cost equal to the fully observed cost: estimation error contributes a nonnegative quadratic loss. Finite-horizon separation does not require asserting that every infinite-horizon controller is stable. For steady-state limits, appropriate stabilizability, detectability, and Riccati solvability conditions are additional hypotheses. State or action constraints and exponential risk criteria can change the control structure.

\begin{aligned}-\dot P&=A^{\mathsf T}P+PA-PBR^{-1}B^{\mathsf T}P+Q,\quad P(T)=S,\\ \dot\Sigma&=A\Sigma+\Sigma A^{\mathsf T}+GG^{\mathsf T}-\Sigma C^{\mathsf T}(DD^{\mathsf T})^{-1}C\Sigma,\quad\Sigma(0)=\Sigma_0,\\ u^*&=-R^{-1}B^{\mathsf T}P\widehat X.\end{aligned}

The control matrix P runs backward; the estimation-error covariance Σ runs forward from the prior covariance. Feedback acts on the filtered mean.↔ Scroll wide equations horizontally.

Conditioning exposes the uncertainty that feedback cannot remove

For a scalar one-step observation, Gaussian conditioning gives a posterior mean equal to the prior mean plus a gain times the innovation. The gain is prior variance divided by prior variance plus observation-noise variance. The posterior variance is the corresponding reduced uncertainty. These formulas follow from the joint Gaussian conditional distribution; without Gaussianity, the best estimate need not be this linear function of the measurement.

If the next state is current state plus action plus independent zero-mean process noise, conditional quadratic cost splits into a square involving the posterior mean and a constant involving posterior variance and noise variance. The minimizing action depends on the mean and effort weight. The constants still affect the achieved expected cost, so discarding them would conceal the value of accurate sensing. This one-step derivation illustrates separation without replacing its full dynamic theorem.

K=\frac{\Sigma}{\Sigma+R_y},\qquad m^+=m+K(y-m),\qquad \Sigma^+=\frac{\Sigma R_y}{\Sigma+R_y}.

Here R_y denotes observation-noise variance, distinct from the action-effort matrix R.↔ Scroll wide equations horizontally.

Work it through

Filter a noisy measurement, then choose the action

Let the hidden state have prior distribution $N(1,4)$ and observe $Y=X+\nu$ with independent $\nu\sim N(0,1)$ . After observing $Y=3$ , choose u to minimize $\mathbb E[(X+u+w)^2+2u^2\mid Y=3]$ , where independent w has mean zero and variance one half.

The scalar filtering gain is $4/(4+1)=4/5$ . The posterior mean is $1+(4/5)(3-1)=13/5$ , and the posterior variance is $4/5$ . The independent process noise adds another one half to the conditional next-state variance.
The conditional objective is $(13/5+u)^2+2u^2+13/10$ . Differentiating gives $6u+26/5=0$ , so $u^*=-13/15$ . The positive quadratic coefficient proves a unique minimizer.
At the optimum, the mean-dependent part is $(2/3)(13/5)^2$ . Adding the variance constant gives $871/150$ , approximately 5.806667. The same action follows by treating the posterior mean as the state, while the cost still includes uncertainty.

J(u)=3\left(u+\frac{13}{15}\right)^2+\frac{871}{150}.

Completing the square displays the certainty-equivalent action and the full uncertainty-inclusive cost.↔ Scroll wide equations horizontally.

The result

The posterior is N(2.6,0.8), the action is −13/15, and the conditional optimal cost is 871/150.

The living notebook

Explore the model

Interactive lab

Interactive lab · function

Inspect a centered conditional control cost

For a separate conditional mean 1.5 and effort weight two, completing the square gives a control-dependent gap 3(u+0.5)². Put z=u+0.5. Pause and Reset, then set coefficient a to 1 and position x to 0.5; the displayed z² is 0.25, so the original gap is 0.75. For a fresh comparison Reset and set a to 1 and x to 0; the gap vanishes. The plot omits posterior-variance constants and is not a Gaussian filter. Control changes retain time; set both controls after Reset.

y = ax²TangentSigned integral

y at chosen x: 0.608
Tangent slope · 2ax: 1.35
Signed area · ax³/3: 0.182

Coefficient a0.75Position x0.9

t = 0 s

This lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forAn uncertainty term can raise cost while leaving the minimizer of a completed quadratic unchanged.

Pause & reason

Check your understanding

Interactive checks need JavaScript. You can still read the explanation.

Read the checkpoint explanation

The optimal feedback applies the LQ gain to the filtered conditional mean. The covariance determines filtering and uncertainty cost, while a raw measurement is generally not the conditional mean and future information is inadmissible.

Your turn

Try it yourself

Use prior $N(0,1)$ , observation noise variance three, and measurement two. Then minimize $\mathbb E[(X+u+w)^2+u^2\mid Y=2]$ when independent w has variance one quarter. Compute the posterior, action, and conditional cost.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compute the posterior first. Then complete the square in u, retaining both posterior and process variances as constants.

Compare with the solution

The gain is one quarter, so the posterior mean is one half and variance is three quarters. The objective becomes $(u+1/2)^2+u^2+1$ . Its minimizer is $u=-1/4$ , and the minimum is $1+1/8=9/8$ . The uncertainty contribution is one even though it does not change the minimizing action.

Keep the idea

Take it with you

Classical Gaussian separation uses the observation-adapted conditional mean.
Control and estimation Riccati equations have different boundary data and roles.
Certainty-equivalent actions do not erase estimation-error cost or imply general robustness.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.