greatitself.
Continue
advanced39 min studyLesson 864 in Mathematics

Wold decomposition and linear prediction

Separate remote-past predictable structure from orthogonal innovations and compute forecasts within the correct linear information space.

Before you begin
On this page

The question for today

What you’ll understand

  • Define innovations and pure nondeterminism through closed linear past spaces.
  • State the deterministic-plus-innovation Wold decomposition with its mean-square convergence.
  • Distinguish best linear prediction from a full conditional expectation.

Past observations span a collection of possible linear predictors. Orthogonal projection onto its closure gives the best mean-square linear forecast and defines the new part of each observation: its innovation. Wold's theorem organizes a stationary process around these innovations while preserving a component already predictable from the arbitrarily remote past. This is a linear Hilbert-space result. Neither a random-looking path nor a continuous-looking spectrum automatically supplies the pure nondeterminism needed to remove that predictable component.

The past is a closed linear space

Work with a centered real weakly stationary scalar process in L². Let H_t be the closed linear span of all observations up through t. Define epsilon_t as X_t minus its orthogonal projection on H at t−1. The residual is orthogonal to that entire past. Stationarity makes its variance constant, and distinct innovations are uncorrelated. This definition guarantees weak white noise when the variance is positive; it does not make the innovations independent without extra distributional structure.

The intersection of all past spaces is the remote-past space. Pure nondeterminism means that this intersection contains only the zero random variable. It is a condition on linear predictability, not on whether a sample path is a deterministic numerical sequence. For example, a random constant known after one observation lies in every past space. Although its value is random before observing it, it is perfectly predictable from the past and contributes to the deterministic component in this terminology.

Ht=span⁡‾{Xs:s≤t},εt=Xt−PHt−1Xt,pure nondeterminism: ⋂tHt={0}\mathcal H_t=\overline{\operatorname{span}}\{X_s:s\le t\},\qquad\varepsilon_t=X_t-P_{\mathcal H_{t-1}}X_t,\qquad \text{pure nondeterminism: }\bigcap_t\mathcal H_t=\{0\}
The series is centered and square integrable. P denotes orthogonal projection in L², so these are linear information spaces rather than the full sigma-algebras of every measurable past feature.↔ Scroll wide equations horizontally.

The theorem preserves a predictable component

When innovation variance is positive, Wold's decomposition writes X as a deterministic component D plus a one-sided moving average of its innovations, with leading coefficient one and square-summable coefficients. The series converges in L², and D is orthogonal to all innovations and is predictable from the arbitrarily remote linear past. Pure nondeterminism makes D vanish. If innovation variance is zero, the process is entirely deterministic in this linear sense and the normalized positive-variance innovation representation is not the relevant branch.

Square summability is sufficient for the orthogonal series to converge; absolute summability is a stronger condition often supplied by stable rational ARMA models. A deterministic component must not be equated solely with a finite set of spectral lines. The relation to spectral regularity is subtler, involving factorization and log-integrability conditions. The Hilbert-space definition avoids losing that distinction when a continuous spectral density is present but its predictability properties are unusual.

Xt=Dt+∑j=0∞ψjεt−j,ψ0=1,∑jψj2<∞;MSE⁡(X^t+h∣t)=σε2∑j=0h−1ψj2(D=0)X_t=D_t+\sum_{j=0}^\infty\psi_j\varepsilon_{t-j},\quad\psi_0=1,\quad\sum_j\psi_j^2<\infty;\qquad\operatorname{MSE}(\widehat X_{t+h\mid t})=\sigma_\varepsilon^2\sum_{j=0}^{h-1}\psi_j^2\quad(D=0)
The forecast uses the entire infinite linear past. For a purely nondeterministic process, future innovations give exactly this h-step mean-square prediction error.↔ Scroll wide equations horizontally.

Linear optimality does not mean every measurable optimum

Under joint Gaussianity, projecting on the Gaussian observation history agrees with its conditional mean, so linear forecasts are fully mean-square optimal among measurable functions of that history. Without Gaussianity, the conditional expectation can be nonlinear and improve on every linear predictor. A simple illustration takes a symmetric variable supported on minus one, zero and one, with equal probabilities, and predicts its square. The best affine predictor is its constant mean two thirds, while the nonlinear square is known exactly.

Finite datasets also supply less than the infinite past in the theorem. Their optimal projection has initialization effects and may have larger forecast error than the infinite-history formula. If a nonzero mean has not been removed, include the mean as an intercept rather than silently treating it as a random innovation. Estimated means and fitted coefficients add another source of uncertainty. Wold's structural representation does not automatically eliminate those statistical and finite-history limitations.

Work it through

Forecast a short innovation moving average

Let X_t=epsilon_t+0.5epsilon at t−1 be invertible, with innovation variance four and mean zero. Using the infinite past, find one- and two-step forecasts and their mean-square errors.

  1. Invertibility makes current and past innovations recoverable from the infinite observation history. Thus X at t+1 has known part 0.5epsilon_t and new part epsilon at t+1.

  2. The one-step best linear forecast is 0.5epsilon_t and its error variance is four.

  3. At horizon two, both innovation terms are future and orthogonal to the available linear past. The forecast is zero and the error variance is four times (1+0.5²)=5.

The result

The forecast error grows from four to five as the predictable shock contribution leaves the horizon. These are infinite-past linear results, not estimated finite-sample errors.

The living notebook

Explore the model

Interactive lab

Interactive lab · function

See why a nonlinear conditional mean can beat a line

Pause and reset, then set both controls to a = 1 and x = 1, predicting y = 1 on the square function. For a fresh comparison, pause and reset, then set a = 1 and x = 0, predicting y = 0. On the three equally likely inputs −1,0,1, symmetry makes the best affine predictor of the square the constant two thirds, although the square function predicts exactly. The instrument is a deterministic parabola; it illustrates the linear-versus-measurable distinction and does not run a Wold decomposition or generate a time series.

See why a nonlinear conditional mean can beat a lineThis lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Current readings: y at chosen x: 0.608; Tangent slope · 2ax: 1.35; Signed area · ax³/3: 0.182. All parameters are adjustable with the labeled controls below.−2.4−1.201.22.4−9−4.504.59xy(x, ax²)
y = ax²TangentSigned integral
y at chosen x
0.608
Tangent slope · 2ax
1.35
Signed area · ax³/3
0.182
t = 0 s

This lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forOrthogonal linear prediction and unrestricted conditional prediction agree only with additional structure.

Pause & reason

Check your understanding

What does pure nondeterminism mean in the centered L² formulation?

Your turn

Try it yourself

Let X_t=Z+epsilon_t, where Z is a centered random constant of variance four and epsilon is independent iid centered noise of variance one, independent of Z. Identify the remote-past component and infinite-past one-step prediction error.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

A long past average converges in L² to Z because the average noise variance tends to zero.

Compare with the solution

For every fixed cutoff, averaging more observations from its past estimates Z with noise variance tending to zero. Thus Z belongs to every closed past space. The decomposition has D_t=Z, innovation epsilon_t, and only innovation coefficient psi-zero one. The infinite-past predictor of X at t+1 is Z, with error variance one, rather than the total variance five. The process is not purely nondeterministic, even though it includes genuinely unpredictable noise at every time. This example separates the deterministic-component terminology from the ordinary meaning of a completely nonrandom series.

Keep the idea

Take it with you

  • Innovations are residuals from a closed linear past and are initially only orthogonal white noise.
  • Pure nondeterminism removes the remote-past component from Wold's representation.
  • Infinite-history linear forecasting and finite-data conditional forecasting require separate qualifications.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.