Autocovariance and positive definite sequences

Center the real weakly stationary series by its population mean. Define gamma(h)=E[(X at t+h minus mu)(X at t minus mu)]. Symmetry gives gamma(−h)=gamma(h), and gamma(0) is its variance. The covariance matrix of consecutive observations has entries gamma(i−j), so each diagonal is constant: it is Toeplitz. If the variance is positive, dividing gamma by gamma(0) gives the autocorrelation function, with value one at lag zero.

For any real weights, the associated quadratic form is the variance of a weighted observation sum. It must be nonnegative for every finite length. We call this positive semidefiniteness; some time-series texts call the corresponding sequence positive definite even when a nonzero weight vector can have zero variance. Strict positivity is needed before using an ordinary matrix inverse in prediction. A deterministic linear relation can make a legitimate stationary covariance matrix singular.

A useful way to produce a valid sequence is to filter white noise through known coefficients and calculate the covariance directly. For X_t=epsilon_t+theta epsilon at t−1 with unit-variance weak white noise, gamma(0)=1+theta², gamma(1)=theta and all higher nonzero lags vanish. This construction automatically satisfies every finite quadratic-form constraint. It does not require Gaussian noise for the second-moment calculation.

Conversely, a symmetric real sequence with all finite Toeplitz matrices positive semidefinite is an admissible covariance sequence: one can construct a centered Gaussian process with those covariances through compatible finite-dimensional Gaussian laws. This is an existence statement about a process, not a unique distributional model for every non-Gaussian series sharing the covariance. Second moments determine optimal linear projections, while nonlinear prediction can still depend on information the covariance does not record.

Given a finite centered data vector, extend it by zeros outside the observation window and use a common denominator n for every sample autocovariance lag. The resulting sequence is positive semidefinite because its quadratic form is a sum of squares of filtered data. Subtracting the sample mean before this construction does not remove that algebraic property. Using n minus the lag instead may reduce some finite-sample bias under particular mean assumptions, but need not preserve positive semidefiniteness.

Estimating a mean also changes the expectation of the sample autocovariance, so describing an estimator as unbiased requires stating exactly what was centered and which population assumptions hold. Sample autocorrelations at different lags are dependent estimates; a collection of individually plausible confidence intervals is not a proof of a valid joint covariance model. For forecasting, one should check that the fitted covariance matrices and the chosen innovation algorithm remain compatible and numerically well conditioned.