Build lagged features around the forecast origin

For a regular discrete series, let origin t denote the latest period that is permitted as observed under the stated availability policy. A feature vector can contain values at t, t−1, and t−2 while its target is y at t+h. If the dataframe is indexed by target time s instead, the same most recent feature is y at s−h. A one-row shift is correct for one-step prediction under this convention, but it is not automatically correct for every horizon.

Group by the complete series identity before shifting: site, metric, and aggregation definition may all be needed. Sort within that group and resolve duplicate timestamps according to the version contract. A global shift over a table sorted by site can copy the final observation of one site into the first row of another. An apparently reasonable validation score cannot excuse this error; compare selected feature rows with hand-built examples at entity boundaries and after missing timestamps.

A rolling mean must specify included timestamps and how missing values affect its denominator. For an origin after period t has become usable, a length-three window may include t, t−1, and t−2. For a row indexed by the next target t+1, that same calculation requires shifting before rolling. A centered window generally includes future observations and is unsuitable for causal online features unless its output is delayed enough to make every constituent genuinely available.

Row count and elapsed time are different window definitions. The last twenty observations may cover twenty minutes during normal reporting and several hours during an outage. A one-hour average over irregular samples also needs a measurement interpretation: an arithmetic average of event samples is not necessarily a time-weighted average of a continuously held rate. Store sample count and elapsed coverage with aggregates. These auxiliary features help the model distinguish low demand from weak observation support.

Scaling, imputation statistics, feature selection, and learned embeddings are parts of the model pipeline. Fit them using only the training information allowed at each replay cutoff. A standardization mean computed over the entire series leaks later level changes into earlier folds even when every explicit lag points backward. For a global model, also decide whether each entity’s scale is estimated independently and how a newly appearing series obtains a scale without access to its future history.

Version the feature definition along with the model. Changing a window’s closure, timezone conversion, minimum-observation threshold, or revision selection can alter predictions without modifying learned weights. In pandas 2.3.3, distinguish shifting values by periods from shifting index labels with a frequency; relabeling timestamps does not by itself build the intended lagged alignment. Test full training-serving parity using a fixed historical snapshot and compare each intermediate feature, not merely the final forecast.