Forecast from what the system actually knew

Observation time identifies the period or event being measured. Publication time records when a provider released a value. Ingestion time records receipt inside your infrastructure, while usable time records when validation and feature materialization made that version available to the forecasting path. These can differ substantially. A forecast cutoff must be compared with the timestamp governing the actual serving contract, rather than whichever timestamp is easiest to query. A source publication time alone does not prove internal availability.

Store a logical observation key together with version identity and usable time. A correction should create another version rather than overwrite the only evidence of what the model previously saw. For a historical query, first exclude versions unavailable at the decision cutoff, then choose the winning eligible version using a deterministic rule. A tie needs an explicit sequence or revision number. Keep deletion and invalidation records because removing a bad observation also changes the historical information set.

An event-time join can prevent a measurement from being joined before its observation period and still leak a revision published later. The stronger replay combines the event-time relationship with usable-version eligibility. Feast documents point-in-time historical joins around entity timestamps and feature event timestamps; that mechanism should not be assumed to recover a separate internal availability history unless that history is actually represented. Test the query with a deliberately late correction, not only with neatly ordered events.

The St. Louis Fed’s real-time periods provide a concrete example of data vintages: the value known at a past date can differ from today’s revised value for the same observation date. Apply that distinction to infrastructure telemetry, business aggregates, and external predictors. A backfilled table may describe the past more completely without describing what an online predictor could have known. Preserve both views when they serve different purposes, and label the evaluation’s vintage policy explicitly.

Record the forecast issue time, feature snapshot identity, source revision IDs, model version, and transformation version. If a feature pipeline runs every five minutes, distinguish the requested cutoff from its last completed materialization. A request at 10:10 may consume a snapshot completed at 10:08. Log that snapshot instead of claiming all data through 10:10 was used. This makes an apparent model regression diagnosable as stale inputs, changed transformations, or an actual change in predictive behavior.

Future calendar information is a useful exception to a simplistic rule that every feature timestamp must precede the target. Tomorrow’s weekday is knowable today; tomorrow’s actual queue length is not. Planned maintenance is usable only as the plan known at issue time, with later schedule changes preserved as new versions. Validate eligibility by knowledge time and feature meaning. Such checks let a model use legitimate future-known predictors while excluding observations that merely acquired old event timestamps during backfill.