Online and offline feature parity

Bind the model to a transformation revision and any learned preprocessing artifacts, such as normalization statistics, vocabulary maps, clipping bounds, or imputation values. Reusing source code is useful but insufficient if one environment loads a different mean vector or evaluates a different timezone. Pin relevant dependencies and numerical types where they affect results. A production request can also supply values unavailable in a historical table, so offline evaluation must reconstruct their actual historic meaning instead of substituting a later enriched record.

Feast offers on-demand transformations that can be applied in historical and online retrieval, but this does not absolve an application from testing the deployed path and backend. Other transformations may run outside the feature store entirely. Describe which stage owns each operation to avoid normalizing twice or leaving one path unnormalized. A registry entry can identify a definition while an old serving worker still holds cached metadata. Validate the actual resolved version and preprocessing artifact in request traces, not merely the definition currently visible in a control-plane interface.

For each fixture, fix typed entity keys, decision time, source snapshot, transformation revision, availability cutoff, and late-data policy. Decide whether the comparison concerns corrected final values or exactly what was known online. An online request at noon cannot be compared fairly to a later historical recomputation that includes a correction first published at 12:05. Retain event and publication metadata or logged serving values where necessary. Without them, label the audit’s limits rather than treating a discrepancy as proof that one arithmetic implementation is wrong.

Test missing, null, zero, nonfinite values where supported, unknown categories, empty windows, daylight-saving boundaries, and duplicated or late events. A numerical tolerance should cover justified representation differences, not semantic changes such as cents versus dollars or the wrong entity. Compare categorical outputs and missingness masks exactly when their contract requires it. Aggregate mismatch rates by feature and entity class; a low overall rate can hide a concentrated error among cold-start users or a model’s most sensitive feature.

Capture enough intermediate evidence to distinguish source selection, temporal eligibility, aggregation, preprocessing, serialization, and model-input assembly. If the raw selected observation already differs, changing a floating-point tolerance is not a repair. If raw values agree but normalized outputs differ, inspect the preprocessing artifact and operation order. If both agree but predictions differ, inspect feature ordering and model version. Run candidate serving logic in a controlled shadow path, preserving existing request semantics and documenting any extra latency or dependency load introduced by comparison.

The function instrument demonstrates a controlled transformation mismatch. Reset, set both a=0.5 and x=1.5, predict y=1.125, then Play or Step. Separately Reset, set both a=0.55 and x=1.5, predict y=1.2375, then Play or Step. The known coefficient change causes a difference of 0.1125 for the same input. This fixed quadratic is not an arbitrary feature executor or a parity-testing framework. Its value is that one changed definition parameter has a checkable consequence while the input is held constant.