Make simple forecasts operationally honest

For a regular series with seasonal period m, seasonal naive forecasting repeats the most recent observed value from the same seasonal position. For horizons longer than m, it continues repeating the last observed cycle rather than reading future actuals from the evaluation table. The index formula should point to a time no later than the forecast origin. Write a short trace for horizons m, m+1, and 2m; these boundary cases reveal many incorrect implementations of a superficially simple method.

The seasonal position must match the operational calendar. A seven-day lag may preserve weekday but miss a movable holiday; a twenty-four-hour elapsed-time lag can differ from the same local clock hour around daylight-saving changes. Define whether the baseline uses elapsed slots, local calendar matching, or a holiday-aware lookup. Do not add calendar exceptions after seeing a test failure and then describe the result as an untouched benchmark. Version each exception and select its policy on earlier data.

Availability can invalidate the usual seasonal anchor. If the relevant last-cycle value is not yet usable, decide in advance whether to use an older same-position value, a recent-level baseline, or a degraded response. This is a different policy from textbook seasonal naive with complete history, so name it clearly in evaluation. Record which anchor was actually chosen and its usable version. Otherwise an availability-aware baseline can be accidentally compared with a learned model that saw richer inputs.

A hierarchy of fallbacks should terminate. For example, try one-cycle-old same-position demand, then two-cycle-old demand, then a training-only median, and finally an explicit unavailable state. Set a maximum allowed anchor age and a rule for new entities with no history. A baseline that always emits a number is not automatically more reliable than one that reports insufficient evidence. Include forecast coverage and degraded-mode frequency in the comparison, alongside the accuracy of values that were actually issued.

Issue baseline and model predictions for the same origins, targets, truth versions, and serving deadlines. Include cases in which either path fails. A model that improves mean absolute error only when its large feature join succeeds may still lose operationally after its fallback behavior is included. Measure end-to-end decision quality under the intended action policy, while retaining pure forecasting metrics so the source of any benefit can be understood.

Break baseline performance down by season, regime, entity age, and forecast horizon. A seasonal repeater can be strong during stable weekly operations and weak after a permanent workload migration. A recent-value baseline may react faster to a level shift but miss a predictable weekday pattern. These failures provide useful diagnostic structure for a learned model: beating one aggregate number is less informative than demonstrating which predictable patterns it captures and which operational conditions still force a fallback.