Carry temporal state only within its declared stream

Define a state key using the source stream, episode or session, and relevant model/preprocessing version. Also define how timestamp discontinuities, resolution changes, missing chunks, and an explicit new episode are handled. A reset may require warmup before predictions are comparable with a fully conditioned stream. Logging only a request identifier can hide state reuse between cameras or users; log the state identity and reset reason alongside the output interval.

A simple recurrence illustrates why the initial state matters. Real video systems may instead cache several past feature maps, as in MoViNets' stream-buffer design, but those buffers still represent particular preceding observations. Do not assume buffers are portable between different model weights, spatial shapes, frame cadences, or preprocessing configurations. Either validate compatibility explicitly or reset with a documented loss of context. State restoration must reproduce the configuration and prior evidence that originally defined it.

Causal temporal operations use no future inputs relative to their declared prediction boundary. A centered temporal convolution or bidirectional attention window generally has a different contract because it consumes later evidence. For a regular verified cadence, a known right-context count implies a minimum media lookahead. Variable timestamps require the actual time difference to the last required frame instead of a universal frame-count formula. Decoder reordering, transport, batching, and computation can add further elapsed delay.

Keep media time and feature-ready elapsed time distinct when testing causality. If an output needs a frame not yet decoded and available, its earliest emission cannot precede that dependency's availability plus the remaining serial computation. A model trained on centered offline clips cannot become causal merely by calling its function on a live stream. It needs a compatible architecture and training/evaluation setup, or an explicitly delayed decision contract that reports the lookahead.

For a fixed deterministic causal model, compare one continuous execution with several chunk partitions while carrying exactly the required state. Fix evaluation mode, numerical tolerance, padding rules, and the initial state. Include one-frame chunks, a long gap, a new source, and a changed configuration. An intentional reset can change boundary predictions, so compare it with the corresponding reset oracle rather than declaring every difference a numerical bug. Recovery tests should also establish whether restored state contains all required prior context.

The optimization lab isolates dependence on an initial value without pretending to execute a video model. Reset, set both η=0.5,x₀=2, predict x₂=0.5, then take two Steps. Reset and set both η=0.5,x₀=0; the value remains zero. Each Step is one iteration. This model has no incoming frames, additive input term, stream keys, or feature buffers, so it does not implement the recurrence in the worked example. It only makes one consequence of carrying an initial condition observable.