Retaining late-data state is part of the answer's contract

An event-time window can emit an initial result when progress passes its trigger and still retain state for allowed late arrivals. Flink 2.1's allowed-lateness setting is attached to event-time window behavior, with cleanup tied to window timestamps and the configured lateness. A later eligible event can cause an updated result. Downstream consumers need a replacement, retraction, or delta contract to avoid counting every correction as a fresh independent result. Once the window state is removed, the ordinary incremental update path no longer has the history needed to produce the same corrected answer.

Allowed lateness is not simply a per-record rule saying watermark minus event timestamp must be below one threshold. It is related to the retained window's cleanup boundary under the operator's rules. A very old event can still belong to a retained window, while a seemingly similar event in an already expired window cannot use its vanished state. Discrete timestamp maxima and exact comparison rules matter near the edge. Keep illustrative examples away from equality unless teaching the specific implementation's boundary, and test that boundary explicitly in production code.

Flink 2.1's managed State TTL support uses processing time, with configurable update and visibility behavior. That is different from an event-time window's allowed lateness. A replay that runs slowly or quickly can therefore encounter different processing-time expiry even for identical event timestamps. State visibility after expiration and physical cleanup timing are also separate: hiding an expired value does not prove its bytes have already disappeared from every backend file or retained checkpoint. Read the selected backend and TTL policy before using a memory graph as evidence that a semantic retention promise was met.

For an application-level join or deduplication set, define why state can expire and which clock establishes that reasoning. An arbitrary shorter TTL can make previously matchable events fail to join or old duplicates appear new. A longer TTL can increase state cost without improving an answer if the source no longer retains the needed replay data. Align event-time policy, source retention, checkpoint availability, and downstream correction horizon. Expired data may need an audit side output or a batch reconciliation path, but that path must be designed and monitored rather than assumed to happen automatically.

Measure retained keys or events, cleanup lag, late arrivals accepted, late arrivals rejected, correction outputs, and reconciliation backlog. Report these by source or key population where skew matters. A job can meet low output latency by dropping late records while quietly reducing completeness for a slow input. Tests should restore from a checkpoint, replay old event timestamps, and compare the expected retained-state policy with actual visibility. Explicitly test policy changes across upgrades; increasing a TTL cannot reconstruct already deleted state, and changing its semantics can alter results even when code still deserializes.

The cache lab illustrates forgetting through capacity, not time. Reset, set BOTH C=4 and s=4, predict twelve hits among N=16 requests at t=0, then Play or Step. Reset and re-set BOTH C=3 and s=4; predict zero hits because each four-identity replay follows eviction. The request count is capped at min(4096,16+floor(2t)); edits retain t and recompute from empty. There is no TTL, watermark, event-time join, or late correction. Explain why forgotten identities can change replay behavior without claiming that this LRU trace measures a real retention horizon.