Live state storage and durable checkpoint storage are different layers

A state backend chooses how managed state is represented and accessed during execution. Flink 2.1 documents heap-based and embedded key-value-store choices with different serialization, memory, and I/O tradeoffs, as well as additional backend options with their own restrictions. Checkpoint storage identifies where durable snapshot material and metadata are retained. A local backend directory is not automatically a disaster-resilient checkpoint. Conversely, placing checkpoints in durable object storage does not imply each state read performs an object-store request. Read the selected version's backend and checkpoint configuration as two related decisions.

Measure access patterns: random keyed reads, sequential scans, updates, timers, value sizes, and skew. A backend that holds large state efficiently can still pay CPU costs for serialization, compaction, or cache misses. More memory can change cache behavior while leaving state cardinality unchanged. A heap payload estimate and a serialized file size do not include the same overhead. Choose a backend based on representative processing and recovery measurements, not solely the largest state size it can theoretically hold. Operational limits also include local disk capacity and checkpoint-storage throughput.

For an incremental backend mechanism such as the documented RocksDB checkpoint path, unchanged immutable artifacts can be referenced rather than uploaded again. Newly produced files and metadata are persisted as required. This is a physical-storage mechanism, not necessarily a log containing only the keys the application updated. Compaction can produce new files containing values that were not logically changed, and metadata can reference artifacts shared across checkpoints. A small logical update rate therefore does not guarantee a proportionally small upload, especially while the store reorganizes data.

The latest completed checkpoint describes the artifacts needed for recovery. Shared files must remain available while referenced; cleanup cannot delete them merely because they originated in an older checkpoint. Restoring an incremental checkpoint does not mean starting from its newly uploaded delta alone. Nor should one assume every backend reconstructs state through a universal chronological chain of application-level patches. Use the actual manifest and backend rules. Validate serialization and state-schema compatibility before restoring into changed code; a durable artifact that the new program cannot interpret is not a successful upgrade strategy.

Track checkpoint duration, new and referenced bytes, failed or expired checkpoints, local compaction, state-access latency, restore download, and catch-up time. An asynchronous checkpoint still consumes CPU, I/O, and network resources while processing continues. Check recovery when local files are unavailable rather than validating only a favorable local-reuse case. Retention and source replay windows must cover the recovery plan. A job that checkpoints quickly but takes too long to restore and catch up can miss the availability objective despite a green checkpoint-success counter.

The cache lab isolates locality, not a real backend. For EACH run Reset, explicitly set BOTH C and s, predict, then Play or Step. At t=0/N=16, (C,s)=(4,2) yields zero hits across eight identities; (8,2) yields eight hits after the first pass. Requests are capped at min(4096,16+floor(2t)); edits retain t and recompute from empty. This fully associative LRU toy has no serialization, disk, compaction, checkpoint manifest, or recovery. Use it to explain why the same logical state-access stream can respond differently to cache capacity, without claiming measured backend speed.