Eviction releases reusable storage only when the required state remains recoverable

An idle full-prefix block with zero active references may remain in a lookup structure for future reuse. Evicting it removes that reusable identity before the physical slot is repurposed. A later compatible request can recompute the missing state, so eviction changes future work rather than the current output of a request that no longer owns it. Active or in-flight references require protection. Reusing the bytes while another kernel still reads them is a lifetime error, even if a scheduler has already marked the corresponding request canceled.

A live request can be preempted through an explicit protocol that stops its further launches, resolves outstanding access, retains its reconstruction inputs, and releases or offloads its KV allocation. The original PagedAttention system discusses recomputation and swapping as recovery choices under memory pressure. That original design should not be treated as a statement that every current engine uses the same policy or supports all combinations. Separate a block-pool admission failure from a device allocation failure: a pool may be full while the underlying GPU tensor remains fixed in size.

To replay a dense prefix, retain the exact token history and every other input that affects the state: model and adapter revisions, positions, attention-mask semantics, multimodal features or reproducible inputs, and relevant configuration. Restoring token strings through a changed tokenizer is not equivalent to restoring token IDs. If exact continuation is required, preserve sampling and stopping state as well; rebuilding KV alone does not restore a random generator's position or an already streamed output boundary. Replay under compatible numerical execution can reproduce the mathematical state while still differing at floating-point roundoff level.

Do not drop arbitrary old KV positions and continue as if the full state remained available. A model explicitly designed for a sliding attention window can have a different valid retention policy, but that is an architectural contract rather than an eviction shortcut. If compressed or quantized state is reconstructed differently, define the accepted numerical and quality tolerance. Tests should compare replayed continuations with a controlled nonpreempted reference and include cancellation, multimodal inputs, changed adapters, and prefix identity. Recovery correctness needs evidence beyond a successful allocation after the pause.

A cheap-to-recompute prefix can be a better eviction candidate than an equally sized expensive one, but reuse probability, queue pressure, and fairness also matter. Measure prefill and transfer latency at the actual batch and context shapes. A policy that repeatedly admits work then preempts the same long requests can waste capacity and violate latency goals despite high instantaneous utilization. Track bytes reclaimed, replay tokens, repeated victims, and time-to-first-token or inter-token stalls. An idle cache hit rate alone does not describe the cost imposed on active requests.

The probability lab isolates a hypothetical independent reuse count after eviction. Reset, explicitly set BOTH reuse probability p=0.25 and n=20, predict five reuses on average, then Play or Step. Reset and re-set BOTH p=0.75 and n=20; predict fifteen. Edits retain t; playback highlights exact binomial counts without replaying a model. The toy has no cache capacity, prefix lengths, recovery times, ownership, or scheduler. Its stipulated reuse probability cannot be inferred from an eviction event itself, and correlated conversations would violate the independent identical-trial assumption.