Keep cache validity aligned with the committed token history

Many autoregressive loops retain KV for all processed tokens while the most recently emitted token is pending as the next input. Under that convention, committed text length can exceed cache length by one. Other APIs can expose a different boundary, so read the supported cache contract rather than assuming one universal integer relationship. Before a speculative round, identify the known prefix and any pending token. After verification, preserve precisely the processed states consistent with the accepted history, and ensure the correction or bonus token is processed before its hidden state is needed.

In a linear round with a fully processed prefix of length L and γ scored candidate tokens, suppose k<γ proposals are accepted before rejection. The correction produces a committed continuation of k+1 tokens, but its KV has not been computed by the candidate pass because that pass processed a different token at the failing position. Keep prefix plus the k accepted proposal states and treat the correction as pending. Pinned Transformers 4.56.2's assisted path crops target cache to the new output length minus one under its loop convention. Do not generalize that expression to unrelated cache APIs without checking their processed boundary.

Keys and values for a rejected token and its descendants were computed using a history that will not be committed. A linear implementation can truncate the valid sequence range; a tree implementation may gather the accepted path into the cache layout and remove sibling references. Updating an attention length without fixing parent/path mappings can still expose stale states. Physical allocations may remain reserved for reuse, but the valid logical cache must exclude them. In paged systems, block tables, ownership, and reference counts determine which physical memory corresponds to valid sequence positions.

Target and draft states are different model computations unless a supported architecture deliberately shares them. The draft may have advanced farther along its own proposed path and must also be realigned with the corrected token history using its own processed-state convention. Merely copying target KV into an unrelated draft is not valid. Shared-prefix caching needs request isolation: rollback of one request must not mutate another request's committed prefix. Copy-on-write or other supported ownership mechanisms address physical sharing, while the decoder still maintains the semantic token and position mapping.

Compare small runs with a no-cache or independently recomputed target reference. Force rejection at the first, middle, and last proposal; test all-accepted bonus output, EOS, output caps, repeated token IDs, and tree branch switches. Check the next target distribution after rollback, not only the emitted prefix. Validate position IDs, attention masks, valid lengths, and model-specific cache types under the pinned version. A memory leak and a history-contamination bug require different diagnoses, even though both can appear after repeated speculative rounds.

The cache lab illustrates retention without supplying these semantic invariants. For each separate view, Reset, set both capacity C and stride s=4, predict the initial counts, then inspect. Capacity four yields twelve hits and four misses; capacity two yields sixteen misses. This exact LRU address model is not a KV cache, append/rollback algorithm, or page-table validator. It has no token history or pending state. Its limitation matters: a favorable reuse count cannot establish that a cached representation belongs to the committed context, so real rollback tests must compare actual model conditionals.