Diverging generation branches copy the shared writable tail

Suppose several branches begin from a prompt with full prefix blocks and one partial tail. The full blocks need no changes during append-only generation and can stay shared. The partial tail has unused slots, but those slots belong to a shared allocation whose logical contents will diverge as branches append. A branch must first obtain exclusive write ownership of that tail, usually by allocating a new block and copying the valid existing state. Empty tail capacity is not permission to mutate an allocation still visible through another branch's table.

The original PagedAttention paper describes block-granularity copy-on-write for parallel sampling and beam search. Do not confuse that mechanism with every later engine's prefix-cache policy: the separately cited vLLM 0.9.1 V1 prefix cache admits full blocks and documents append-only block tables with possible temporary duplicate full blocks. A lesson about the original branch-sharing design is not a promise that any particular V1 API exposes the same branching operation. Check the engine version, decoding support, and ownership implementation before using the accounting as an operational plan.

For several branches sharing one partial tail, the first writer copies it and drops its reference to the original. Other writers repeat that process while the original remains multiply owned. When only one writer owns the original and no other hold forbids mutation, that writer can append in place. Allocate and complete the copy before publishing the writer's new table mapping to its mutation kernel. A data race or premature reference decrement can produce plausible but incorrect generated output instead of an obvious crash.

After each branch has private tail storage, later appends within that tail require no additional copy merely because the prompt was once shared. Crossing a block boundary allocates a new private block. Beam pruning or cancellation drops the abandoned branch's references; it does not invalidate shared blocks still used by surviving branches. Speculative execution adds temporary branch states that need the same ownership discipline when proposals are accepted or rolled back. Count transient copies and completion delays when reserving memory rather than budgeting only the final steady-state block graph.

Construct small deterministic tests in the actual engine: branch from a known prefix, force different continuations, cancel one branch, and compare surviving state with an independently recomputed reference. Inspect table mappings and ownership transitions around the tail boundary and copy completion. Memory savings alone cannot reveal a shared-tail corruption bug. Test cases should include exactly full prompts, one-slot tails, several simultaneous writers, and pruning while kernels are in flight. The number of logical branches is not itself the number of full prompt copies that must physically exist.

The cache lab provides only a private-working-set capacity analogy. Reset, explicitly set BOTH C=4 and s=2, predict zero hits in the initial sixteen requests, then Play or Step. Reset and re-set BOTH C=8 and s=2; predict eight hits because the cyclic working set has eight identities. N=min(4096,16+floor(2t)); edits retain t and recompute from empty. This model has no branches, aliases, copy-on-write, or asynchronous completion. It illustrates why additional distinct state can exceed a pool's capacity, while the branch and copy counts must come from the explicit example.