A KV block table maps logical token ranges to a managed device pool

Let a block hold a fixed number of token positions across the relevant cache layout. A token's zero-based logical index determines a block number and an offset within that block. A per-request table then supplies the corresponding physical pool identifier, so consecutive logical blocks can live far apart in the device pool. Attention kernels must follow this mapping while preserving the original sequence order and valid token lengths. Reordering physical addresses does not permit changing causal positions or interpreting uninitialized tail slots as real cached tokens.

The original PagedAttention work applies the analogy of virtual-memory pages to application-managed attention storage. The analogy explains indirection, fixed allocation units, and sharing; it is not an identity between GPU KV blocks and OS virtual pages. A KV block can span many hardware pages and tensors, and its token capacity is a serving configuration choice. Host offload, device transfer, and preemption need explicit mechanisms. The kernel and scheduler must agree on block-table representation, head layout, dtype, and lifetime for the indirection to be correct.

With interchangeable equal-size blocks, a request can use any available pool entries without requiring one large contiguous region inside that pool. Its final block may still have unused token slots, and block tables, reference counts, hashes, alignment, and per-layer layouts consume additional memory. An allocator can reserve a large device tensor for the pool even while many application blocks are idle, so framework allocated bytes can remain high after request completion. Measure pool occupancy separately from the amount of device memory reserved to hold the entire pool.

A smaller block reduces worst-case tail waste but increases table entries and may make transfers or kernel access less efficient. A larger block can reduce metadata overhead while wasting more space for many short requests. The best size depends on measured sequence lengths and kernel behavior rather than a universal percentage rule. Cached full prefixes can also leave zero-reference blocks holding reusable content until reclaimed. Such blocks occupy physical pool slots, yet their contents may be evictable; distinguish them from actively referenced blocks when counting admission headroom.

Allocate a new pool block before publishing a table entry that kernels can use, initialize the required state, and retain ownership until outstanding accesses finish. A stale table entry after a physical block has been recycled can silently read another request's KV values rather than produce a clean allocation error. Coordinate table updates, launches, completion events, and cleanup so reclamation occurs at a safe boundary. The vLLM 0.9.1 prefix-cache design distinguishes the block pool, free queue, cached-block mapping, and per-request block lists; those roles illustrate why free and unreferenced are operational states with explicit transitions.

The cache lab only models a bounded pool of reusable identities. Reset, explicitly set BOTH capacity C=4 and stride s=4, predict twelve hits among sixteen requests at t=0, then Play or Step. Reset and re-set BOTH C=3 and s=4; predict zero hits. Playback uses N=min(4096,16+floor(2t)); edits retain t and recompute from empty. There are no token positions, block tables, device addresses, live-reference pins, or kernel completion events. Its equal-size entries illustrate a capacity boundary without implementing safe KV allocation or OS paging.