Variable lengths create several different forms of unusable capacity

In a fixed-token block pool, round each unshared positive request length upward separately. Rounding the sum of all lengths can underestimate capacity because a private tail usually cannot be filled with another request's state. Each request wastes fewer than one block of token slots under this narrow on-demand model, but many short or nearly full-plus-one lengths can make total slack significant. Sharing, speculative reserves, sliding windows, and preallocation change the relevant count, so state which are excluded before applying a simple bound.

The average slack is not automatically half a block. That familiar result requires a suitable distribution of integer length remainders. Workloads with templated prompts, fixed maximum generation lengths, or many boundary-adjacent completions can be strongly nonuniform. Measure actual retained lengths and active-request counts over time rather than only final output lengths. A large fraction of one block can be small in bytes for one model and costly for another, because layer count, KV heads, width, and dtype determine each token slot's physical payload.

A general contiguous allocator can have eight MiB free as holes of three, three, and two MiB yet fail a four-MiB allocation without a way to combine or remap those holes. An equal-size KV pool avoids that particular requirement for contiguous logical request storage: any suitable free block can satisfy the next block request. This does not eliminate fragmentation in the allocator backing the pool, other workspaces, or separate pools with incompatible layouts. An idle block from one cache dtype or layer group may not be interchangeable with a required block from another.

PyTorch 2.9 distinguishes native allocator behavior from its cudaMallocAsync backend, and some tuning options are backend-specific. Its native split-block statistics can help diagnose particular OOM patterns; a large reserved-minus-allocated number alone does not prove fragmentation. The difference can be ordinary reusable cache, pending frees, or incompatible sizes. Inspect allocation histories and pool categories before applying a tuning change. Similarly, reserving maximum output capacity is an admission policy choice, not an allocator defect, even though it can leave substantial device capacity unavailable to other requests.

Smaller blocks reduce the maximum tail slack but increase table entries and can alter kernel access patterns, copy overhead, and prefix-reuse granularity. Compare total bytes including metadata and measured service latency on representative traces. Keep block count, token slots, and bytes in separate columns so a format change cannot masquerade as a workload improvement. Test long and short requests together: a configuration that looks efficient on a uniform batch can behave differently under arrivals, completions, branches, and cancellation. There is no universal block size that simultaneously minimizes every cost.

The modular lab tests the assumption about remainders. Reset, explicitly set BOTH m=8 and a=3, predict all eight residues in one orbit, then Play or Step; interpreted on paper as length remainders, their slacks average 3.5 slots. Reset and re-set BOTH m=8 and a=4; only residues zero and four occur, averaging two slack slots. Edits retain t; iteration n=min(2000000,floor(2t)). This orbit is not an allocation trace or a workload distribution. It shows why different supported remainder sets need not justify the same half-block expectation.