Register pressure and spilling

Register demand depends on live ranges, unrolling, inlining, indexing, and generated instructions, not simply the number of variables declared in source. Large or dynamically indexed thread-private arrays may reside in local memory even without a simple register-exhaustion story. Inspect compilation resource reports and final generated code for the actual architecture. PTX is an intermediate representation, so later compilation can change placement. A favorable source rewrite can unexpectedly add registers through temporaries or a different instruction selection.

CUDA local memory is private in its address-space semantics but backed by device memory, not a private on-chip register extension. Spilled accesses can therefore add traffic and latency, subject to caching and their access pattern. Threads accessing corresponding local offsets can have coalesced accesses, but that does not make spills free. Do not confuse local memory with block-shared memory, whose ownership, lifetime, and synchronization rules are different. The word local describes visibility here, not physical proximity.

A small register increase can cross an allocation boundary and reduce resident blocks abruptly. Other limits, including threads, warps, shared memory, and architectural block caps, may bind first. Thus dividing total registers by threads and a reported per-thread count is not a complete occupancy calculation. CUDA’s occupancy APIs and profiling tools can account for target-specific constraints. State which bottleneck your arithmetic includes and which it deliberately omits before reporting a resident-block prediction.

The CUDA 12.8.1 best-practices guide gives a Volta compute-capability 7.0 example with 256-register allocation granularity per warp. Use that scoped rule to understand rounding rather than assuming every newer target shares every detail. Reducing a compiler register cap may raise nominal residency while forcing more spills or instructions. Conversely, a lower-occupancy kernel can still perform well when it has sufficient independent work and reuse. Compare execution time, memory traffic, and numerical behavior after each meaningful change.

Tune fusion, tile dimensions, or unrolling based on measured bottlenecks. Shorter live ranges can reduce pressure without reducing useful parallelism, while storing values in shared memory trades registers for another limited resource and introduces access and synchronization costs. A kernel that passes a small benchmark can spill heavily for another template specialization or input shape. Record compile options, target architecture, launch size, and specialization alongside resource reports so regressions can be traced to the code actually running.

The Roofline lab provides only a traffic analogy. Reset, set intensity to 8 FLOP per byte and bandwidth to 80 GB/s, predict the synthetic ceiling, then Step. Compare a fresh Reset with both intensity at 4 and bandwidth at 80, interpreting the changed intensity as an assumed increase in bottleneck traffic. Its compute ceiling remains 1000 GFLOP/s. It has no registers, warps, occupancy, or spill compiler; explain why a lower traffic ceiling cannot determine whether a real register-cap change is beneficial.