SM residency: account for every binding resource

A block consumes some effective register allocation, some shared memory, and a number of thread and warp slots. Several blocks fit only when all required capacities remain within limits. If registers allow four blocks and shared memory allows two, the useful residency bound is two, not their average or sum. A zero-sized allocation does not create a meaningful divide-by-zero bound; treat that resource as nonbinding instead. Other architecture-specific constraints can make a simplified bound optimistic.

Registers are allocated at architecture-dependent granularities, often involving warp or block allocation units. Multiplying a compiler-reported register count by active threads is therefore not always the final reserved amount. Shared-memory alignment, carveout configuration, dynamic shared memory, kernel attributes, and maximum block dimensions also matter. Query the actual device and kernel, inspect compiler resource reports, and use the occupancy API for the selected configuration. A number copied from an old tuning example is not an implementation-independent limit.

Reducing registers can allow another block to reside, but forcing a low register cap can spill values into local memory and increase memory traffic. Increasing shared-memory reuse can reduce global loads while limiting resident blocks. Larger blocks can share work efficiently yet waste lanes on a tail or constrain scheduling. These are tradeoffs rather than rules that every kernel should maximize one resource ratio. Compile and measure the complete kernel after a proposed resource change.

A kernel may also fail to launch if the requested per-block resources exceed supported limits. A valid launch with lower residency is different from an invalid resource request. Static resource estimates do not establish which resident warps are ready or whether another workload competes for the device. Keep launch validity, theoretical occupancy, achieved occupancy, and useful throughput in separate reports. When tuning, identify whether a change affected resource allocation, executed work, or the measured application’s critical path.

Record GPU model and compute capability, CUDA/compiler version, block dimensions, registers, static and dynamic shared memory, and relevant kernel attributes. Repeat resource inspection after changing compiler flags, templates, or inlining because generated code can change. A single occupancy API result is useful configuration evidence, but it does not validate out-of-bounds accesses or barriers. Test correctness before interpreting performance, and distinguish total application allocations in device memory from per-SM shared-memory or register limits.

The reliability lab offers a small arithmetic analogy for finite capacity. Reset, set n=8 and k=3, predict five available items, then Step. The model actually counts available replicas and a majority threshold; it does not allocate registers, model SM blocks, or calculate occupancy. Use only its explicit available-count readout to separate total capacity from unavailable capacity. The real residency calculation below uses its own resource units and minimum rule rather than the lab’s unrelated majority condition.