Cooperative grids: synchronize only within a supported launch

If a kernel launches more blocks than can reside, some blocks wait for earlier ones to release SM resources. Suppose every resident block spins until a counter says all grid blocks arrived. The unscheduled blocks cannot increment that counter, and the resident blocks never leave to make room. Correct atomic increments do not fix this scheduling deadlock. A fence also does not allocate missing residency. Use separate correctly ordered kernel phases or a supported cooperative protocol rather than relying on a favorable scheduling accident.

CUDA Cooperative Groups can represent a grid group whose sync operation is valid under the documented cooperative launch requirements. Use the cooperative launch API and query device support. Every participating group member must follow the collective contract; an early return by some required participants can invalidate the synchronization. Do not infer support for multi-device or newer cluster collectives from single-device grid support. These are distinct features with architecture, runtime, and launch requirements of their own.

B_active depends on block dimensions, compiled registers, static and dynamic shared memory, and other device constraints. Increasing a tile’s shared memory can reduce the maximum legal cooperative grid even when the ordinary launch would remain valid. Use the occupancy query for the specific kernel and dynamic shared-memory amount, then apply the cooperative launch requirements. A remembered device-wide thread count is not enough. Another build or kernel attribute can change the answer without any change to the application’s logical problem size.

An all-resident cooperative grid can process more data than its thread count by assigning multiple elements to threads or iterating over tiles. That changes the work mapping, not the residency limit. Ensure all participants still reach each grid synchronization in the required order while guarding data accesses. Tail handling that causes some blocks to exit before a grid barrier can break correctness. Measure whether the saved phase launches justify longer-lived blocks, additional synchronization, and the constrained scheduling footprint.

Separate kernels ordered within one stream provide a natural phase dependency without requiring all blocks from both phases to reside together. They also add launch and intermediate-state costs. A cooperative kernel can preserve some state across phases and synchronize internally, but is more constrained. Choose based on correctness, queried support, resource budgets, and measured application behavior. Include the smallest and largest supported problem shapes in tests, especially when dynamic shared-memory requirements change with shape.

The reliability lab illustrates only finite simultaneous availability. Reset, set n=6 and k=2, predict four available replicas, then Step. These are replica counts and a majority readout, not SMs or resident blocks. The lab has no cooperative launch, spinning kernel, or grid barrier, and its majority rule does not determine GPU validity. Use the explicit residency formula and API queries for the cooperative grid. The analogy simply warns against assuming that every nominal participant is simultaneously available to make progress.