Shared-memory reuse: stage a tile and prove who may overwrite it

For a square output tile of width T and a reduction chunk of width T, the block needs one T-by-T tile of A and one of B. The resulting T cubed multiply-add contributions reuse each A value across output columns and each B value across output rows. Cooperative loads assign parts of those inputs to threads, which then share the staged values. This counts mathematical reuse inside one block. It does not imply each input is fetched from device memory exactly once across all blocks or that every local access is equally cheap.

A simple stage-load intensity counts only the bytes bringing A and B into the staging region and the arithmetic using that stage. It deliberately excludes output stores and any old-C reads, so it must not be compared directly with a whole-operation tensor-byte count without adjustment. Data type conversion, padding, alignment, and implementation metadata can increase staging storage. Neighboring blocks can still load overlapping operands, with caches potentially satisfying some traffic. Name the load boundary and counting assumptions every time the ratio is used.

In a straightforward CUDA block algorithm, cooperating threads finish loading shared input before dependent threads consume it. Before the same storage is reused for the next K chunk, all relevant consumers must have finished reading the old tile. A barrier can establish these phase relationships when every required thread follows the protocol. An out-of-range output thread may still be needed to perform a cooperative load or participate in a barrier, so an early return based solely on its output coordinate can break correctness.

Asynchronous copies and double buffering change the schedule rather than eliminating the need for ordering. A copy must be issued with the supported alignment and size conditions, and consumers must wait for the corresponding stage to become available. Before producers reuse a buffer slot, the consumers of its prior generation must be done. CUDA 12.8 documents architecture-dependent asynchronous mechanisms; do not assume identical support or synchronization semantics on every GPU. Draw the stage-generation timeline and match each wait or barrier to the data it protects.

Increasing tile size or stage count can improve reuse or overlap while consuming more shared memory and registers per block. That can reduce resident blocks, limit independent warps, or prevent a launch if resource limits are exceeded. Occupancy itself is a constraint and diagnostic, not a universal performance score: a lower-occupancy kernel can still be faster if it reduces traffic or uses instructions more effectively. Measure the complete kernel with architecture-specific counters and a correctness check after each schedule change.

The roofline lab illustrates only the potential benefit of higher intensity. Reset, set I=4 and B=100, predict ceiling 400 GFLOP/s, then Play or Step. Separately Reset, set I=8 and B=100, predict 800 GFLOP/s, then Play or Step. Its fixed compute ceiling is 1,000 GFLOP/s, and the controls do not allocate shared memory or tune stages. Use the doubled synthetic bound to motivate a reuse hypothesis, then explain why extra synchronization, resource pressure, or actual traffic can prevent that gain in a real kernel.