Peer-memory access and locality

Query peer capability for the device pair and relevant direction instead of assuming any two visible GPUs can access each other’s memory. Enable the needed peer-access relationship for ordinary allocations where supported. Unified virtual addressing can make a pointer recognizable without granting access permission or establishing locality. A peer path can also have a different bandwidth and latency from a device’s own memory. Record the topology and access direction; symmetric application code does not prove symmetric capabilities or performance.

Stream-ordered memory-pool allocations require their documented pool-access configuration, including cudaMemPoolSetAccess for the intended devices. Merely calling cudaDeviceEnablePeerAccess is not the corresponding permission mechanism for those pool allocations. Device-native peer atomics and ordering guarantees also need explicit capability and memory-model checks. Do not turn the existence of a working peer read into a claim that every atomic type, scope, or multi-device synchronization pattern is supported. Allocation provenance is part of the access contract, not an incidental implementation detail.

A producer kernel on GPU zero must finish the writes required by a consumer on GPU one before that consumer reads them. Establish the documented dependency, for example a suitable recorded event and supported cross-device stream wait, or an appropriate host synchronization before submission. A pointer arriving in the consumer’s launch arguments is not a completion signal. Separate access permissions from ordering and visibility so that a later performance change does not accidentally remove the dependency that made the original program correct.

The allocation must remain valid until every device that may access it has finished. Freeing on the owner device after its own last local use is insufficient if a peer still reads the tensor. Similarly, a remote write or atomic protocol needs the correct scope rather than an assumption that the interconnect provides application-level synchronization automatically. For an inference weight shard, immutable reads after one ordered publication are simpler than concurrent mutation, but even immutable data still require publication and lifetime ownership.

Remote access can avoid a copy when only a small portion of a tensor is consumed once. Copying a whole tensor locally can amortize its transfer when many full scans follow, but it uses additional capacity and introduces a copy-completion boundary. A real design may instead partition work, use a collective communication primitive, or cache selected blocks. Compare representative tensor sizes and reuse counts under concurrent load, because isolated link throughput does not capture contention with other collectives or transfers.

The Roofline lab illustrates only an assumed bandwidth change. Reset, set operational intensity to 2 and bandwidth to 80, predict the synthetic ceiling, then Step. Compare a fresh Reset with both intensity at 2 and bandwidth at 40 before Step. The ceilings are 160 and 80 GFLOP/s under a fixed synthetic compute limit of 1000. These values do not represent a particular peer link, GPU, copy operation, permissions check, or cross-device memory-order implementation.