Unified Memory page migration

CUDA exposes device attributes that describe managed-memory and concurrent-access support. Read those attributes together with the operating-system and platform requirements before choosing an access protocol. A Linux discrete GPU with supported concurrent managed access does not establish the rules for every Windows configuration, integrated GPU, or coherent CPU-GPU system. HMM and address-translation services can change how system allocations are accessed. Hardware coherence on a platform also does not make unsynchronized conflicting program accesses valid under the required memory model.

Keep four questions separate: can a processor address the allocation, where are its current data, which coherence mechanism maintains the image, and what orders a particular producer before a consumer? A kernel and CPU thread may require an event or stream synchronization even when they share a pointer. An application that relies on an unsupported concurrent CPU access can fail for reasons unrelated to bandwidth. Document the platform capabilities alongside the source-level ownership protocol rather than hiding them behind the phrase managed memory.

A first GPU access can trigger fault handling and data movement on a suitable demand-paged platform. Subsequent accesses may reuse resident data, but alternating CPU and GPU ownership can repeatedly move a working set. Oversubscription adds another problem: a working set larger than available local memory can cause eviction and refaulting. Fault counts alone do not completely describe performance because batching, migration size, remote accesses, and overlap matter. Use profiling evidence to distinguish one-time startup costs from repeated movement during steady-state training.

Prefetch and memory advice express documented requests or policy information; they are not blanket guarantees that every later access stays local. An asynchronous prefetch must also be ordered before the consumer if its completion is required. Organize work into phases that reuse tensors near the processor performing repeated computation, and avoid fine-grained CPU inspection that unintentionally creates frequent ownership changes. This is an optimization hypothesis, not a rule that every managed allocation should always be copied in its entirety to a GPU.

Compare end-to-end execution with representative tensor sizes, not merely a small allocation that remains resident after warmup. Record GPU and CPU access phases, memory pressure, prefetch dependencies, and capabilities. A workload that performs CPU preprocessing once and many GPU updates has a different locality story from one that alternates each layer between processors. If results vary with memory pressure, test whether local capacity or another resource explains the change before treating migration policy as the only cause.

The cache lab illustrates the narrower idea of a repeatedly reused working set. Reset, set capacity to 4 and stride to 4, predict the result at seventeen requests, then Step once. Compare a fresh Reset with both capacity at 2 and stride at 4 before Step. The toy starts empty but initially counts sixteen requests; one Step adds the seventeenth. It models deterministic fully associative LRU, not managed pages, hardware coherence, prefetch, page faults, or CPU-GPU ownership transfers.