NUMA placement and remote memory

A CPU-affinity mask restricts where a thread may execute. A memory policy influences eligible nodes and allocation choices for pages. Neither description alone proves the placement of an already populated array. A thread can move after initialization while its pages remain where they were, and an allocation request can reserve virtual addresses without immediately allocating every physical page. Inspect actual page distribution and execution placement rather than assuming the current worker owns nearby memory.

First-touch is a useful shorthand for placement influenced by the thread that first causes physical allocation under a particular policy. It is not an unconditional law. Read faults may map shared zero pages, policies can interleave or bind allocations, and automatic balancing or explicit migration can later change placement. Parallel initialization often helps align ownership with later computation, provided the same workers and partitioning remain relevant. Record the operating-system policy and initialization operations that make the claim true.

Private partitions with frequent reuse may benefit from placing each partition near its owning worker. A shared read-mostly dataset can sometimes be replicated if the memory budget and update contract permit it. A broad streaming workload may instead benefit from distributing allocations across memory controllers, even though some individual accesses are remote. A single local-versus-remote latency number cannot decide these cases because aggregate bandwidth and contention depend on the full placement and access matrix.

Migration also has a cost: bytes must move, mappings may need updates, and caches or translations may be disrupted. For a short-lived task, spending time moving an array may exceed any later savings. Replication creates consistency and freshness questions for mutable data. Compare total job time, capacity use, and communication volume, including initialization and movement when they are part of the real workflow. A steady-state microbenchmark can hide costs that users actually pay.

Begin with a topology description, worker affinity, allocation policy, and the observed node distribution of relevant pages. Compare local, remote, and interleaved arrangements with identical useful work and a stable problem size. Separate pointer-chasing latency tests from streaming bandwidth tests because they expose different bottlenecks. On a shared machine, external memory traffic can alter results; repeat and report variability rather than presenting one low sample as a property of the entire topology.

The vectors lab represents one two-dimensional vector using angle and magnitude, with x and y projections. Set a magnitude, predict how changing the angle redistributes the components, and verify their squared sum. This provides a limited accounting analogy for a workload divided between two categories, not a model of NUMA traffic: remote fractions are nonnegative counts, not signed vector components. It cannot infer page placement, latency, bandwidth, or the best policy for a real machine.