NUMA: Non-Uniform Memory Access
On multi-socket nodes, memory local to a core is faster than remote memory; ignoring this non-uniformity silently halves performance.
Memory is not equidistant
A modern compute node often has several processor sockets, each with its own attached memory. A core can access any memory in the node's shared address space, but its local memory (attached to its own socket) is faster and higher-bandwidth than remote memory across the socket interconnect. This is Non-Uniform Memory Access, NUMA.
First-touch placement
On common operating systems, a memory page is physically placed near the core that first writes to it, not the core that allocates it. This first-touch policy means the initialization pattern determines data placement. Initializing an array with one thread, then processing it with many, leaves most data remote to most threads, a frequent, invisible performance loss.
Getting it right
- Initialize data with the same thread layout that will process it
- Pin threads to cores so they do not migrate away from their data
- Keep each thread working mostly on its local memory
Thread affinity
Operating systems may move threads between cores, which can strand a thread far from its data and disrupt cache reuse. Affinity settings (through OpenMP's OMP_PROC_BIND and OMP_PLACES, or numactl) pin threads to specific cores or sockets, preserving locality. Correct affinity is often worth a substantial speedup on multi-socket nodes.
The bigger picture
NUMA is the within-node echo of the same principle that governs whole clusters: data movement is expensive, and locality is the lever. Treating a node's memory as uniformly fast is a common mistake; respecting its structure is part of writing scalable shared-memory code.