Shared-Memory Parallelism
In shared-memory systems many cores read and write one common address space, communicating implicitly through memory rather than explicit messages.
One address space, many cores
A shared-memory machine gives every processor access to a single memory address space. Threads communicate simply by reading and writing shared variables; there is no explicit send or receive. This model is natural to program and underlies multi-core CPUs and the threads within a single compute node.
Threads and their pitfalls
Because threads share data, two threads updating the same location without coordination create a race condition: the result depends on timing and is not reproducible. Correct shared-memory code protects shared state with locks, atomics, or barriers, and minimizes the amount of shared mutable state.
Programming models
- OpenMP: compiler-directive parallel loops and tasks
- POSIX threads (pthreads): explicit low-level threading
- Threading building blocks and parallel standard-library algorithms
The scaling wall: memory bandwidth
Adding cores helps only while memory can feed them. Many cores share the same memory controllers and last-level cache, so a bandwidth-bound kernel stops speeding up well before all cores are used. On larger nodes, memory is non-uniform (NUMA): a core's local memory is faster than a remote socket's, and ignoring this costs performance.
Where it fits
Shared memory is the model within a node. To span many nodes, it is combined with distributed memory and message passing, giving the common hybrid: MPI between nodes, OpenMP or threads inside each node. This hybrid matches the hardware directly, since a node genuinely shares memory while separate nodes do not, and it keeps the number of communicating processes manageable at very large scale.