SIMD Units
A SIMD unit applies one instruction to many data elements at once, packing several values into a wide register and processing them in lockstep.
Single Instruction, Many Data
SIMD stands for single instruction, multiple data. Instead of one add per instruction, a SIMD unit packs several numbers into a wide vector register, say eight 32-bit floats in a 256-bit register, and adds all eight lanes in parallel with a single instruction. When the same operation applies to a whole array, this multiplies throughput without multiplying instruction-fetch work.
Vector Registers and Lanes
SIMD hardware is organized into lanes, each an independent slice of the datapath that processes one element. A 512-bit unit has sixteen 32-bit lanes or eight 64-bit lanes. Instruction sets like x86 SSE and AVX, ARM NEON and SVE, and RISC-V's vector extension expose these registers and the operations over them.
- Element-wise arithmetic across all lanes at once
- Shuffles and permutes that move data between lanes
- Masked operations that disable selected lanes
Where It Helps and Where It Does Not
SIMD shines on regular, data-parallel work: image and signal processing, linear algebra, physics simulation, and the inner loops of machine learning. It struggles when the data is irregular, when lanes need to branch differently, or when memory accesses are scattered rather than contiguous, since a single instruction cannot easily do different things in different lanes.
Programming Model
Compilers can auto-vectorize simple loops, but complex kernels are often written with intrinsics or hand-tuned assembly to control layout and avoid cross-lane penalties. Predicate masks let code handle loop remainders and conditional work by disabling lanes rather than branching. In scientific computing, SIMD is a core reason a modern CPU can sustain many floating-point operations per cycle.
SIMD sits between scalar execution and the massively parallel model of GPUs: wider than a scalar unit, but tightly coupled to a single instruction stream.