SIMD and Vectorization
SIMD applies one instruction to several data elements at once; vectorization is the art of shaping code so the compiler or CPU can use it.
One instruction, many lanes
SIMD stands for Single Instruction, Multiple Data. A vector register holds several values (for example, eight single-precision floats), and one vector instruction operates on all of them together. This multiplies arithmetic throughput without adding cores, and it is the finest-grained form of data parallelism.
Vector instruction sets
- x86 SSE, AVX, AVX-512 with 128- to 512-bit registers
- Arm NEON and the scalable SVE
- Each doubling of width doubles the elements processed per instruction
Getting code to vectorize
Compilers auto-vectorize loops when they can prove it is safe. This requires loops with no data dependence between iterations, unit-stride (contiguous) memory access, no aliasing between pointers, and a known trip count. Restructuring data from arrays-of-structs to structs-of-arrays often enables vectorization by making access contiguous.
Obstacles
- Loop-carried dependencies that force sequential order
- Conditional branches inside the loop body
- Non-contiguous or gather/scatter memory patterns
- Function calls the compiler cannot inline
Why it matters
A CPU core's peak floating-point rate assumes full vector width. Scalar code that ignores SIMD may reach only a fraction, sometimes an eighth or a sixteenth, of peak. Because vectorized kernels are also more cache- and bandwidth-friendly, vectorization is one of the highest-leverage single-core optimizations, and a prerequisite before scaling out across cores and nodes. Compilers report which loops vectorized and why others did not, so inspecting those reports is often the fastest route to finding and removing the obstacle.