GPU Architecture
A GPU packs thousands of simple cores that run threads in lockstep groups, trading single-thread speed for enormous parallel throughput.
Throughput Over Latency
A CPU spends most of its transistors making a single thread fast: caches, out-of-order execution, branch prediction. A GPU makes the opposite bet. It fills the chip with thousands of simple arithmetic units and runs enormous numbers of threads, tolerating each thread being slow because so many run at once. The design goal is total throughput, not the latency of any one thread.
Warps and SIMT
GPU threads are grouped into fixed-size bundles (a warp or wavefront, commonly 32 or 64 threads) that execute the same instruction in lockstep across different data. This model, single instruction, multiple threads (SIMT), is like SIMD but the parallelism is expressed as many threads rather than explicit vector operations. Many warps are grouped onto a streaming multiprocessor, the GPU's core building block.
- Warp: a group of threads running one instruction in lockstep
- Streaming multiprocessor: schedules many warps over shared units
- Massive thread count hides memory latency
Hiding Latency
When a warp stalls waiting on memory, the multiprocessor's scheduler instantly switches to another ready warp, keeping the arithmetic units busy. With enough warps resident, memory latency is hidden entirely behind useful work. This is why GPUs devote hardware to holding thousands of thread contexts rather than to caches: their answer to slow memory is more parallelism, not lower latency.
Divergence and Memory
The SIMT model falters when threads in a warp take different branches: the hardware must execute both paths with some lanes masked off, called branch divergence, wasting throughput. Memory performance depends on coalescing: when threads in a warp access adjacent addresses, the hardware merges them into few wide transactions. GPUs excel at regular, data-parallel numerical work, which is why large-scale scientific simulation, including plasma and magnetics modeling relevant to fusion research, maps well onto them.