Accelerators: GPUs, TPUs, and FPGAs
Accelerators are specialized processors that outperform general CPUs on specific workloads by trading flexibility for raw throughput or efficiency.
Why specialize
A general-purpose CPU spends much of its silicon on flexibility: control logic, caches, and speculation. An accelerator strips that away and dedicates hardware to a narrower class of work, gaining throughput and energy efficiency. The trade is programmability and generality for speed on the target workload.
The main families
- GPUs: thousands of cores for data-parallel floating-point work; the general-purpose accelerator
- Tensor / matrix engines: hardware for dense matrix multiply-accumulate, the core of deep learning
- FPGAs: reconfigurable logic wired to match a specific dataflow; strong for streaming and low latency
- ASICs: fixed-function chips (such as ML inference accelerators) that give the best efficiency for one task
Tensor and matrix units
Modern accelerators include dedicated units that multiply small matrices in one operation, often at reduced precision. Because neural-network training and inference are dominated by matrix multiplication, these units deliver most of the peak throughput on such workloads. They are effective only when the computation is cast as dense matrix math.
The programmability spectrum
GPUs are the most programmable accelerators and dominate general HPC. FPGAs demand hardware-description or high-level-synthesis tooling and long build times, but excel where a fixed pipeline and predictable latency matter. ASICs are least flexible and most efficient, justified only at large deployment scale.
Heterogeneous reality
No single device wins everywhere, so systems are heterogeneous: CPUs orchestrate and run irregular code, accelerators do the arithmetic-heavy inner loops. The engineering challenge is partitioning work and managing data movement across memory spaces so the accelerators stay fed.