Tensor Cores
Tensor cores are specialized units that compute small matrix multiply-accumulate operations in one step, at much higher throughput than general cores.
Matrix engines
A tensor core is a hardware unit that performs a fused matrix multiply-accumulate on small tiles, computing D = A*B + C for fixed-size sub-matrices in a single operation. Because matrix multiplication is the dominant kernel in deep learning and appears throughout dense linear algebra, dedicating silicon to it yields far higher throughput than issuing the same multiplies and adds on general-purpose cores.
Mixed precision by design
Tensor cores typically multiply inputs in a reduced-precision format (FP16, BF16, or lower) while accumulating the products in a wider format (FP32). This mixed-precision structure is deliberate: the multiplications tolerate low precision, but summing many products in low precision would lose accuracy, so the accumulator stays wide. The result is high speed with error controlled well enough for training and many scientific uses.
- Compute a small matrix multiply-accumulate tile per operation.
- Reduced-precision multiply, wider-precision accumulate.
- Programmed via warp-level primitives (WMMA) or library calls.
- Best reached through tuned libraries rather than hand-written kernels.
How code reaches them
Most applications use tensor cores indirectly through libraries such as tuned GEMM (matrix multiply) and convolution routines, or through deep-learning frameworks that map their operations onto them. Direct programming uses warp-level fragment APIs that load tiles into registers across a warp, issue the matrix operation, and store results. Getting good utilization requires the right data types, tile alignment, and enough work to keep the units fed.
Beyond deep learning
Because tensor cores are just fast matrix engines, they benefit any workload that can be cast as dense small-matrix products: batched linear algebra, certain stencil and spectral operations, and mixed-precision iterative refinement solvers. In fusion modeling, dense linear-algebra steps and surrogate neural models trained on Hyperion simulation data can both use tensor cores, provided the precision strategy is validated against a full-precision reference.