Model Quantization
Quantization shrinks and speeds up models by storing and computing weights in low-precision integers instead of floats.
Fewer bits per number
Quantization reduces the numerical precision used to represent a model weights and activations, most commonly from 32-bit floating point to 8-bit integers, and sometimes lower. Because memory, bandwidth, and arithmetic all scale with bit width, quantization cuts model size and speeds up inference, often several-fold, with modest accuracy loss, which makes it central to deploying models on phones and edge hardware.
How values are mapped
Quantization maps a continuous range of real values onto a small set of discrete levels using a scale and a zero point: an integer q relates to a real value r by r = scale * (q - zero_point). The scale is chosen to cover the observed range of the tensor. Per-channel scales, one per output channel of a layer, preserve more accuracy than a single per-tensor scale because ranges vary across channels.
Two workflows
- Post-training quantization: quantize an already-trained model, optionally using a small calibration set to set ranges; fast and simple
- Quantization-aware training: simulate quantization during training so the model learns weights robust to it; more work but higher accuracy, especially at very low bit widths
Trade-offs and hardware
The accuracy cost grows as precision drops; 8-bit integer inference is often nearly lossless, while 4-bit and below usually require quantization-aware training or careful outlier handling. Realized speedups depend on hardware support for low-precision arithmetic, so gains vary by target device. Activations are harder to quantize than weights because their ranges shift with each input. Quantization stacks with pruning and distillation to compress models further for deployment.
For very large models, weight-only quantization reduces memory footprint even when activations stay in higher precision, easing the memory bottleneck of loading the model.