GPU Occupancy
Occupancy is the ratio of active warps on a streaming multiprocessor to the hardware maximum, a proxy for how well latency is hidden.
Definition
Occupancy measures how many warps are resident and eligible to run on a streaming multiprocessor relative to the architectural maximum. High occupancy means the SM has many warps to switch among, so when one warp stalls on a memory access another can issue, keeping the execution units busy. It is the primary mechanism by which GPUs hide the long latency of global memory.
What limits it
Occupancy is capped by whichever per-SM resource runs out first as more blocks are placed on the SM: the number of registers each thread uses, the amount of shared memory each block requests, or the hard limits on resident blocks and warps. A kernel that uses many registers per thread can host fewer threads, lowering occupancy. Compilers report register and shared-memory usage so this can be tuned.
- Occupancy = active warps / maximum warps per SM.
- Register pressure, shared-memory use, and block limits cap it.
- Higher occupancy hides memory latency but is not the sole goal.
- Occupancy calculators predict the limit from resource usage.
The nuance: more is not always better
Maximizing occupancy is a means, not an end. A kernel with low occupancy but heavy instruction-level parallelism (independent operations within each thread) can fully hide latency with fewer warps. Some highly tuned kernels deliberately trade occupancy for more registers per thread to keep more data on-chip. The right target is enough occupancy to hide latency, after which further increases give nothing while the register cut may cost reuse.
In practice
When porting a Hyperion transport kernel to GPUs, the first tuning pass checks whether register spilling has dropped occupancy so low that the SM stalls. Trimming per-thread state to lift occupancy to the point of latency hiding, then stopping, is a typical and disciplined path rather than chasing 100 percent.