Warps, Occupancy, and Divergence
GPUs execute threads in lockstep groups called warps; keeping enough warps resident and avoiding divergence within them drives performance.
The warp
A GPU executes threads in fixed-size groups, a warp of 32 threads (or a wavefront on some hardware), which run in lockstep: all threads in a warp execute the same instruction at the same time on their own data. This is how a GPU achieves its throughput, but it also creates two performance considerations, occupancy and divergence.
Occupancy
Occupancy is the ratio of active warps to the maximum a streaming multiprocessor can hold. A GPU hides memory latency by switching among ready warps, so higher occupancy means more warps available to cover stalls. Occupancy is limited by resource use per thread: registers and shared memory. Using too many registers per thread reduces the warps that fit.
Branch divergence
Because a warp shares one instruction stream, an if-else where some threads take one branch and others take the other forces the warp to execute both paths serially, masking off the inactive threads in each. This divergence can halve or worse the throughput of that region. Structuring code so warps take uniform branches avoids the penalty.
Practical guidance
- Keep enough warps resident to hide memory latency
- Limit per-thread register and shared-memory use to raise occupancy
- Avoid data-dependent branches that split a warp
- Coalesce memory access so a warp's loads combine into few transactions
The balance
Maximum occupancy is not always optimal; a kernel using more registers per thread may run faster despite lower occupancy if it hides latency another way. As always, profiling the actual kernel, not chasing a single metric, decides what to tune.