Pipelining Hazards and Forwarding
Pipelining overlaps instruction stages for throughput, but data, control, and structural hazards can stall it unless resolved.
Overlapping Work
A pipelined processor splits instruction execution into stages, a classic set being fetch, decode, execute, memory, and write-back. While one instruction executes, the next decodes and the one after fetches. The clock period shrinks to one stage's delay, so ideally one instruction completes every cycle even though each takes several cycles end to end.
The catch is that instructions are not independent. A hazard is any situation that prevents the next instruction from executing in its designated cycle.
Three Kinds of Hazard
- Data hazard: an instruction needs a result the previous instruction has not yet produced
- Control hazard: a branch changes the instruction flow before the target is known
- Structural hazard: two instructions need the same hardware resource at once
Forwarding
Many data hazards are cured by forwarding (bypassing): the result of an instruction is routed directly from the pipeline stage that produced it to the stage that needs it, before it is written back to the register file. Extra multiplexers at the execute stage's inputs select either the register value or a freshly computed result. Forwarding eliminates most stalls between dependent arithmetic instructions.
When Forwarding Is Not Enough
A load-use hazard is the stubborn case: an instruction needs data that a load is still fetching from memory, which is not available in time even with forwarding. The pipeline must insert a one-cycle stall (bubble). Control hazards are handled by branch prediction and, when a prediction is wrong, by flushing the wrongly fetched instructions. Structural hazards are usually designed away by duplicating resources, such as separate instruction and data caches.
The art of pipeline design is keeping the machine full: deeper pipelines allow higher clock rates but make each mispredicted branch and each stall more expensive, because more in-flight work must be discarded.