Backpropagation Through Time

BPTT trains recurrent networks by unrolling the loop over time and applying the chain rule across every step.

Unrolling the recurrence

A recurrent network reuses the same weights at each time step. To compute gradients, we conceptually unroll the loop into a deep feedforward network, one copy of the cell per step, sharing weights. Backpropagation through time (BPTT) runs the standard backward pass over this unrolled graph and then sums the gradient contributions to the shared weights across all steps.

The gradient sum

Kronos motion — confinement time

Because a weight matrix appears at every step, its total gradient is the sum of the gradients from each step. The loss at a late time step depends on hidden states from many earlier steps, so its gradient flows back through the whole chain. This chained dependence is what lets an RNN learn how early inputs affect later outputs, but it is also the source of numerical trouble.

Why gradients misbehave

Propagating a gradient back across k steps involves multiplying by the recurrent Jacobian k times. If the relevant eigenvalues are below one, the product decays exponentially and the gradient vanishes; if above one, it grows exponentially and explodes. Vanishing gradients mean the network effectively forgets distant inputs; exploding gradients cause unstable updates.

Truncated BPTT

Unrolling over a very long sequence is expensive in memory and time. Truncated BPTT splits the sequence into chunks and backpropagates only within each chunk (or a fixed window), carrying the hidden state forward but stopping the gradient at the boundary. This bounds memory and computation at the cost of not learning dependencies longer than the truncation window.

python

# gradient clipping guards against exploding gradients
import torch
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

Practical remedies

Exploding gradients are handled by gradient clipping, which rescales the gradient when its norm exceeds a threshold. Vanishing gradients are harder and are addressed architecturally: LSTM and GRU introduce gated pathways that let gradients flow across many steps without repeated shrinking. Good initialization and normalization also help keep the unrolled network trainable.

Unroll the recurrence, then apply backprop.
Shared-weight gradients are summed across all steps.
Repeated Jacobian products cause vanishing or exploding gradients.
Truncation bounds cost; clipping and gating stabilize training.