Recurrent Neural Network

A neural network with feedback connections that maintains state to process sequences of varying length.

Definition

A recurrent neural network (RNN) processes a sequence one element at a time, maintaining a hidden state that carries information from earlier elements forward. This lets it handle inputs of variable length such as text, audio, or time series.

Because they process sequences step by step, RNNs cannot be parallelized across time during training, which limits their speed on long sequences. This sequential bottleneck, more than accuracy, is a major reason attention-based models displaced them for large-scale tasks.

Bidirectional variants process a sequence in both directions to use future as well as past context, useful when the whole sequence is available at once. For streaming applications, where input arrives over time, the forward-only form is required. The choice reflects a general principle: architecture should match how data is available at deployment, not only how it looks in training.

It is trained by backpropagation through time, unrolling the recurrence across the sequence.

The vanishing gradient problem

Plain RNNs struggle to learn long-range dependencies because gradients shrink or explode over many steps, a form of the vanishing gradient problem. Gated variants such as the LSTM were designed to fix this.

Why it matters

RNNs were the standard for sequence modeling before transformers, and they remain efficient for streaming and low-resource settings where processing one step at a time is an advantage.

Fusion connection

Recurrent models forecast the evolution of plasma diagnostic time series in simulation, a step toward predicting instabilities before they grow.