Reinforcement-Learning Control of Tokamaks
Learning control policies that command magnetic coils to shape and stabilize a plasma, as demonstrated in published TCV work.
The control problem
A tokamak plasma must be held in a target shape and position by adjusting many magnetic-field coils together, faster than a human can react and against instabilities that grow in milliseconds. Traditionally this uses hand-designed feedback controllers built from a linearized plasma model.
The reinforcement-learning approach
Reinforcement learning (RL) trains a control policy by trial and error against a simulator, rewarding it for keeping the plasma near targets and penalizing constraint violations. Published work on the TCV tokamak demonstrated an RL policy that controlled the coil voltages to achieve a range of plasma shapes, including configurations that are hard to hand-tune.
- State: magnetic measurements and plasma shape estimate
- Actions: voltage commands to the shaping coils
- Reward: closeness to target shape, position, and current, with penalties
Why train in simulation
Exploratory actions on real hardware are risky and discharges are scarce. Policies are therefore trained against a fast, accurate plasma-and-circuit simulator, then transferred to the device. The fidelity of the simulator, and how the policy handles the gap between simulation and reality, largely determine success.
Strengths and limits
RL can discover control strategies for configurations without a good hand-designed controller, and can pursue multiple objectives at once. But it inherits any bias in the simulator, can behave unpredictably outside trained conditions, and needs safety constraints so that exploration and deployment never command physically unsafe actions.
Design context
RL control is a general method applicable to tokamak designs, including the compact, negative-triangularity geometry of the Kronos breeder. Any such use is simulation-based control study for a design, not operation of built hardware.