The Adjoint Method
A technique to compute the gradient of a scalar objective with respect to many parameters at a cost independent of the parameter count.
The sensitivity problem
In design and inverse problems one has a scalar objective (drag, error, energy) that depends on many parameters through the solution of a governing equation, such as a PDE. Naively, computing the gradient by perturbing each parameter and re-solving would cost one full simulation per parameter, which is prohibitive when parameters number in the thousands or millions. The adjoint method computes the entire gradient with essentially one extra solve, regardless of the number of parameters.
How it works
The objective is subject to the constraint that the state satisfies the governing equation. Introducing a Lagrange multiplier field (the adjoint variable) and requiring the Lagrangian to be stationary yields an adjoint equation: a linear equation, driven by the objective's sensitivity to the state, that runs backward relative to the forward problem. Solving it once gives the adjoint field, from which the gradient with respect to every parameter follows by a cheap inner product.
- One forward solve for the state
- One adjoint (linear, backward) solve for the multiplier
- A cheap projection to obtain the full gradient over all parameters
- Total cost independent of parameter count
Continuous vs discrete adjoint
The continuous adjoint derives the adjoint equation from the governing PDE and then discretizes it. The discrete adjoint differentiates the discretized equations directly, which is exactly what reverse-mode automatic differentiation produces. The discrete adjoint gives gradients consistent with the actual discrete solver (important for optimizers), while the continuous adjoint can be cleaner to derive and interpret.
Where it matters
The adjoint method is the engine of large-scale gradient-based optimization: aerodynamic shape design, seismic inversion, data assimilation in weather models, and parameter estimation in plasma transport. Any time a simulation must be tuned against data or an objective over a high-dimensional design space, the adjoint method is what makes gradient descent affordable.