Pontryagin's Maximum Principle
Pontryagin's principle gives necessary conditions for optimal control through a Hamiltonian and an adjoint costate system.
Necessary Conditions for Optimality
Pontryagin's maximum principle, developed in the 1950s, provides the necessary conditions that any optimal control trajectory must satisfy. It generalizes the calculus of variations to problems with input constraints, which is essential because real actuators are always bounded.
The Hamiltonian
Central to the principle is the Hamiltonian, which combines the running cost with the dynamics weighted by an adjoint variable called the costate. The optimal control at each instant is the one that minimizes (or, in the original sign convention, maximizes) the Hamiltonian over the allowed set of inputs. This pointwise condition is what gives the principle its name.
The three conditions
- State equation: the state evolves forward by the system dynamics.
- Costate equation: the costate evolves backward, driven by the derivative of the Hamiltonian with respect to the state.
- Minimum condition: at each time the input minimizes the Hamiltonian, respecting input constraints.
- Boundary conditions tie the two-point problem together, with transversality conditions at the endpoints.
Why constraints matter
When inputs are bounded, the optimal control often sits at a limit rather than at an interior stationary point. The maximum principle handles this directly: the input that minimizes the Hamiltonian may be at the boundary of the allowed set. This produces bang-bang control, where the input switches between extremes, common in minimum-time problems.
The two-point boundary-value problem
Applying the principle yields a two-point boundary-value problem: the state has initial conditions, the costate has terminal conditions, and both must be solved together. This coupling forward and backward in time makes such problems harder than a simple simulation, and they are usually solved by numerical shooting or collocation methods.
Pontryagin's principle is the trajectory-based counterpart to dynamic programming's value-function approach. It is especially powerful for open-loop optimal trajectories and minimum-time or minimum-fuel problems, where its bang-bang structure gives clean, interpretable solutions.