Calculus of Variations for Control

The calculus of variations finds functions that minimize integral cost functionals, the mathematical foundation of optimal control.

Optimizing Over Functions

Ordinary calculus finds the point that minimizes a function. The calculus of variations finds the entire function that minimizes a functional, a quantity defined by an integral over a trajectory. Because optimal control seeks an input signal (a function of time) that minimizes a cost integral, the calculus of variations is its direct mathematical ancestor.

The Euler-Lagrange equation

The central result is the Euler-Lagrange equation. For a functional given by the integral of a running cost L that depends on a trajectory and its derivative, the minimizing trajectory must satisfy: the partial derivative of L with respect to the trajectory equals the time derivative of the partial derivative of L with respect to its rate. This differential equation is the necessary condition for an extremum.

From variations to control

• Treat the state trajectory and control input as the functions to be chosen.
• Adjoin the system dynamics as constraints using Lagrange multipliers, which become the costate variables.
• The resulting stationarity conditions are exactly Pontryagin's principle for the unconstrained-input case.
• Input constraints, which the classical calculus of variations does not handle, are what Pontryagin's principle adds.

Transversality conditions

When the endpoints of the trajectory are not fully fixed, additional boundary requirements called transversality conditions determine the free endpoints. These specify, for example, that the costate vanishes at a free terminal time or matches the gradient of a terminal cost, completing the two-point boundary-value problem.

Historical role

The calculus of variations predates control theory by centuries, arising from problems such as the brachistochrone, the curve of fastest descent. Its extension to constrained dynamic systems in the twentieth century produced modern optimal control. Understanding it clarifies why the costate and Hamiltonian appear and where the boundary conditions come from.

In practice the calculus of variations is rarely applied by hand to real designs; instead its conditions are embedded in numerical optimal-control solvers. But it remains the conceptual bedrock on which Pontryagin's principle and the Hamilton-Jacobi-Bellman equation rest.