Dual Control
Dual control balances two competing goals: controlling the plant well now and probing it to reduce uncertainty for better control later.
The exploration-exploitation tension
When a controller must act on an uncertain model, aggressive control that regulates tightly may suppress the very signal variation needed to identify the plant, while probing to learn the model injects disturbances that hurt current performance. Feldbaum's dual control recognizes both effects and optimizes them jointly: control actions serve both to regulate and to inform.
The optimal but intractable ideal
The exact dual-control problem is a stochastic dynamic program over the joint distribution of state and parameters, the hyperstate. Its solution automatically trades caution, exploitation, and probing, exploration. Unfortunately the dynamic program is intractable except for toy problems, because the hyperstate is high-dimensional and the recursion suffers the curse of dimensionality.
Practical approximations
Real dual controllers approximate the ideal. Certainty-equivalence control ignores the probing value entirely and often under-excites, letting estimates stall. Cautious control accounts for uncertainty in the cost but can become overly timid, even switching off, the turn-off phenomenon. Explicit dual methods add a probing term, or a constraint on estimation accuracy, to deliberately maintain excitation. Approximate dynamic programming and wide-sense dual schemes offer computable middle grounds.
- Control actions both regulate and identify
- Exact solution is an intractable stochastic dynamic program
- Certainty equivalence under-probes; cautious control over-cautions
- Practical schemes add explicit probing to maintain excitation
Dual control clarifies why self-tuning regulators and adaptive controllers sometimes fail: pure certainty equivalence neglects the learning value of inputs, so estimates degrade under good regulation. Adding modest probing restores identifiability.
The framework is conceptually central to adaptive and learning-based control, including modern reinforcement-learning controllers, which face the same exploration-exploitation trade-off. For any design-stage adaptive loop, dual-control thinking guides how much to probe.