Imitation Learning and Behavior Cloning
Behavior cloning turns imitation into supervised learning: predict the expert's action from the state.
Learning from demonstration
Imitation learning trains a policy from expert demonstrations rather than a reward signal. It is attractive when demonstrations are easy to collect but a reward is hard to specify, for example teaching a robot a manipulation skill by teleoperation.
Behavior cloning
The simplest approach is behavior cloning (BC): treat each demonstrated (state, action) pair as a labeled example and fit the policy with ordinary supervised learning. For discrete actions this is classification; for continuous actions, regression or a learned action distribution.
# Behavior cloning = supervised learning
# dataset of expert (state, action) pairs
loss = cross_entropy(policy(state), expert_action) # discrete
# or mse(policy(state), expert_action) # continuousCompounding error
BC's weakness is covariate shift. The policy is trained only on states the expert visited, but any small error moves it toward states the expert never saw, where its predictions are unreliable, causing further drift. Errors compound quadratically in the horizon rather than linearly, so long tasks are fragile.
- No reward needed, no environment interaction during training
- Fast and stable, since it is pure supervised learning
- Cannot exceed the expert, and degrades off the demonstrated distribution
Beyond cloning
Remedies address the distribution problem directly. DAgger iteratively collects expert labels on states the learner actually visits. Inverse RL and adversarial imitation recover a reward so the agent can recover from mistakes. Offline RL uses demonstrations plus returns to improve on the expert. BC remains the standard baseline and a common initialization for these richer methods.