Behavioural Cloning
Imagine learning to drive by watching an expert’s steering, braking, and lane choices, then practising those same responses yourself. Behavioural cloning takes this direct approach: rather than discovering good behaviour through rewards and trial-and-error, it learns to copy decisions found in demonstrations.
How it learns
Behavioural cloning treats imitation as a supervised-learning problem. A demonstration dataset contains pairs of an observed situation and the expert action taken there: (state, action). The model, called a policy, is trained to predict the action from the state:
- Input: camera images, robot joint positions, game state, or other observations.
- Target: the expert’s steering command, grasp motion, joystick action, and so on.
- Objective: make the learned policy choose actions close to those in the demonstrations.
The central weakness: drifting off course
The difficult part is not fitting the recorded examples; neural policies can do that well. The problem is distribution shift. During training, the policy sees states visited by the expert. At deployment, one small mistake can put it in an unfamiliar state—slightly off the road, for example—where it has no learned recovery action. That error leads to another unfamiliar state, and failures compound across a long task. A driving policy that predicts expert steering accurately on held-out clips can therefore crash when it must control the car continuously.
Why it matters in reinforcement learning
Behavioural cloning avoids designing a reward function, which is valuable when “do this well” is easier to demonstrate than to specify. It is widely used to initialize agents before reinforcement learning, including robot policies trained in simulation or from teleoperation data. But it does not independently improve beyond its demonstrations, explore better actions, or know which outcomes are safe. Methods such as DAgger address its main failure mode by repeatedly collecting states visited by the learner and asking an expert for the correct recovery action. Behavioural cloning is therefore a fast, practical starting point—not a guarantee of robust autonomous behaviour.
Behavioural cloning is an imitation-learning method that trains a policy by supervised learning on expert state–action demonstrations, predicting the action an expert would take in each observed state. It avoids designing a reward function and can initialise competent behaviour from offline data. Its main limitation is distribution shift: small errors move the policy into unfamiliar states, where errors compound without corrective feedback.
Imagine learning to make coffee by watching a skilled barista and copying each step: grind the beans, steam the milk, pour carefully. You are not told the deeper goal behind every movement; you simply learn to reproduce what good examples look like.
Behavioural cloning does this for AI. It trains a system by showing it recordings of an expert’s choices, such as a human driving a car or controlling a robot. The system learns to imitate those choices in similar situations. This can be useful when it is hard to describe every rule or reward. But it may struggle in unfamiliar situations, where the expert examples do not show what to do.