Imitation Learning

Imitation learning starts from a practical idea: rather than painstakingly describing what “good” behaviour looks like with a reward function, show the agent examples of an expert doing the task. The agent then learns to turn observations—camera images, robot positions, game states—into actions that resemble those demonstrations.

How it works
The simplest form, behavioural cloning, treats each demonstration as a supervised-learning pair: state in, expert action out. A driving policy, for example, learns steering and braking from recorded human driving. This avoids inventing a reward that a car could exploit, such as driving quickly while ignoring safety.

The central difficulty: drifting off the expert path
A cloned policy is trained on states the expert visited. But one small steering error can put it in a state absent from the data; its next prediction is then less reliable, causing further errors. This is called distribution shift or compounding error. DAgger addresses it by repeatedly letting the learner act, asking an expert for the correct action in the states it actually reaches, and adding those corrections to the dataset. The result trains on recovery situations, not just perfect demonstrations.

Beyond copying actions
Other approaches infer what the expert was trying to achieve rather than copying each move:

Inverse reinforcement learning learns a reward function under which the demonstrated behaviour looks good, then uses RL to optimize that reward.
Generative Adversarial Imitation Learning (GAIL) trains a discriminator to distinguish expert from learner trajectories; the learner receives a reward for producing trajectories the discriminator cannot tell apart.

These methods help when several actions are valid or when matching whole trajectories matters. They still depend on demonstration coverage: an agent cannot reliably learn emergency recovery, rare obstacles, or a shortcut the expert never showed. In Gymnasium robotics tasks or MuJoCo control, imitation can provide a strong initial policy before reward-based methods such as PPO refine it—especially when real-world trial-and-error would be slow, expensive, or unsafe.

Imitation learning trains an agent to reproduce behaviour demonstrated by an expert, replacing or supplementing hand-designed reward signals with example trajectories. It can learn a policy directly through behavioural cloning or infer objectives from demonstrations through inverse or adversarial methods. It matters when desired behaviour is easier to show than to specify as a reward, though performance depends on demonstration quality and coverage of the states the agent encounters.

Imitation learning is like learning to cook by watching a skilled chef rather than being told exactly what makes a meal “good.” You watch what they chop, when they stir, and how they react when something goes wrong—then try to copy those choices.

For AI, the “chef” might be an expert driver, a surgeon, or a person controlling a robot. Instead of inventing a detailed list of rewards and penalties, developers provide examples of good behaviour. The system learns patterns from those demonstrations and uses them to act in similar situations. This matters when the goal is easy to recognise in an expert’s actions but hard to describe as a simple rule.