World Model
A world model is an agent’s internal predictive sketch of its environment: a learned system for imagining what will happen next. Rather than learning only that an action was rewarded in the past, the agent learns how actions change the situation and uses that knowledge to think ahead.
What it learns
A world model is trained from the agent’s interaction data: observations, actions, rewards, and episode endings. Given its current internal state and a proposed action, it predicts key consequences, such as:
- the next observation or a compact latent state representing it,
- the expected reward, and
- whether the episode is likely to end.
Many environments are too visually detailed to predict pixel by pixel. Systems such as Dreamer therefore compress camera images into a latent state, learn how that state evolves, and learn a policy inside this compact imagined world.
Planning through imagination
With a world model, an agent can test action sequences without physically performing every one. A robot can imagine moving left versus right before risking a fall; a game-playing agent can roll out several possible futures and choose the action with the best predicted return. This makes learning far more data-efficient when real trials are slow, expensive, or unsafe. In Dyna-style methods, imagined transitions supplement real experience. MuZero takes a focused approach: it learns only the predictions needed for planning—reward, value, and policy—not a fully human-interpretable simulation.
Why accuracy matters
The model is useful only where its predictions remain trustworthy. Small one-step errors can compound across a long imagined rollout, producing a plan that looks excellent in the model but fails in the real environment. An agent can also exploit a model’s blind spot, like finding a shortcut that its internal simulator wrongly predicts will earn reward. Practical systems limit rollout length, continually retrain on fresh real data, represent uncertainty, or verify promising plans through actual interaction. A world model therefore adds a powerful capability beyond reward-only trial and error: it lets the agent use experience not just to react, but to mentally rehearse consequences before acting.
A world model is a learned internal representation that predicts how an environment’s state, observations, and rewards change in response to actions. In reinforcement learning, it lets an agent simulate possible futures and evaluate actions before executing them. This enables planning and can improve sample efficiency by extracting more learning from limited real interaction.
Imagine learning to play a new video game while also building a rough mental map of it: “If I jump here, I’ll probably land there; if I touch that enemy, I lose health.” That mental map helps you think ahead before taking a risky move.
A world model is an AI’s version of that map. It is an internal picture of how its surroundings tend to behave: what may happen after different choices, and which situations are likely to lead to good or bad results. Instead of only learning from the move it makes next, the AI can use this picture to imagine possible futures and choose more wisely. This is especially useful when real-world trial and error is slow, expensive, or dangerous.