AlphaZero

AlphaZero is a striking example of an agent becoming highly skilled without being shown expert moves. It begins knowing only a game’s rules and learns by repeatedly playing against itself, turning wins and losses into better decisions.

How it learns and plans

AlphaZero combines a deep neural network with Monte Carlo Tree Search (MCTS). Given a board position, its network produces two outputs:

a policy: which moves look promising;
a value: an estimate of whether the current player will eventually win.

MCTS uses these predictions to focus its search on useful continuations rather than exploring every legal move equally. It simulates many possible future lines under the actual game rules, then chooses a move from the search results. In this sense, AlphaZero plans before acting rather than merely reacting from a position.

Self-play creates the training data

After a self-play game ends, every visited position is stored with the final outcome: win, loss, or draw. The network is trained both to predict that outcome and to imitate the improved move distribution produced by MCTS. This is important: the search supplies a stronger training target than the network’s first guess. Repeating the loop—network guides search, search generates games, games improve the network—produces steadily stronger play.

Why AlphaZero mattered

AlphaZero reached elite performance in chess, Go, and shogi without human game records, hand-designed positional features, or separate opening books. It showed that reward from game outcomes can be paired with planning to solve a difficult credit-assignment problem: a move’s value may not become clear until dozens of turns later.

Its limits

AlphaZero relies on an accurate, fast simulator of the environment: it can apply the known rules while searching. That is practical for board games but not for many real settings, where dynamics are unknown, expensive, or unsafe to test. Its large search and self-play budget are also computationally demanding; a policy that plays well after deep search can be much weaker when forced to act without it.

AlphaZero is a self-play reinforcement-learning system that learns a policy and value function directly from game outcomes, then uses Monte Carlo Tree Search guided by those learned predictions to select moves. Starting only from game rules, it achieved superhuman play in Go, chess, and shogi. It matters because it tightly couples planning with learning: search produces stronger training targets, and improved networks make future search more effective.

Imagine learning chess by playing thousands of games against yourself. After each game, you only learn whether you won or lost. Over time, you notice which kinds of moves and plans tend to lead to victory.

AlphaZero is an AI system that learned to play chess, shogi, and Go in this way. It was not taught human opening books or expert strategies. Instead, it played itself, learned from the results, and looked ahead at possible moves before choosing one. This mattered because it showed that a system could discover remarkably strong strategies from simple rules and feedback alone—sometimes finding moves that surprised even top human players.