AlphaGo

AlphaGo became famous for defeating elite human Go players, but its deeper achievement was showing how learning and planning can reinforce each other. Go offers an enormous number of possible moves, so a system cannot simply examine every continuation until the game ends.

How it chose moves
AlphaGo combined neural networks with Monte Carlo Tree Search (MCTS). A policy network estimated which moves were promising, concentrating search on a small fraction of legal moves. A value network estimated the chance that a board position would eventually win. MCTS then repeatedly explored candidate move sequences, using those estimates to decide where more computation was worthwhile. The result was not just “pick the network’s highest-scoring move”: AlphaGo looked ahead, compared possible futures, and selected a move supported by search.

How it learned from reward
Its training had two stages:

First, it learned from expert human games, giving its policy a strong starting point.
Then it improved through self-play reinforcement learning: two versions of the system played Go against each other, and the final win or loss supplied the reward.

That reward is extremely sparse—nothing says whether a move on turn 37 was brilliant or harmful until the game ends. Self-play creates a stream of increasingly challenging opponents, while MCTS helps generate stronger moves than the policy network could produce alone. Those search-improved decisions become new training targets for the networks.

Why it matters
AlphaGo illustrates a central RL idea: a learned evaluator can guide planning, and planning can produce better data for learning. It used the known rules of Go as its environment model, rather than learning those rules from scratch. Without search, the policy can miss a tactical consequence far ahead; without reliable value estimates, search wastes effort or commits to misleading branches. AlphaGo also made clear that high performance comes from the whole interaction loop—self-play, reward, value learning, and planning—not from a neural network in isolation. Its successor AlphaZero removed the initial human-game training and learned Go entirely through self-play.

AlphaGo is DeepMind’s Go-playing system that combines deep neural networks with Monte Carlo Tree Search. A policy network guides which moves to examine, while a value network estimates positions’ chances of winning; self-play and game outcomes train both. AlphaGo demonstrated that search guided by learned models can achieve superhuman sequential decision-making from reward signals, culminating in its 2016 victory over Lee Sedol.

Imagine learning chess by playing thousands of games, noticing which moves lead to wins, and gradually getting better. AlphaGo was an AI system that learned to play Go, an ancient board game famous for having an enormous number of possible moves.

It did not just copy a list of expert instructions. It learned from examples of strong players, played huge numbers of games against itself, and used the results—winning or losing—as feedback. Before making a move, it could also look ahead at likely future positions, much as a skilled player thinks through several possibilities. AlphaGo mattered because it showed that AI could master a task long seen as too intuitive and complex for computers.