Tree-of-Thought Reasoning
Ask a model to reason step by step and it commits to one line of thinking: whatever it says first becomes the ground everything else is built on, mistake or not. Tree-of-thought reasoning breaks that commitment. Instead of one chain, the model generates several candidate next steps, judges how promising each looks, and expands only the branches worth continuing — searching a tree of partial solutions rather than walking a single path.
The loop it runs
- Generate — propose several distinct next thoughts from the current partial solution.
- Evaluate — score each candidate, usually by asking the model itself whether that state looks promising, solvable or doomed.
- Search — keep the best branches and expand them, breadth-first, depth-first or beam, backtracking out of dead ends.
What it buys, and what it costs
The technique earns its keep where an early wrong turn is fatal and progress can be judged before the end: puzzles, constrained writing, multi-step arithmetic, a tool plan whose first step decides whether steps two to five make any sense. Introduced in the 2023 Tree of Thoughts paper, it lifted accuracy sharply on tasks where plain chain-of-thought prompting largely failed. The cost is the obvious one: many model calls per answer instead of one, so latency and spend multiply, and the evaluator becomes the weak link if it cannot tell a promising branch from a plausible-sounding one. Agents therefore apply it selectively, on the few decisions where exploring alternatives is cheaper than acting on a bad plan.
Tree-of-thought reasoning is a prompting and search strategy in which a model generates several candidate next steps, scores how promising each partial solution looks, and expands only the better branches — backtracking out of dead ends instead of committing to a single chain. It raises accuracy on planning and puzzle-style tasks where an early wrong turn cannot be recovered, at the cost of many model calls per answer.
Planning a route through an unfamiliar city, you do not take the first turn and hope. You consider two or three options, picture where each one leads, and drop the ones that look wrong. Tree-of-thought reasoning gives AI the same habit.
Normally the AI thinks in a straight line: one step, then the next. If the first step was a mistake, everything after it is built on that mistake. This approach has it come up with several possible next steps, judge which look promising, and continue only with those — going back if a path dead-ends. It gets noticeably better answers on tricky problems. It also costs more, because the AI has to think many times over instead of once.