Context Window Management

Every agent runs inside a hard limit: the model can only see so many tokens at once, and a loop that appends a thought, a tool call and a result on every pass walks towards that limit from the first step. Context window management is the bookkeeping that keeps a run inside it.

The levers
Four of them, usually combined:

• Measure — count tokens per turn and track how close the transcript is to the ceiling, including the room the response itself needs.
• Trim — cap individual tool results before they land; one untruncated log or whole-file read can swallow a quarter of the window.
• Evict — drop or summarise the oldest turns, the sliding-window approach, and fall back on compaction when a summary has to stand in for the entire early history.
• Offload — keep durable material outside the window, in files or a vector store, and retrieve the specific piece a step needs.

Position matters as much as size
A window that fits is not the same as a window that works. Material in the middle of a long context is attended to least reliably, so the goal, the live constraints and the latest observation are deliberately kept at the edges rather than left wherever they happened to arrive.

What failure looks like
Unmanaged, a run dies with a hard error at the limit, mid-task, having paid for everything it did first. Managed carelessly it survives but degrades — re-reading a file it already read, repeating a step whose record was evicted, dropping a requirement nobody retired on purpose. Runtimes such as LangGraph expose these controls per step, which turns the limit into a design decision instead of a surprise.