Dynamic Tool Loading
Every tool an agent can call has to be described in its prompt, and those descriptions are not free — a hundred tools with schemas can eat tens of thousands of tokens before the agent has done anything. Dynamic tool loading is the fix: keep the catalogue outside the prompt and put only the handful of tools relevant to the current step in front of the model.
How the handful gets chosen
The selection happens before the model call, by code the agent author controls:
- Retrieval over tool descriptions — embed the task or the latest user turn, and pull the nearest tools from the catalogue.
- Phase-scoped sets — a research step sees search and fetch; a write step sees the file and commit tools and nothing else.
- Hierarchical discovery — expose a small set of groups, and load a group’s real tools only once the agent picks it.
- Runtime registration — an MCP server connected mid-session contributes its tools immediately, without a restart.
Why it matters
Beyond the token saving, a smaller menu measurably improves tool choice: selection accuracy degrades as the list grows, and two similarly named tools in the same prompt are a reliable source of wrong calls. The failure mode of getting it wrong is quiet — the agent cannot see the tool it needed, so instead of reporting a missing capability it improvises with whatever was loaded, and you get a plausible answer produced the wrong way. Keeping a small always-on core alongside the retrieved set is the usual guard.
Dynamic tool loading is the practice of placing only the tools relevant to the current step in an agent’s prompt, selecting them at runtime from a larger catalogue by retrieval, task phase or hierarchical discovery instead of declaring everything up front. It keeps schema descriptions from consuming the context window and improves selection accuracy, which falls as the tool list grows. Load the wrong subset and the agent improvises rather than reporting the missing capability.
Think of a tradesman arriving at a job. He does not carry every tool he owns up the stairs — he looks at the task, picks what he needs for this part, and leaves the rest in the van.
Dynamic tool loading is software doing the same thing. A program working on its own might have hundreds of capabilities available, but showing it all of them at once is wasteful and confusing, so it is given just the few that suit the step it is on.
The result is cheaper and more reliable. Faced with five sensible options it usually picks well; faced with two hundred, it picks the wrong one and carries on as if it were right.