Centralized Training Decentralized Execution

Imagine a team of robots learning to carry a heavy object. During practice, a coach can watch every robot and explain how their movements affected one another. Once deployed, though, each robot must act from its own sensors and messages. Centralized Training Decentralized Execution (CTDE) builds that practical split directly into multi-agent reinforcement learning.

What is centralized, and what is decentralized?
During training, the learning algorithm can use information from the whole team: every agent’s observations, actions, rewards, and sometimes the full environment state. This richer view helps solve the credit-assignment problem: did the team succeed because agent A explored, agent B yielded, or both coordinated? During execution, each agent’s policy chooses an action using only information it will genuinely have at deployment—usually its local observation, internal memory, and permitted communication. No central controller or privileged state is required.

How CTDE is implemented
A common arrangement uses decentralized actors and a centralized critic:

Each actor learns a local policy, such as “given my camera view, which direction should I move?”
The critic evaluates actions using joint information, producing a more informative learning signal.
After training, the critic is discarded; only the local actors run.

In MADDPG, for example, each agent has a critic conditioned on all agents’ actions and observations. In cooperative settings, QMIX learns per-agent value functions while combining them into a team-level value during training.

Why the split matters
Each learner changes the environment seen by the others, making independent training unstable: a policy can appear worse simply because teammates changed. Centralized information makes this moving-target problem easier to learn from, while decentralized execution preserves scalability, privacy, and robustness when agents cannot share everything in real time. CTDE does not guarantee coordination: a reward that accidentally favors one robot blocking another can still produce a bad team strategy. Its key discipline is that training may use extra information, but the final policy must succeed without relying on information that disappears in deployment.

Centralized Training Decentralized Execution (CTDE) is a multi-agent reinforcement-learning framework in which policies are trained using centralized information, such as all agents’ observations and actions, but each agent acts at deployment using only its own locally available information. This enables more accurate learning despite other agents changing behavior, while preserving independent execution when communication or global state is unavailable.

Imagine a sports team reviewing game footage together after a match. Everyone can see the whole field, discuss what each teammate did, and learn better team tactics. But during the next match, each player must make quick decisions from their own position, without a coach whispering every move.

Centralized Training Decentralized Execution uses that same idea for teams of AI agents. While learning, the agents can use shared information to understand how their choices affect the group. When acting in the real world, each agent decides independently using only what it can observe. This helps a team cooperate even when constant communication is impossible or unreliable.