Embodied Agent
Everything gets harder when the agent has a body. A software agent that calls the wrong tool can retry; a robot that places a mug at the wrong height drops it, and no retry puts the pieces back. An embodied agent is an agent whose action space is physical — motors, grippers, a camera feed — rather than function calls returning JSON.
What the body changes
The plan–act–observe loop still holds, but every stage is strained by physics:
- Observation is raw and noisy — pixels, depth, joint positions — not a clean tool result the model can read.
- Actions are continuous and irreversible, and the world keeps moving whether or not the agent is ready.
- Latency is a safety property: a planner that thinks for four seconds cannot be the thing closing a gripper.
- Grounding is required — “the red one on the left” has to resolve to coordinates in the actual scene.
How it is built, and why it matters
The working pattern is a hierarchy: a language model handles high-level task decomposition — “clear the table” becomes a sequence of picks and places — while trained low-level controllers execute each primitive at speed. Google’s RT-2 and the open OpenVLA models push this further, mapping vision and instruction directly to robot actions. Embodiment matters because it is the honest test of agentic planning: there is no undo, the environment is not a sandbox you reset, and the gap between a plan that sounds right and one that works is immediately visible.
An embodied agent is an agent that acts in the physical world through a body — sensors, motors, a gripper — rather than through software tool calls, so its observations are raw perception and its actions are continuous and irreversible. The standard design pairs a language model doing high-level task decomposition with fast trained controllers for each movement. Embodiment removes the retry: a dropped object cannot be undone, which makes planning quality and latency safety-critical.
Think of the difference between an assistant who can only send emails and one who can actually walk into your kitchen and load the dishwasher. An embodied agent is the second kind: software that has been given a body — wheels, arms, a camera — so the things it decides to do happen in the real world.
That changes how careful it has to be. A program that makes a mistake online can try again. A robot that knocks a glass off the counter cannot un-break it. So embodied agents pair the part that plans what to do next with fast, well-practised control of the body, and they have to cope with a world that keeps moving while they think.