Action Space
An agent cannot learn to act until it knows what “acting” means in its environment. The action space is the complete set of choices the agent is allowed to make at each decision point—like the legal moves in a game, or the permitted steering and throttle commands for a robot.
What the agent can control
In a Markov decision process, an action is usually written as a ∈ A, where A is the action space. After the agent selects an action, the environment produces a new state and a reward. The action space therefore defines the output of the policy: a policy maps the current state or observation to an action, or to probabilities over possible actions.
- Discrete action spaces contain separate choices: left/right in CartPole, or up/down/left/right in a grid world. DQN is designed for this setting because it can estimate a value for every available action.
- Continuous action spaces contain numeric controls within ranges: a steering angle from −1 to 1 and an acceleration from 0 to 1. PPO and similar policy-gradient methods can output distributions over these values.
- Structured spaces combine choices: a robot might control several joint torques at once, while a strategy game might require selecting a unit and then a destination.
Why its design matters
The action space is not a minor interface detail; it determines what behavior is even possible and which algorithms fit. A driving agent with only “turn left,” “turn right,” and “go straight” cannot make fine corrections. Giving it unrestricted continuous controls creates the opposite problem: it must search a vastly larger set of possibilities. Invalid or poorly scaled actions can also destabilize training—for example, a simulated robot receiving torque commands far beyond realistic limits can exploit simulator quirks rather than learn stable locomotion. In Gymnasium, environments expose this contract through action_space, allowing libraries such as Stable Baselines3 to choose and validate compatible policies.
Learning through permitted experiments
Reward tells the agent which outcomes were desirable, but the action space determines the experiments it can perform to discover those outcomes. Exploration means trying actions within this space; exploitation means repeatedly choosing those already estimated to pay off. A well-specified action space makes that trade-off learnable rather than arbitrary.
The action space is the set of all actions an agent is allowed to choose from at each decision point. It can be discrete, such as move left or right, or continuous, such as a steering angle or motor torque. It defines the agent’s control over the environment: policies select actions from this space, and its structure determines which learning and exploration methods are practical.
Think of teaching a dog a trick. At any moment, the dog can sit, lie down, bark, walk away, or try something else. That menu of possible choices is its action space.
For a learning AI, the action space is simply everything it is allowed to do in a situation. A game-playing program might choose “move left,” “move right,” or “jump.” A robot might move an arm, turn a wheel, or stop. The AI learns which choices tend to bring rewards, but it cannot choose actions outside its menu. So the action space defines the range of decisions the learner can explore and improve.