Action

An action is the choice an agent makes at a particular moment: turn left, apply a force, select an item, or do nothing. It is the part of reinforcement learning where a model stops merely predicting and starts changing what happens next.

At time step t, the agent receives an observation or state s_t, chooses an action a_t, and the environment responds with a reward and a new state. A policy, written as π(a|s), is the rule that maps what the agent sees to actions. The policy can choose one action deterministically or assign probabilities across several choices, allowing purposeful exploration.

• Discrete actions: a small set of named choices, such as up/down/left/right in a Gymnasium gridworld or “fire” versus “do not fire” in Atari. DQN is designed around this kind of action space.
• Continuous actions: numeric controls such as steering angle, joint torque, or throttle. A MuJoCo robot might choose a vector of forces, one for each motor. PPO can learn policies for these spaces.
• Structured or constrained actions: choices with rules, such as selecting a valid move in a board game. Invalid actions must be masked or prevented; otherwise the agent wastes experience learning about impossible behavior.

An action is not judged only by its immediate reward. Pressing a button that gives no reward now could open a path to a large reward later, while a tempting shortcut could exploit a badly designed reward signal and score well without accomplishing the real task. The agent must balance exploration—trying actions whose consequences remain uncertain—with exploitation—repeating actions already known to pay off. Because each action changes the next state and therefore the data collected, poor action selection can trap learning in a narrow, misleading part of the environment.