Policy

A policy is the part of a reinforcement-learning agent that decides what to do next. Picture a robot at a hallway junction: its policy is the rule that turns what it currently sees—its location, camera image, battery level, or other state information—into a choice such as “go left,” “go right,” or “stop.”

How a policy chooses actions

Formally, a policy maps a state (or observation) to an action. A deterministic policy, written as π(a | s) in its simplest conceptual form, always picks the same action in the same state. A stochastic policy instead gives probabilities to actions: at a junction, it might go left 80% of the time and right 20% of the time. That controlled randomness is useful because the agent must discover actions whose rewards are not yet known.

In a table-based problem, a policy can explicitly store an action or action probabilities for every state.
In a visual control task, a neural network can represent the policy, taking pixels as input and outputting steering, braking, or action probabilities.
In PPO, the policy network is directly adjusted to make reward-producing actions more likely while limiting overly abrupt changes.

Learning the behavior, not just predicting

Unlike supervised learning, there is no dataset containing the correct action for each situation. The policy creates the data it will learn from: it acts, receives rewards and later consequences, then changes itself to favor choices associated with higher return, meaning cumulative future reward. A game-playing agent can discover that an untested move is valuable only by trying it; repeating its current favorite move forever would prevent that discovery. This is the practical tension between exploration and exploitation.

Why policy quality matters

The policy is the agent’s actual behavior at deployment time. A value estimate can correctly rank options, but without a policy that uses that information, the agent does nothing useful. Poorly configured policies can also exploit accidental reward loopholes—for example, finding a simulator shortcut that earns points without completing the intended task—or perform well during training and fail when slightly different real-world dynamics appear. Reinforcement learning therefore evaluates success through the behavior a policy produces across full trajectories, not through prediction accuracy on a fixed labeled set.

A policy specifies an agent’s behavior: it maps each state or observation to an action, either deterministically or as a probability distribution over actions. In reinforcement learning, the policy determines the trajectories and rewards the agent experiences. Learning seeks a policy that maximizes expected cumulative reward, while effective exploration and stable improvement depend on how the policy selects actions.

Think of a policy as a player’s personal rulebook for a game: “When I see this situation, I’ll make that move.” A cyclist might use a simple policy such as, “If the road is slippery, slow down.”

For a learning system, a policy is its guide for choosing what to do next in each situation. It might tell a robot which way to move, or a game-playing program when to attack or wait. At first, its rulebook may be poor. By trying actions and seeing which ones lead to better rewards, it gradually develops a policy that makes better choices more often.