Episodic Task

An episodic task is an RL problem with a natural finish line. An agent starts an attempt, interacts for a limited sequence of steps, reaches a terminal outcome, and then begins a fresh attempt—like playing one game of chess, navigating one maze, or completing one robot pick-and-place job.

What defines an episode
Each episode has an initial state, a trajectory of actions, states, and rewards, and a terminal state. The agent’s goal is to maximize the total reward collected before termination, commonly written as the return: G = r₁ + γr₂ + γ²r₃ + .... At termination, there are no further rewards from that episode, so the value of the terminal state is conventionally zero. A Gymnasium environment such as CartPole-v1 is episodic: the episode ends when the pole falls, the cart leaves its allowed range, or a time limit is reached.

Why the ending matters
Termination tells an RL algorithm where future consequences stop. In temporal-difference learning, a DQN update uses the next-state value to estimate what comes next—but it must not bootstrap beyond a true terminal state:

For a nonterminal transition: target = reward + γ × estimated next-state value.
For a terminal transition: target = reward alone.

Getting this wrong lets imagined rewards leak past the end of the task, producing distorted value estimates and unstable learning. It is also important to distinguish termination from truncation: a genuine success, failure, or game-over state terminates the task; an externally imposed time limit merely stops data collection and can still require bootstrapping.

Practical consequences
Episodes make evaluation intuitive: report success rate, episode return, or steps to completion across many independent runs. They also let an agent learn delayed credit assignment—for example, a sparse reward of +1 only when a maze exit is reached. PPO and Stable Baselines3 commonly train on batches of such rollouts, resetting each environment after an episode ends. The reset is not a minor engineering detail: it defines the distribution of starting situations the policy learns to handle.

An episodic task is a reinforcement-learning problem divided into finite interactions called episodes, each beginning from an initial state and ending at a terminal state. The agent’s objective is to maximize expected cumulative reward within each episode, such as completing a game or reaching a goal. Episode boundaries define when returns, resets, and performance evaluation occur, simplifying credit assignment and learning targets.

Think of playing a board game: it has a clear beginning, you make choices along the way, and eventually the game ends. You might win, lose, or earn a score. Then you reset the board and play again.

An episodic task is that kind of learning situation for an AI. Each attempt is a separate, complete “episode” with a starting point and an ending point. For example, a robot might try to navigate a maze, or a program might play one round of a game. Once the attempt ends, it can learn from the outcome before starting fresh on the next one.