Episode

An episode is one complete attempt by an agent to interact with its environment: it starts from an initial situation, continues through actions and consequences, and ends when the task reaches a stopping point. A game of chess, one warehouse robot delivery run, or a single simulated lap around a track can each be treated as an episode.

At each time step, the agent observes a state or observation, selects an action, receives a reward, and reaches a new situation. The resulting sequence is called a trajectory:

• initial observation or state;
• action chosen by the agent;
• reward returned by the environment;
• next observation, repeated until the episode ends.

The end is signalled by a terminal state, such as a robot reaching its destination, a game ending, or a cart falling over in Gymnasium’s CartPole environment. Environments also impose truncation: a run can be stopped because it hit a time limit, even though the underlying task did not truly end. Keeping termination and truncation distinct matters when calculating learning targets.

For episodic tasks, the agent is commonly judged by its episode return: the sum of rewards collected from the beginning to the end, frequently with later rewards discounted. This is what lets reinforcement learning assign credit across time. An action that earns no immediate reward can still be valuable because it enables a high-reward outcome near the episode’s end.

Episodes also define practical training units. PPO commonly gathers batches of complete or fixed-length trajectory fragments before updating its policy; DQN stores individual transitions collected across many episodes in replay memory. A rising average episode return is useful evidence of progress, but it can conceal a reward-design bug: an agent might end episodes quickly because a shortcut accidentally pays more than completing the intended task.

Not every reinforcement-learning problem has a natural finish. Temperature control or server scheduling can run indefinitely; these are continuing tasks. Training still uses artificial rollout boundaries for computation, but those boundaries should not be mistaken for genuine task completion. In an episodic setting, the reset after an episode is central: it provides fresh starting conditions and prevents the agent from treating one finished attempt as though it were part of the next.