Trajectory

A trajectory is the record of one run of an agent interacting with its environment: what it observed, which actions it chose, and which rewards followed. Think of it as the agent’s play-by-play log for an episode, from the first decision until the task ends.

What a trajectory contains
In a standard episodic setting, a trajectory is written as something like τ = (s₀, a₀, r₁, s₁, a₁, r₂, …). At each time step, the agent receives a state s, takes an action a, then gets a reward r and a new state. The final state may be terminal, such as a robot reaching its goal, a game ending, or a simulated vehicle crashing. In partially observable problems, the log may contain observations rather than the environment’s full underlying state.

Why the whole sequence matters
A single reward rarely explains whether one action was good. A trajectory preserves the delayed chain of consequences needed for credit assignment. RL algorithms use it to calculate return: the discounted sum of rewards earned after a decision. For example:

A maze agent receives +1 only on reaching the exit. Its full trajectory reveals which earlier turns led there.
A policy-gradient method such as PPO collects trajectories, then increases the probability of actions that appeared in high-return runs.
DQN breaks stored trajectories into individual transitions—(state, action, reward, next state)—and samples them from replay memory.

Data the agent creates itself
Unlike a fixed labelled dataset, trajectories depend on the current policy and the environment’s randomness. Change the policy, and the agent visits different states and gathers different evidence. This is why exploration matters: repeatedly following a known rewarding route can leave an untested shortcut undiscovered. It is also why evaluation uses fresh trajectories; a policy that scores well on the situations it encountered during training can fail when the dynamics shift slightly. A trajectory is therefore not just a log—it is the raw experience from which an RL agent learns what actions lead to worthwhile futures.

A trajectory is the ordered sequence of states, actions, and rewards generated as an agent interacts with an environment over time, such as (s₀, a₀, r₁, s₁, ...). It records an episode or segment of experience under a policy. Trajectories matter because reinforcement-learning objectives, return estimates, and policy updates are computed from the rewards and consequences observed along them.

Think of recording every move in a game from start to finish: what you saw, what you chose to do, what happened next, and whether it went well. That complete “story” is a trajectory.

For a learning system, a trajectory is one run through an experience. A robot might begin in a room, turn left, bump into a chair, change direction, and finally reach a charging station. All those moments together form its trajectory.

Trajectories matter because the system learns from whole chains of consequences. A choice that seems unimportant at first may lead, several steps later, to success or failure.