Return

A single reward tells an agent whether the last moment felt good or bad. Return answers the larger question: how good was the rest of the journey from this moment onward? That distinction is what lets reinforcement learning handle choices whose real consequences appear much later.

What return measures

At time step t, the return, written G_t, combines the rewards received after that point. In the common discounted form:

G_t = R_(t+1) + γR_(t+2) + γ²R_(t+3) + ...

Here, R is a reward and γ, the discount factor, lies between 0 and 1. A reward one step away counts fully; a reward farther away is multiplied by progressively more powers of γ. For a finite episode, the sum ends when the episode ends. With γ = 1, return is simply the undiscounted total reward.

Why discount future rewards?

Discounting makes an unbounded future into a manageable quantity and expresses a preference for reliable, sooner outcomes. It also shapes what the agent learns:

A small γ makes an agent short-sighted: it prioritizes immediate reward.
A γ near 1 gives distant consequences substantial weight, but makes credit assignment and learning harder.
In a robot task with a costly fall at the end, return carries that penalty backward to earlier balance decisions, even when those decisions earned no immediate reward.

How algorithms use it

The objective is to maximize expected return, not to greedily maximize the next reward. A value function estimates expected return from a state; an action-value function estimates it after choosing an action. In Monte Carlo learning, an episode’s actual return can train these estimates directly. In DQN, the target uses a one-step reward plus a discounted estimate of the next state’s return. This is powerful, but errors in that bootstrapped estimate can propagate backward. Return is therefore the score that connects an agent’s local actions to the long-term behavior its reward design truly encourages.

Return is the total reward an agent receives from a time step onward, usually defined as a discounted sum of future rewards: \(G_t = \sum_{k=0}^{\infty}\gamma^k R_{t+k+1}\). The discount factor γ determines how strongly later rewards count. Return is the objective used to evaluate actions and policies, enabling agents to trade immediate reward against longer-term consequences.

Imagine judging a road trip not by whether the first few minutes were pleasant, but by how good the whole journey turns out to be: the sights, delays, fuel cost, and whether you reach the destination. In reinforcement learning, that big-picture score is called the return.

A return is the total reward an AI receives after making a choice, including rewards that arrive later. This matters because a move that seems bad right now can lead to something much better later—like taking a longer route to avoid traffic. The AI aims to choose actions that lead to the best long-term return, not merely the quickest immediate reward.