Cumulative Reward

Think of cumulative reward as the score an agent earns across a whole stretch of interaction, not just for its latest move. A single action can look good immediately yet create trouble later, so reinforcement learning judges behaviour by the rewards that add up over time.

What is being added
In an episode with rewards r₁, r₂, …, r_T, the cumulative reward is their total:

R = r₁ + r₂ + ... + rT

For example, a robot receives +1 each time it moves closer to a goal, −10 if it crashes, and +100 on arrival. Its cumulative reward captures the complete trade-off: a route that reaches the goal quickly and safely should beat one that gathers a few early points before crashing. In a finite game or a Gymnasium episode, reporting “episode reward” usually means this quantity.

Its relation to return
Cumulative reward commonly means the plain, undiscounted total. RL algorithms also use discounted return, which weights later rewards less:

Gₜ = rₜ₊₁ + γrₜ₊₂ + γ²rₜ₊₃ + ...

Here γ is the discount factor. With γ below 1, immediate rewards count more and infinite interactions have a bounded objective. People use “return” for this discounted quantity, though usage varies; checking whether discounting is included prevents a frequent source of confusion.

Why it matters in practice
The agent’s policy is trained to maximize expected cumulative reward or expected return, not to greedily maximize the next reward. That distinction enables delayed-reward tasks such as navigation, games, and control, but it also exposes reward-design mistakes:

• An agent might discover a loophole that repeatedly earns small reward without completing the intended task.
• Sparse rewards make credit assignment hard: a success score at the end gives little direct guidance about earlier decisions.
• In DQN or PPO, a rising training cumulative reward is useful evidence of progress, but it can hide a brittle policy that fails when the environment changes slightly.

The number therefore acts as the long-horizon yardstick for what “good behaviour” means, while the reward function determines whether that yardstick measures what was truly intended.