Reward Function

A reward function is the environment’s way of telling an agent, “that outcome was better or worse.” It does not tell the agent which action was correct or provide a labelled answer; it supplies a number, and the agent must discover which patterns of behaviour lead to high numbers over time.

In a Markov decision process, the reward function describes the immediate payoff associated with an interaction. It is commonly written as R(s, a, s'): the reward received after taking action a in state s and arriving in state s'. In a stochastic environment, it can specify an expected reward rather than a fixed one. The agent’s real target is not merely the next reward, but the return: the discounted sum of future rewards. This is why a chess agent can sacrifice material now for a later win, or a robot can take a longer route that avoids a costly collision.

The reward function is the practical definition of success. Small omissions change what the agent learns to do. For example:

• A navigation agent rewarded only for speed may cut through unsafe areas.
• A game-playing agent rewarded for points can find a loophole that increases its score without completing the intended task: reward hacking.
• A robot given only a reward for finishing a task may receive useful feedback too rarely to learn; intermediate signals can make learning feasible.

These intermediate signals are called reward shaping. Useful shaping guides exploration without changing which final behaviour is genuinely best; careless shaping creates a policy that chases the helper reward instead of the real goal.

Algorithms such as DQN and PPO use reward observations to update value estimates or policies from the agent’s own experience. A delayed reward creates a credit-assignment problem: the learner must work out which earlier decisions caused it. A badly scaled, sparse, noisy, or exploitable reward can make training unstable, slow, or deceptively successful in simulation while producing useless real behaviour. The reward function is therefore not a minor implementation detail—it is the bridge between an intended task and the behaviour reinforcement learning actually produces.