Radial Basis Function (RL)

Imagine placing a set of soft “landmarks” across a continuous state space. A radial basis function (RBF) tells the agent how close its current state is to each landmark, letting it reuse what it learned nearby instead of treating every position as entirely new.

How the representation works

An RBF feature is usually a bell-shaped function centred at a chosen point c. For a state s, a common form is Gaussian RBF: its activation is high when s is near c, then falls smoothly as distance grows. A value function can combine many such features:

V(s) = Σ_i w_i φ_i(s)

Here, φ_i(s) is the activation of the i-th RBF and w_i is its learned weight. For action values, an agent can use separate weights or features for each action, producing Q(s, a). The feature centres determine where the representation has detail; their widths determine how broadly experience generalises.

Learning from reward

RBFs do not replace reinforcement-learning updates; they provide the inputs those updates adjust. In a TD method, receiving a reward and seeing the next state creates a prediction error. The algorithm changes the weights of RBFs active near the visited state. Learning that a robot should slow down near one location therefore also influences nearby locations, which is crucial when a continuous environment cannot be visited in exactly the same state twice.

Strengths and practical limits

RBFs are a transparent, useful choice for low-dimensional control tasks: position and velocity in Mountain Car, for example, can be covered with a grid of centres and trained using semi-gradient SARSA. They give smoother generalisation than a lookup table and are simpler to diagnose than a neural network. Their weakness is scale: covering a high-dimensional observation space needs an explosive number of centres. Poorly placed or overly wide RBFs blur genuinely different situations; narrow ones behave like a sparse lookup table and learn slowly. Unlike a deep network, fixed RBF features also cannot discover a better representation from raw images.

A radial basis function (RBF) is a feature whose value depends on a state’s distance from a chosen centre, typically decreasing smoothly as distance grows. In reinforcement learning, multiple RBFs represent value functions or policies over continuous state spaces, allowing estimates learned near one state to generalise to similar states. Their centres and widths control the locality and smoothness of this generalisation.

Imagine learning your way around a city by remembering familiar landmarks. When you are near a coffee shop, you may remember that this neighbourhood usually leads toward the station. Places farther away do not affect that memory much.

A radial basis function, or RBF, gives a learning system a similar kind of local “landmark.” It lets the system treat nearby situations as alike while keeping very different situations separate. For example, a robot that learns a good move with its arm at one angle can apply that lesson to similar angles, without assuming it works everywhere.

This matters when there are too many possible situations to learn each one separately.