Reduce LR on Plateau

Training can reach a point where the model is still improving, but its current step size is too large to make careful progress. Reduce LR on Plateau is a learning-rate schedule that watches a chosen metric and lowers the learning rate only when that metric has stopped improving for a while.

How the schedule reacts
After each epoch, the scheduler receives a value such as validation loss. It keeps track of the best value seen so far. If the metric fails to improve by a meaningful amount for a configured number of epochs—called patience—it multiplies the learning rate by a factor, such as 0.1. A learning rate of 0.001 would then become 0.0001. Common controls include:

mode: whether lower is better (min, for loss) or higher is better (max, for accuracy).
threshold: the minimum change counted as a real improvement.
cooldown: epochs to wait after a reduction before judging another plateau.
min_lr: a floor that prevents updates from becoming uselessly tiny.

Why smaller steps help
Early in training, large updates move quickly through broad regions of the loss landscape. Near a good solution, those same updates can bounce around a narrow valley instead of settling into it. Reducing the rate makes the optimiser—such as Adam or SGD—take finer steps. In PyTorch, torch.optim.lr_scheduler.ReduceLROnPlateau is called with the monitored metric after validation, unlike schedules stepped purely by epoch count.

Reading a real training run
If validation loss falls rapidly, then stays around 0.42 for five epochs while training loss still declines, a reduction can unlock further validation improvement. Monitoring validation rather than training loss matters: training loss can keep falling while generalisation worsens. This scheduler is not early stopping; it gives optimisation another chance at a gentler scale. Set patience too low and noisy metrics trigger needless reductions; set it too high and the run wastes epochs at an ineffective rate. A factor that is too aggressive can also freeze learning before the model has found a useful region.

Reduce LR on Plateau is a learning-rate schedule that monitors a validation metric and reduces the optimizer’s learning rate when improvement stops for a specified number of evaluations. The reduction, typically by a fixed factor, lets training take smaller parameter updates once progress has stalled. It matters because a learning rate that remains too high can prevent convergence near a good solution; plateau-triggered decay supports finer late-stage optimization.

Imagine trying to park a car in a tight space. At first, you make big steering moves to get close. Once you are nearly in place, big moves become risky, so you switch to tiny adjustments.

Reduce LR on Plateau gives an AI learning system the same instinct. The learning rate is how big a change the system makes after each lesson. When its results stop improving for a while—a plateau—the schedule lowers that rate. This lets the system stop making overly bold changes and carefully refine what it already knows, helping it improve when progress has stalled.