LambdaMART
Ranking metrics care about position, and position is discrete: move an item from fourth place to third and NDCG jumps a step, with nothing smooth in between to descend. LambdaMART sidesteps the problem rather than solving it — instead of differentiating the metric, it writes down the gradient it wants and boosts trees against it.
Where the lambdas come from
For each pair of items in a list with different relevance, the model computes a lambda: a force pushing the better item up and the worse one down, scaled by the absolute change in NDCG that swapping the two would cause. Summing those forces over every pair gives each item a gradient, and MART — multiple additive regression trees — fits a regression tree to those gradients, adds it to the ensemble, and repeats. The loss being minimised is never written down; only its gradient exists.
Why it is still the default re-ranker
- It consumes hand-built features — recency, price, past click-through rate, text match, the retrieval model's own score — which is exactly what a re-ranking stage has on hand.
- Trees absorb mixed scales, missing values and monotone relationships without normalisation, and train in minutes on millions of rows in LightGBM or XGBoost.
- The metric weighting concentrates learning on the top few slots, the only ones a user examines.
Where it sits
Second. A two-tower model or implicit ALS retrieves a few hundred candidates; LambdaMART orders them inside the latency budget. Its blind spot is the stage's own: training consumes logged impressions, so an item the previous ranker never showed generates no lambda at all, and the list narrows over time unless exploration is injected deliberately.
LambdaMART is a gradient-boosted tree ranker trained on pairwise preferences whose gradients — the lambdas — are weighted by the change in NDCG that swapping each pair would cause, which makes a position-sensitive, non-differentiable metric trainable. It remains the standard model for the re-ranking stage over hand-engineered features: strong on small tabular feature sets, fast to train, and tuned directly at the top of the list the user actually sees.
Imagine a shop assistant arranging a display. Ten items are worth showing, but only the first two get noticed, so moving the best one to the front matters far more than tidying the back of the shelf.
LambdaMART teaches software that lesson. It learns by comparing items two at a time — should this one come before that one — and pays most attention to the swaps that would change what you see first. Each round of training adds another small decision tree to a growing committee, and the committee's votes decide the final order. It is an old idea in this field and still one of the most used, because it works well without mountains of data.