Notes
🔍

MAE (RecSys)

Mean absolute error answers a blunt question: on average, how far off is a predicted rating? Take the gap between each predicted and actual rating, drop the sign, average the lot. There is nothing more to it, which is most of its appeal.

How it behaves
MAE is reported in the units of the rating scale, so it reads directly: an MAE of 0.72 on the MovieLens 1–5 scale means the typical prediction is about three-quarters of a star out. Because the penalty grows linearly with the error, one catastrophic miss costs no more than the sum of several small ones — the opposite of RMSE, which squares errors and is therefore dominated by the tail of the error distribution. Two models can post the same MAE with very different worst cases, which is why the pair is reported together.

Where it fits, and where it misleads
MAE belongs to the rating-prediction framing and only there — it needs a held-out number to subtract from. In the implicit setting most systems now run on there are clicks and plays, no ratings, and nothing for MAE to measure. Even where ratings exist it is a weak proxy for what the user experiences:

  • Every prediction counts the same, so an error on an item the reader would never scroll to weighs as much as an error in the first slot.
  • It is blind to ordering, and the ordering is the product.

Why it survives anyway
The Netflix Prize made rating error famous, and MAE remains the honest way to report calibration — whether predicted scores mean what they claim on the scale they are stated in. Where a system shows a predicted rating to the user, or feeds one into a downstream rule, that calibration matters independently of the ranking, and Precision@k or NDCG will not reveal it.

MAE is the mean absolute difference between predicted and actual ratings, reported in the units of the rating scale. It measures rating-prediction accuracy only, and penalises errors linearly, so unlike RMSE it is not dominated by a few large misses. It says nothing about the order of a recommended list, which is why top-k and rank-aware metrics replaced it as the primary offline measure.

Imagine guessing how much each of your friends will like a film, from one star to five, and then checking the real answer afterwards. For every guess you note how far out you were — half a star here, a star and a half there — and then take the average. That average is MAE.

It is a plain, honest score of how close the guesses land. What it cannot tell you is whether the right films ended up at the top of the list, and that is what people actually notice. A system can guess everyone's star ratings quite well and still lead with something dull, so this number is now one check among several rather than the headline.