Attack Detection (RecSys)

A recommender believes whatever its logs tell it, and nothing in the training step asks where those logs came from. A few hundred accounts that rate a target item alongside well-known ones are enough to push that item into many neighbourhoods. Attack detection is the job of noticing those accounts before the model learns from them.

What gives an attack away
Fake profiles have to be effective, and being effective makes them unusual. Detection leans on that tension:

Profile-level signals — ratings that sit far from each item's mean more consistently than a real user's do, a suspiciously high overlap with popular items, an extreme score on exactly one item.
Group-level signals — accounts whose profiles are near-identical to each other, created in bursts, sharing one target item. A single fake profile is cheap to hide; a coordinated set is not.
Item-level monitoring — a jump in an item's rating volume or mean that no campaign or release explains.

How it is actually run
Detection sits upstream of training. Classifiers trained on synthetically generated attacks score each profile, and unsupervised methods — clustering, or looking at the principal components a tight group of profiles collapses onto — find coordinated sets without needing labels for the specific campaign. Flagged profiles are removed or down-weighted in the interaction matrix rather than banned outright, since false positives are real users.

Why it matters
Item-item neighbourhood methods are the most exposed, because a handful of profiles can rewrite one item's similarity list and that list is then served to everyone. Detection is what keeps robustness from depending on the goodwill of whoever can create accounts. It also gets harder as feedback becomes implicit: a fake click has no rating pattern to look odd, so the signal moves to timing, device and session structure.

Attack detection identifies fake accounts injected into a recommender to promote or demote an item, so their interactions can be removed or down-weighted before training. It works from profile-level anomalies, such as unusual deviation from item means, and from group-level ones, such as near-identical profiles created together around a shared target. Without it, a few hundred coordinated profiles can rewrite an item's similarity neighbourhood and that corrupted list is then served to every user.

Think of a restaurant that ranks its dishes by what regulars order. Now imagine the owner's cousin brings in fifty friends who all order one dish, every day, for a week. The ranking is no longer about taste — it has been gamed.

Attack detection is the staff noticing. Software that recommends things learns from the accounts it sees, so fake accounts set up to push one product can poison its suggestions. Detection looks for the tell-tale signs: accounts behaving almost identically to each other, created at the same time, all strangely enthusiastic about the same thing. Those accounts get discounted, so the recommendations go back to reflecting what real people actually chose.