Feature Pipeline (RecSys)
A ranking model is only as good as the numbers put in front of it, and in recommendation almost none of those numbers exist until something computes them. The feature pipeline is that something: the machinery that turns raw events into the exact feature vector a model sees, both during training and while serving a live request.
What it actually does
Clicks, plays, purchases and impressions arrive as an event stream. The pipeline aggregates them into what a ranker consumes — a user's last twenty interacted items, an item's click-through rate over seven days, how many times this pair has co-occurred — then encodes, hashes and normalises the result. Some features are precomputed in batch and written to a store keyed by user or item. Others must be computed at request time because they depend on the session in progress.
The failure it exists to prevent
Training and serving must produce the same value for the same feature. When they do not — a different window, different null handling, an aggregate that quietly included data from after the label — the result is training-serving skew: offline metrics improve and live performance does not follow. This is why feature stores such as Feast exist, and why mature systems define each feature once and let both paths read that definition.
Why it matters
The pipeline sets what the ranking stage can know and how fresh that knowledge is. A store lookup plus a few aggregations has to fit inside a latency budget of tens of milliseconds, which caps feature complexity far more tightly than model architecture does. Most measurable gains in a production recommender come from better and fresher features, not a bigger model.
A feature pipeline is the machinery that turns raw interaction events into the exact feature vector a recommender consumes, in training and in serving. It aggregates behaviour into values such as recent item history, seven-day click-through rate and co-occurrence counts, precomputing what it can and deriving session-dependent features per request. Using one definition for both paths is what prevents training-serving skew, where offline gains never appear in live traffic.
A kitchen cannot cook straight from the delivery crate. Someone has to wash, chop and measure before anything reaches the pan. A feature pipeline is that preparation work for a recommendation system.
Raw records of taps, plays and purchases are not useful as they stand. The pipeline turns them into tidy summaries — what you watched most recently, how popular an item has been this week — so the part that actually chooses can decide in a fraction of a second. The crucial rule is that the preparation must be done the same way every time. If the practice kitchen chops one way and the busy service kitchen another, the dish that tested beautifully will disappoint real diners.