Retrieval for recommendation systems

Both are, structurally, “given a query representation, find the nearest items in a large corpus by vector similarity” — recommendation retrieval just replaces “a text question” with “a user” and “a document chunk” with “an item” (product, video, post), and reuses the exact same ANN-index machinery (HNSW, IVF, PQ) and the same distance-metric considerations from the earlier subtopics. The real divergence is the training signal and objective: RAG embeddings are trained so semantic textual similarity predicts relevance; recommendation embeddings are trained so user–item interaction (click, purchase, watch-through) predicts relevance, which means the notion of “similar” is behavioral, not semantic — two items can be textually unrelated but strongly co- recommended because the same users engage with both. Recommendation also typically layers on business objectives (diversity, freshness, fairness, revenue) that plain text retrieval rarely needs to weigh explicitly.

A two-tower model has one neural network (the “user tower”) that encodes user features (history, profile, context) into a vector, and a separate network (the “item tower”) that encodes item features into a vector in the same space, trained so that the dot product (or cosine) between a user’s vector and an item’s vector predicts engagement likelihood — architecturally identical in spirit to a bi-encoder for text retrieval, and for the same reason: item vectors can be pre-computed once for the entire catalog, indexed in an ANN structure, and a user’s vector computed at request time is then searched against that index — exactly the offline-index / online-query split that makes text bi- encoder retrieval scale. The alternative (scoring every user-item pair with a single combined model, the way a cross-encoder scores query-document pairs) is far more accurate per prediction but cannot scan a catalog of millions of items in real time, which is why two-tower retrieval plus a heavier re-ranker on the shortlist is the standard split, directly paralleling bi-encoder-retrieve-then- cross-encoder-rerank in text RAG.

MIPS finds the items whose vectors have the largest raw dot product with the query/user vector, without normalizing either side — unlike cosine, magnitude is allowed to matter. This is often deliberate in recommendation: an item’s embedding norm can be trained to reflect something like general popularity or engagement propensity, so a larger norm systematically biases an item toward being retrieved more often across many users, which can be a genuinely desired inductive bias (popular items surface more readily) rather than noise to normalize away, unlike typical text similarity where magnitude differences are usually an artifact you want to remove (see the distance-metrics subtopic’s cosine-vs-dot discussion). MIPS is also the native formulation many two-tower training objectives directly optimize for (maximize inner product of matched pairs), so using it at retrieval time keeps train/serve consistent — switching to cosine at serve time would retrieve a different ranking than the one the model was actually trained to produce.

• Candidate generation (retrieval) — cheaply narrow millions or billions of items down to a few hundred/thousand plausible candidates, optimized for recall and speed, tolerant of some imprecision since a downstream stage will refine it. This is the two-tower / ANN-index stage.
• Ranking — take the shortlist and score it with a much richer, more expensive model that can use full cross-features (user-item interaction features computed jointly, not just two independent embeddings), optimized for precision and the actual business objective (predicted engagement, revenue, multiple weighted objectives combined).

This is exactly the retrieve-then-rerank split from text RAG (bi-encoder/BM25 for recall, cross- encoder for precision) — the same underlying reason applies in both domains: the accurate scoring function is too expensive to run over the full corpus, so a cheap approximate stage narrows the field first. Recommendation systems often add a third re-ranking / business-logic stage after ranking (diversity injection, deduplication, freshness boosting, ads insertion) that has no direct text-RAG analogue but conceptually mirrors context-assembly’s dedup/ordering concerns in RAG.

A RAG system’s retrieval objective is close to purely “find the passages most relevant to answering this question” — near-duplicate relevant chunks are a minor inefficiency (wasted tokens), not a correctness problem, because the generator reads all of them and produces one answer. A recommendation system instead shows a ranked list the user scans, and pure relevance-maximization tends to produce a list dominated by near-duplicate variations of whatever single item/genre scored highest (the “filter bubble” / redundancy problem) — which measurably hurts user satisfaction and engagement over a session even though each individual item was “relevant.” This motivates explicit diversity mechanisms analogous to but more central than RAG’s optional MMR re-ranking: penalizing similarity to already-selected items in the list, deliberately injecting some exploration/novelty, and balancing multiple objectives (relevance, diversity, freshness, business constraints) rather than optimizing a single relevance score end to end.

Cold start — a new user with no interaction history, or a new item with no engagement data yet — means the two-tower model has no learned signal to place that user or item accurately in the embedding space, since the whole architecture is trained on historical interactions. It differs from a thin RAG corpus (where the fix is straightforwardly “add more documents”) because the missing signal here is behavioral data that does not exist yet by definition for a brand-new user/item — you cannot retroactively generate the interactions. Mitigations are correspondingly different from RAG’s: fall back to content-based features (item metadata, category, text description — often embedded with a text/vision model much like RAG’s own embeddings) until enough interaction data accumulates; use popularity or editorial fallback rankings for new users; and explicitly explore (show a broader, less personalized set) early to accelerate collecting the signal the personalized model needs — a fundamentally different problem shape than RAG’s “ingest more documents” answer, even though both are described as “cold start” informally.