Retrieval, search & RAG
Lexical and semantic retrieval — BM25, embeddings, vector indexes, distance metrics, Elasticsearch, hybrid search, and the RAG pipeline around them.
Questions I use to pressure-test how retrieval actually works underneath a RAG system — what BM25 is computing, what a semantic vector is and which distances are meaningful on it, how ANN indexes trade accuracy for speed, and how lexical and dense retrieval get fused.
Overlaps deliberately with the LLMs topic’s Context subtopic (which covers what happens once retrieved text is in the prompt); this topic is the retrieval layer itself.
Subtopics
- RAG architecture and the retrieval pipelineSeniorWhat RAG is really doing, the full pipeline from query to answer, query transformation, re-ranking, and where RAG systems fail.9 concepts
- BM25 and lexical searchSeniorThe inverted index, TF-IDF, the BM25 scoring formula with term saturation and length normalization, analyzers, and where lexical still beats dense.7 concepts
- Embeddings and vector indexesSeniorWhat a semantic vector is, bi-encoders vs cross-encoders, pooling and normalization, and how flat / IVF / HNSW / PQ indexes trade recall for speed.10 concepts
- Distance and similarity metrics on vectorsSeniorCosine, dot product, Euclidean, Manhattan, and the exotic ones — the equations, how they relate on normalized vectors, and when each is the right choice.8 concepts
- Elasticsearch and hybrid searchSeniorLucene/Elasticsearch architecture, search engine vs vector DB (and which vector DB), and how lexical and dense results get fused (RRF, weighted score combination).8 concepts
- Late interaction and multi-modal retrievalSeniorColBERT's token-level scoring, why it sits between bi-encoders and cross-encoders, and how CLIP-style models make images and text searchable in one space.6 concepts
- Chunking and retrieval qualitySeniorChunk size and overlap trade-offs, semantic and structural chunking, small-to-big, metadata, and handling tables, code, and duplicates.8 concepts
- Knowledge graphs and GraphRAGSeniorEntity/relation extraction, graph traversal vs vector similarity, when structure beats embeddings, and the real costs of graph-based retrieval.6 concepts
- Retrieval evaluation metricsSeniorRecall@k, precision@k, hit@k, MRR, nDCG, and MAP — the formulas, a worked example, what each optimizes for, and the pitfalls when comparing them.7 concepts
- Evaluating retrieval and RAGSeniorWhy retrieval and generation are evaluated separately, building a golden set, generation metrics (faithfulness, groundedness), needle-in-a-haystack, and catching regressions in production.6 concepts
- Retrieval for recommendation systemsSeniorTwo-tower models, MIPS, and how candidate generation vs ranking splits the recommendation problem — where it converges with and diverges from RAG retrieval.6 concepts