Retrieval Engineering II — Production RAG

Module 3 left you with something rare: a RAG system with a labeled retrieval test set and metrics wired in from the first commit. This module spends that investment. Every technique here — hybrid fusion, cross-encoder reranking, query rewriting, metadata scoping, incremental ingestion — is an intervention whose value you will measure against your baseline, not assert. That is the difference between this track and the tutorial ecosystem: production RAG is not a bag of tricks, it is a sequence of evidence-backed upgrades to a pipeline you can already score.

The 2026 production consensus is stable on where the leverage lives: hybrid search plus reranking are the two biggest retrieval-quality levers available without touching your data, and component-level failure diagnosis is what separates teams that fix their RAG systems from teams that re-tune prompts at random. Employers agree — "reranking" and enterprise retrieval patterns (knowledge bases over OpenSearch and pgvector with ACL scoping) now appear verbatim in postings for senior DS/ML roles.

What You'll Learn

• BM25 + vector fusion with RRFwhy dense retrieval alone fails on exact identifiers and rare terms, with a worked fusion calculation and runnable config
• The retrieve 20 → rerank → top 5 cross-encoder patternwhy bi-encoders can't do what cross-encoders do, and how to budget the latency
• Query rewriting, decomposition, and multi-turn condensationfixing retrieval before retrieval runs
• Metadata architecture and ACL-aware retrievalfilters as index-level security, not prompt-level hope
• Ingestion at scaleincremental upserts, CDC-driven sync, deletion propagation, and when a full blue-green reindex is mandatory
• A component-level failure taxonomyingestion-miss, retrieval-miss, grounding-miss, generation-miss, with a trace-driven triage procedure you'll rehearse in the failure-clinic lab

Why Dense Retrieval Isn't Enough

Embedding-based retrieval earns its place by matching meaning: "how do I cancel my subscription" finds a document titled "Ending your plan." But experienced practitioners hit its failure modes within the first week of real traffic, and they are systematic, not random:

• Exact identifiers. Query: "error E4022 on ingest job." Your embedding model has no privileged representation of E4022; it's a rare token mapped into a crowded region of embedding space. BM25 treats it as what it is — a rare, high-IDF term that should dominate the match.
• Domain jargon and internal names. Product codenames, table names, acronyms coined last quarter — all effectively out-of-vocabulary for a general-purpose embedding model, all trivially matched lexically.
• Short keyword queries. Users trained by search engines type "vacation policy india" — a query with almost no semantic structure for an embedding to exploit.

Lexical search has the inverse profile: brittle to paraphrase, blind to synonyms, unbeatable on exact terms. The production answer is not to choose — it is to run both and fuse.

Reciprocal Rank Fusion, worked

RRF is the default fusion method because it needs no score normalization — BM25 scores and cosine similarities live on incomparable scales, but ranks are always comparable. Each document's fused score is the sum, across retrievers, of 1 / (k + rank), with k = 60 the conventional damping constant:

(api-reference.md#c12, ranked well by both retrievers, actually tops the list at 0.0318 — the ordering insight of RRF: agreement between retrievers beats a solo #1 from either.) A document that only one retriever found still survives with a diminished score, which is exactly the behavior you want: hybrid retrieval is a recall-widening move, and the reranker downstream will handle precision.

In Qdrant, hybrid retrieval with RRF is a single query — dense and sparse (BM25-style) vectors stored side by side:

from qdrant_client import QdrantClient, models

results = client.query_points(
    collection_name="kb_chunks",
    prefetch=[
        models.Prefetch(query=models.SparseVector(**bm25_query), using="sparse", limit=50),
        models.Prefetch(query=dense_query_vector, using="dense", limit=50),
    ],
    query=models.FusionQuery(fusion=models.Fusion.RRF),
    limit=20,   # feed the reranker, not the LLM
)

OpenSearch offers the equivalent via a hybrid query clause with a normalization processor in the search pipeline. Either way, the config decision that matters is the per-retriever limit: fetch generously (50 per arm is a sane default) because fusion and reranking exist precisely to sift a wide candidate pool.

Cross-Encoder Reranking: Retrieve 20 → Rerank → Top 5

Your first-stage retriever — even hybrid — is a bi-encoder system: query and documents are embedded independently, and relevance is approximated by vector proximity computed with no interaction between the query's tokens and the document's tokens. That independence is what makes it fast enough to scan millions of chunks, and it is also why it saturates: "compare these two texts without reading them together" has a ceiling.

A cross-encoder removes the independence. It takes the pair — query and candidate concatenated — through a transformer with full cross-attention and outputs a relevance score. It reads them together. The cost is that it must run once per candidate pair, so it cannot search a corpus; it can only re-order a shortlist. Hence the canonical production pattern:

1. 1Retrieve a wide candidate set with hybrid search — top 20 (up to 100 for hard recall problems).
2. 2Rerank every candidate with a cross-encoder — score each (query, chunk) pair.
3. 3Keep the top 3–5 reranked chunks as the LLM context.

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("BAAI/bge-reranker-v2-m3")  # open weights, runs on CPU/free GPU
pairs = [(query, hit.payload["text"]) for hit in candidates]   # 20 pairs
scores = reranker.predict(pairs)
top5 = [c for c, _ in sorted(zip(candidates, scores), key=lambda x: -x[1])[:5]]

Three engineering notes that tutorials skip:

• The reranker changes what "k" means. Before reranking, precision pressure forced you to keep k small (every irrelevant chunk pollutes the context). After, first-stage k is a recall knob you can turn up almost freely — the reranker absorbs the precision cost. Re-tune your chunk budget after adding one; teams routinely find retrieve-50 → top-4 beats their old retrieve-8.
• Latency is per-pair and mostly parallel. Scoring 20 pairs with a bge-class reranker is tens-to-hundreds of milliseconds on modest hardware — usually noise against the LLM generation call. Measure it in your traces rather than assuming; if it bites, shrink the candidate set or batch the pairs.
• Evaluate the stages separately. Track recall@20 for the first stage and recall@5 after reranking on your labeled set. If first-stage recall@20 is 0.93 but post-rerank recall@5 is 0.71, your reranker is demoting gold chunks — a model-domain mismatch you'd never see in an end-to-end score.

Query Understanding: Fix Retrieval Before It Runs

Everything so far assumed the query is worth retrieving for. Production queries frequently aren't — and no amount of fusion or reranking rescues a retrieval executed against the wrong question. Three transforms cover most of the gap, all implemented as a cheap LLM call before retrieval:

• Rewritingstrip conversational packaging, expand known acronyms and aliases from a maintained glossary, and normalize to the corpus's vocabulary. "hey — the thing we discussed where TTV numbers looked off in the exec dash?" → "time-to-value metric discrepancy executive dashboard." The rewrite prompt should receive your domain glossary as context; this is the highest-ROI five lines of prompt in the pipeline.
• Decompositionmulti-part questions retrieve badly as a single vector: "compare our churn definition with the one finance uses, and which dashboards use each?" is three retrievals wearing one query's coat. Decompose into sub-queries, retrieve per sub-query, take the union into the reranker, and let the generator synthesize.
• Multi-turn condensationin conversation, the retrievable question is smeared across turns: "what about for enterprise accounts?" is meaningless standalone. Condense chat history plus the new turn into one self-contained query before retrieval — the single most common omission in conversational RAG, and the first thing to check when your assistant answers follow-ups worse than opening questions.

HyDE — generating a hypothetical answer document and embedding that for retrieval — belongs in your vocabulary as a fallback for short, underspecified queries where query-document vocabulary mismatch is extreme. Treat it as an experiment to run against your test set, not a default: it adds an LLM call of latency to every retrieval and helps unevenly across corpora.

Metadata Architecture and ACL-Aware Retrieval

Enterprise retrieval questions are rarely "find similar text." They are "find similar text from the current policy year, in documents this user is allowed to read, preferring the runbook over the Slack export." That's metadata, and it has to be designed at ingestion, not bolted on at query time.

A production chunk payload looks like this:

{
  "text": "...chunk content...",
  "doc_id": "runbook-ingest",
  "parent_span": [1240, 2180],
  "source_system": "confluence",
  "doc_type": "runbook",
  "updated_at": "2026-05-14",
  "version": "v3",
  "acl_groups": ["data-platform", "sre"],
  "language": "en"
}

Which enables the retrieval patterns that generic tutorials never reach:

• Scoped retrieval. Filter on doc_type or source_system when the query classifier detects intent ("how do I…" → runbooks and how-tos, not meeting notes). Recency filters (updated_at > …) beat hoping the embedding notices a document is stale.
• Parent-document retrieval. Embed small chunks (precise matching) but return the enclosing section or parent span (sufficient context for generation). Store the parent pointer in the payload; resolve it after reranking. This decouples the matching granularity from the reading granularity — one of the cheapest quality wins available.
• ACL-aware retrieval. Resolve the user's group memberships at query time and apply acl_groups as a pre-filter inside the vector search, so unauthorized chunks are never candidates. Filtering after retrieval — or worse, instructing the LLM "don't reveal documents the user can't access" — is a security incident with extra steps: the content has already entered the candidate set and, in the prompt-level version, the context window. Modern vector DBs execute filtered HNSW search efficiently; there is no performance excuse for post-filtering. And ACL sync is an ingestion obligation — when a document's permissions change at the source, the index payload must update on the same sync cycle as the content.

For genuinely multi-hop questions — "which dashboards break if we deprecate the orders_v2 table?" — flat chunk retrieval hits a structural wall: no single chunk contains the answer, and the reasoning path runs across entities. That is the GraphRAG motivation: extract entities and relations at ingestion into a property graph, retrieve by graph traversal plus vector search, and answer from the combined evidence. It is a significant ingestion investment; adopt it when your failure taxonomy (below) shows a persistent cluster of multi-hop failures that retrieval tuning didn't dent — not because it's interesting.

Ingestion at Scale: The Unglamorous Half

Every RAG demo ingests once and retrieves forever. Every production corpus is alive — documents are edited, superseded, deleted, re-permissioned — and an index that drifts from its sources produces a uniquely corrosive failure: confident answers, correctly grounded, in stale truth. Ingestion is a data engineering problem, and your pipeline instincts transfer directly.

• Incremental sync, not re-crawl. Full re-ingestion is O(corpus) in embedding cost and time; production sync is O(changes). Detect changes via source-system webhooks or CDC streams where available, content-hash comparison on a schedule where not. Key every chunk deterministically — hash(doc_id, version, chunk_index) — so a changed document upserts its new chunks and tombstones the old ones atomically.
• Deletion propagation is the failure mode that gets you paged. An edited document that half-updates is embarrassing; a deleted document whose chunks survive in the index is a compliance problem — the retracted policy, the offboarded customer's data, the document whose ACL was tightened. Deletions and permission changes must propagate with the same SLA as additions, and your sync job needs an explicit reconciliation pass (index doc list vs. source doc list) to catch missed delete events.
• Dedup and canonicalization. Enterprise corpora are full of near-duplicates — the same policy in Confluence, a PDF export, and a Slack paste. Near-dupes waste context budget and, worse, can crowd all top-k slots with copies of the same content, starving the context of the second relevant document. Deduplicate at ingestion (content hashing for exact, embedding-similarity clustering for near) and keep the canonical source.
• When incremental isn't allowed: blue-green reindex. Changing the embedding model, chunking strategy, or payload schema invalidates the whole index — vectors from different embedding models are not comparable, so a mixed index is silently broken. The procedure mirrors a blue-green deploy: build the new index alongside the old, run your retrieval test set against both, cut traffic over when the new index wins, keep the old one warm for rollback. Your M3 labeled set just became your reindex acceptance test — this is why it was built first.

Component-Level Failure Diagnosis

A RAG system that answers badly is not one problem — it is four possible problems wearing the same symptom, and the fix for each is useless against the others. The production skill, rehearsed in lab L7, is attributing each failure to a component from the trace before touching anything:

The triage procedure per failing trace is a fixed interrogation order — cheap checks first:

1. 1Does the answer exist in the corpus? Search the source documents (not the index) by hand. No → ingestion-miss (or a genuinely unanswerable question — tag it; those belong in your eval set too).
2. 2Is the gold chunk in the retrieved top-k? Look at the trace's retrieval span. No → retrieval-miss. Check the transformed query here: if the rewrite mangled the question, log it as a rewrite failure, a retrieval-miss subtype.
3. 3Did the answer use the gold chunk? It's in the context — does the answer reflect it? Contradicted or ignored → grounding-miss.
4. 4Everything present, still failing → generation-miss; the defect is in what you asked the generator to do.

Run this over a batch of failures and the counts become a Pareto chart that dictates your roadmap. A real-shaped clinic outcome from a 60-failure triage:

Note what the table does: it converts "the bot is unreliable" into two engineering tickets that address two-thirds of all failures. It also connects forward — the attribution logic runs on exactly the component metrics (context recall for retrieval, faithfulness for grounding) that Modules 5 and 6 will let you compute automatically over every trace instead of by hand.

Check yourself: A user's query "error E4022 on ingest job" returns a generic non-answer, even though a runbook with E4022 in a heading is in the corpus. Before you retrain, re-chunk, or re-prompt, what do you determine first — and what is the likely fix?

Exercises

1. 1Hybrid ablation on your corpus. Using your M3 pipeline and labeled retrieval set, produce the three-column table — BM25-only, dense-only, hybrid-RRF — for recall@10 and MRR. Write five sentences: which query types each arm wins, and what that says about your corpus's vocabulary profile.
2. 2Rerank and re-budget. Add an open cross-encoder (bge-reranker class) behind a widened first stage (k=50). Report first-stage recall@50, post-rerank recall@5, and end-to-end faithfulness against your pre-rerank baseline. Then answer: what first-stage k maximizes post-rerank recall@5 before latency or scores degrade?
3. 3Break your own sync. Simulate corpus drift: edit three source documents, delete one, and tighten the ACL on another — then run your ingestion sync and write assertions that verify all five changes propagated (new chunks present, tombstones applied, ACL payload updated). Document the one that failed silently, because one usually does.
4. 4Twenty-trace triage. Collect 20 failing traces from your pipeline (or the lab's provided broken app). Attribute each to ingestion / retrieval / grounding / generation using the four-step interrogation, produce the Pareto table, and propose the single fix with the largest expected recovery. State which metric will prove the fix worked.

Your playbooks & labs

The playbook library for this module packages the reusable assets: the RRF fusion config recipes for Qdrant and OpenSearch, the retrieve-20→rerank→top-5 implementation card with stage-wise evaluation queries, the query-rewrite prompt with glossary injection, the chunk-payload metadata schema with ACL pre-filter patterns, and the ingestion-sync checklist (upsert keys, deletion reconciliation, blue-green reindex procedure). Do this at work this week → pb-24 Hybrid Search, Reranking, and Query Understanding (the fix for the E4022 retrieval-miss), pb-25 Beyond Flat Vector Search: Metadata Architecture, Parent-Document Retrieval, and GraphRAG (ACL pre-filters and parent-document retrieval), and pb-26 Production RAG Health: Failure-Mode Triage and Ingestion at Scale (the component triage Priya demanded, plus living-corpus sync). Two graded labs anchor the module: L6 upgrades your Module 3 pipeline to hybrid + reranked retrieval and demands the before/after evidence table, and L7 is the failure clinic — 60 traced failures on a deliberately broken app, graded on attribution accuracy and metric recovery after your fixes. Do L6 first; the clinic assumes you know what a healthy retrieval stage looks like.

Further Reading

• •
  Qdrant — Hybrid Queries — official reference for multi-vector prefetch and RRF/score fusion used in this module's configs
• •
  OpenSearch — Hybrid search — hybrid query clauses and normalization pipelines on the other index family named in enterprise postings
• •
  Sentence Transformers — Retrieve & Re-Rank — official documentation of the bi-encoder/cross-encoder division of labor
• •
  LlamaIndex — Query Transform Cookbook — rewriting, decomposition, and HyDE implementations
• •
  Microsoft GraphRAG — official project docs for graph-based retrieval when multi-hop failures persist
• •
  Ragas documentation — the component metrics (context recall, faithfulness) that automate this module's manual triage in M5
• •
  Arize Phoenix documentation — the OSS tracing tool used to capture the retrieval spans this module's diagnosis reads