Hybrid Search, Reranking, and Query Understanding

Tier: Core · Workflow area: RAG & Retrieval Engineering

When to use

• •
  pb-21 diagnostics show retrieval misses on exact terms — IDs, error codes, product names, legal citations — that dense vectors blur (the classic BM25 win).
• •
  Recall@20 is good but recall@5 is weak: the right chunk is retrieved, just ranked too low → reranking.
• •
  Multi-turn chat where follow-ups ("what about for enterprise?") retrieve garbage → query condensation; compound analytical questions → decomposition.

Prerequisites

• •
  pb-21 harness + scorecard; pb-22/23 winners frozen.
• •
  A store with native hybrid support (Qdrant, OpenSearch, Weaviate) or rank_bm25 alongside your dense index.
• •
  A cross-encoder reranker: BAAI/bge-reranker-v2-m3 (open, runs on CPU/free GPU) as default.

The workflow

1. 1.

   Add the sparse leg and fuse with RRF. Run BM25 and dense retrieval in parallel, k=50 each, and fuse with Reciprocal Rank Fusion — rank-based, so no score-scale tuning:

   Copy

   def rrf(rankings: list[list[str]], k: int = 60) -> list[str]:
       scores = defaultdict(float)
       for ranking in rankings:
           for rank, doc_id in enumerate(ranking):
               scores[doc_id] += 1.0 / (k + rank + 1)
       return [d for d, _ in sorted(scores.items(), key=lambda x: -x[1])]
   

   Measure hybrid vs dense-only on the pb-21 set before adding the reranker — you want each lever's contribution attributable. Expect the biggest wins on keyword-ish queries; slice accordingly.

2. 2.

   Rerank the fused candidates. The retrieve-wide-then-rerank shape: fetch ~50–100 candidates cheaply, cross-encode the top set, keep 3–5 for generation:

   Copy

   from sentence_transformers import CrossEncoder
   reranker = CrossEncoder("BAAI/bge-reranker-v2-m3", max_length=512)
   pairs = [(query, chunk_text[c]) for c in fused[:50]]
   ranked = [c for _, c in sorted(zip(reranker.predict(pairs), fused[:50]), reverse=True)]
   context = ranked[:4]
   

   Cross-encoders read query and chunk together — they fix precisely the "right chunk at rank 12" failure. Measure recall@k of the final context set and nDCG@5; also record added latency (batch the pairs; a T4 reranks 50 pairs in tens of ms, CPU in the low hundreds).

3. 3.

   Condense multi-turn queries. Before retrieval, rewrite the conversational turn into a standalone query with a cheap-tier structured call:

   Copy

   class CondensedQuery(BaseModel):
       standalone_query: str
       needs_retrieval: bool     # small talk / meta-questions skip retrieval entirely
   

   Evaluate on a multi-turn slice of your eval set (build 10–15 conversation snippets if you have none — this failure mode is invisible to single-turn evals).

4. 4.

   Decompose compound questions. "Compare X's and Y's refund policies and their 2025 changes" → sub-queries, retrieved independently, contexts merged (dedup by chunk ID) with per-sub-query attribution. Gate decomposition behind a classifier field (is_compound: bool) — decomposing simple questions adds latency and noise.

5. 5.

   HyDE — test, don't assume. Generate a hypothetical answer, embed it, retrieve by similarity to the hypothetical. It shines when queries and documents live in different registers (layman questions ↔ technical docs); it hurts when the model hallucinates domain specifics. It's one flag in your harness — let the eval decide per corpus.

6. 6.

   Ablate and ship the stack. Final table: baseline → +hybrid → +rerank → +query-understanding, each row with recall@5, nDCG@5, RAGAS answer metrics, p95 latency, cost/query. Ship the config where marginal quality justifies marginal latency; it is common and legitimate to ship hybrid+rerank always, and query transforms only on the chat surface.

Tool variants

• Qdrant: native hybrid (sparse+dense named vectors, server-side fusion); OpenSearch: BM25 + k-NN with RRF processors — the common enterprise landing zone; pgvector: pair with tsvector/BM25 extension and fuse in app code.
• Rerankers: bge-reranker-v2-m3 (open, multilingual) default; hosted Cohere Rerank / Voyage rerank when you can't run inference; LLM-as-reranker (cheap tier, listwise) as an option to benchmark against — usually costlier at equal quality.
• Query transforms: LangChain MultiQueryRetriever / LlamaIndex QueryTransform classes package these; the structured-output versions above keep you framework-independent.
• Frameworks: all of the above compose in either LangChain or LlamaIndex; keep the ablation harness outside the framework so components stay swappable.

Verification

• •
  Each lever measured in isolation (ablation rows), on the full set and on its target slice (keyword queries for hybrid, rank-12 cases for rerank, multi-turn for condensation).
• •
  Final stack beats pb-21 scorecard on recall@5 and nDCG@5 outside the bootstrap CI, with p95 latency within the agreed budget.
• •
  needs_retrieval=false and is_compound gates fire correctly on ≥90% of a 20-case spot check.
• •
  No regression on the hard-negative (unanswerable) questions — more retrieval firepower must not create more confident hallucination.

Pitfalls

• Fusing scores instead of ranks. BM25 and cosine scores live on different scales; weighted score-sums need per-corpus tuning and break silently. RRF first; tune weighted fusion only if RRF demonstrably underperforms.
• Reranking a tiny candidate pool. Cross-encoding the dense top-5 just reorders the same misses; the pattern only pays when the first stage casts a wide net.
• Truncation inside the reranker. 512-token cross-encoders silently truncate long chunks — align max_length with your pb-22 chunk size or rerank on chunk head + heading.
• Query rewriting that drifts. Condensation models sometimes "improve" the question into a different question; log original + rewritten pairs and review a sample weekly.
• Stacking all levers untested. Each adds latency and a failure mode; the ablation table is what stops the stack from becoming un-debuggable.

Restricted-environment variant

• •
  Hybrid + rerank is fully local (BM25, dense, bge-reranker all self-hosted) — zero client text leaves the environment; this is the default stack for India services engagements.
• •
  Query transforms need a generator: use the local 7–8B model for condensation/decomposition; quality is adequate for rewriting even where it isn't for final answers.
• •
  Free-compute path: the entire ablation on a 10k-chunk corpus runs on Kaggle T4 within free weekly hours; reranker on CPU if GPU is contended.

Success criteria

• •
  An ablation table attributing quality and latency to each lever, committed beside the pb-21 scorecard.
• •
  Hybrid + rerank in production shape: wide retrieval, batched cross-encoding, 3–5-chunk contexts, with per-stage latency logged.
• •
  Multi-turn retrieval demonstrably fixed on the conversational slice, with rewrite logging in place.
• •
  A one-line answer, backed by your own numbers, to "should we use HyDE/decomposition here?"