Advanced RAG: retrieval, reranking and grounded answers
A relevant-looking passage is not necessarily enough to answer a question. Diagnose failures at ingestion, retrieval, ranking and generation before changing the model.
- •Separate corpus, retrieval, reranking and generation failures.
- •Calculate rank-fusion scores and evidence hit rate.
- •Compare long context and corrective retrieval under a fixed budget.
Sources & further reading
Lost in the Middle ↗Self-RAG ↗Corrective Retrieval Augmented Generation ↗In-Depth Architecture & Mechanics
Interview brief
Your support assistant answers product questions from manuals and release notes. Keyword lookups work, but questions about version-specific limits fail. Explain how you would investigate, then propose the smallest useful change.
Diagnose the pipeline
Trace the query, document versions, candidate passages, reranker scores, final context and cited claims. Label whether the corpus contains the answer. If it does, check whether a supporting passage reached the candidate set, survived reranking and appeared in the prompt. Finally inspect whether the answer actually follows the evidence. These stages need different fixes.
Retrieval is a budget allocation problem
Use lexical retrieval for exact identifiers and dense retrieval for paraphrases. Reciprocal rank fusion combines ranks without pretending unlike scores are calibrated. For an illustrative RRF constant of 60, a document at lexical rank 1 and dense rank 20 receives 1/61 + 1/80 = 0.02889. A document ranked 3 in both gets 2/63 = 0.03175. These are fusion scores, not relevance probabilities. Tune candidate count and fusion settings on held-out queries.
Measure before adding another retrieval loop
Suppose 200 of 250 answerable questions have supporting evidence in the top 20. Evidence hit rate@20 is 80%; this is not document-level recall unless every relevant document is labeled. If only 150 answers are correct, a stronger generator cannot repair the 50 questions missing evidence. Test with oracle context to separate retrieval and generation errors. Track citation support and completeness separately.
Long context and corrective retrieval
Packing more passages can add cost and distract from important evidence. Evaluate answer quality as you vary passage order, token budget and redundant content. Self-RAG and CRAG are research approaches with distinct learned components; a hand-written retry loop is not a reproduction. In an application, bound retries and permit a clarifying question or an honest abstention.
Follow-up exercise
Design a test set containing exact error codes, ambiguous versions, missing answers, tables and conflicting release notes. Compare lexical-only, dense-only, hybrid and hybrid-plus-reranker configurations at the same latency budget. Explain which errors remain and why.
Query → version and permission filters → lexical + dense candidates
→ rank fusion → reranker → evidence budget → grounded answer
↘ insufficient evidence → clarify / abstaindef rrf(ranks, constant=60):
return sum(1 / (constant + rank) for rank in ranks)
exact_match = rrf([1, 20])
balanced = rrf([3, 3])
print(f"Exact-match candidate: {exact_match:.5f}")
print(f"Balanced candidate: {balanced:.5f}")
assert balanced > exact_match
# RRF scores rank candidates; they are not probabilities.