Reranking
A second, slower pass that reorders retrieved results by actually reading them — usually the cheapest large improvement available to a RAG system.
Reviewed July 11, 2026Stable
When not to use it
- When first-stage recall is the problem. If the right passage isn't in the candidate set, reranking cannot help. Measure recall@k first; if it's low, fix retrieval or chunking instead.
- When latency is genuinely tight. Sub-100ms budgets may not have room. Be honest about whether yours actually is.
- When you retrieve three passages and use three. There's nothing to rerank. Reranking needs a surplus to discard.
- When chunking is broken. Same reasoning as recall: a reranker cannot reorder its way to an answer that no chunk contains.
Reach for something else instead
- Hybrid retrieval — BM25 plus dense. Improves first-stage recall, which reranking cannot.
- Better chunking — often the actual problem, and free.
- LLM-as-reranker — hand the passages to a model and ask it to order them. Works; costs more.
- Late interaction (ColBERT-style) — one stage, between the two in cost and accuracy.
Read more on the blog
- Beyond vector search: how RAG actually works in 2026RAG stopped being "vector database plus a language model" a while ago. Here's how retrieval-augmented generation actually works now, chunking, embeddings, reranking, knowledge graphs, and agentic retrieval, and where each piece quietly breaks.
- How to reduce AI hallucinations: what actually worksYou cannot instruct a model into being truthful, because the process that invents a fact is the same one that recalls a real one. Every technique that measurably reduces hallucination works by adding something outside the model. Here is what works, in order of impact, and what only appears to.
- Your RAG system isn't hallucinating. It never found the answer.When a RAG system gives a bad answer, almost everyone blames the model. Usually the right passage was never retrieved, and that changes everything about how you fix it.
- Context engineering: the skill that replaced prompt engineeringPrompt engineering didn't die, it got absorbed. The bottleneck moved from how you phrase a request to what information surrounds it. Here's what context engineering actually is, why it took over in 2026, and the discipline underneath the buzzword.
Further reading
- Nogueira & Cho (2019), Passage Re-ranking with BERT — the paper that made cross-encoder reranking standard.
- Khattab & Zaharia (2020), ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT — the middle ground between bi- and cross-encoders.
- Robertson & Zaragoza (2009), The Probabilistic Relevance Framework: BM25 and Beyond — the first-stage retriever a reranker most often sits on top of.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reranking a candidate set that's too small. Retrieve 50, rerank to 5 — not retrieve 5, rerank to 5.
- Adding a reranker while first-stage recall is unmeasured, then not knowing whether it helped or why.
- Assuming it fixes hallucination. If retrieval never found the answer, better ordering of wrong passages produces a better-ordered wrong answer.
- Passing all 50 reranked passages to the model. Crowding is real; the point of reranking is to discard.
At a glance
FieldLanguage & LLMs
Stagesecond pass, after retrieval
Model typecross-encoder
Typical shaperetrieve 50 → rerank → keep 3–5
Latency cost~50–200ms
Ceilingfirst-stage recall@k
DifficultyIntermediate
Often compared with
Reranking vs. better embeddings — reranking improves the order of what you found; better embeddings improve what you find. If the answer isn't in the candidates, only the second one helps.
Where this sits
A destination. 28 concepts lead here, and nothing in the corpus depends on it.
12Levelsteps in
28Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstRetrieval-Augmented Generation
Computed from the prerequisite graph, not assigned. How this works