Retrieval-Augmented Generation (RAG)
Letting an AI answer from a specific set of documents by looking them up as it responds — instead of relying only on what it memorized.
Reviewed July 10, 2026Stable
When not to use it
- When the knowledge is small and stable. If it fits comfortably in the prompt, put it in the prompt. RAG adds a retrieval system, a vector store, chunking decisions, and a whole new class of bugs.
- When you need the model to behave differently, not know more. RAG supplies facts; it doesn't change tone, format, or skill. That's a fine-tuning or prompting job.
- When retrieval quality will be poor. RAG on a messy, contradictory document pile produces confidently wrong answers with citations attached, which is worse than no answer at all.
Reach for something else instead
- A longer prompt. Context windows are large now. If your knowledge base is a handful of documents, skip the infrastructure.
- Fine-tuning when you need consistent style, format, or a narrow skill baked in — not fresh facts.
- Plain search with a human reading the results. Sometimes the honest answer is that people want the source document, not a paraphrase of it.
Read more on the blog
- Beyond vector search: how RAG actually works in 2026RAG stopped being "vector database plus a language model" a while ago. Here's how retrieval-augmented generation actually works now, chunking, embeddings, reranking, knowledge graphs, and agentic retrieval, and where each piece quietly breaks.
- Your RAG system isn't hallucinating. It never found the answer.When a RAG system gives a bad answer, almost everyone blames the model. Usually the right passage was never retrieved, and that changes everything about how you fix it.
- RAG vs fine-tuning: the decision, honestlyThe most common question in applied AI, answered without the hedging: when do you use retrieval, when do you fine-tune, when do you need both, and what almost every team gets wrong about the choice.
- AI in journalism: 45% of news answers had a flawTwenty-two broadcasters in eighteen countries evaluated 3,000 AI answers about the news. Forty-five per cent carried a significant issue, and the worst performer failed on 76%.
RAG: retrieve first, then let the model answer from what it found.
The question is used to retrieve relevant chunks from a document store; those chunks are placed in the model's context alongside the question, and the model answers from them rather than its frozen memory. This is how you get current, sourceable answers — and why retrieval quality, not the model, is usually what determines whether RAG works.
Further reading
- Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — the paper that named it. Worth knowing it describes a different system to today's: DPR and a BART generator fine-tuned jointly, not a frozen model with text in the prompt.
- Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts — evidence that stuffing the context window is not the same as the model using it.
- Karpukhin et al. (2020), Dense Passage Retrieval for Open-Domain Question Answering — the retrieval half, which is where most RAG systems actually fail.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming retrieval fixes hallucination. It reduces it. The model can still ignore, misread, or blend the retrieved text.
- Chunking badly and blaming the model. Most disappointing RAG systems are retrieval failures wearing a generation costume — the right passage was never fetched.
- Skipping evaluation of the retrieval step on its own. Measure whether the right chunk comes back before you judge the answer.
At a glance
FieldLanguage & LLMs
Core idearetrieve-then-read
Addscurrent & private knowledge, citations
Fails whenretrieval fetches the wrong passage
DifficultyIntermediate
Often compared with
RAG vs. Fine-tuning — adding knowledge at answer time vs. changing the model itself.
Where this sits
27 concepts come first. Understanding it opens up 4 more.
11Levelsteps in
27Needs firstconcepts
4Opens up1% of 310
2Areascrosses fields
Computed from the prerequisite graph, not assigned. How this works