Hallucination
When an AI produces something fluent and confident that is simply false — fluency is not the same as accuracy.
When not to use it
- As a catch-all for every model error. Calling a formatting failure or a retrieval miss a "hallucination" hides the real bug and stops you fixing it.
- As a reason to distrust the model on everything. Hallucination rates vary enormously by task. Summarising a document you supplied is not the same risk as recalling a citation from memory.
Reach for something else instead
- Grounding with retrieval so answers come from supplied text rather than memory.
- Constrained generation — pick from a list, fill a schema — removes the room to invent.
- Verification passes, where a second call checks the first against the source, catch more than prompt-based pleading.
Sources & further reading
- Ji et al. (2022), Survey of Hallucination in Natural Language Generation — the taxonomy worth having before you use the word.
- Maynez et al. (2020), On Faithfulness and Factuality in Abstractive Summarization — hallucination measured on a task where the source text was right there.
- Bender et al. (2021), On the Dangers of Stochastic Parrots — the argument that fluency without grounding is the design, not the bug.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Asking the model not to hallucinate. It cannot tell when it is; that's what makes it a hallucination.
- Trusting cited sources without checking. Fabricated citations often have real-looking authors, journals, and DOIs.
- Assuming RAG solved it. Retrieval reduces hallucination; models still blend, misread, and over-extend the retrieved text.
At a glance
FieldLanguage & LLMs
Core ideafluent but false output
Root causetrained for plausibility, not truth
Main defensegrounding + human verification
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4