Temperature
A single setting that controls how random or predictable an AI's output is — low for focused, high for creative.
When not to use it
- As a quality dial. Lower temperature doesn't make a model more correct — it makes it more predictable. A confidently wrong answer at temperature 0 is still wrong, just reliably so.
- To fix hallucination. Determinism isn't accuracy; the model still generates from the same flawed distribution, just less adventurously.
- Without pinning it. An unset temperature means your outputs change between runs and you'll debug ghosts.
Reach for something else instead
- Top-p / nucleus sampling gives finer control over the tail than temperature alone, and the two interact.
- Structured output — schemas or constrained decoding — when what you actually wanted was reliable format, not low randomness.
- Better prompts or examples when what you actually wanted was better content.
Sources & further reading
- Holtzman et al. (2019), The Curious Case of Neural Text Degeneration — where nucleus (top-p) sampling comes from, and why pure likelihood produces bad text.
- Guo et al. (2017), On Calibration of Modern Neural Networks — model confidence is not probability.
- Fan, Lewis & Dauphin (2018), Hierarchical Neural Story Generation — top-k sampling, the other lever people reach for.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Setting temperature to 0 and expecting perfect reproducibility. Batching, hardware, and floating-point nondeterminism can still shift output.
- Cranking it up for "creativity" and getting incoherence. High temperature buys variety, not imagination.
- Adjusting temperature and top-p simultaneously, then not knowing which one changed anything.
At a glance
FieldLanguage & LLMs
Core idearandomness dial
Lowfocused/consistent
Highcreative/varied
Notedoesn't affect factual accuracy
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4