Small Language Model (SLM)
A language model small enough to run somewhere a big one can't — and the demonstration that most of the size was never doing the work.
Reviewed July 16, 2026Stable
When not to use it
- Open-ended reasoning across broad knowledge. This is precisely where the parameters were doing work, and small models fail here by confabulating.
- Tasks you can't define. SLMs win by being narrow; if you don't know the job, you're paying in accuracy for flexibility you'll need.
- Long-context work. Small models degrade over long inputs faster than their benchmark scores suggest.
Reach for something else instead
- A frontier model with caching may be cheaper than it looks once prompt caching is on, and it's less engineering.
- A fine-tuned SLM beats a prompted large model on narrow tasks — this is the comparison people skip.
- Routing — small model by default, large model on escalation — is what most mature systems converge on.
Sources & further reading
- Hoffmann et al. (2022), Training Compute-Optimal Large Language Models — Chinchilla; existing models were oversized and undertrained.
- Gunasekar et al. (2023), Textbooks Are All You Need — phi-1; data quality substituting for scale, and the contamination questions that came with it.
- Hinton, Vinyals & Dean (2015), Distilling the Knowledge in a Neural Network — the mechanism by which small models inherit large ones' behaviour.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Comparing an SLM to a frontier model on frontier tasks and concluding SLMs don't work. Compare them on your task, at your volume, with fine-tuning on, or the comparison means nothing.
- Taking phi-style results at face value. The training data was substantially generated by a much larger model, so contamination and distillation are hard to separate from the data-quality claim.
- Expecting recall instead of giving retrieval. A small model doesn't know less because it's badly made; it knows less because it's small — so hand it the facts.
At a glance
FieldLanguage & LLMs
Runs onlaptop, phone, single GPU
Wins oncost, latency, privacy, stability
Loses onbroad knowledge, hard reasoning
DifficultyBeginner