Foundation Model
A large model trained broadly once and adapted to many tasks — a term coined to name a shift in how AI gets built, and contested from the day it was proposed.
When not to use it
- As a synonym for "LLM". Foundation models include image, audio, and multimodal systems; the term names a role in a stack, not an architecture.
- As a synonym for "frontier model". One is about how it's built and used, the other about capability and regulatory attention.
- For narrow, well-specified problems with plentiful labels. A small supervised model is cheaper, faster, more predictable, and easier to defend.
Reach for something else instead
- Task-specific supervised models still win on narrow, stable, high-volume problems, and are far easier to reason about.
- Small language models give you most of the adaptation story without the dependency, when the task is narrow enough.
- Classical methods — a regex, a lookup, a gradient-boosted tree — remain the correct answer more often than the discourse suggests.
Sources & further reading
- Bommasani et al. (2021), On the Opportunities and Risks of Foundation Models — the paper that coined the term and made the homogenisation argument.
- Brown et al. (2020), Language Models are Few-Shot Learners — the result that made one-model-many-tasks credible.
- Radford et al. (2021), Learning Transferable Visual Models From Natural Language Supervision — the same shift outside text.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating the foundation as a stable dependency. It's a vendor's product; it changes, and everything you built inherits the change whether you tested for it or not.
- Assuming breadth means competence. Broad pretraining buys transfer, not accuracy on your specific problem, and the gap only shows up in your own evals.
- Missing the homogenisation risk the coining paper actually led with. When everyone builds on the same three foundations, everyone shares their blind spots — and "many vendors" is not diversity if they all wrap the same weights.