Distributional Hypothesis
Firth's claim that you shall know a word by the company it keeps — the idea that meaning can be recovered from co-occurrence patterns, which is why a model trained only on text can represent meaning at all.
Reviewed July 17, 2026Stable
When not to use it
- As a complete theory of meaning. It is a working assumption that has proven productive, not a settled result.
- To explain failures on physical or spatial reasoning, where the relevant facts are rarely written down and distribution cannot capture them.
Reach for something else instead
- Symbolic semantics with hand-built lexical resources such as WordNet, which is precise, sparse and expensive to maintain.
- Grounded semantics, tying words to perception or action rather than to other words.
- Multimodal training, which supplements distribution with signal from other channels.
Sources & further reading
- Firth (1957), A synopsis of linguistic theory — the formulation the field quotes.
- Mikolov et al. (2013), Efficient Estimation of Word Representations in Vector Space — word2vec, and the hypothesis at scale.
- Levy & Goldberg (2014), Neural Word Embedding as Implicit Matrix Factorization — the link between neural embeddings and the count-based tradition.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming embedding proximity means semantic similarity. Antonyms are distributionally near-identical and embed close together.
- Treating one embedding model as neutral. It encodes the distribution of whatever corpus produced it.
- Reading the hypothesis as proven because embeddings work. That it is useful is established; that it is sufficient is not.
At a glance
FieldLanguage & LLMs
OriginFirth (1957)
Claimmeaning from co-occurrence
Underliesevery embedding
Known blind spotantonyms
DifficultyPractical
Where this sits
A destination. 7 concepts lead here, and nothing in the corpus depends on it.
5Levelsteps in
7Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstEmbeddings
Computed from the prerequisite graph, not assigned. How this works