Distributional Hypothesis
Firth's claim that you shall know a word by the company it keeps — the idea that meaning can be recovered from co-occurrence patterns, which is why a model trained only on text can represent meaning at all.
Reviewed July 16, 2026Stable
When not to use it
- As a complete theory of meaning. It is a working assumption that has proven productive, not a settled result.
- To explain failures on physical or spatial reasoning, where the relevant facts are rarely written down and distribution cannot capture them.
Reach for something else instead
- Symbolic semantics with hand-built lexical resources such as WordNet, which is precise, sparse and expensive to maintain.
- Grounded semantics, tying words to perception or action rather than to other words.
- Multimodal training, which supplements distribution with signal from other channels.
Read more on the blog
- Why AI thinks hot and cold mean the same thingWord embeddings score "accept" and "reject" at 0.73 similarity. The reason is the idea embeddings are built on, and it explains why cosine similarity is a weaker signal than most systems treat it as.
- Does it understand? The argument, properly statedBoth sides of this debate are usually presented by their opponents. Here is the sceptical case at full strength, the case for at full strength, why the two keep missing each other, and what would actually settle it.
- What linguistics predicts about where AI failsLinguistic features stopped improving models around 2019, because models learn the structure themselves. The categories survived anyway, as the best available map of where these systems break.
- Why AI answers the question you asked, not the one you meantAsk a model whether it can do something and it may tell you rather than doing it. The gap between literal meaning and intended meaning has a name, a fifty-year literature, and a pattern in where models fail.
Further reading
- Firth (1957), A synopsis of linguistic theory — the formulation the field quotes.
- Mikolov et al. (2013), Efficient Estimation of Word Representations in Vector Space — word2vec, and the hypothesis at scale.
- Levy & Goldberg (2014), Neural Word Embedding as Implicit Matrix Factorization — the link between neural embeddings and the count-based tradition.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming embedding proximity means semantic similarity. Antonyms are distributionally near-identical and embed close together.
- Treating one embedding model as neutral. It encodes the distribution of whatever corpus produced it.
- Reading the hypothesis as proven because embeddings work. That it is useful is established; that it is sufficient is not.
At a glance
FieldLanguage & LLMs
OriginFirth (1957)
Claimmeaning from co-occurrence
Underliesevery embedding
Known blind spotantonyms
DifficultyPractical
Where this sits
7 concepts come first. Understanding it opens up 1 more.
5Levelsteps in
7Needs firstconcepts
1Opens up0% of 310
2Areascrosses fields
Learn these firstEmbeddings
Computed from the prerequisite graph, not assigned. How this works