Embeddings
Turning words (or images, or anything) into lists of numbers, arranged so that similar meanings end up close together.
When not to use it
- Exact matching. If you need to find an invoice number, a SKU, or a legal citation, embeddings will helpfully return things that are similar — which is precisely wrong. Use exact or keyword search.
- Small collections. Under a few hundred documents, keyword search plus a decent ranking is faster to build, easier to debug, and often just as good.
- Anything where you must explain the match. "These vectors were close" is not an answer a compliance team will accept.
Reach for something else instead
- Keyword search (BM25) is still the strongest baseline for a huge share of real search problems, and it's transparent about why it matched.
- Hybrid search — keywords and embeddings together, results merged — beats either alone often enough that it's the sensible default for production retrieval.
- Fine-tuned classifiers are better than embedding similarity when your categories are fixed and you have labelled examples.
Sources & further reading
- Mikolov et al. (2013), Efficient Estimation of Word Representations in Vector Space — word2vec, and the origin of the king−man+woman analogy.
- Reimers & Gurevych (2019), Sentence-BERT — the shift from word vectors to sentence embeddings that made semantic search practical.
- Radford et al. (2021), Learning Transferable Visual Models From Natural Language Supervision — CLIP, the shared image-and-text embedding space.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Mixing embedding models. Vectors from two different models live in incompatible spaces; comparing them produces confident nonsense. Documents and queries must use the same model.
- Embedding documents whole. A 40-page PDF becomes one blurry average of everything it says. Chunk into passages that each hold one idea.
- Treating cosine similarity as truth. It measures "these look related in this model's geometry," which is not the same as relevance to your user's actual question.
At a glance
FieldDeep Learning
Core ideameaning as geometry
Compared bycosine similarity
Powerssemantic search, RAG, recommendations
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4