What are embeddings? How AI turns meaning into numbers

When a search engine returns the right result even though your query and the document share no words, when a chatbot retrieves the relevant passage from a huge knowledge base, when a store recommends an item that simply fits your taste, the technology quietly doing the work is almost always the same: embeddings. They are one of the most important and least understood ideas in modern AI, the connective tissue behind semantic search, retrieval, recommendations, and much of how machines handle meaning. And they rest on a single idea that is worth understanding on its own, because once you grasp it, a whole layer of AI infrastructure suddenly makes sense: embeddings turn meaning into geometry.

This guide explains what an embedding is, the idea of meaning-as-geometry that makes it useful, how embeddings are created, how they evolved from crude word lists into the context-aware representations used today, how similarity between them is measured, why the same technique works across text, images, and audio, and what embeddings power in practice, along with their limits. By the end, the phrase "vector embedding" should stop being jargon and become a tool you can reason about: a way of placing meaning in a space so that a computer can navigate it.

What an embedding is

An embedding is a list of numbers, called a vector, that represents a piece of data, such as a word, a sentence, a document, or an image, in a form a computer can compare. Instead of leaving a word as text, which a computer can only match character by character, an embedding turns it into a point in a high-dimensional space, a coordinate made of hundreds or thousands of numbers. In fact, inside a large language model, the first thing that happens to your input is that each token is turned into an embedding, which is how the model represents text internally in the first place. On its own, one such vector means little. The power comes from how these points are arranged relative to each other: things with similar meaning are placed close together, and things with different meanings are placed far apart.

That arrangement is the entire point. An embedding is not just any numerical encoding of data; it is one where position encodes meaning. Two sentences that say the same thing in different words land near each other. A picture of a dog and the word "puppy" can sit close together. The distance between two embeddings is a measure of how related they are. This is what separates an embedding from an arbitrary list of numbers, and it is what makes the whole idea useful.

The core idea: meaning as geometry

Here is the insight to hold onto. Meaning is famously slippery and hard to pin down, and for decades getting computers to judge whether two pieces of text "mean the same thing" was extremely difficult, because they could only compare surface form, the actual characters and words. Embeddings solve this by converting meaning into location. Once every piece of data is a point in a space arranged so that closeness reflects similarity, the hard, fuzzy question of "do these two things mean something similar?" becomes the easy, precise question of "are these two points close together?" And measuring the distance between points is something a computer does effortlessly.

This is the trick, and its consequences are far-reaching: by turning meaning into geometry, embeddings convert an impossibly vague problem into simple arithmetic. Similarity becomes distance. Search becomes finding nearby points. Recommendation becomes finding points near the things you liked. Clustering becomes grouping points that huddle together. An enormous range of AI capabilities that seem to require understanding meaning are, underneath, just geometry in a space where meaning has been laid out as position. That is why embeddings show up everywhere: they are the bridge between the messy world of meaning and the precise world of numbers.

Why this beats keyword matching

To see why this was such a leap, compare it with the older way of searching, which matched keywords. Traditional search looked for documents containing the same words as your query. If you searched for "fruit," it would find documents with the word "fruit," but not necessarily a document about apples and oranges that never used the word itself. It matched surface form, the literal words, and was blind to meaning. Synonyms, paraphrases, and related concepts slipped through the cracks.

Embeddings changed this by matching meaning instead. Because the embedding for "fruit" sits close to the embeddings for "apples" and "oranges" in the space, a search based on embeddings will find that document even though it shares no words with the query. This is called semantic search, and it is a genuine change in capability: the query and the document can have zero words in common and still be recognised as related, because relatedness is judged by position in meaning-space rather than by overlapping text. Nearly every modern system that "just finds the right thing" is doing this underneath, and it is why embeddings became foundational infrastructure rather than a niche technique.

How embeddings are created

Embeddings are produced by a special kind of neural network called an embedding model, which takes a piece of data as input and outputs its vector. The model is not told by hand where to place each thing; it learns the arrangement from data, which is what makes the result meaningful rather than arbitrary. The typical way this learning works is through contrastive training: the model is shown many pairs of things, some similar in meaning and some not, and it is adjusted so that it pulls the vectors of similar pairs closer together and pushes the vectors of dissimilar pairs farther apart. Repeated across an enormous number of examples, this pressure gradually organises the whole space so that geometry lines up with meaning.

The result is that the model learns a set of directions, the axes of the space, that capture meaningful patterns of variation in the data, without anyone specifying what those axes should be. This is the same principle of learned representations that underlies deep learning generally, applied specifically to produce a space where distance is meaning. Different embedding models, trained on different data with different objectives, produce different spaces that emphasise different things, which is why the choice of embedding model matters for any application built on them.

From static words to context: how embeddings evolved

Embeddings have a history worth knowing, because it explains why modern ones are so much better. The early influential embedding methods, with names like word2vec and GloVe, produced a single fixed vector for each word. Every occurrence of a word got the same embedding regardless of how it was used. This was a breakthrough, but it had a clear flaw: many words mean different things in different contexts. The word "bank" refers to a riverbank or a financial institution depending on the sentence, yet a static embedding had to cram both meanings into one vector, blurring them together.

Modern embeddings, built on the transformer architecture, fixed this by becoming contextual. Instead of a fixed vector per word, a contextual embedding model looks at the whole surrounding text and produces a vector for a word or passage that depends on its context. Now "bank" in a sentence about rivers gets a different vector from "bank" in a sentence about money, because the model reads the context and places each occurrence where it belongs. This shift from static to contextual embeddings is a large part of why semantic search and retrieval work as well as they do today, and it is why the embeddings behind current systems capture nuance that the earlier generation could not.

The structure of the space: relationships as directions

One of the most striking discoveries about embedding spaces is that they encode not just similarity but relationships, as consistent directions in the space. The famous early illustration came from word embeddings: if you take the vector for "king," subtract the vector for "man," and add the vector for "woman," you land near the vector for "queen." The relationship "male to female" turned out to be a consistent direction you could add or subtract, and the same held for other relationships, like country to capital city. Analogies became vector arithmetic.

This revealed something deep about what embeddings capture. The space is not just a jumble where similar things happen to be near each other; it has learned structure, with meaningful relationships laid out as geometric patterns. While modern contextual embeddings are more complex than these clean early examples suggest, the underlying point holds: the space encodes the structure of meaning, which is why so much can be computed by moving through it.

Measuring similarity: cosine similarity

If similarity is closeness in the space, you need a way to measure closeness, and the standard tool is cosine similarity. Rather than measuring the straight-line distance between two vectors, cosine similarity measures the angle between them, which captures whether they point in the same direction regardless of their length. This matters because it makes the comparison robust to differences in magnitude, such as one text being longer than another. The result is a score that is easy to interpret: a cosine similarity of one means the two vectors point in exactly the same direction and are as semantically alike as possible, a value near zero means they are unrelated, and a negative value means they point in opposing directions. Other measures like dot product and straight-line distance are used in particular cases, but cosine similarity is the default for comparing text embeddings, and it is the small piece of arithmetic that turns two vectors into a single number saying how alike they are.

One space for everything: multimodal embeddings

A powerful extension is that embeddings do not have to be limited to text. The same approach works for images, audio, and other data, and, importantly, different kinds of data can be embedded into the same shared space. When a model is trained so that an image of a dog and the text "a photo of a dog" land near each other, you get the ability to search images with text, or text with images, because both live in one common meaning-space. This is the foundation of much multimodal AI, where a single space holds text, pictures, and more, and proximity across modalities means the same thing it means within one: relatedness of meaning. It is the same idea of meaning-as-geometry, extended so that the geometry spans more than one type of data.

What embeddings power

The applications of embeddings are, essentially, everything that involves finding or organising things by meaning. Semantic search, as described, retrieves documents by meaning rather than keywords. Retrieval-augmented generation, the technique that gives language models access to external knowledge, runs on embeddings: documents are split into chunks, each chunk is embedded and stored, and at query time the question is embedded and used to retrieve the most similar chunks to feed to the model. Recommendation systems embed items and users and suggest things whose vectors are near what you have liked. Clustering and classification group data by proximity in the space. Deduplication and anomaly detection find things that are suspiciously close or far. Storing and searching these vectors efficiently at scale is the job of a vector database, which is built to find the nearest vectors to a query among millions, using fast nearest-neighbour methods. Wherever an application needs to judge that two things mean something similar, embeddings are likely underneath.

The limits worth knowing

Embeddings are powerful but not magic, and using them well means knowing their limits. An embedding is only as good as the model that produced it, and different models, trained on different data, encode different notions of similarity, so one that works well for general web text may perform poorly on specialised domains like law or medicine. The meaning of "similar" is defined by the model's training objective, which may not match what you consider relevant for your task, so two things the model places close together are not guaranteed to be close in the sense you care about. Embeddings also inherit the biases present in their training data, encoding and potentially amplifying stereotypical associations, which matters when they drive consequential decisions. And at scale, the storage and computation of large numbers of high-dimensional vectors become a real cost, often the dominant one in a retrieval system. None of these undermine the technique, but they are the difference between using embeddings naively and using them well.

The short version

An embedding is a vector, a list of numbers, that represents a piece of data such as a word, sentence, or image, arranged so that things with similar meaning sit close together in a high-dimensional space and dissimilar things sit far apart. This turns the slippery notion of meaning into geometry: similarity becomes distance, which a computer can measure precisely, so search becomes finding nearby points and recommendation becomes finding points near what you liked. Embeddings are produced by neural networks trained to pull similar things together and push dissimilar things apart, and modern transformer-based embeddings are contextual, giving the same word different vectors depending on its context, which fixed the ambiguity of older static methods. Similarity is usually measured with cosine similarity, the angle between vectors. The same idea extends across text, images, and audio in a shared space, and it powers semantic search, retrieval-augmented generation, recommendations, and clustering.

The idea to hold onto is that embeddings work by turning meaning into position in a space, so that the hard, fuzzy problem of judging whether two things mean something similar collapses into the easy, precise problem of measuring how close two points are, which is why a single technique underlies semantic search, retrieval, recommendations, and most of how modern AI finds the right thing. Meaning becomes a map, and once you have the map, finding what you want is just a matter of measuring distance.

Common questions

What are embeddings in AI? Embeddings are numerical representations of data, given as vectors, that capture meaning in a form computers can compare. An embedding turns a word, sentence, document, or image into a point in a high-dimensional space, arranged so that items with similar meaning are close together and different meanings are far apart. This lets a computer judge how related two things are by measuring the distance between their vectors. Embeddings are the technology behind semantic search, retrieval-augmented generation, recommendation systems, and clustering, because they convert the difficult problem of comparing meaning into the simple problem of comparing positions in space.

How do embeddings work? Embeddings work by turning meaning into geometry. An embedding model, a trained neural network, maps each input to a vector so that similar inputs land near each other and dissimilar ones land far apart. The model learns this arrangement through contrastive training, being adjusted to pull similar pairs together and push dissimilar pairs apart across huge amounts of data, until the geometry of the space reflects meaning. Once data is embedded this way, judging similarity is just measuring the distance between points, so tasks like search and recommendation reduce to finding nearby vectors, which computers do quickly and precisely.

What is a vector embedding? A vector embedding is the same thing as an embedding: a fixed-length list of numbers that represents a piece of data as a point in a high-dimensional space. The term emphasises that the representation is a vector, an array of hundreds to thousands of numbers, where each number is a coordinate along a learned dimension of the space. What makes it a useful embedding rather than an arbitrary vector is that the space is organised so proximity reflects semantic similarity. Modern text embeddings typically have a few hundred to a few thousand dimensions, with more dimensions allowing finer distinctions at the cost of greater storage and computation.

What is cosine similarity? Cosine similarity is the standard way to measure how similar two embeddings are. Instead of the straight-line distance between two vectors, it measures the angle between them, capturing whether they point in the same direction regardless of their length, which makes it robust to differences like one text being longer than another. A cosine similarity of one means the vectors point the same way and are as semantically alike as possible; a value near zero means they are unrelated; a negative value means they point in opposite directions. It is the small calculation that turns a pair of embeddings into a single, interpretable similarity score.

What is the difference between word and contextual embeddings? Word embeddings, from early methods like word2vec, assign a single fixed vector to each word regardless of context, so every use of a word gets the same representation. This blurs words with multiple meanings, since "bank" as a riverbank and "bank" as a financial institution are forced into one vector. Contextual embeddings, built on transformers, instead produce a vector for a word or passage based on its surrounding text, so the same word gets different vectors in different contexts and its intended meaning is captured. This shift from static to contextual embeddings is a major reason modern semantic search and retrieval work far better than earlier approaches.

What are embeddings used for? Embeddings power almost anything that involves finding or organising data by meaning. They enable semantic search, which retrieves results by meaning rather than shared keywords, and retrieval-augmented generation, which retrieves relevant document chunks to give a language model external knowledge. They drive recommendation systems by placing items and preferences near each other in the space, and they support clustering, classification, deduplication, and anomaly detection. Because different data types can share one embedding space, they also enable multimodal applications like searching images with text. Vector databases store and search embeddings at scale, finding the nearest vectors to a query among millions.

What are the limitations of embeddings? Embeddings have several limits worth knowing. An embedding is only as good as the model that made it, and different models encode different notions of similarity, so one suited to general text may fail on specialised domains. The meaning of "similar" is set by the model's training objective, which may not match what you consider relevant, so two things placed close together are not always close in the way you need. Embeddings also inherit and can amplify biases from their training data. And storing and searching large numbers of high-dimensional vectors becomes a significant cost at scale, often the dominant expense in a retrieval system.