Embeddings Explained: Meaning as Distance

Typing "when will my money come back" into a help-center search can return an article titled "When refunds arrive". No word in the query appears in the title, and the search still found the right page. That behavior comes from embeddings, numerical representations of text in which similar meanings sit close together. The idea is simple to state and easy to misuse, because closeness in vector space is not the same as relevance in every case.

Meaning as position

An embedding model reads a piece of text and returns a list of numbers, a vector with many dimensions. The model is trained so that texts with related meanings receive vectors that point in similar directions. Similarity is usually measured with a cosine or a dot product, but the underlying idea is geometric: two passages are similar when their vectors are close, meaning the angle between them is small. Individual dimensions have no meaning a reader can interpret, and they do not need to be interpreted. What matters is the relationship between vectors.

A useful picture is a map of ideas in which nearby places share a theme. A question about money coming back lands near an article about refunds, even though the words differ, because training taught the model that those phrasings often describe the same situation.

How search uses the vectors

To search, the system embeds every document chunk in advance and stores the vectors in an index. When a query arrives, it is embedded the same way, and the chunks with the nearest vectors are returned. The lookup stays fast on large collections because specialized indexes avoid comparing the query with every stored vector one by one, at a small cost in exactness. The same geometry supports grouping: items with nearby vectors can be clustered without anyone naming the groups in advance.

Embeddings are only as good as the model that produced them for the material at hand. The same query can rank differently under two embedding models, and switching models means rebuilding the index. Preprocessing matters too. Lowercasing, stripping markup and deciding whether headings are included all change the vectors, and therefore the results. Record the embedding model, its version and the preprocessing steps beside the index, the way a database keeps its schema, so that a later change can be traced to its cause.

What distance does not capture

Closeness approximates meaning, and the approximation fails in predictable places. Negation is one. "The refund was approved" and "the refund was not approved" share most of their words and can sit close together. Exact identifiers are another. An order number, an error code or a part number carries its meaning in its exact characters, and a vector model may treat two different codes as near neighbors because they look alike. Rare technical terms, internal product names and abbreviations used only inside one organization can be poorly placed, because the model saw little of them in training. When the domain differs from the training data, the geometry can drift, and retrieval quality drops without any visible error message.

Chunking before embedding

Embedding a long document as a single unit produces a vector that averages many topics, and an average matches nothing particularly well. Splitting the document into chunks solves that problem, but chunk size is a trade-off. Small chunks are precise but lose the surrounding context that gives a sentence its meaning. Large chunks keep context but dilute the signal. Overlap between chunks helps keep a sentence and its qualifier together. Headings should travel with their text, and metadata such as product, date or document type allows filters to narrow the search before similarity is computed.

Test chunk sizes on real queries. The best size depends on the documents and the questions, and a default that works for one collection can fail on another.

Combining embeddings with keyword search

Many production search systems combine two methods. Keyword search matches the exact terms in a query, and it handles identifiers and rare words reliably. Vector search handles paraphrase and vocabulary mismatch. A hybrid system runs both, merges the candidate lists, and often adds a reranking step that reads the query and each candidate more closely before ordering the final results. The combination takes more work to build, but it fails in fewer ways than either method does alone. Reranking also costs time, since it reads each candidate alongside the query, so its benefit has to be weighed against the latency budget.

Keyword search is also the better tool on its own in some cases. A small collection with exact technical vocabulary, a codebase searched for function names, and legal text where a specific phrase matters may all work better with plain matching. Choosing the simpler tool is a sound engineering decision, not a step backward.

Practice: find where the correct answer is lost

Collect thirty real queries from a domain you know, and for each one identify the document that should be returned. Run every query through keyword search, vector search and the hybrid, and note whether the correct document appears in the top five results for each method. Then look at the queries where a method failed and sort the failures: identifiers, negation, paraphrase, rare terms and chunking problems. The pattern shows whether the next improvement belongs in the index, the chunking, the query or the ranking. Keep the set, because it is the test needed whenever any part of the pipeline changes.