Embeddings

Embeddings map text, images, or other objects into vectors so nearby points represent model-learned similarity. In this section they power vector databases, retrieval pipelines, hybrid retrieval, and sometimes agent memory. They are best understood as learned similarity functions with operational constraints.

Embedding similarity

An encoder maps an item $x$ to $e=f_{\theta }(x)\in {\mathbb{R}}^d$. Cosine similarity is

$$\cos (a,b)=\frac{a^{\top }b}{\Vert a{\Vert }_2\Vert b{\Vert }_2}.$$

The training objective determines what similarity means; a search embedding is not automatically a clustering or classification embedding.

For normalized vectors, cosine similarity and dot product give the same ranking. For unnormalized vectors, vector length can change ranking behavior. Production indexes should document the distance function, normalization, embedding model, and version because all four affect retrieval.

Worked example

Suppose a query embedding points mostly toward the “refund” direction, with some “billing” component and no “weather” component. Cosine similarity ranks documents by vector angle:

Candidate	Vector intuition	Cosine to query	Retrieval meaning
A refund policy	Strong refund component, small billing component.	0.992	Best semantic match.
B invoice payment	Strong billing component, weak refund component.	0.409	Related business topic, but weaker.
C weekend forecast	Weather component unrelated to the query.	0.031	Semantically off-topic.

The query vector points toward the refund document, so vector search retrieves it first. A reranking step can still change the order after reading full query-document pairs, especially when exact dates, names, or policy constraints matter.

Design choice	Effect
Normalize embeddings before cosine search	Makes angle rather than vector length drive ranking.
Use a domain-tuned embedding model	Improves similarity for local jargon and document structure.
Keep metadata filters with vectors	Prevents semantically similar but unauthorized or stale passages from entering context.

Choosing embedding inputs

The text sent to the embedding model should be the text you want similarity to operate on. A chunk containing only:

Section 4.2: Exceptions

is usually a poor embedding input. A better record includes title, path, normalized metadata, and the chunk body:

Refund policy > Enterprise accounts > Approval thresholds.
Enterprise refunds above 5000 EUR require finance approval.

This improves retrieval because the vector now carries both local content and document context. It also helps query rewriting match user language such as “approval threshold” to the policy wording.

Evaluation

Evaluate embeddings by retrieval task, not by visual intuition. Use a set of queries with known answer-bearing chunks and measure recall before reranking, recall after metadata filters, and answer support after context packing. Slice by language, document type, freshness, and rare identifiers. A model that works for English FAQs may fail on code snippets, tables, legal clauses, or product IDs.

Caveats

Embeddings can miss exact constraints, names, and numbers. Similarity can also retrieve topically related but non-answering chunks. Evaluate by task, corpus, language, and update cadence; combine dense vectors with lexical or structured retrieval when exact terms matter.

References

• OpenAI API documentation: Embeddings
• Muennighoff et al., 2022, MTEB
• Karpukhin et al., 2020, Dense Passage Retrieval

Nav

Section — Generative AI and Agentic Systems

← RAG Chunking →