A vector embedding is a fixed-length list of numbers that represents a piece of content — a sentence, a document, an image — such that similar content produces nearby vectors. That is the whole idea. Everything else is engineering around it.
Why nearness is useful
Keyword search matches characters. A search for "cancel my subscription" misses a document titled "ending your plan" because they share no words. Embeddings place both near each other because they mean the same thing, which is what makes semantic search work.
How similarity is measured
- Cosine similarity — the angle between two vectors, ignoring magnitude. The default for text, and what most models are trained to optimise.
- Dot product — cosine weighted by magnitude. Equivalent to cosine when vectors are normalised, which most are.
- Euclidean (L2) distance — straight-line distance. Sometimes right for image embeddings, rarely the best default for text.
Use the metric the embedding model was trained with. Mixing them is a quiet source of mediocre retrieval that looks like a chunking problem.
The decisions that actually affect quality
- Chunk size. Too large and one chunk covers several topics, so its vector means nothing in particular. Too small and it loses the context needed to be interpretable. Paragraph-ish, respecting document structure, is a reasonable start.
- Chunk overlap. A little overlap stops a sentence at a boundary from losing its neighbours. Too much inflates your index and returns near-duplicates.
- Dimensions. Higher dimensions capture more nuance and cost more to store and search. Many production systems find the smaller variant of a model indistinguishable in quality for their corpus.
- What you embed. Embedding a document's title plus a summary often retrieves better than embedding its raw body, because the vector then represents the topic rather than an arbitrary passage.
Storing them
-- Postgres with pgvector: embeddings live beside the data they describe
CREATE TABLE document_chunk (
id bigserial PRIMARY KEY,
document_id bigint NOT NULL REFERENCES document(id) ON DELETE CASCADE,
content text NOT NULL,
embedding vector(1536) NOT NULL
);
-- Approximate index: fast, and deliberately not exact
CREATE INDEX ON document_chunk
USING hnsw (embedding vector_cosine_ops);A dedicated vector database earns its place at scale or when you need features Postgres lacks. Below that threshold, keeping vectors next to the rows they describe means one database to operate, one backup, and joins that let you filter by tenant before searching.
Embeddings are model-specific and not portable
Vectors from one model are meaningless to another. Changing embedding models means re-embedding the entire corpus — there is no migration path. Record which model and version produced every vector, or a future upgrade becomes an archaeology project.
Where semantic search alone falls short
Embeddings are weak on exact identifiers: order numbers, SKUs, error codes, names. "Show me ticket ABC-4417" is a keyword query, and a pure vector search will return things that are semantically similar and factually wrong. Hybrid search — keyword and vector combined, then reranked — is the usual answer.
Frequently Asked Questions
How many dimensions do I need?
Fewer than you think. Test the smaller variant of your model against your own retrieval evals before paying for the larger one; on many corpora the difference is not measurable.
Do I need a vector database?
Not initially. Postgres with pgvector handles a great deal, and keeping vectors beside your relational data simplifies filtering and operations. Move when you have a measured reason.
Can I embed images and text together?
With a multimodal model trained for it, yes — that is how image search by text description works. Embeddings from separate text and image models do not share a space and cannot be compared.
References
About Jishu Labs
Jishu Labs is a software development company founded in 2016. We build custom software, AI/ML systems, and full-stack web and mobile applications for clients, and we make eight AI tools for software teams.