What Are Embeddings? How AI Turns Text into Numbers

1,536 numbers. Hand the sentence "What do I do if an invoice is late?" to an AI system and the first thing it produces is not an answer. It is a list of 1,536 decimal numbers. The sentence is processed only after it has become that list, and everything else, the search, the "assistant that knows our documents," the product recommendation, is built on top of it. The list is called an embedding, and in most AI projects it is the smallest line on the invoice and the most decisive one for the result.

So, what are embeddings? The short answer: a way of representing the meaning of text as numbers. The long answer covers how many numbers, which model produces them, how well they work outside English and what it costs to convert a 10,000-page archive. This article gives the long answer at the level of detail someone approving a purchase actually needs. For the map as a whole, our non-technical leader's guide to AI infrastructure decisions covers the surrounding choices; here we zoom in on one square of it, the moment text turns into numbers.

What are embeddings, and how do numbers carry meaning?

An embedding represents the meaning of a word, sentence or document as a vector: a list of hundreds or thousands of numbers. AI models do not read text directly. They convert every piece of text into such a list and measure meaning as distance between lists. Two passages that mean similar things get vectors that sit close together; unrelated passages sit far apart.

Make it concrete. The vectors for "cat" and "dog" are near each other; "cat" and "invoice" are not. The famous example from the 2013 word2vec paper shows the arithmetic: take the vector for king, subtract man, add woman, and you land very close to queen. The numbers are not meaning itself, but they preserve the relationships of meaning with surprising fidelity.

The most useful business analogy is giving every document a GPS coordinate. In a library shelved alphabetically, someone searching for "late payment penalty" never finds the file titled "default interest"; the letters do not match. In a library shelved by coordinate, the two files sit side by side, because they are about the same thing. Keyword search matches letters; embedding search matches meaning. The cost and quality consequences of that difference follow below, in numbers.

How do you choose an embedding model, and what do dimensions mean?

An embedding model is a ready-made system that turns text into numbers, and choosing one comes down to three criteria: vector size, price and language performance. Size, or dimensions, is the number of entries in the list, and it directly drives storage and search speed. Price is quoted per million tokens. Language performance is the one you have to test yourself if your documents are not in English.

The figures that third-party comparison sites agree on at the time of writing look like this. OpenAI's text-embedding-3-small produces 1,536 dimensions at $0.02 per million tokens; text-embedding-3-large produces 3,072 at $0.13. Voyage's voyage-3-large gives 1,024 dimensions at $0.06. Cohere's Embed v4 gives 1,536 at $0.12 and accepts documents of up to 128,000 tokens in one pass. Google's Gemini embedding model produces 3,072 dimensions; the price figures we found for it were contradictory, so we leave it out and point you to the official pricing page.

On dimensions there is a widespread misconception: bigger is not always better. OpenAI's own measurement from January 2024 makes the point. text-embedding-3-large cut down to 256 dimensions scores 62.0 on the MTEB benchmark, ahead of the previous-generation ada-002 at its full 1,536 dimensions (61.0). A vector six times smaller, and a better result. The technique behind it is Matryoshka representation learning: the vector can be truncated from the front while keeping most of its meaning, so you can trade cost, speed and quality without changing models.

An independent benchmark in March 2026 put ten models side by side and summarised the outcome in one line: no model wins every round. There is no "best embedding model." There is a best model for your workload.

On the open-source side, two names stand out. BAAI's BGE-M3 supports more than 100 languages, produces 1,024 dimensions and can do both meaning-based and keyword-style matching in a single model. Alibaba's Qwen3-Embedding family comes in three sizes from 0.6 to 8 billion parameters; according to its own technical report, the 8-billion version scores 75.22 on the English MTEB benchmark against Gemini's 73.30. That is a vendor measurement, and independent verification is a separate matter. Both run free on your own server.

What does it cost to embed a 10,000-page archive?

About one to two dollars. The model fee for converting a 10,000-page archive into embeddings stays under two dollars even with the most expensive commercial model. Next to the rest of the project, that is a rounding error. The real cost accumulates in the database that stores the vectors, in the engineering of how documents are split, and in the language-model call on the query side.

Here is the arithmetic, with approximate assumptions. Say a page holds 500 words. English text runs at roughly 1.3 tokens per word, so 650 to 700 tokens a page and about 7 million tokens for the archive. Languages with richer word structure inflate that; Turkish, for instance, runs at roughly 1.5 to 2 times the token count, so the same archive is nearer 12 million tokens. Taking the higher figure:

• text-embedding-3-small ($0.02/M): about $0.25.
• voyage-3-large ($0.06/M): about $0.70.
• Cohere Embed v4 ($0.12/M): about $1.45.
• text-embedding-3-large ($0.13/M): about $1.55.

Storage is small too. Twelve million tokens split into 400-token chunks, with overlap, comes to roughly 35,000 vectors. A 1,536-dimension vector at 4 bytes per number is 6 kilobytes; 35,000 of them is about 200 megabytes. At 3,072 dimensions, 400 megabytes; at 256, around 35. All of it fits in a laptop's memory. The dimension decision matters at millions of documents, not tens of thousands.

The query side shows the same asymmetry. Each search query is 20 to 50 tokens; a million queries costs under a dollar on the small model. But the language-model call that answers the query consumes several thousand tokens each time and costs a hundred to a thousand times more than the embedding. When the budget is on the table, look at the language-model line, not the embedding line. For where to keep the vectors, our piece on whether you actually need a vector database walks through which scale needs what; for tens of thousands of vectors, pgvector on an existing PostgreSQL is usually enough.

How well do embeddings work outside English?

Commercial models are "good enough" in most major languages; in open source, BGE-M3 tends to lead multilingual comparisons; and language-specific fine-tuned models report better results on particular tasks. But the international leaderboards such as MTEB are English-heavy, so for any other language the reliable way to decide is a 50-to-100-question test on your own data.

Turkish makes a useful case study because the work there accelerated in late 2025. A model called TurkEmbed4Retrieval, published in November 2025, retrains the deeper layers of a multilingual base on Turkish search data and reports beating a Turkish ColBERT variant by 19 to 26 percent on a scientific retrieval test, by its authors' own measurement. A Turkish-specific evaluation suite, TR-TEB, is also in progress. These are single-source findings; they point a direction rather than settle a decision, and the same pattern is repeating in other languages.

Non-English languages also carry a hidden cost: token inflation. The same meaning can take one and a half to two times as many tokens, and that flows straight into both the embedding bill and the language-model bill. Half of the 12-million-token figure above comes from that inflation. There is a structural limit as well: Microsoft's multilingual-e5-large cuts off at 512 tokens, and a single page in a token-heavy language exceeds it, which makes chunking mandatory rather than optional.

Five things people get wrong about embeddings

The five misconceptions below are the ones we hear most often in meetings where embeddings come up. Under each is why the numbers say otherwise. What they share is a tendency to imagine embeddings as more expensive, more intelligent or more permanent than they are.

• "Embedding means training a model." It does not. You send text to a ready model and receive a vector; no training happens. Fine-tuned models such as TurkEmbed4Retrieval are a separate, rarely needed step.
• "More dimensions are better." A 256-dimension new model beats a 1,536-dimension old one. Dimensions measure storage and latency, not quality.
• "It replaces keyword search entirely." For product codes, personal names and exact phrases, keyword search still wins. The industry standard is hybrid search, which is why BGE-M3 offers both methods in one model.
• "The model understands the text." It learns statistical closeness. "Refundable" and "non-refundable" can land uncomfortably near each other; negation and numeric comparison are weak spots.
• "Embed once and you're done." Change the model and the whole archive must be re-embedded; vectors from different models cannot be compared. When a provider retires a model, that cost arrives on its own. Record the model name and version next to every vector.

A concrete scenario: three weeks at a 40-person law firm (figures rounded)

A 40-person law firm wanted to move fifteen years of contracts and pleadings into a system that could search by meaning. The archive held about 9,000 documents, 14,000 pages. The first week went on converting documents to text; a quarter of the scanned PDFs had no text layer and needed optical character recognition. The embedding itself took an hour that week, and the model fee stayed under three dollars.

The second week was model comparison. The firm hand-labelled 80 questions its lawyers actually ask, each paired with the document holding the right answer. Three models were tried: a large commercial one, BGE-M3, and a language-specific fine-tuned model. The rate of finding the right document in the top five results came out at 84, 79 and 81 percent respectively. The commercial model led, but narrowly; for data-sensitivity reasons the firm went with BGE-M3 on its own server and layered keyword search on top. The hybrid setup pushed the rate to 88 percent. A search for "default interest" now also returns documents that say "late payment penalty," while a case-number search still matches character for character.

The third week tuned chunk size. At 800-token chunks, clause numbers were bleeding into one another; dropping to 350 tokens and prefixing each chunk with the document name and clause heading cleaned the results up noticeably. Total infrastructure cost: installing pgvector on the existing PostgreSQL server and two days of engineering. The embedding fee remained the cheapest line in the project. The most expensive line was the lawyers' time spent building the 80-question test set, and that time was the only thing that determined the project's quality.

Frequently asked questions

What's the difference between an embedding and a vector database?

The embedding is the act of converting text to numbers, and its output. The vector database is the store that holds those numbers and finds the nearest neighbour fast: pgvector, Qdrant, Milvus, Pinecone, Weaviate. One produces, the other shelves. For tens of thousands of vectors, adding pgvector to an existing PostgreSQL is enough for most small businesses.

What happens if the embedding model changes?

The old vectors become incompatible with the new model and the whole archive is re-embedded. At 10,000 pages that is a few dollars and a few hours; at millions of documents it is an operation to plan. Storing the model and version with each vector keeps the migration manageable.

Why does chunking matter so much?

The model compresses a chunk's meaning into a single vector; squeeze a 50-page document into one vector and the detail is gone. Typical practice is 200-to-500-token chunks with 10 to 20 percent overlap, each prefixed with context such as the document name. In the law-firm example, the biggest quality jump came from this setting.

Are RAG and embeddings the same thing?

No; embedding is RAG's first step. RAG uses the query's vector to find the nearest document chunks, hands them to the language model, and the model answers from those chunks. Our non-technical guide to RAG covers the whole chain and its cost.

Should I run it on my own server or use an API?

High data sensitivity or high volume points to an open-source model on your own server; a quick start at low volume points to an API. Since the embedding fee is negligible either way, the decision rests on data location and operational load, not price.

So what should you actually do?

• Build a 50-to-100-question test set first: the questions your team really asks, paired with the documents holding the answers. Model choice follows this set, not a leaderboard.
• Try three models on the same set: one commercial, BGE-M3, and a language-specific one if your documents are not in English. Total cost a few dollars, one day of time.
• Set up hybrid search: combine meaning-based and keyword search; do not leave code and name lookups to embeddings alone.
• Experiment with chunk size: start at 300 to 400 tokens, prefix each chunk with the document title, measure on the test set.
• Record the model version: note which model and version produced every vector, so you know the scope when a re-embedding day comes.

Back to the 1,536 numbers at the top. The list itself is nearly free; the model choice, the chunking decision and the test set that produce it are everything that determines the project's quality. If you have an archive waiting to be converted, we would be glad to think through how you build that 80-question set. Telling us what kind of documents you hold is enough to start.