Embedding Models Compared: OpenAI, Cohere, and Open-Source Models for Production Search

The embedding model is the first thing that runs in your retrieval pipeline and the last thing most teams audit when results are poor.

It deserves more attention than it gets. Model choice affects recall quality directly, drives storage costs through vector dimensionality, determines whether multilingual queries degrade gracefully, and creates long-term operational risk if you need to re-embed a million documents after a provider deprecation.

This article gives you enough signal to make a real decision for production search and RAG systems.

What Embeddings Actually Represent

An embedding is a learned projection from text into a dense vector space where semantic similarity maps to geometric proximity. Two sentences that mean the same thing should cluster close together. Two sentences with different meanings should be far apart.

The quality of that mapping depends on:

• Training data (size, domain coverage, query-document pair quality)
• Model architecture (transformer depth, attention head count, pooling strategy)
• Training objective (contrastive loss, in-batch negatives, hard negative mining)
• Fine-tuning (whether the model was specialized for retrieval vs classification vs clustering)

This matters because a model trained primarily on sentence pairs for semantic similarity performs differently on asymmetric retrieval (short query, long document) than one trained specifically on search data. MTEB scores reflect this distinction across task types.

The MTEB Benchmark

The Massive Text Embedding Benchmark (MTEB) is the best public reference for comparing models. It covers 56 datasets across 8 task types: retrieval, clustering, classification, pair classification, reranking, STS (semantic textual similarity), summarization, and bitext mining.

For RAG and search, focus on the retrieval subtask scores (NDCG@10) and the reranking subtask. Average MTEB score includes classification and STS tasks that are irrelevant for retrieval workloads, and optimizing for average score often misleads.

At time of writing, top performers on MTEB retrieval:

Model	MTEB Retrieval NDCG@10	Dimensions	Multilingual
text-embedding-3-large	~54.9	3072 (truncatable)	No
text-embedding-3-small	~62.3	1536 (truncatable)	No
embed-english-v3.0	~54.9	1024	No
embed-multilingual-v3.0	~54.9	1024	Yes (100+ languages)
nomic-embed-text-v1.5	~62.4	768	No
BGE-large-en-v1.5	~54.3	1024	No
BGE-M3	~54.9	1024	Yes

Raw MTEB numbers are directional, not definitive. Always run evals on your own query logs and corpus before committing.

Provider Comparison

OpenAI text-embedding-3-small and text-embedding-3-large

The v3 models added Matryoshka Representation Learning (MRL), which lets you truncate output dimensions without retraining. text-embedding-3-small at 1536 dimensions and text-embedding-3-large at 3072 can both be truncated to as few as 256 dimensions with partial but predictable quality degradation.

This is practically useful. If you want to trade retrieval quality for storage cost and query latency, you can truncate to 512 or 256 dimensions and test the quality drop on your actual workload.

The API is stable and the latency profile is predictable. The main operational risk is dependency on one provider: if the model gets deprecated or pricing changes, you need to re-embed everything.

text-embedding-3-small vs text-embedding-3-large: small is cheaper (~5x) and faster; large produces marginally better retrieval in some tasks. For most production workloads, small is the right default and large is worth testing only when you have evidence that quality is insufficient.

Cost: ~$0.02 per million tokens (small), ~$0.13 per million tokens (large).

Cohere embed-v3

Cohere offers two relevant models: embed-english-v3.0 for English-only and embed-multilingual-v3.0 for 100+ languages. Both produce 1024-dimensional vectors.

The v3 models require you to specify an input_type parameter: search_document, search_query, classification, or clustering. This affects how the model encodes input and improves asymmetric retrieval quality. It adds a step to your pipeline but it is the right default.

Cohere also exposes int8 and binary quantization at the API level, returning quantized vectors directly. Binary embeddings at 1/32 the storage cost of float32 can be useful for prefiltering large corpora before a full-precision reranking pass.

MTEB retrieval scores for embed-v3 are competitive with text-embedding-3-small on English. The multilingual model outperforms alternatives for non-English queries. Fine-tuning is available through Cohere’s API if you have labeled query-document pairs.

Cost: ~$0.10 per million tokens (both variants).

Open-source options

The open-source landscape has materially improved. Three models are worth knowing.

sentence-transformers / all-mpnet-base-v2 and all-MiniLM-L6-v2

The sentence-transformers library from Hugging Face is the standard library for running embedding models locally. all-MiniLM-L6-v2 (384 dimensions) is fast and cheap to run; all-mpnet-base-v2 (768 dimensions) scores better on most retrieval tasks. Both are useful for prototyping, smaller-scale deployments, or latency-critical paths where you want the model co-located with the service.

nomic-embed-text-v1.5

nomic-embed-text-v1.5 supports MRL (same as OpenAI v3), is fully open-source (Apache 2.0), and scores well on MTEB retrieval. It supports a 8192 token context window, which is larger than most alternatives and relevant for long document embedding. Available on Hugging Face or via Nomic Atlas API.

BGE (BAAI General Embedding)

The BGE family from Beijing Academy of AI covers English-only (BGE-large-en-v1.5) and multilingual (BGE-M3) variants. BGE-M3 supports 100+ languages and 8192 token context. Both are on Hugging Face with permissive licensing. BGE models have fine-tuning documentation and are widely deployed in production.

TypeScript Examples

OpenAI

import OpenAI from "openai";

const client = new OpenAI();

async function embedDocuments(texts: string[]): Promise<number[][]> {
  const response = await client.embeddings.create({
    model: "text-embedding-3-small",
    input: texts,
    dimensions: 512, // MRL truncation: trade quality for storage cost
  });

  return response.data.map((item) => item.embedding);
}

async function embedQuery(query: string): Promise<number[]> {
  const response = await client.embeddings.create({
    model: "text-embedding-3-small",
    input: query,
    dimensions: 512,
  });

  return response.data[0].embedding;
}

Cohere

import { CohereClient } from "cohere-ai";

const cohere = new CohereClient({ token: process.env.COHERE_API_KEY! });

async function embedDocuments(texts: string[]): Promise<number[][]> {
  const response = await cohere.embed({
    model: "embed-english-v3.0",
    texts,
    inputType: "search_document",
    embeddingTypes: ["float"],
  });

  if (!response.embeddings.float) {
    throw new Error("Expected float embeddings");
  }

  return response.embeddings.float;
}

async function embedQuery(query: string): Promise<number[]> {
  const response = await cohere.embed({
    model: "embed-english-v3.0",
    texts: [query],
    inputType: "search_query",
    embeddingTypes: ["float"],
  });

  if (!response.embeddings.float) {
    throw new Error("Expected float embeddings");
  }

  return response.embeddings.float[0];
}

Hugging Face (sentence-transformers, self-hosted via inference endpoint)

async function embedWithHuggingFace(
  texts: string[],
  endpoint: string,
  token: string
): Promise<number[][]> {
  const response = await fetch(`${endpoint}/embeddings`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ inputs: texts }),
  });

  if (!response.ok) {
    throw new Error(`Embedding request failed: ${response.status}`);
  }

  return response.json() as Promise<number[][]>;
}

For local inference with sentence-transformers in a Python sidecar, the TypeScript side just calls an internal HTTP endpoint. The embedding logic stays in Python where the ML ecosystem is richer.

Batching Strategies

All three providers accept batched inputs. Batch size limits differ:

• OpenAI: up to 2048 inputs per request
• Cohere: up to 96 inputs per request
• Hugging Face Inference API: varies by deployment; typically 32-256

For ingestion pipelines, always batch. A loop that sends individual embed calls will hit rate limits and cost 10-50x more in latency than a batched pipeline.

async function batchEmbed(
  texts: string[],
  batchSize: number,
  embedFn: (batch: string[]) => Promise<number[][]>
): Promise<number[][]> {
  const results: number[][] = [];

  for (let i = 0; i < texts.length; i += batchSize) {
    const batch = texts.slice(i, i + batchSize);
    const embeddings = await embedFn(batch);
    results.push(...embeddings);
  }

  return results;
}

For high-volume ingestion, add a concurrency limit with a semaphore or p-limit to run multiple batches in parallel without overwhelming rate limits.

Caching Embeddings

Query-side caching is often worth doing. Identical or near-identical queries repeat in most search products (navigational queries, autocomplete, common support questions).

A simple approach: hash the input text and model identifier, check a cache (Redis or KV store), return cached vector on hit, embed on miss and cache with a TTL.

import { createHash } from "crypto";

function embeddingCacheKey(text: string, model: string): string {
  return createHash("sha256")
    .update(`${model}::${text}`)
    .digest("hex");
}

async function cachedEmbed(
  text: string,
  model: string,
  cache: Map<string, number[]>,
  embedFn: (t: string) => Promise<number[]>
): Promise<number[]> {
  const key = embeddingCacheKey(text, model);

  const cached = cache.get(key);
  if (cached) return cached;

  const embedding = await embedFn(text);
  cache.set(key, embedding);
  return embedding;
}

For document-side embeddings, store them in your vector database with a stable chunk ID. Re-embed only when the source content or model version changes. Track model version in document metadata.

Handling Model Version Changes

This is where teams get hurt. Embeddings from different model versions are not compatible. You cannot mix vectors from text-embedding-3-small and text-embedding-ada-002 in the same index. Cross-model cosine similarity produces garbage results.

When a model version changes:

1. Re-embed the full corpus with the new model.
2. Write new vectors to a shadow index.
3. Run A/B retrieval tests comparing old and new indexes on a sampled query set.
4. Cut traffic to the new index only after verifying no retrieval regression.
5. Delete the old index.

This is non-trivial at large scale. One mitigation: keep raw text chunks stored durably and separately from the index, so you can always re-embed from source without re-extracting from PDFs or re-crawling data sources.

Store the embedding model identifier and version in every document record. You will need it during migration.

When to Fine-Tune

Off-the-shelf models underperform when:

• Your domain has heavy jargon or rare terminology not well-represented in the training data (legal, clinical, industrial equipment).
• Your queries are structurally unusual (code identifiers, chemical formulas, part numbers).
• Your document-query asymmetry is extreme (very short queries, very long documents).

Fine-tuning requires labeled data: (query, positive document, negative document) triplets. You need at minimum a few thousand examples to see meaningful gains; tens of thousands for robust improvements.

Options:

• Cohere: supports fine-tuning via their API; you upload triplets and get a custom model back.
• Open-source: fine-tune any sentence-transformers model using the SentenceTransformerTrainer class with MultipleNegativesRankingLoss.
• OpenAI: does not currently expose embedding model fine-tuning.

Fine-tuning adds operational overhead: you now own a model artifact, need to version it, and need to re-embed when you retrain. Make sure the retrieval quality improvement is worth that cost before committing.

If you have a specialized domain but limited labeled data, look at domain-adaptive pretraining (DAPT): continue pretraining a base model on unlabeled in-domain text before the retrieval fine-tuning step. This can improve results when labeled pairs are scarce.

Dimensionality and Storage Costs

Dimensionality has a direct cost impact at scale.

For 10 million documents:

Dimensions	float32 bytes	float16 bytes	int8 bytes
3072	~117 GB	~58 GB	~29 GB
1536	~58 GB	~29 GB	~15 GB
1024	~39 GB	~19 GB	~10 GB
768	~29 GB	~15 GB	~7.5 GB
512	~19 GB	~10 GB	~5 GB

This affects:

• Vector database storage costs (especially for managed providers)
• Memory-mapped index size (critical for HNSW which is memory-resident)
• Query latency (higher dimensions = more compute per similarity calculation)

For most workloads at 10M documents or below, 768 or 1024 dimensions with float32 is manageable. Above 50M documents, quantization (int8 or binary) becomes necessary to keep index memory within reason.

OpenAI’s MRL truncation and Cohere’s int8/binary output are both useful levers here. Test quality at each compression point on your actual queries before committing.

Decision Framework

Semantic search (English, public API acceptable, <50M docs)

text-embedding-3-small is the practical default. Stable API, reasonable cost, MRL gives you dimensionality flexibility. Use large only if you can demonstrate quality improvement on your eval set.

RAG over multilingual content

embed-multilingual-v3.0 or BGE-M3 depending on whether you want a managed API or self-hosted. Both support 100+ languages with competitive MTEB multilingual scores. Cohere’s API is easier to operate; BGE-M3 gives you full model ownership.

High-volume ingestion where cost dominates

text-embedding-3-small with dimension truncation, or nomic-embed or BGE self-hosted. At >100M tokens per day, API costs become significant. Self-hosted models on a GPU instance often cost less at scale and eliminate per-token pricing.

Specialized domain (legal, medical, industrial)

Start with a strong open-source base (BGE-large-en or nomic-embed) and fine-tune on domain data. The base model gives you good general retrieval; fine-tuning recovers the domain-specific signal. Using a managed API for a domain-specific model is harder because you either get no fine-tuning option (OpenAI) or need to manage a custom model artifact through a vendor API.

Classification or clustering tasks (not retrieval)

MTEB retrieval scores are not predictive here. Evaluate specifically on classification tasks. Cohere’s input_type: "classification" and input_type: "clustering" options show real quality differences from search_document mode. OpenAI’s models also perform well on classification tasks with no parameter change needed.

Prototype or offline analysis (no latency requirement)

all-MiniLM-L6-v2 or all-mpnet-base-v2 via sentence-transformers locally. No API cost, runs on CPU, easy to iterate. Swap to a production model when you move to deployment.

Production Checklist

Before shipping:

• Store embedding model identifier and version in every indexed document
• Test batching at your expected peak ingestion throughput
• Evaluate retrieval quality on your actual query logs, not synthetic test sets
• Establish a migration plan for model version changes before you need one
• Set up caching on the query path for repeated or near-identical queries
• Monitor embedding latency separately from query/retrieval latency in your traces

The model choice matters less than having clean chunks, good metadata filtering, and real retrieval evals. A good eval harness with labeled relevance will reveal problems your benchmark scores cannot.

Start simple. Measure on your actual queries. Change what the data tells you to change.