Building Hybrid Search: Combining Vector Embeddings and BM25 for Production Retrieval

The Search Problem Nobody Talks About

Most teams treat search as a binary decision: keyword search or vector search. Deploy Elasticsearch for BM25, or spin up a vector database for semantic retrieval. Pick one, move on.

This framing is wrong, and it costs you relevance.

Keyword search misses intent. A query for “heart attack symptoms” retrieves documents containing exactly those words, not documents about “myocardial infarction” or “chest pain with radiating arm pain.” BM25 has no model of meaning.

Vector search misses precision. A query for “Python 3.11 walrus operator” might surface documents about Python in general, or even programming concepts unrelated to that specific syntax. Embeddings encode semantic neighborhoods, not exact term matches. Ask a model for “Redis ZADD” and you may get results about sorted sets that never mention the actual command.

The sweet spot is hybrid retrieval: run both, fuse the results, and optionally rerank. Teams that implement this correctly see relevance improvements of 15-30% over either system alone on benchmark datasets like BEIR. This article walks through the full production implementation.

---

How BM25 and Vector Search Fail Differently

Understanding where each approach breaks tells you exactly why combining them works.

BM25 failure modes:

• Vocabulary mismatch: user says “fast car,” document says “high-performance vehicle”
• No paraphrase handling: “fix bug” vs. “resolve defect”
• No domain knowledge: medical or legal synonyms not in training data

Vector search failure modes:

• Exact match blindness: model embeds “Python 3.11” and “Python 3.8” close together because the text is similar, even though they’re distinct
• Hallucinated proximity: unrelated concepts occasionally cluster in embedding space
• Short query degradation: single-word or acronym queries produce poor embeddings
• Rare entity drift: product names, version strings, or proper nouns often land in the wrong neighborhood

These failure modes are largely non-overlapping. BM25 fails on semantic intent; vector search fails on lexical precision. A result that ranks high in both systems is almost certainly relevant. A result that ranks high in only one is more ambiguous. Fusion exploits this property.

---

Reciprocal Rank Fusion

Reciprocal Rank Fusion (RRF) is the simplest and most robust score combination strategy. It was introduced by Cormack, Clarke, and Buettcher in 2009 and has held up remarkably well.

The formula for a document d across multiple ranked lists:

RRF(d) = sum over each ranker: 1 / (k + rank(d))

k is a smoothing constant, typically 60. It prevents very top-ranked documents from dominating the fusion and makes the system robust to rank position noise.

Compared to linear score combination, RRF has two advantages. First, you do not need to normalize scores across systems (BM25 scores and cosine similarity are on completely different scales). Second, it degrades gracefully when a document is absent from one result list (you simply skip it for that ranker).

Here is a TypeScript implementation of RRF:

interface RankedResult {
  id: string;
  rank: number;
}

interface FusedResult {
  id: string;
  score: number;
}

function reciprocalRankFusion(
  rankedLists: RankedResult[][],
  k: number = 60
): FusedResult[] {
  const scores = new Map<string, number>();

  for (const list of rankedLists) {
    for (const { id, rank } of list) {
      const current = scores.get(id) ?? 0;
      scores.set(id, current + 1 / (k + rank));
    }
  }

  return Array.from(scores.entries())
    .map(([id, score]) => ({ id, score }))
    .sort((a, b) => b.score - a.score);
}

---

Postgres as the Unified Backend

You do not need two separate systems. Postgres with pgvector handles both retrieval modes, which eliminates the operational overhead of running separate Elasticsearch and vector database clusters. For most production workloads under 10 million documents, this is the correct default architecture.

First, set up the schema:

CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm; -- optional, for fuzzy matching

CREATE TABLE documents (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  content     TEXT NOT NULL,
  metadata    JSONB,
  embedding   vector(1536),
  ts_content  TSVECTOR GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
  created_at  TIMESTAMPTZ DEFAULT now()
);

CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
CREATE INDEX ON documents USING GIN (ts_content);

Now the retrieval functions in TypeScript, using postgres (the npm package, not pg):

import postgres from "postgres";

const sql = postgres(process.env.DATABASE_URL!);

interface SearchResult {
  id: string;
  content: string;
  metadata: Record<string, unknown>;
}

async function vectorSearch(
  embedding: number[],
  limit: number = 20
): Promise<Array<SearchResult & { rank: number }>> {
  const results = await sql<Array<SearchResult & { rank: number }>>`
    SELECT
      id,
      content,
      metadata,
      ROW_NUMBER() OVER (ORDER BY embedding <=> ${JSON.stringify(embedding)}::vector) AS rank
    FROM documents
    ORDER BY embedding <=> ${JSON.stringify(embedding)}::vector
    LIMIT ${limit}
  `;
  return results;
}

async function bm25Search(
  query: string,
  limit: number = 20
): Promise<Array<SearchResult & { rank: number }>> {
  const results = await sql<Array<SearchResult & { rank: number }>>`
    SELECT
      id,
      content,
      metadata,
      ROW_NUMBER() OVER (
        ORDER BY ts_rank_cd(ts_content, query) DESC
      ) AS rank
    FROM documents,
         to_tsquery('english', ${query.split(" ").join(" & ")}) AS query
    WHERE ts_content @@ query
    ORDER BY ts_rank_cd(ts_content, query) DESC
    LIMIT ${limit}
  `;
  return results;
}

async function hybridSearch(
  query: string,
  embedding: number[],
  limit: number = 10
): Promise<SearchResult[]> {
  const [vectorResults, keywordResults] = await Promise.all([
    vectorSearch(embedding, 20),
    bm25Search(query, 20),
  ]);

  const vectorRanked = vectorResults.map(({ id, rank }) => ({ id, rank }));
  const keywordRanked = keywordResults.map(({ id, rank }) => ({ id, rank }));

  const fused = reciprocalRankFusion([vectorRanked, keywordRanked]);
  const topIds = fused.slice(0, limit).map((r) => r.id);

  // Fetch full documents for top IDs, preserving RRF order
  const docs = await sql<SearchResult[]>`
    SELECT id, content, metadata
    FROM documents
    WHERE id = ANY(${topIds}::uuid[])
  `;

  const docMap = new Map(docs.map((d) => [d.id, d]));
  return topIds.map((id) => docMap.get(id)!).filter(Boolean);
}

---

Embedding Model Selection

The embedding model is the single largest lever on retrieval quality. Choosing poorly here degrades vector search so much that the hybrid barely beats keyword-only.

Key dimensions to evaluate:

Model	Dimensions	Context tokens	MTEB Score	Cost/1M tokens	Best for
text-embedding-3-small	1536	8191	62.3	$0.02	General use, cost-sensitive
text-embedding-3-large	3072	8191	64.6	$0.13	Higher accuracy, slower
voyage-large-2	1536	16000	67.1	$0.12	Long documents, retrieval tasks
nomic-embed-text-v1.5	768	8192	62.4	self-hosted	On-prem, privacy-sensitive
bge-m3	1024	8192	68.5	self-hosted	Multilingual, SOTA open-source

MTEB (Massive Text Embedding Benchmark) is the standard evaluation suite. Always check MTEB scores for the specific retrieval task category, not just the overall average. A model that excels at classification can underperform on retrieval.

Practical rules:

• Chunk size matters more than you think. Most embedding models degrade on documents over 512 tokens. Chunk at sentence boundaries, not arbitrary character counts, and include overlap (typically 20-30% of chunk size) to avoid splitting context across chunks.
• Embed at index time with the same model you use at query time. Sounds obvious, but model version changes mid-deployment require full re-indexing.
• Normalize embeddings before storage if you plan to use dot product similarity instead of cosine. Cosine is more forgiving of unnormalized inputs.

import OpenAI from "openai";

const openai = new OpenAI();

async function embedText(text: string): Promise<number[]> {
  const response = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: text.slice(0, 8000), // hard cap below context limit
  });
  return response.data[0].embedding;
}

function chunkText(
  text: string,
  maxTokens: number = 400,
  overlapTokens: number = 80
): string[] {
  // Approximate: 1 token ~ 4 characters for English
  const maxChars = maxTokens * 4;
  const overlapChars = overlapTokens * 4;
  const chunks: string[] = [];
  let start = 0;

  while (start < text.length) {
    const end = start + maxChars;
    // Find sentence boundary
    const slice = text.slice(start, end);
    const lastPeriod = slice.lastIndexOf(". ");
    const cutAt = lastPeriod > maxChars * 0.7 ? lastPeriod + 1 : slice.length;
    chunks.push(text.slice(start, start + cutAt).trim());
    start = start + cutAt - overlapChars;
  }

  return chunks.filter((c) => c.length > 50);
}

---

Reranking with Cross-Encoders

Retrieval gives you a candidate set. Reranking refines the order using a more expensive model that sees the query and document together.

Bi-encoders (standard embedding models) encode query and document independently and compare the resulting vectors. Fast, but limited: the model never sees both texts simultaneously, so it cannot reason about nuanced query-document relevance.

Cross-encoders take the concatenated query and document as input and output a single relevance score. They are 100-1000x slower but significantly more accurate. The standard production pattern is to retrieve 50-100 candidates via hybrid search and rerank the top 50, then return the top 10.

interface RerankedResult {
  id: string;
  content: string;
  score: number;
}

async function rerankWithCrossEncoder(
  query: string,
  candidates: SearchResult[],
  topK: number = 10
): Promise<RerankedResult[]> {
  // Using Cohere's rerank API as an example
  const response = await fetch("https://api.cohere.com/v1/rerank", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.COHERE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "rerank-english-v3.0",
      query,
      documents: candidates.map((c) => c.content),
      top_n: topK,
    }),
  });

  const data = await response.json();

  return data.results.map((r: { index: number; relevance_score: number }) => ({
    id: candidates[r.index].id,
    content: candidates[r.index].content,
    score: r.relevance_score,
  }));
}

Self-hosted cross-encoder options include ms-marco-MiniLM-L-6-v2 (fast, 80MB) and bge-reranker-large (slower, higher accuracy). For most applications, a hosted reranker is operationally simpler and the per-query cost is negligible compared to the LLM generation step downstream.

---

Tradeoffs at a Glance

Approach	Latency	Accuracy on exact match	Accuracy on semantic queries	Ops complexity	Cost
BM25 only	~5ms	High	Low	Low	Very low
Vector only	~15ms	Low	High	Medium	Low-medium
Hybrid (no rerank)	~20ms	High	High	Medium	Low-medium
Hybrid + rerank	~200ms	Very high	Very high	Medium	Medium
Hybrid + rerank + LLM	~1s+	Highest	Highest	High	High

Latency numbers are for a 1M document Postgres index on standard cloud hardware. Your numbers will differ based on index size, hardware, and network.

The right level for your application depends on the query budget you can absorb. An internal knowledge base tool can tolerate 200ms. A search-as-you-type autocomplete cannot. For interactive search, run hybrid without reranking and add reranking only for a final “did you mean” or “top result” display.

---

Production Tuning

Index configuration matters. The ivfflat index in pgvector requires tuning lists (number of Voronoi cells) and probes (cells searched at query time). A common starting point is lists = sqrt(row_count) and probes = lists / 10. More probes improves recall at the cost of latency. For high-recall applications, consider hnsw instead:

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

HNSW has higher memory overhead but better recall-latency tradeoffs than IVFFlat for most workloads.

Query preprocessing is often ignored. Raw user queries are noisy. Strip stopwords before BM25 search (Postgres to_tsquery handles this with the language dictionary). For vector search, query expansion helps: rephrase the query into multiple forms and average the embeddings, or use a small LLM to generate a “hypothetical answer” (HyDE technique) and embed that instead.

async function hydeEmbed(query: string): Promise<number[]> {
  const completion = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [
      {
        role: "system",
        content:
          "Write a short, factual passage that would answer the following question. Do not include the question itself.",
      },
      { role: "user", content: query },
    ],
    max_tokens: 200,
  });

  const hypotheticalAnswer = completion.choices[0].message.content ?? query;
  return embedText(hypotheticalAnswer);
}

Monitor recall, not just latency. In production, it is easy to have fast retrieval that returns irrelevant results. Track click-through rate on search results, or use LLM-as-judge to periodically evaluate a sample of query-result pairs. Set up alerting if click-through drops more than 10% week-over-week.

Batch embedding at indexing time. Never embed documents one at a time. Batch in groups of 100-500 and write directly to Postgres. For large initial imports, use a queue (PgBoss or BullMQ work well) and process in parallel workers to saturate the embedding API rate limits.

Handle missing embeddings gracefully. If a document has no embedding (e.g., the embedding job is still queued), fall back to BM25 only. Do not return an error. The user gets a slightly less relevant result, but the system remains functional.

---

Closing

Hybrid search is not complicated once you see the architecture clearly: two retrievers running in parallel, fused with RRF, and optionally reranked. The biggest practical wins come from embedding model selection and chunk boundary decisions, not from tuning fusion weights. Start with equal weighting (default RRF k=60), measure, and adjust only if evaluation data shows a clear improvement from weighting one signal higher.

Postgres with pgvector handles this workload up to tens of millions of documents without a dedicated vector database. When you outgrow it, the retrieval logic ports directly to any system that supports both ANN search and full-text search.