How Elasticsearch Works Internally: Inverted Indexes, Segment Merging, and Distributed Search Across Shards

Elasticsearch sits at an interesting intersection of two hard problems: low-latency full-text search over large corpora and distributed systems coordination at scale. Most engineers know it as “the search cluster” and interact with it through query DSL. But the behaviors that bite in production, slow queries under high write load, mapping explosions that cause OOM kills, relevance drift after index changes, are all explained by its internal mechanics. Understanding how Lucene segments work, how the two-phase distributed search protocol operates, and how the cluster coordinates shard placement makes the difference between an Elasticsearch cluster that hums along and one that pages you at 3am.

This article traces the path from an index request to a searchable document, then follows a query from the coordinating node through shard-level execution back to ranked results. It covers Lucene’s segment model, merge policy, the translog, BM25 scoring internals, cluster coordination, and the operational concerns that matter most at scale. TypeScript examples use the official @elastic/elasticsearch client.

The Lucene Foundation: Inverted Indexes and Doc Values

Elasticsearch is a distributed layer over Apache Lucene. Every primary shard is a Lucene index. Understanding what Lucene does with a document is the prerequisite for understanding everything else.

When Lucene analyzes a field, it constructs an inverted index: a mapping from each term to the list of document IDs that contain that term, called a postings list. For a field containing "distributed systems", the analyzer emits the tokens distributed and systems. Lucene stores distributed -> [doc1, doc4, doc7] and systems -> [doc1, doc2, doc7]. A query for "distributed" becomes a postings list lookup followed by optional intersection or scoring.

The postings list is not a simple array. For efficient phrase queries and proximity scoring, Lucene also stores term positions and optional offsets within each document. The cost is storage; the benefit is that phrase queries like "distributed systems" can verify adjacency without fetching source documents.

For sorting, aggregations, and numeric range queries, the inverted index is the wrong data structure. Elasticsearch uses doc values instead: a column-oriented store that maps document IDs to field values. If you have 10 million documents and want to sort by price or bucket by category, doc values allow sequential reads through a single column rather than chasing pointers across an inverted index. Doc values are persisted to disk and memory-mapped at query time.

A third structure, the term dictionary, indexes terms themselves with a finite state transducer (FST) so Lucene can answer prefix and fuzzy queries without a full scan. This is what powers wildcard and match_phrase_prefix queries efficiently.

Segment Immutability

Lucene writes data in segments: immutable files that each represent a self-contained mini-index. Once written, a segment never changes. Deletions are recorded in a separate .del file that marks document IDs as deleted; they remain in the segment until a merge removes them. Updates in Elasticsearch are internally a delete plus a re-index.

Immutability simplifies concurrency dramatically. Readers never contend with writers because writers never modify what readers are reading. The operating system page cache handles segment files efficiently since they are never invalidated. The cost is that small writes produce many small segments, and many small segments mean slower searches because each query fans out across all segments in the shard.

The Write Path: From Index Request to Searchable Segment

import { Client } from "@elastic/elasticsearch";

const client = new Client({ node: "http://localhost:9200" });

await client.index({
  index: "events",
  id: "evt-001",
  document: {
    userId: "u-42",
    action: "checkout",
    amount: 149.99,
    timestamp: "2026-06-01T10:30:00Z",
  },
});

When this call lands on an Elasticsearch node, the following sequence occurs.

Step 1: Routing. The coordinating node computes hash(document_id) % number_of_primary_shards to determine which primary shard owns the document. The request is forwarded to the node holding that primary.

Step 2: Translog write. Before acknowledging the write, the primary appends the operation to its translog (transaction log), a sequential append-only file on disk. This is Elasticsearch’s durability guarantee. If the node crashes before a segment is flushed, the translog replays on startup to recover in-flight documents. The translog is fsync’d either on every write (index.translog.durability: request, the default) or asynchronously (async). Async gives higher write throughput at the cost of potentially losing up to sync_interval seconds of data on a hard crash.

Step 3: In-memory buffer. The document is written to an in-memory indexing buffer backed by a Lucene IndexWriter. Documents in this buffer are not yet searchable.

Step 4: Refresh. At the refresh interval (default 1 second), Elasticsearch calls IndexWriter.flush(), which writes the in-memory buffer to a new Lucene segment on disk and makes it visible to search. This is the origin of Elasticsearch’s “near real-time” search claim: writes are visible within roughly one second, not immediately. You can force a refresh with ?refresh=true on the index call, but doing this on every write destroys write throughput by preventing batch segment creation.

Step 5: Flush. Periodically (or when the translog reaches a size threshold), Elasticsearch calls a full Lucene commit, which writes a new segment to disk and truncates the translog up to that point. After a flush, the segment is durable and the translog entries before it are no longer needed for recovery.

Step 6: Replica forwarding. In parallel with the translog write, the primary forwards the operation to all replica shards. The write is acknowledged to the client only after the configured wait_for_active_shards replicas confirm success.

Merge Policy

Small segments accumulate with each refresh cycle. Lucene’s TieredMergePolicy (the default) groups segments into tiers by size and triggers background merges that combine smaller segments into larger ones. This reduces segment count, physically removes deleted documents, and keeps per-query segment fan-out bounded.

Merge I/O competes with query I/O. During bulk indexing, merge threads can saturate disk bandwidth and slow search. index.merge.scheduler.max_thread_count controls parallelism. On SSDs, increasing it is usually safe. On spinning disks, a single merge thread is safer. The merge policy’s max_merged_segment setting caps how large a merged segment grows, and segments_per_tier controls how aggressively merges trigger.

// Force a merge (expensive: blocks caller until complete)
// Use only for read-only indices (e.g., after a time window closes)
await client.indices.forcemerge({
  index: "events-2026-05",
  max_num_segments: 1,
});

Force-merging a finalized time-based index to a single segment is one of the most impactful optimizations for read-heavy historical data: it eliminates inter-segment overhead entirely and allows maximum OS page cache efficiency.

Distributed Search: Query-Then-Fetch

A search request reaching a cluster follows the query-then-fetch protocol, a two-phase process designed to produce globally ranked results from distributed data.

const result = await client.search({
  index: "events",
  query: {
    bool: {
      must: { match: { action: "checkout" } },
      filter: { range: { amount: { gte: 100 } } },
    },
  },
  sort: [{ timestamp: "desc" }],
  size: 10,
});

Phase 1: Query phase. The coordinating node broadcasts the query to one copy (primary or replica) of every shard in the index. Each shard executes the query locally against its Lucene index, scores matching documents using BM25, and returns a lightweight result set: the top from + size document IDs and their scores. It does not return source fields. The coordinating node collects these partial result sets, merges them into a globally sorted list, and identifies the final top size document IDs.

Phase 2: Fetch phase. The coordinating node sends a multi-get request to each shard that owns documents in the final result set. Each shard fetches the stored fields or _source for those specific IDs and returns them. The coordinating node assembles the final response.

This design keeps network transfer in Phase 1 minimal (IDs and scores only) at the cost of a second round trip. The tradeoff matters when from is large: deep pagination requires every shard to compute and return from + size candidates, so a request for page 1000 with size: 10 forces each shard to score and sort 10,010 documents locally. This is why search_after pagination with a sort tiebreaker exists: it avoids the from overhead entirely.

BM25 Scoring

Elasticsearch uses Okapi BM25 as its default relevance model. BM25 scores a document for a query term as a function of:

• TF (term frequency): How often the term appears in the document, with diminishing returns controlled by the k1 parameter (default 1.2). A document with the term 100 times scores only slightly higher than one with 20 times.
• IDF (inverse document frequency): How rare the term is across the index. Terms appearing in 90% of documents contribute little signal.
• Field length normalization: Shorter fields get a boost. A match in a 3-word title is more significant than the same match in a 500-word body. Controlled by the b parameter (default 0.75).

BM25 scoring is per-shard. Each shard computes IDF from its own document set. On a cluster with many small shards, IDF can diverge between shards and produce unexpected ranking where a rare term appears frequently in one shard but not others. The fix is search_type=dfs_query_then_fetch, which adds a pre-phase that collects global term frequencies before scoring. It adds a round trip but produces consistent scores regardless of shard distribution.

Shard Routing and Custom Routing

By default, Elasticsearch routes documents round-robin by ID hash across primary shards. Custom routing pins documents to a specific shard, which can dramatically reduce scatter-gather fan-out for tenant-isolated workloads.

// Index with custom routing so all events for tenant "acme" land on the same shard
await client.index({
  index: "events",
  id: "evt-002",
  routing: "acme",
  document: { tenantId: "acme", action: "login", timestamp: "2026-06-01T11:00:00Z" },
});

// Query with routing: Elasticsearch skips all other shards
const result = await client.search({
  index: "events",
  routing: "acme",
  query: { term: { tenantId: { value: "acme" } } },
});

Custom routing trades flexibility for efficiency. If routing values are skewed (one tenant has 80% of documents), the target shard becomes a hotspot.

Cluster Coordination: Master Nodes, Shard Allocation, and Rebalancing

Master Election

Elasticsearch uses a custom consensus protocol called Raft-adjacent coordination (not stock Raft) implemented via the cluster.initial_master_nodes quorum bootstrap. The elected master is the sole writer of cluster state: the global metadata including index mappings, shard assignments, and node membership. All other nodes receive cluster state updates from the master.

Master-eligible nodes form a quorum. For a three-node cluster, two must be available to elect a master and accept cluster state writes. Split-brain is prevented by requiring a quorum: with minimum_master_nodes (pre-7.x) or the newer voting configuration mechanism, a minority partition cannot form a second master.

Shard Allocation

When an index is created with number_of_shards: 5 and number_of_replicas: 1, the master allocates 10 shard copies (5 primaries, 5 replicas) across available nodes. Allocation respects hard constraints first: a primary and its replica cannot share a node (single-node failure would lose both). Then it applies soft balancing preferences: spread shards evenly by count and by total index size to prevent any node from becoming a storage hotspot.

When a node leaves the cluster, its shards become unassigned. The master triggers a recovery: replica copies on surviving nodes are promoted to primary, and new replicas are allocated elsewhere. If no replica exists (replica count was 0), the primary is unassigned and that shard’s data is unavailable until the node returns.

Rebalancing fires automatically when nodes join or leave and is throttled by cluster.routing.allocation.node_concurrent_recoveries to avoid saturating network and disk with simultaneous shard movements.

Production Considerations

Shard Sizing

The JVM heap required for a node scales with the number of shards it holds, not primarily with data volume. Each shard carries per-shard memory overhead for segment metadata, field caches, and the query cache. Elasticsearch recommends keeping shard count below 20 per GB of heap and targeting shard sizes between 10 GB and 50 GB for most workloads. Hundreds of tiny shards on a node are a common cause of slow search and GC pressure. If an index has accumulated too many shards via daily time-based splitting, the Rollup or ILM shrink actions can reduce shard count without reindexing.

// Check shard count and sizes via cat shards API
const shards = await client.cat.shards({
  index: "events-*",
  h: ["index", "shard", "prirep", "state", "docs", "store", "node"],
  format: "json",
});

Mapping Explosion

Dynamic mapping creates new field mappings automatically. In log pipelines where payloads contain unpredictable JSON keys, this can produce indices with thousands of fields. Each field consumes heap for its field data structures and contributes to cluster state size, which is held in memory on every node. The mitigations are dynamic: strict to reject unknown fields, dynamic: false to index but not make unknown fields searchable, or explicit mappings with dynamic_templates for predictable field patterns.

Slow Query Diagnosis

Elasticsearch has a built-in slow log that records queries exceeding configurable thresholds.

await client.indices.putSettings({
  index: "events",
  settings: {
    "index.search.slowlog.threshold.query.warn": "2s",
    "index.search.slowlog.threshold.query.info": "500ms",
    "index.search.slowlog.threshold.fetch.warn": "1s",
  },
});

Slow queries are usually caused by one of: unbounded wildcard queries that scan the entire term dictionary, high-cardinality terms aggregations loading large field data into heap, deep pagination with large from values, or _source filtering on indices without stored fields where Elasticsearch must decompress and parse the full source JSON to extract a subset.

The Profile API provides per-query timing breakdowns showing which query clauses and which aggregation buckets consume time.

const profileResult = await client.search({
  index: "events",
  profile: true,
  query: { match: { action: "checkout" } },
});

// profileResult.profile.shards[N].searches[0].query shows per-clause timing

Hot-Warm-Cold Tiering

Index Lifecycle Management (ILM) automates shard migration across node tiers based on age or size. A typical configuration keeps the current day’s index on hot nodes (NVMe SSDs, high CPU), moves yesterday’s indices to warm nodes (SATA SSDs, lower CPU), and archives older indices to cold nodes (spinning disks or object storage via searchable snapshots).

await client.ilm.putLifecycle({
  name: "events-policy",
  policy: {
    phases: {
      hot: {
        min_age: "0ms",
        actions: { rollover: { max_primary_shard_size: "30gb", max_age: "1d" } },
      },
      warm: {
        min_age: "3d",
        actions: { shrink: { number_of_shards: 1 }, forcemerge: { max_num_segments: 1 } },
      },
      cold: {
        min_age: "30d",
        actions: { searchable_snapshot: { snapshot_repository: "s3-archive" } },
      },
      delete: { min_age: "365d", actions: { delete: {} } },
    },
  },
});

The warm phase shrink and force-merge combination is particularly effective: shrink reduces shard count on warm nodes, and the force-merge to one segment removes the last deleted documents and collapses segment overhead entirely before the data goes cold.

Tradeoffs

Dimension	Elasticsearch	OpenSearch	Typesense	Meilisearch	PostgreSQL FTS
Query language	Full JSON DSL, aggregations pipeline	Elasticsearch-compatible DSL	Simple JSON params	Simple JSON params	SQL with tsvector / tsquery
Relevance model	BM25 (configurable, LTR via plugin)	BM25	BM25 with typo tolerance	BM25 with typo tolerance	BM25 via ts_rank
Horizontal scaling	Native: shards and replicas	Native (fork of ES)	Single-node (clustering in beta)	Single-node (experimental clustering)	Via Citus or manual sharding
Write throughput	High with async refresh; lower with refresh=true	Same as ES	High	Moderate (batch indexed)	Depends on index count and tsvector trigger overhead
Aggregations	Rich: metrics, buckets, pipelines, composite	Same as ES	Basic faceting	Basic faceting	Limited via SQL GROUP BY
Operational complexity	High: JVM tuning, shard sizing, ILM, mapping management	Same as ES	Low	Low	Low (runs in existing Postgres)
Fuzzy search	Yes, configurable fuzziness	Yes	Yes, built-in typo tolerance	Yes, built-in typo tolerance	Limited (trigram indexes)
Vector search	kNN via HNSW (8.x+)	k-NN plugin	Yes (built-in)	Yes (built-in)	pgvector extension
License	Server Side Public License (SSPL)	Apache 2.0	Apache 2.0	MIT	PostgreSQL License
Sweet spot	Log analytics, large-scale search with complex aggregations	ES workloads requiring open-source licensing	App search with fast iteration	Developer-friendly search with typo tolerance	Small-medium datasets already on Postgres

Elasticsearch’s complexity is justified when you need its aggregations pipeline, when data volume exceeds what a single-node search engine can hold, or when you need ILM to manage time-series indices automatically. For applications that just need relevance-ranked full-text search over a manageable dataset, Typesense or Meilisearch impose a fraction of the operational burden with comparable end-user quality. PostgreSQL full-text search is often underestimated: tsvector with a GIN index handles millions of rows acceptably and requires no additional infrastructure.

The Lucene segment model, the translog-backed write path, and the query-then-fetch distribution protocol are not arbitrary design choices. Each solves a specific constraint: immutable segments enable lock-free reads, the translog decouples durability from write amplification, and two-phase search keeps network transfer proportional to result size rather than corpus size. When something goes wrong in a production Elasticsearch cluster, the failure mode almost always traces back to one of these three mechanisms being pushed past its designed operating range.