How Elasticsearch Works: Inverted Indexes, Segment Merges, and Distributed Search From Query to Hit

Most engineers reach for Elasticsearch because search is hard and it ships a working solution fast. The trouble starts when the cluster misbehaves: queries that should be fast take seconds, indexing throughput falls off a cliff, or a node leaves the cluster and suddenly you’re staring at red health. Understanding what Elasticsearch is actually doing internally is the difference between cargo-culting settings and reasoning about them.

This article works through the full stack: the Lucene layer underneath, how writes flow from memory to disk, how a query travels across shards and comes back as ranked hits, and what production clusters actually require to stay healthy.

The Lucene Foundation

Elasticsearch is a distribution layer over Apache Lucene. Each Elasticsearch shard is a self-contained Lucene index. Understanding what Lucene gives you is the starting point.

Inverted Indexes and Term Dictionaries

A forward index maps documents to the words they contain. An inverted index flips this: it maps each unique term to the list of documents containing that term. Lucene stores this as two tightly related structures: the term dictionary and the postings list.

The term dictionary is a sorted, compressed structure of every unique term across all documents in a segment. It supports fast lookup by keeping a small in-memory FST (finite state transducer) that points into the on-disk term dictionary. When you search for "latency", Lucene seeks into this dictionary and retrieves a pointer to the postings list for that term.

The postings list contains the sorted list of document IDs where the term appears, plus additional per-document data depending on index configuration: term frequency (how many times the term appears in that document), positions (the ordinal position of each occurrence, for phrase queries), and offsets (byte positions, for highlighting). Postings lists are delta-encoded and compressed, typically with FOR (Frame Of Reference) or VByte encoding since doc IDs are integers and delta compression is effective on sorted lists.

Doc Values for Columnar Access

Keyword aggregations, sorting, and numeric range filters operate over doc values, not the inverted index. Doc values are a columnar, on-disk representation of field values, stored per-segment alongside the postings. Where the inverted index is term-to-doc, doc values are doc-to-value. Reading field values for sorting or aggregating 10,000 matched documents out of the inverted index would require random lookups for every document. Doc values make this sequential.

Doc values are enabled by default for keyword, numeric, date, and ip fields. They are disabled for text fields (which use the inverted index for full-text search) and can be disabled on keyword fields if you never need sorting or aggregations on that field.

The Segment-Based Storage Model

Lucene writes are immutable. You cannot update a document in place. This design choice has far-reaching consequences.

Write Path

When you index a document in Elasticsearch, it does not go directly to disk. The write path has several stages:

1. The document is appended to the translog (write-ahead log) and simultaneously written to the in-memory index buffer.
2. Periodically (controlled by index.refresh_interval, default 1 second), Elasticsearch calls a Lucene refresh. This flushes the in-memory buffer into a new segment in the Lucene directory. The segment is written as a set of files in the OS filesystem cache but not fsync’d to disk yet. After a refresh, documents are visible to searches.
3. Periodically, a flush occurs: Lucene commits all segments to disk (fsync), and the translog is cleared because the data is now durably persisted.

This is what Elasticsearch means by “near-real-time” search. Documents are not immediately searchable after indexing (there is a ~1 second window), but they do not require a full commit to become searchable either.

// Rough illustration of what happens when you bulk index documents
const client = new Client({ node: "http://localhost:9200" });

const body = documents.flatMap((doc) => [
  { index: { _index: "products", _id: doc.id } },
  doc,
]);

// This write is acknowledged after translog append + index buffer write.
// The documents are NOT yet searchable. They become searchable after
// the next refresh (default: 1 second).
const response = await client.bulk({ refresh: false, body });

// If you need immediate visibility:
// refresh: "wait_for" blocks until the next refresh cycle completes.
// refresh: true forces an immediate refresh (expensive; avoid on hot paths).

Segments and Immutability

Each Lucene commit produces a set of segment files. A segment is immutable once written. Deletes are recorded as a separate bitset (.del file), not as in-place removals. Updates are a delete plus a new insert. Segment files are never modified after creation.

This immutability is what makes Lucene’s OS cache utilization excellent: the kernel can keep segment files in the page cache indefinitely because they never change. It also simplifies crash recovery significantly.

The cost is segment proliferation. Each refresh creates a new segment. Without merging, a busy index accumulates thousands of tiny segments, and search has to open and query all of them in parallel, which is expensive.

Segment Merges and Merge Policies

Lucene solves segment proliferation with background merges. The merge process selects a set of small segments, combines them into a single larger segment, then deletes the originals. Merged segments compact away deleted documents, actually reclaiming disk space.

The default merge policy is TieredMergePolicy. It groups segments into size tiers and triggers a merge when a tier has more than max_merge_at_once (default: 10) segments. Tunable parameters include segments_per_tier (how many segments per tier are acceptable) and max_merged_segment (the size cap for a single merged segment, default 5GB).

Merges run in background threads. On write-heavy workloads, merges can saturate disk IO. You can limit merge throughput with indices.store.throttle.max_bytes_per_sec in older Elasticsearch versions, or via thread pool sizing in newer ones.

forcemerge is a manual API call that reduces an index to a target number of segments. It is a legitimate tool for read-only indexes (time-series data that has rolled over, archived indices) where you want to minimize segment count for query performance. Calling forcemerge on a live, actively-written index is almost always a mistake: it competes with write IO, produces large segments that will need to be re-merged when new data arrives, and can cause long GC pauses if the merged segment is very large.

Distributed Architecture

Shards as Independent Lucene Indexes

A single Elasticsearch index is split into N primary shards. Each shard is an independent Lucene index. When you index a document, Elasticsearch routes it to a shard using: shard = hash(_routing) % number_of_shards. The _routing field defaults to _id.

The number of primary shards is fixed at index creation time and cannot be changed without reindexing. This constraint shapes shard sizing strategy more than almost anything else. If you create a 1-shard index and it grows to 500GB, your options are reindex or accept the degraded query performance.

Primary-Replica Model

Each primary shard has zero or more replica shards. Replicas serve two purposes: redundancy (a node failure does not lose data) and read throughput (search requests can be served from any replica).

Writes always go to the primary. The primary validates, indexes, and then forwards to all replicas. The index operation is acknowledged to the client only after the replicas have confirmed the write, unless you configure wait_for_active_shards otherwise.

Shard Allocation and Master Election

Elasticsearch uses a dedicated master node (or master-eligible nodes) to manage cluster state: shard allocation, index creation, node join/leave events. The master does not handle search or indexing traffic; it manages metadata.

Split-brain prevention is handled by quorum. With master-eligible nodes M, a quorum requires floor(M/2) + 1 votes to elect a master. This is configured via discovery.zen.minimum_master_nodes in older versions, and is handled automatically by the Zen2 algorithm in Elasticsearch 7+. Zen2 eliminates the need to set minimum_master_nodes manually by implementing a proper distributed consensus protocol.

# elasticsearch.yml: cluster settings for a 3-node master-eligible cluster
cluster.name: production-cluster
node.master: true
node.data: true
discovery.seed_hosts:
  - es-node-1:9300
  - es-node-2:9300
  - es-node-3:9300
cluster.initial_master_nodes:
  - es-node-1
  - es-node-2
  - es-node-3

With three master-eligible nodes, the quorum is 2. Losing one node does not affect master election. Losing two nodes causes the cluster to enter a read-only state to prevent split-brain.

Search Execution Path

Coordinating Node Scatter-Gather

When a search request arrives at any Elasticsearch node, that node becomes the coordinating node for that request. It does not necessarily hold any of the relevant data itself.

Search proceeds in two phases: the query phase and the fetch phase.

Query phase: The coordinating node broadcasts the query to one shard copy (primary or replica) from each shard group in the target index. Each shard executes the query locally against its Lucene index and returns a lightweight result set: document IDs and scores for the top-N hits. The coordinating node merges these N shard result sets, re-ranks by score, and determines the globally top-N document IDs.

Fetch phase: The coordinating node issues targeted fetch requests to the specific shards holding the globally top-N documents. Each shard returns the full _source for its requested documents. The coordinating node assembles the final response.

This two-phase design minimizes network transfer: the query phase only moves IDs and scores, not full documents. A search requesting the top 10 results across 10 shards transfers at most 10×10 = 100 (ID, score) pairs in the query phase, then fetches exactly 10 full documents in the fetch phase.

Distributed Scoring and DFS Query-Then-Fetch

BM25 scoring uses IDF (inverse document frequency) as a component. IDF measures how rare a term is across the corpus. In a distributed index, each shard computes IDF using only the documents in that shard, not the global corpus. For large, evenly distributed indexes this approximation is acceptable: shard-local term frequencies roughly reflect global frequencies. For small indexes or skewed sharding, this can produce surprising ranking.

dfs_query_then_fetch solves this by adding a pre-phase: before the query phase, the coordinating node collects term and field statistics from all shards, computes global IDF, and passes those global statistics to each shard during the query phase. This improves scoring accuracy at the cost of an extra round-trip per query. Use it when result relevance quality matters more than raw query latency, typically for small indexes or testing.

Relevance Scoring: BM25

Elasticsearch uses BM25 as its default similarity function (replacing TF-IDF in version 5.0). BM25 scores a document D for query term T as:

score(D, T) = IDF(T) * (TF(D,T) * (k1 + 1)) / (TF(D,T) + k1 * (1 - b + b * |D| / avgdl))

Where:

• IDF(T) penalizes common terms and rewards rare ones
• TF(D,T) is the term frequency in document D
• k1 controls term frequency saturation (default 1.2): high k1 means more TF weight, low k1 flattens TF impact
• b controls field-length normalization (default 0.75): b=1.0 is full normalization (long documents penalized), b=0.0 disables length normalization
• |D| is the document field length in tokens
• avgdl is the average field length across all documents in the index

You can tune k1 and b per-field in the index mapping. For title fields where length is consistently short, setting b=0.0 avoids punishing slightly longer titles.

// Custom BM25 similarity per field
const indexSettings = {
  settings: {
    similarity: {
      title_similarity: {
        type: "BM25",
        k1: 1.5,
        b: 0.2,
      },
    },
  },
  mappings: {
    properties: {
      title: {
        type: "text",
        similarity: "title_similarity",
      },
      body: {
        type: "text",
        // Uses default BM25 (k1=1.2, b=0.75)
      },
    },
  },
};

Near-Real-Time Search

The index.refresh_interval setting controls how often Elasticsearch calls refresh on each index. The default is 1s. Setting it to -1 disables automatic refresh entirely, which is useful during bulk initial indexing: no refresh means no segment creation until you call refresh explicitly, which means no merge overhead during the bulk load.

The translog is the durability guarantee. Between flushes, if a node crashes, Elasticsearch replays the translog on restart to recover documents that were in the index buffer but not yet committed to a Lucene segment. The translog itself is fsync’d by default after every operation (index.translog.durability: request), which is safe but slow. Setting index.translog.durability: async with index.translog.sync_interval: 5s trades durability for throughput: up to 5 seconds of data is at risk on a hard crash.

// Disable refresh for bulk initial load, then re-enable
await client.indices.putSettings({
  index: "products",
  body: {
    "index.refresh_interval": "-1",
    "index.number_of_replicas": "0", // Also disable replicas during bulk load
  },
});

// ... bulk index all documents ...

await client.indices.putSettings({
  index: "products",
  body: {
    "index.refresh_interval": "1s",
    "index.number_of_replicas": "1",
  },
});

await client.indices.refresh({ index: "products" });
await client.indices.forcemerge({ index: "products", max_num_segments: 1 });

Cluster Scaling Patterns

Hot-Warm-Cold Architecture

Time-series workloads (logs, metrics, events) have a clear data access pattern: recent data is queried frequently, older data rarely. Hot-warm-cold tiering maps data lifecycle to node characteristics.

Hot nodes are write-heavy: SSDs, high CPU, high memory. They hold the current and recent indices.

Warm nodes handle infrequent reads: larger HDDs, lower cost per GB. Historical indices are moved here after the active write window closes.

Cold nodes (Elasticsearch 7.8+) hold read-only, searchable snapshots stored in object storage (S3, GCS). They serve queries with higher latency than warm nodes but at a fraction of the cost. Cold tier data is not kept on local disk between searches.

Node tiers are assigned via node.attr.data or the node.roles setting in modern Elasticsearch.

Index Lifecycle Management

ILM automates the hot-warm-cold transition. A policy defines phases (hot, warm, cold, delete), transitions (when to move: after age or size), and actions per phase (rollover, shrink, force-merge, freeze, delete).

// ILM policy for logs
const ilmPolicy = {
  policy: {
    phases: {
      hot: {
        min_age: "0ms",
        actions: {
          rollover: {
            max_age: "1d",
            max_primary_shard_size: "50gb",
          },
        },
      },
      warm: {
        min_age: "3d",
        actions: {
          shrink: { number_of_shards: 1 },
          forcemerge: { max_num_segments: 1 },
        },
      },
      cold: {
        min_age: "30d",
        actions: {
          searchable_snapshot: {
            snapshot_repository: "s3-archive",
          },
        },
      },
      delete: {
        min_age: "365d",
        actions: { delete: {} },
      },
    },
  },
};

Rollover for Time-Series Data

Instead of creating indices by calendar date (a common anti-pattern that produces unpredictably sized shards), use rollover: start with an alias pointing to a single index, and roll over to a new index when the current one crosses a size or age threshold. This decouples your index rotation from the clock, keeps shards in the 10-50GB target range, and plays well with ILM.

Tradeoffs Table

Dimension	Elasticsearch	OpenSearch	Meilisearch	Typesense	PostgreSQL FTS
Indexing throughput	Very high (distributed)	Very high (same core)	High (single-node)	High (single-node)	Moderate
Query latency (p50)	Low (10-50ms at scale)	Low (same)	Very low (<10ms, small data)	Very low (<10ms)	High (seconds on large data)
Operational complexity	High: cluster management, JVM tuning, shard sizing	High (same)	Low: single binary	Low: single binary	Low: already have Postgres
Relevance tuning	Full (BM25, custom similarity, boosting, rescoring)	Full (same)	Limited (typo-tolerance, ranking rules)	Limited (ranking rules)	Limited (ts_rank, weights)
Schema flexibility	High (dynamic mapping, but watch for explosions)	High	Moderate	Moderate	Fixed schema
Horizontal scaling	First-class: shards, replicas, cross-cluster	First-class (same)	Limited (v1 multi-node in progress)	Moderate (Raft-based clustering)	Hard: requires Citus or manual partitioning
Cost at scale	High: JVM memory, SSD per hot node	Same as ES	Low until multi-node	Low until large dataset	Low if already operational

The most common mistake is using Elasticsearch for a dataset that would be fine with PostgreSQL tsvector. If your data fits on one server, you need prefix/fuzzy matching, and your query volume is under a few hundred per second, PostgreSQL FTS covers this and eliminates an entire operational dependency.

Production Considerations

Shard sizing. Target 10-50GB per shard. Under 1GB shards have high overhead per shard (each shard has a JVM heap footprint, file descriptor allocation, and merge thread participation). Over 50GB shards slow recovery time after a node failure and push JVM heap requirements up. For time-series data, use ILM rollover to hit this range automatically.

Mapping explosions. Dynamic mapping is convenient and dangerous. If your documents contain user-supplied JSON with arbitrary keys, Elasticsearch will create a new field mapping for every unique key it sees. Tens of thousands of field mappings cause cluster state to balloon (it is stored in-memory on every node), mapping updates to slow, and eventually the cluster to become unstable. Use dynamic: strict on production indexes and define all fields explicitly. For arbitrary JSON payloads, map the field as object with enabled: false to store it without indexing.

Circuit breakers. Elasticsearch has a set of circuit breakers that prevent operations from taking the JVM heap over the configured limit. The field data circuit breaker is the one that catches you first: loading text field doc values (fielddata) for aggregations can consume massive heap. Keep indices.fielddata.cache.size bounded (default is 40% of heap). If a query would breach the circuit, it throws a CircuitBreakingException rather than crashing the node. Monitor /_nodes/stats/breaker to see if breakers are tripping frequently.

Slow query diagnosis. Enable the slow log for queries that exceed a threshold:

PUT /my-index/_settings
{
  "index.search.slowlog.threshold.query.warn": "10s",
  "index.search.slowlog.threshold.query.info": "1s",
  "index.search.slowlog.threshold.fetch.warn": "2s"
}

Combine slow logs with profile: true on suspect queries. The profile API returns per-shard execution trees showing exactly which part of the query (term lookup, scoring, aggregation) is consuming time.

JVM heap sizing. Allocate no more than 50% of available RAM to the JVM heap, with a hard cap at 31GB. Beyond 31GB, the JVM switches from compressed ordinary object pointers to full 64-bit pointers, which roughly doubles object overhead. The other 50% of RAM goes to the OS filesystem cache, which Lucene uses heavily for segment file access. An Elasticsearch node with 64GB RAM should have 31GB JVM heap and 33GB available to the OS cache.

---

Elasticsearch rewards operators who understand its internals. The inverted index and BM25 layer explain why relevance behaves the way it does. The segment model explains why write amplification and merge IO exist. The scatter-gather search path explains why shard count affects query latency even at small data sizes. Most production incidents trace back to one of three root causes: wrong shard sizing at index creation, dynamic mapping left on for user-provided data, or JVM heap misconfiguration. Getting those three right before the first document is indexed removes the majority of failure modes.