How Amazon S3 Works Internally: Object Storage Architecture, Request Routing, and Durability Engineering at Planetary Scale

Amazon S3 stores over 350 trillion objects and handles millions of requests per second. Most engineers interact with it through a thin SDK wrapper and treat it as an infinite key-value store that never loses data. That mental model works until you need to reason about request latency, consistency guarantees, cost anomalies, or failure modes. Understanding what actually happens between a PutObject call and the acknowledgment changes how you design systems that depend on it.

The Flat Namespace and Bucket-Key Addressing

S3 exposes a flat namespace. Every object lives at a globally unique address formed by combining an account-scoped bucket name with an arbitrary key string. There are no real directories. The / character in keys is a UI convention that the S3 console uses to simulate folder hierarchy, but the storage layer sees logs/2026/06/app.log and README.md as equally flat entries.

This flatness is intentional. Hierarchical filesystems require locking on intermediate nodes during writes, which becomes a serialization bottleneck at scale. A flat namespace lets S3 distribute objects across an enormous number of independent storage nodes with no shared coordination on the write path.

Bucket names must be globally unique across all AWS accounts, which is how S3 can use them in virtual-hosted-style URLs (bucket.s3.amazonaws.com) and route requests without consulting per-account metadata on every call. The bucket is also the unit of configuration: policies, versioning, replication rules, and lifecycle rules all attach to the bucket, not to individual keys.

Request Routing: The Front-End Fleet and the Partition Index

When your application calls GetObject, the request hits one of S3’s load-balancing front-end nodes. These nodes do not store data. Their job is authentication, authorization (checking bucket policies and IAM), request parsing, and routing to the correct storage node.

The routing decision relies on a service S3 calls the index layer. The index maps each (bucket, key) tuple to a specific partition, and each partition to a set of storage nodes in one or more Availability Zones. The index itself is a distributed B-tree-like structure partitioned by prefix hash, which is why S3 historically recommended randomizing key prefixes: sequential prefixes like timestamps land in the same index partition and create hot spots. As of 2018, S3’s index automatically scales hot partitions, so the prefix-randomization advice is less urgent today, but the underlying mechanic is unchanged.

The front-end node queries the index, gets the list of storage nodes holding the object’s erasure-coded chunks, and coordinates the read or write. For reads, it fetches enough chunks to reconstruct the object. For writes, it streams the object to storage nodes and waits for a quorum before acknowledging the client.

import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const s3 = new S3Client({ region: "us-east-1" });

// Standard PUT: S3 acknowledges only after the quorum write completes.
// The 200 OK you receive means durability is guaranteed from that moment forward.
async function uploadObject(bucket: string, key: string, body: Buffer): Promise<void> {
  await s3.send(
    new PutObjectCommand({
      Bucket: bucket,
      Key: key,
      Body: body,
      ContentType: "application/octet-stream",
      // Server-side encryption with a customer-managed KMS key
      ServerSideEncryption: "aws:kms",
      SSEKMSKeyId: process.env.KMS_KEY_ID,
    })
  );
}

// Pre-signed URL: the signing happens client-side; credentials never leave your server.
async function presignGet(bucket: string, key: string, expiresIn = 3600): Promise<string> {
  return getSignedUrl(s3, new GetObjectCommand({ Bucket: bucket, Key: key }), { expiresIn });
}

Durability Engineering: Erasure Coding Across Availability Zones

S3 Standard promises 99.999999999% (eleven nines) annual durability. This is not achieved through replication alone. S3 uses erasure coding, specifically a Reed-Solomon variant, to distribute an object across storage nodes spread across at least three Availability Zones within a region.

The mechanics: an object is split into k data chunks and m parity chunks. Any k of the k + m total chunks are sufficient to reconstruct the original object. A common configuration uses something in the range of 6 data chunks and 2 or 3 parity chunks, though Amazon has not published the exact parameters. The critical property is that losing an entire AZ, which represents losing a correlated subset of chunks, still leaves enough surviving chunks for reconstruction.

This is fundamentally different from replication. A 3x replication scheme uses 3x storage overhead. An erasure coding scheme with 6 data and 2 parity chunks uses 1.33x overhead while tolerating two simultaneous chunk failures anywhere in the pool. At S3’s scale, the storage savings are enormous, and the durability is higher because you need to lose m + 1 independent chunks before data is unrecoverable.

The trade-off is read amplification. Reconstructing an object requires fetching k chunks and running the decode computation. S3 hides this latency for small objects by keeping recently accessed hot objects in a separate cache tier. For large objects, the parallel chunk fetches across AZs can actually reduce latency compared to a single sequential read from one storage node.

When a storage node fails, S3’s background repair process detects the missing chunks via the index layer, fetches the surviving chunks, recomputes the missing ones, and writes them to a replacement node. The target is to repair degraded chunks faster than the probability of a second failure makes data loss likely.

Consistency Model: From Eventual to Strong

Before December 2020, S3 offered read-after-write consistency only for new object PUTs, and eventual consistency for overwrites and DELETEs. This meant a PutObject followed immediately by a GetObject on the same key could return the old version or a 404, depending on which replica you hit.

Amazon shipped strong consistency across all S3 operations in December 2020, with no performance or cost penalty. The mechanism is an upgrade to the index layer: writes now update the index synchronously as part of the quorum write, and all subsequent reads consult the same strongly consistent index before locating storage nodes. The index acts as the linearization point.

The practical consequence is that you can now build read-your-writes workflows without defensive retries, use S3 as a coordination mechanism for distributed jobs (read a manifest, process, write a completion marker), and trust that a ListObjectsV2 call after a batch of writes reflects the complete batch. Applications that were written with eventual consistency workarounds can simplify their retry logic.

One subtlety remains: versioning. If a bucket has versioning enabled and you PUT an object, you get a new version ID. If you immediately GET without specifying the version ID, you get the latest version, which is strongly consistent. If another process PUTs between your write and your GET, you may get their version. This is not a consistency violation; it is concurrent writers and the application needs to handle it with version IDs or conditional writes.

Storage Class Tiering: What Happens to Your Bits

S3 exposes several storage classes that trade access latency and cost:

S3 Standard keeps all erasure-coded chunks in high-performance spinning or flash storage across three AZs. First-byte latency is in the single-digit milliseconds.

S3 Standard-IA (Infrequent Access) uses the same erasure coding and AZ spread as Standard but stores chunks on lower-cost storage and charges a per-GB retrieval fee. The infrastructure is identical; the pricing model is different.

S3 Glacier Instant Retrieval stores objects in archival storage but maintains the metadata in the hot index layer. Retrieval is milliseconds, the same as Standard-IA, but storage cost drops another step. Objects go through a background tiering pipeline that moves the actual bytes off fast storage while keeping a pointer in the index.

S3 Glacier Flexible Retrieval and Deep Archive break the millisecond retrieval contract. These tiers write objects to tape or cold spinning disk systems, and the index entry marks them as archived. A restore operation triggers a background job to move the bytes back to a hot storage tier before reads are served. Restore times range from minutes (Expedited) to 12 hours (Standard) to 48 hours (Deep Archive bulk). You pay for restore bandwidth and staging storage on top of the archival storage rate.

S3 Express One Zone launched in 2023 as a high-performance single-AZ tier. It uses a purpose-built data plane separate from the multi-AZ S3 data plane, stores objects in a single AZ with a different bucket type called a directory bucket, and targets sub-10-millisecond first-byte latency. The trade-off is durability: losing the AZ means losing access to the data until the AZ recovers. It is designed for latency-sensitive workloads where durability can be provided at the application layer (re-derivable data, replicated caches, ML training scratch space).

Multipart Upload: The Protocol Behind Large Object Writes

S3 limits a single PutObject request to 5 GB. For larger objects, or for parallel upload throughput, you use the multipart upload protocol.

The protocol is a three-phase state machine. You initiate an upload and get back an UploadId. You upload parts independently in any order; each part must be at least 5 MB except the final one. Each successful part upload returns an ETag. You complete the upload by sending the UploadId and the ordered list of (PartNumber, ETag) pairs. S3 assembles the parts into a single object on the storage layer, updating the index to point to the assembled object.

import {
  CreateMultipartUploadCommand,
  UploadPartCommand,
  CompleteMultipartUploadCommand,
  CompletedPart,
} from "@aws-sdk/client-s3";

const PART_SIZE = 10 * 1024 * 1024; // 10 MB

async function multipartUpload(
  bucket: string,
  key: string,
  data: Buffer
): Promise<void> {
  const { UploadId } = await s3.send(
    new CreateMultipartUploadCommand({ Bucket: bucket, Key: key })
  );

  const parts: CompletedPart[] = [];
  const totalParts = Math.ceil(data.length / PART_SIZE);

  // Upload parts in parallel, bounded by a concurrency limit in production.
  await Promise.all(
    Array.from({ length: totalParts }, async (_, i) => {
      const start = i * PART_SIZE;
      const partBody = data.subarray(start, start + PART_SIZE);

      const { ETag } = await s3.send(
        new UploadPartCommand({
          Bucket: bucket,
          Key: key,
          UploadId,
          PartNumber: i + 1,
          Body: partBody,
        })
      );

      parts.push({ PartNumber: i + 1, ETag: ETag! });
    })
  );

  // Sort parts by part number before completing.
  parts.sort((a, b) => a.PartNumber! - b.PartNumber!);

  await s3.send(
    new CompleteMultipartUploadCommand({
      Bucket: bucket,
      Key: key,
      UploadId,
      MultipartUpload: { Parts: parts },
    })
  );
}

Incomplete multipart uploads accumulate as invisible storage that you are billed for. The standard mitigation is a lifecycle rule on the bucket that aborts incomplete uploads after a configured number of days.

S3 Select: Pushdown Query Execution

S3 Select lets you push a SQL-like query into the storage layer and receive only the matching subset of an object, rather than downloading the full object and filtering client-side. It supports CSV, JSON, and Parquet objects, and the query runs on the storage node before the data leaves S3’s network.

The mechanism is closer to a scan filter than a full query engine. S3 Select does not build indexes or maintain statistics. It reads the object sequentially (or uses Parquet row-group metadata to skip blocks) and evaluates the WHERE clause on each record. The result is streamed back to the client. For Parquet, column pruning means S3 only reads the columns referenced in the SELECT list, which can dramatically reduce the bytes scanned.

import { SelectObjectContentCommand, ExpressionType } from "@aws-sdk/client-s3";
import { Readable } from "stream";

async function selectFromCSV(
  bucket: string,
  key: string,
  query: string
): Promise<string> {
  const response = await s3.send(
    new SelectObjectContentCommand({
      Bucket: bucket,
      Key: key,
      ExpressionType: ExpressionType.SQL,
      Expression: query,
      InputSerialization: {
        CSV: { FileHeaderInfo: "USE", FieldDelimiter: "," },
        CompressionType: "NONE",
      },
      OutputSerialization: { CSV: {} },
    })
  );

  const chunks: Buffer[] = [];
  for await (const event of response.Payload!) {
    if (event.Records?.Payload) {
      chunks.push(Buffer.from(event.Records.Payload));
    }
  }
  return Buffer.concat(chunks).toString("utf-8");
}

// Usage: pull only high-value events from a large CSV log file.
// selectFromCSV("my-bucket", "events/2026-06-03.csv", "SELECT * FROM S3Object WHERE value > 1000");

The practical use case is log analytics and ETL pre-filtering. If you store structured logs as gzip-compressed CSV or Parquet in S3, S3 Select can filter to 1% of records and return them in a fraction of the time and cost of a full download. It does not replace a dedicated query engine like Athena for complex joins or aggregations, but it eliminates unnecessary data transfer for simple filter operations.

Event Notification Architecture

S3 can publish events (object created, object deleted, replication failure) to SNS, SQS, Lambda, or EventBridge. The architecture is a fanout pipeline inside the S3 data plane.

When a write completes its quorum and the index is updated, the write path emits an event record to an internal journal. A delivery subsystem reads that journal, looks up the notification configuration for the bucket, and delivers the event to the configured destination. The delivery is at-least-once: if the destination is temporarily unavailable, S3 retries with exponential backoff, and you may receive duplicate events.

The key architectural constraint is that event delivery is asynchronous and decoupled from the write acknowledgment. The client receives a 200 OK for a PUT before the event has been delivered to SQS or Lambda. Systems that need to react to new objects should not assume event delivery is instantaneous and should be idempotent with respect to duplicate events.

For workflows requiring stronger guarantees, EventBridge S3 integration provides a richer filtering syntax (route only ObjectCreated events for keys matching reports/*.parquet) and integrates with EventBridge’s cross-account and archive-replay capabilities.

Production Considerations

Prefix distribution and request rate scaling. S3 scales request throughput per prefix: 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix. If your access pattern concentrates on one prefix (e.g., all objects under uploads/), you cap out at those limits. Design key schemas that spread load across multiple prefixes proportionally to your expected request rate.

Eventual consistency in cross-region replication. S3 replication (SRR and CRR) is asynchronous. The source bucket has strong consistency; the destination bucket reflects the object only after replication completes, which can take seconds to minutes depending on object size and region distance. Replication Time Control (RTC) offers a 15-minute SLA with replication metrics, but it is not synchronous.

ListObjectsV2 pagination at scale. Listing a bucket with millions of objects is expensive. ListObjectsV2 returns up to 1,000 keys per call and must be paginated using a continuation token. The list operation scans the index and is eventually consistent for concurrent deletes (a key deleted while you are paginating may or may not appear). For large-scale inventory workflows, S3 Inventory is cheaper and more reliable: it generates a daily or weekly CSV/ORC manifest of all objects in a bucket.

Conditional writes for distributed coordination. As of 2024, S3 supports If-None-Match: * on PutObject, enabling optimistic locking. Two processes racing to create the same key: one wins with a 200, the other gets a 412. This is a lightweight distributed mutex for small coordination payloads without a database.

Object Lambda for on-the-fly transformation. S3 Object Lambda intercepts GetObject requests and routes them through a Lambda function before returning the response to the client. The function can resize images, redact PII, or decompress archives transparently. The client sees a normal S3 URL and has no knowledge of the transformation layer.

Tradeoffs: S3 vs GCS vs Azure Blob vs R2 vs MinIO

Dimension	S3	GCS	Azure Blob	R2	MinIO
Consistency	Strong (since 2020)	Strong	Strong	Strong	Strong
Durability	11 nines, multi-AZ erasure coding	11 nines, multi-region option	11-16 nines depending on tier	11 nines (Cloudflare infra)	Operator-configured erasure coding
Egress cost	Per-GB charged to the caller	Per-GB to internet, free within GCP	Per-GB, first 5 GB/month free	Zero egress fees	Self-hosted, infrastructure cost only
Storage classes	Standard, IA, Glacier, Deep Archive, Express One Zone	Standard, Nearline, Coldline, Archive	Hot, Cool, Cold, Archive	Standard only	Single tier, plugins for tiering
Query pushdown	S3 Select (CSV, JSON, Parquet)	BigQuery Storage Read API (separate product)	None native	None	None
Event notifications	SNS, SQS, Lambda, EventBridge	Pub/Sub, Cloud Functions	Event Grid, Service Bus	Workers, Webhooks	Webhook, Kafka, NATS
Ecosystem integration	Broadest (every AWS service)	Native GCP	Native Azure	Cloudflare Workers ecosystem	Self-hosted, S3-compatible API
Operational overhead	Zero (fully managed)	Zero (fully managed)	Zero (fully managed)	Zero (fully managed)	High (you run the cluster)
Sweet spot	Default choice for AWS-native workloads	GCP-native analytics pipelines	Azure-native workloads	Edge-heavy workloads with zero egress tolerance	On-premise or air-gapped environments

R2’s zero egress fees make it compelling for high-read workloads like public media assets. MinIO’s S3-compatible API means migrations are a config change, but you absorb the operational burden of running a distributed storage cluster and replicating durability guarantees that AWS provides by default.

Closing

S3 looks simple because Amazon has spent two decades making it look that way. The flat namespace hides a partitioned index that routes billions of requests daily. The 200 OK on a PUT represents a completed quorum write across erasure-coded chunks in three AZs. Strong consistency rests on an index layer that serializes reads and writes through a single authoritative view. Understanding these mechanics does not change most day-to-day usage patterns, but it does change how you size workloads, design key schemas, handle replication lag, and reason about the cost model at the edges where simple abstractions break down.