System Design — Page 2 of 21
Browsing page 2 of 21 — 243 articles on system design.
How Amazon SQS Works Internally: Distributed Message Storage, Visibility Timeouts, and the Architecture Behind the Most Widely Deployed Message Queue
A deep dive into SQS internals covering distributed redundant message storage across availability zones, the visibility timeout mechanism and consumer semantics, standard vs FIFO queue architectures with deduplication and ordering, long polling mechanics, dead-letter queues, the full SendMessage-to-DeleteMessage lifecycle, batching APIs, delay queues, server-side encryption, and a tradeoffs comparison against Kafka, RabbitMQ, NATS JetStream, and Redpanda.
How Dragonfly Works Internally: Shared-Nothing Multi-Threading, Dash Hash Tables, and the Architecture That Outperforms Redis on a Single Node
A deep-dive into Dragonfly internals for senior engineers. Covers the io_uring event loop with shared-nothing thread-per-core architecture, the Dash hash table with lock-free concurrent access, mimalloc per-shard memory management, fork-free snapshot persistence via fiber serialization, multi-shard transaction coordination, and production tradeoffs vs Redis, Valkey, KeyDB, and Memcached.
How Redis Works Internally: Single-Threaded Event Loop, Data Structures, Persistence, and Replication
A deep-dive into Redis internals for senior engineers. Covers the single-threaded event loop with epoll/kqueue multiplexing, the actual in-memory data structures behind each Redis type, RDB and AOF persistence mechanics, replication with partial resync, Redis Cluster with hash slots and MOVED/ASK redirections, and production tuning for memory and eviction.
How ScyllaDB Works Internally: Shard-Per-Core Architecture, LSM Storage, and the C++ Engine That Outperforms Cassandra
A deep dive into ScyllaDB internals covering the shard-per-core thread model built on the Seastar framework, shared-nothing design that eliminates cross-core locking, LSM-tree storage with size-tiered and leveled compaction, CQL compatibility with Cassandra, gossip-based cluster membership and consistent hashing, lightweight transactions via Raft consensus, I/O scheduling and memory allocation, streaming and repair, and a tradeoffs comparison table across ScyllaDB, Apache Cassandra, Amazon DynamoDB, CockroachDB, and FoundationDB.
How Grafana Loki Works Internally: Label-Based Indexing, Log Chunk Storage, and the Cost-Efficient Architecture That Stores Logs Like Prometheus Stores Metrics
A deep dive into Loki's internal architecture covering label-based indexing that avoids full-text inversion, the write path through distributors and ingesters with WAL-backed chunk buffering, the read path through query frontends and queriers with LogQL execution, the boltdb-shipper and TSDB index backends, compaction and retention mechanics, and tradeoffs against Elasticsearch, Datadog, and Splunk.
How Prometheus Works Internally: Pull-Based Scraping, the TSDB Storage Engine, and PromQL Query Execution
A deep dive into Prometheus internals for senior engineers. Covers the pull-based scraping model, service discovery integration, the custom TSDB with head block and compacted blocks, WAL-based crash recovery, PromQL evaluation from parse tree to result, recording rules, the Alertmanager pipeline, federation, and production considerations including cardinality management, retention tuning, and remote write for long-term storage.
How RabbitMQ Works Internally: AMQP Protocol, Exchange Routing, and Quorum Queues From Connection to Consumer
A deep dive into RabbitMQ internals covering the AMQP 0-9-1 connection and channel model, binary frame structure, exchange types with binding-based routing, queue storage with Erlang process internals and persistence mechanics, credit-based flow control, consumer acknowledgment and prefetch semantics, the evolution from classic mirrored queues to Raft-based quorum queues, virtual hosts for multi-tenancy, and a tradeoffs comparison table across RabbitMQ, Kafka, NATS, Apache Pulsar, and Redpanda.
How Apache Airflow Works Internally: DAG Parsing, Task Scheduling, and the Executor Architecture Behind Production Workflow Orchestration
A deep dive into Apache Airflow's internals covering DAG file parsing and serialization, the scheduler tight loop, executor architectures, the metastore schema, the triggerer component for async deferrals, and production tuning guidance for teams running Airflow at scale.
How HAProxy Works Internally: Multi-Threaded Event Model, Connection Routing, and the Load Balancing Engine Behind High-Availability Infrastructure
A deep dive into how HAProxy works internally: its evolution from single-process to multi-threaded event-driven model, frontend/backend/server configuration flow, connection multiplexing with HTTP/2, health checking subsystems, stick tables for session persistence and rate limiting, ACL-based content switching, and production tuning for high-throughput deployments.
How Deno Works Internally: The V8 Sandbox, Rust Runtime, and the Permission Model That Rethinks Server-Side JavaScript
A deep dive into Deno's internals covering the V8 integration through rusty_v8, the Tokio-based async runtime, the capability-based permission system, URL-native module resolution, swc-based TypeScript transpilation, the FFI layer, Deno KV, and production considerations for memory management and cold-start optimization.
How Nginx Works Internally: Event Loop, Master-Worker Architecture, and Request Processing from Accept to Upstream
A deep-dive into how Nginx handles tens of thousands of concurrent connections with a handful of worker processes: the master-worker process model, epoll/kqueue event loop, multi-phase request pipeline, upstream connection pooling, shared memory zones, and production tuning for worker count, buffers, and keepalive.
How Redpanda Works Internally: Thread-Per-Core Architecture, Raft-Based Replication, and the C++ Streaming Engine That Replaces Kafka Without the JVM
A deep dive into how Redpanda works internally: its Seastar thread-per-core execution model, Raft consensus for partition replication, shadow indexing for tiered storage, Wasm-based transforms, and what the absence of the JVM actually means in production.