System Design — Page 3 of 21
Browsing page 3 of 21 — 243 articles on system design.
How Apache Pulsar Works Internally: Tiered Storage, BookKeeper-Backed Persistence, and the Multi-Tenant Messaging Engine That Separates Compute from Storage
A deep dive into Apache Pulsar internals covering the broker-BookKeeper separation that decouples compute from storage, the write path through managed ledgers and journal plus entrylog persistence, the read path with cursor tracking and catch-up versus tailing consumers, topic partitioning and bundle-based load balancing, native multi-tenancy with tenant/namespace/topic hierarchy, tiered storage offloading to S3 or GCS, geo-replication via replication cursors, and a tradeoffs comparison against Kafka, RabbitMQ, NATS, and Redpanda.
How RocksDB Works Internally: LSM-Tree Storage, Compaction Strategies, and the Embedded Engine Behind Modern Distributed Databases
A deep dive into RocksDB internals covering the LSM-tree write path through WAL and MemTable to SST files, the read path with bloom filters and block cache, leveled versus universal versus FIFO compaction and their amplification tradeoffs, column families, write stall mechanics, and production tuning guidance.
How Vitess Works Internally: VTGate Query Routing, VTTablet Management, and the Sharding Middleware That Scales MySQL to Planetary Proportions
A deep dive into how Vitess horizontally scales MySQL through VTGate query routing, VTTablet sidecar management, VSchema, scatter-gather query rewriting, Online DDL, VReplication, and live resharding workflows.
How Bun Works Internally: The JavaScriptCore Engine, Zig Runtime, and the Architecture Decisions That Make It Fast
A deep dive into Bun's internals covering why it chose JavaScriptCore over V8, how the Zig runtime manages memory and async I/O without libuv, the bundler and transpiler architecture, native SQLite integration, the HTTP server implementation, the test runner, and the real tradeoffs compared to Node.js and Deno.
How MongoDB Works Internally: The WiredTiger Storage Engine, Document Model, and Replication Protocol Behind the Most Popular NoSQL Database
A deep dive into MongoDB internals covering WiredTiger B-tree storage and MVCC, BSON encoding, compound and wildcard indexes, the query planner's trial-period competition, oplog-based replication, Raft-inspired elections, sharding chunk mechanics, and tunable read/write concerns.
How ZooKeeper Works Internally: ZAB Protocol, Znodes, Watches, and the Coordination Engine Behind Distributed Systems
A deep dive into Apache ZooKeeper's internals covering the ZAB atomic broadcast protocol, the znode data model, watch notification semantics, session management, the request processing pipeline, and snapshot-based persistence. Includes a tradeoffs comparison table against etcd, Consul, and Chubby.
How Apache Spark Works Internally: Driver Architecture, DAG Scheduling, Shuffle Mechanics, and the Catalyst Optimizer Behind Distributed Data Processing
A deep dive into Apache Spark internals covering the driver-executor model, RDD lineage, DAG scheduling, sort-based shuffle, the Catalyst optimizer, Tungsten memory management, and adaptive query execution.
How ClickHouse Works Internally: MergeTree Storage, Vectorized Execution, and Distributed Query Processing
A deep-dive into ClickHouse's architecture: MergeTree storage with sparse indexes and columnar compression, vectorized query execution with SIMD, distributed sharding and replication, materialized views, and production tuning that actually matters.
How DuckDB Works Internally: Vectorized Execution, Columnar Storage, and the In-Process Analytics Engine That Replaced Your Python Scripts
A deep-dive into DuckDB's internal architecture covering the in-process execution model, vectorized operator pipeline, columnar storage with zone maps and lightweight compression, morsel-driven parallelism, MVCC concurrency, and the extension system that makes it a universal analytics frontend.
Designing a Distributed Lock Service: Lease-Based Coordination, Fencing Tokens, and Consensus-Backed Locking at Scale
A deep dive into distributed locking covering lease expiration, the Redlock controversy, ZooKeeper ephemeral sequential nodes, etcd revision-ordered waiters, fencing tokens, and how to choose the right primitive for your consistency requirements.
Designing a Referral and Invite System: Double-Sided Rewards, Fraud Prevention, and Attribution Tracking at Scale
A production guide to referral system design: invite code generation, double-sided reward mechanics, cross-device attribution, fraud prevention with device fingerprinting and abuse ring detection, idempotent reward ledgers, and tradeoffs across invite codes, referral links, in-app sharing, and affiliate models.
How Temporal Works Internally: Event History, Deterministic Replay, and the Architecture Behind Durable Execution
A deep dive into Temporal's internal architecture covering the four cluster services, workflow execution as append-only event history, the deterministic replay model, activity task lifecycle with heartbeating and retry policies, task queue mechanics with sticky execution, timer implementation, signal and query mechanisms, and the Elasticsearch-backed visibility system, with a tradeoffs comparison across Temporal, Inngest, AWS Step Functions, Cloudflare Workflows, and Restate.