System Design — Page 6 of 21
Browsing page 6 of 21 — 243 articles on system design.
Database Sharding Strategies Explained: Range-Based, Hash-Based, and Directory-Based Partitioning with Real Production Examples
A comprehensive guide to database sharding strategies covering range-based, hash-based, and directory-based partitioning, shard key selection, resharding, cross-shard queries, and how Instagram, Discord, and Vitess implement sharding in production. Includes a full tradeoffs comparison table and operational guidance.
How Vector Databases Work: HNSW Indexes, Similarity Search, and the Storage Engine Behind AI-Native Retrieval
A deep dive into vector database internals covering embedding storage, HNSW graph index construction and traversal, IVF and product quantization for memory efficiency, similarity search execution with distance metrics, metadata filtering tradeoffs, the write and query paths, and a comparison of Pinecone, Weaviate, Qdrant, pgvector, and Milvus. With production guidance on recall, latency, quantization, and hybrid search.
How Containers Work: Linux Namespaces, Cgroups, Union Filesystems, and the Runtime Architecture From docker run to Running Process
A deep dive into container internals covering Linux namespaces for process isolation, cgroups v2 for resource limiting, OverlayFS and image layer copy-on-write semantics, the OCI runtime spec and containerd/runc execution flow, container networking with veth pairs and bridge devices, and a tradeoffs comparison against VMs, Firecracker, gVisor, and Wasm containers. With production guidance.
How DynamoDB Works: Partition Management, Request Routing, and Single-Digit Millisecond Reads at Any Scale
A deep dive into DynamoDB internals: consistent-hashing partitions, the request router layer, B-tree storage nodes, read/write capacity modes, single-table design, GSI/LSI tradeoffs, Streams, and transactions.
How MongoDB Works: The Document Model, WiredTiger Storage Engine, and Replication From Write Path to Oplog
A deep dive into MongoDB internals covering the BSON document model, WiredTiger storage engine with B-tree indexes and cache management, the full write path from driver to journal to data files, oplog-based replication with elections and read preferences, sharding with chunk migration, and the aggregation pipeline execution model.
How Kafka Works: Partitioned Logs, Consumer Groups, Replication, and Exactly-Once Delivery From Producer to Consumer
A deep dive into Apache Kafka internals covering the append-only partitioned log, producer partition selection via key hashing and sticky partitioning, consumer group rebalancing protocols, ISR-based replication and leader election, exactly-once semantics via idempotent producers and transactions, log compaction, and production tuning. Includes a tradeoffs table comparing Kafka, RabbitMQ, Amazon SQS, Redpanda, and Pulsar.
How TCP Works: The Three-Way Handshake, Flow Control, Congestion Avoidance, and Reliable Delivery From SYN to FIN
A deep dive into TCP internals covering the connection lifecycle, sequence numbers, sliding window flow control, congestion control algorithms, Nagle's algorithm, head-of-line blocking, and how QUIC solves what TCP cannot. With production tuning guidance for server environments.
How WebSockets Work: The Upgrade Handshake, Frame Protocol, and Bidirectional Communication From HTTP to Persistent Connection
A deep dive into the WebSocket protocol internals: the HTTP upgrade handshake with Sec-WebSocket-Key/Accept computation, the binary frame format, control frames, permessage-deflate compression, and production considerations for horizontal scaling, backpressure, and reconnection.
How PostgreSQL Works: MVCC, the Query Planner, and the Storage Engine Behind the World's Most Advanced Open-Source Database
A deep technical guide to PostgreSQL internals: the process model, heap storage with 8KB pages and TOAST, MVCC via xmin/xmax snapshot isolation, the write-ahead log, the cost-based query planner, index types from B-tree to GIN and BRIN, the buffer manager, vacuum, and production tuning for shared_buffers, work_mem, and connection pooling.
How Elasticsearch Works: Inverted Indexes, Segment Merges, and Distributed Search From Query to Hit
A deep dive into Elasticsearch internals covering the Lucene foundation, immutable segment storage, distributed search execution, BM25 relevance scoring, near-real-time search mechanics, and cluster scaling patterns.
How Kafka Works: Partitions, Replication, Consumer Groups, and the Commit Log
A deep dive into Kafka internals covering the append-only commit log, partition-level ordering, leader-follower replication with ISR mechanics, consumer group rebalancing, exactly-once semantics, and production sizing considerations.
How SQLite Works: B-Tree Storage, WAL Mode, and the Single-File Architecture That Powers Edge Computing
A deep dive into SQLite internals covering the page-based single-file layout, B-tree storage engine, VDBE bytecode execution, WAL mode vs rollback journal, transaction locking states, query planning, and why SQLite has become the default database at the edge.