System Design — Page 15 of 21
Browsing page 15 of 21 — 243 articles on system design.
Designing a Data Lakehouse: Unifying Data Lakes and Warehouses with Iceberg, Delta Lake, and Modern Query Engines
A practical guide to data lakehouse architecture: how Apache Iceberg, Delta Lake, and Apache Hudi unify the storage flexibility of data lakes with the reliability guarantees of warehouses. Covers table formats, catalog services, query engine integration, schema evolution, time travel, and production considerations for startups building analytics infrastructure.
Designing an IoT Data Pipeline: Device Ingestion, Time-Series Storage, and Real-Time Fleet Dashboards at Scale
Building backend infrastructure for device fleets is a different problem than building web APIs. This guide covers the full IoT data pipeline architecture: ingestion protocol selection (MQTT, HTTP, WebSocket), time-series database tradeoffs (TimescaleDB, InfluxDB, QuestDB), stream processing for anomaly detection, and real-time fleet dashboard delivery.
Designing a Health Check System: Liveness Probes, Readiness Gates, and Deep Health Checks for Distributed Services
Most services ship a single /health endpoint that returns 200 OK regardless of what is actually healthy. This article covers how to design a layered health check system with liveness probes, readiness gates, and deep dependency checks, including the production failure modes that overzealous health checks introduce.
Designing a Two-Sided Marketplace: Search, Matching, Trust Scores, and Payment Escrow at Scale
Two-sided marketplaces fail at the architecture layer long before they fail at product-market fit. This guide covers the matching problem, search ranking, trust and reputation systems, payment escrow, dispute resolution, and how to handle cold start without lying to your users.
Designing an A/B Testing Platform: Experiment Assignment, Statistical Analysis, and Feature Flag Integration at Scale
A deep-dive into architecting an A/B testing platform from scratch: consistent user assignment via hashing, mutual exclusion layers, frequentist and Bayesian statistical engines, feature flag integration, and production pitfalls like sample ratio mismatch.
Designing a Distributed Lock Service: Fencing Tokens, Clock Drift, and Correctness Guarantees Beyond Redlock
A deep-dive into building a production distributed lock service. Covers why naive implementations break, Redlock's documented flaws, fencing tokens as the correctness mechanism, and a TypeScript implementation with real tradeoffs.
Designing a Multi-Region Database: Replication Topologies, Conflict Resolution, and Latency-Aware Routing
A practical guide to multi-region database design covering single-leader, multi-leader, and leaderless replication topologies, conflict resolution strategies including last-write-wins and CRDTs, and latency-aware routing with TypeScript implementation examples.
Designing a Video Streaming Pipeline: Transcoding, Adaptive Bitrate, and CDN Delivery at Scale
Video streaming is one of the most infrastructure-intensive problems in software. This article walks through every layer: ingest, transcoding, manifest generation, CDN delivery, and adaptive bitrate selection, with the tradeoffs that actually matter in production.
Designing a Logging and Log Aggregation Pipeline: Collection, Transport, Storage, and Search at Scale
A practical guide to building a production logging pipeline from structured log emission through Kafka transport to Elasticsearch, ClickHouse, or Loki, covering indexing strategies, retention policies, and query patterns.
Designing a Tenant-Aware AI Gateway: Per-Customer Model Routing, Token Quotas, and Cost Attribution in Multi-Tenant SaaS
How to architect an AI gateway layer for multi-tenant SaaS that routes requests to different LLM providers per tenant, enforces per-customer token budgets, and attributes costs accurately.
Designing an Object Storage System: Blob Storage, Metadata Indexing, and Erasure Coding at Scale
How object storage systems like S3 and R2 work under the hood. Covers the separation of metadata and data planes, consistent hashing for blob placement, erasure coding for durability without full replication overhead, metadata indexing for listing and prefix queries, and the tradeoffs between strong consistency and availability.
Designing a Feature Flag System: Targeting Rules, Gradual Rollouts, and Real-Time Evaluation at Scale
A system design deep-dive into how feature flag platforms work internally. Covers the evaluation engine, data model for flags and segments, SDK architecture for low-latency evaluation, gradual rollout strategies, real-time propagation via SSE, auditability, and the tradeoffs between server-side and client-side evaluation.