System Design ·

How DNS Works: Resolution, Caching, and Traffic Routing From Query to Response

A deep dive into DNS resolution covering the full recursive chain, record types and their production uses, TTL mechanics and negative caching, DNS-based traffic management strategies, DNSSEC, and practical guidance for startup engineering teams.

How DNS Works: Resolution, Caching, and Traffic Routing From Query to Response

Every HTTP request your application handles starts with a DNS lookup. Most engineers understand this at a surface level: type a domain, get an IP. But the mechanics underneath that exchange touch caching, distributed consistency, traffic engineering, and security in ways that matter when you are building production systems. Get them wrong and you ship slow migrations, cause unnecessary downtime, or leave yourself open to cache poisoning.

This is a full walkthrough of how DNS works from query to response: the resolution chain, the record types engineers actually use, how caching behaves at each layer, how DNS drives traffic routing, and what you need to know when you are operating DNS for real services.

The Resolution Chain

When your application calls dns.lookup('api.example.com'), the request moves through a chain of resolvers before an IP address comes back. Understanding each hop helps you reason about latency, caching behavior, and failure modes.

Stub Resolver

The stub resolver lives in the operating system. On Linux it reads /etc/resolv.conf to find the address of a recursive resolver, typically a local router or a public resolver like 1.1.1.1 or 8.8.8.8. The stub resolver itself does almost no work. It forwards queries upstream and caches responses in the OS resolver cache.

Recursive Resolver

The recursive resolver (also called a full-service resolver) does the actual legwork. It starts with the query from the stub, checks its own cache, and if it has no cached answer it begins walking down the DNS namespace tree. This resolver is operated by your ISP, your cloud provider, or a third-party DNS service you configure explicitly.

Root Servers

If the recursive resolver has no cached answer, it queries one of the 13 root server clusters (labeled A through M). Root servers do not know the IP for api.example.com. They know which nameservers are authoritative for .com. They return a referral: “ask one of these TLD servers.”

The 13 root server addresses are hardcoded in resolver software as the hints file. Each letter corresponds to a cluster of anycast nodes spread globally, so “13 root servers” actually means hundreds of physical machines.

TLD Servers

The TLD (top-level domain) nameserver for .com is operated by Verisign. It does not know the IP for api.example.com either. It knows which nameservers are authoritative for example.com. It returns another referral.

Authoritative Nameserver

The authoritative nameserver is where your records live. It holds the actual A, CNAME, MX, and other records for your domain. When the recursive resolver reaches the authoritative server and gets a final answer, it caches that answer for the duration of the TTL and returns it to the stub resolver.

The full round trip for a cold cache lookup can take 50-200ms depending on resolver location and network conditions. Most lookups return from recursive resolver cache in under 5ms.

DNS Record Types

DNS stores structured records, not just A-to-IP mappings. Each record type serves a specific purpose and has production implications worth knowing.

A maps a hostname to an IPv4 address. One hostname can have multiple A records for simple load distribution, though round-robin DNS is a poor substitute for a real load balancer.

AAAA maps a hostname to an IPv6 address. Modern resolvers return both A and AAAA and the client picks based on preference (Happy Eyeballs in browsers will race both and use whichever connects first).

CNAME maps a hostname to another hostname. The resolver follows the chain until it hits an A or AAAA record. CNAMEs cannot coexist with other records at the same name. You cannot put a CNAME at your zone apex (example.com) because the apex also needs NS and SOA records. This is why DNS providers invented proprietary ALIAS or ANAME records that behave like CNAMEs but resolve at the nameserver before returning a result.

MX specifies mail servers for a domain with a priority value. Lower priority numbers win. Having two MX records at different priorities gives you mail failover.

TXT holds arbitrary text. In practice it carries SPF records (v=spf1 ...), DKIM public keys (v=DKIM1; k=rsa; p=...), DMARC policy (v=DMARC1; p=reject; ...), and domain ownership verification tokens for cloud services.

SRV encodes service location: protocol, port, weight, and priority. Kubernetes uses SRV records internally for service discovery. Applications that consume SRV records can locate services without hardcoded ports. Format: _service._proto.name TTL class SRV priority weight port target.

NS lists the authoritative nameservers for a zone. These records live at the parent zone (the .com TLD servers hold NS records for example.com) and are replicated inside your zone.

SOA (Start of Authority) holds zone metadata: primary nameserver, admin email, serial number, and timing parameters for zone transfers between primary and secondary nameservers.

Caching at Every Layer

DNS caching is layered, and each layer has different TTL semantics. Misunderstanding this is how engineers accidentally create long-lived incidents during migrations.

Browser cache. Chrome, Firefox, and Safari cache DNS responses in-process. Chrome ignores the record TTL for positive responses and applies its own cap (historically 60 seconds for positive, 15 seconds for negative). Safari respects TTLs more closely. This cache is invisible to OS-level tooling.

OS resolver cache. nscd on older Linux distributions, systemd-resolved on modern ones, dnsmasq on routers. These respect TTLs. You can flush them manually (resolvectl flush-caches on systemd systems) but your users cannot.

Recursive resolver cache. The resolver at your ISP or cloud provider caches answers per their own logic. Most public resolvers (1.1.1.1, 8.8.8.8) respect the authoritative TTL with a minimum floor of a few seconds. They also cache negative responses (NXDOMAIN, NOERROR with empty answer section) for the TTL specified in the SOA’s negative caching field.

Authoritative nameserver TTL. This is the only one you control. It tells downstream caches how long to hold a record. A 300-second TTL means your change propagates everywhere in at most 5 minutes after the old cached entries expire. A 86400-second TTL means stale answers can persist for 24 hours.

Negative Caching

Negative caching (RFC 2308) stores the fact that a name does not exist or a record type is absent. The TTL for negative responses comes from the SOA record’s minimum field. If a service queries for SRV records on a domain that has none, the resolver caches that “no answer” for the SOA minimum TTL. This matters when you add a new record type: resolvers that recently got a negative response will not see your new record until their negative cache entry expires.

Programmatic DNS Lookups in TypeScript

Node’s built-in dns module exposes both the OS resolver path (dns.lookup) and the direct resolver path (dns.resolve). They behave differently.

import dns from "node:dns/promises";

// Uses the OS resolver (getaddrinfo). Respects /etc/hosts, nsswitch.conf.
// Blocks the thread pool. Does not give you TTL.
const address = await dns.lookup("api.example.com");
console.log(address); // { address: '93.184.216.34', family: 4 }

// Bypasses OS resolver. Queries the configured nameserver directly.
// Returns multiple records and TTLs.
const records = await dns.resolve4("api.example.com", { ttl: true });
console.log(records);
// [{ address: '93.184.216.34', ttl: 300 }]

// SRV record lookup for service discovery
const srvRecords = await dns.resolveSrv("_http._tcp.example.com");
// [{ name: 'backend.example.com', port: 8080, priority: 10, weight: 100 }]

// MX records for mail configuration validation
const mxRecords = await dns.resolveMx("example.com");
// [{ exchange: 'mail.example.com', priority: 10 }]

// TXT records for SPF/DKIM verification
const txtRecords = await dns.resolveTxt("example.com");
// [['v=spf1 include:_spf.google.com ~all']]

For health checks that need to validate DNS resolution as part of a readiness probe:

import dns from "node:dns/promises";

interface DnsHealthResult {
  healthy: boolean;
  latencyMs: number;
  addresses: string[];
  error?: string;
}

async function checkDnsResolution(hostname: string): Promise<DnsHealthResult> {
  const start = performance.now();
  try {
    const records = await dns.resolve4(hostname, { ttl: true });
    return {
      healthy: records.length > 0,
      latencyMs: performance.now() - start,
      addresses: records.map((r) => r.address),
    };
  } catch (err) {
    return {
      healthy: false,
      latencyMs: performance.now() - start,
      addresses: [],
      error: err instanceof Error ? err.message : String(err),
    };
  }
}

// Use in a health check endpoint
const result = await checkDnsResolution("api.example.com");
if (!result.healthy) {
  console.error(`DNS resolution failed: ${result.error}`);
  process.exit(1);
}

DNS-Based Traffic Management

DNS is a surprisingly capable traffic routing layer. Cloud DNS providers expose routing policies that let you distribute traffic, implement failover, and route by geography without touching application code.

Weighted routing assigns a numeric weight to each record. The authoritative server returns records proportionally to their weights. A 90/10 split across two IPs sends roughly 90% of queries to the first address. This is the mechanism for canary deployments at the DNS layer: shift 5% of traffic to a new endpoint, watch error rates, then increase the weight.

Geolocation routing returns different records depending on where the resolver’s source IP maps to geographically. Queries from EU resolvers get an EU endpoint; queries from US resolvers get a US endpoint. Accuracy depends on resolver-to-user proximity. When users share a resolver (public DNS providers), the resolver’s location is used, which may differ from the user’s actual location. ECS (EDNS Client Subnet, RFC 7871) partially solves this by forwarding a prefix of the client’s IP to the authoritative server.

Latency-based routing measures round-trip latency from the resolver to each candidate endpoint region and returns the record for the lowest-latency region. AWS Route 53 maintains latency measurements between AWS regions and resolver locations to power this.

Failover routing pairs a primary record with a secondary. A health check monitors the primary endpoint. If the health check fails, the authoritative server stops returning the primary record and returns the secondary. The failover convergence time is bounded by the health check interval plus the TTL of the affected records.

DNS Routing Strategy Tradeoffs

StrategyConvergence TimeAccuracyComplexityUse Case
WeightedTTL-boundedApproximate (caching skews ratios)LowCanary deploys, A/B splits
GeolocationTTL-boundedGood with ECS, rough withoutMediumData residency, regional endpoints
Latency-basedTTL-boundedGood within region clustersMediumMulti-region latency optimization
Failover with health checksHealth check interval + TTLBinary (up/down)MediumDisaster recovery, active/passive HA
Round-robin A recordsImmediate (multiple records)None (no load awareness)Very lowSimple horizontal scale (avoid in prod)

The fundamental constraint across all DNS routing strategies: clients cache the result. Once a client has resolved an address, it will keep using that address until the TTL expires, regardless of what changes you make to DNS. Traffic shifts are always gradual, never instant.

DNSSEC

DNSSEC (DNS Security Extensions) adds a chain of cryptographic signatures to DNS responses. Without DNSSEC, a resolver has no way to verify that the response it received was actually produced by the authoritative nameserver and was not tampered with in transit. Cache poisoning attacks (Kaminsky attack and variants) exploit this gap.

DNSSEC works through a chain of trust:

  1. The authoritative nameserver signs each record set with a Zone Signing Key (ZSK). The signature is stored in an RRSIG record alongside the data.
  2. The ZSK’s public key is published in a DNSKEY record. A second key, the Key Signing Key (KSK), signs the DNSKEY record set.
  3. The parent zone (.com TLD) stores a DS (Delegation Signer) record containing a hash of the child zone’s KSK. This links the parent’s trust to the child’s signing key.
  4. Root servers are the trust anchor. Their DNSKEY is distributed with resolver software.

Resolvers that do DNSSEC validation (DNSSEC-aware resolvers) follow this chain from root to leaf and verify signatures at each step. If any signature is missing or invalid, the resolver returns SERVFAIL.

DNSSEC does not encrypt DNS traffic (that is DNS-over-HTTPS and DNS-over-TLS). It only provides authenticity and integrity.

Split-Horizon DNS

Split-horizon (or split-brain) DNS serves different answers to the same query depending on where the query originates. Internal queries from within your VPC or corporate network get private IP addresses or internal hostnames. External queries get public IPs.

A common setup: db.example.com resolves to 10.0.1.45 for internal services and returns NXDOMAIN (or nothing) for external queries. This keeps internal topology out of public DNS while letting services use consistent hostnames in code regardless of environment.

Implementation options:

  • Run separate authoritative nameservers for internal and external zones, and configure resolvers to use the internal one for *.internal.example.com or similar.
  • Use Route 53 private hosted zones scoped to specific VPCs alongside public hosted zones for the same domain.
  • Use CoreDNS in Kubernetes with custom forward and rewrite rules to handle internal service resolution without leaking to public resolvers.

Production Considerations

DNS propagation timing. “DNS propagation” is shorthand for “waiting for old cached entries to expire.” The time to full propagation is the maximum TTL across all resolvers that have cached the record. You cannot force external caches to flush. Plan migrations around your current TTL: if you are running 3600-second TTLs today, your migration window is at minimum one hour after the change.

Low-TTL strategy for zero-downtime migrations. Before a migration:

  1. Lower the TTL on the record you plan to change to 60-300 seconds. Wait for the current TTL to expire so all caches pick up the new low TTL.
  2. Make the record change.
  3. Monitor for 5-10 minutes. Old caches drain quickly now.
  4. If the migration fails, revert. The low TTL means rollback propagates in minutes.
  5. After the migration stabilizes, raise the TTL back to your normal value (3600 or higher).

Never make the TTL change and the address change in the same step. The two-step process is what gives you the fast rollback window.

DNS observability. DNS failures are silent by default. A misconfigured record or expired delegation causes SERVFAIL or NXDOMAIN responses that surface as connection errors or timeouts in application logs, usually with no mention of DNS. Track:

  • DNS resolution latency from your application’s perspective (use dns.resolve4 with timing instrumentation as shown above)
  • SERVFAIL and NXDOMAIN error rates from your recursive resolver’s logs or metrics
  • TTL headroom on critical records (alert if a certificate or key rotation is approaching and TTLs are still high)
  • Health check status for failover-dependent records
import dns from "node:dns/promises";

// Instrument DNS lookups with OpenTelemetry or your metrics system
async function resolveWithMetrics(
  hostname: string,
  recordType: "A" | "AAAA" = "A"
): Promise<string[]> {
  const start = performance.now();
  let success = false;

  try {
    const records =
      recordType === "A"
        ? await dns.resolve4(hostname)
        : await dns.resolve6(hostname);
    success = true;
    return records;
  } catch (err) {
    const code = err instanceof Error && "code" in err ? (err as NodeJS.ErrnoException).code : "UNKNOWN";
    // Emit to your metrics backend:
    // counter("dns.errors", 1, { hostname, code })
    console.error(`DNS error for ${hostname}: ${code}`);
    throw err;
  } finally {
    const latencyMs = performance.now() - start;
    // Emit to your metrics backend:
    // histogram("dns.resolution.latency_ms", latencyMs, { hostname, success })
    console.debug(`DNS ${hostname} resolved in ${latencyMs.toFixed(1)}ms (success=${success})`);
  }
}

Delegation health. Your domain’s NS records at the TLD level must point to the same nameservers you have configured as authoritative. Mismatches between the parent delegation and your zone’s own NS records cause intermittent resolution failures. Check this after any nameserver change with dig NS example.com @a.gtld-servers.net and compare against dig NS example.com @your-authoritative-ns.

Negative caching gotchas. When adding a new record type to a zone, check your SOA minimum TTL. If it is 3600 seconds and a service has recently queried for that record type and got a negative response, it will not see your new record for up to an hour. Temporarily lower the SOA minimum before adding new record types to high-traffic zones.

Closing

DNS is deceptively simple on the surface and genuinely complex once you operate it at scale. The resolution chain, caching layers, and routing policies each introduce latency and consistency tradeoffs that compound. The engineers who get burned by DNS are almost always the ones who treat it as a static config file rather than a distributed system with its own propagation dynamics. Treat TTLs as a knob you actively manage, instrument resolution from within your application, and build your migration procedures around the caching reality rather than fighting it.

More in System Design

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL
System Design ·

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL

A deep dive into Aurora's storage-compute separation, the log-is-the-database design that pushes redo log processing to storage nodes, quorum-based replication across 6 copies in 3 AZs, protection group architecture, fast cloning, Serverless v2 scaling mechanics, and honest tradeoffs versus RDS, self-managed PostgreSQL, CockroachDB, and Cloud Spanner.

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale
System Design ·

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale

A deep dive into CockroachDB's internals: the 512MB range-based data model with automatic splitting, per-range Raft replication, MVCC timestamp ordering with hybrid logical clocks, DistSQL physical planning, serializable transactions with timestamp refreshes, closed timestamps for follower reads, online schema changes using multi-version schema descriptors, and production considerations for hotspots and write amplification.

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation
System Design ·

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation

Containers are not virtual machines and they are not magic. They are a thin composition of Linux kernel primitives: namespaces, cgroups, and a layered filesystem. This article traces exactly what happens between docker run and a running process, covering the OCI spec, containerd, runc, overlay filesystems, and container networking at the kernel level.

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally
System Design ·

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally

A deep dive into YugabyteDB internals covering the DocDB storage engine built on RocksDB LSM trees, per-tablet Raft consensus, YSQL and YCQL dual query layers, distributed MVCC with hybrid logical clocks, automatic tablet splitting and rebalancing, xCluster multi-region replication, and production considerations for hotspots, connection pooling, and schema design.