How Nginx Works Internally: Event Loop, Master-Worker Architecture, and Request Processing from Accept to Upstream

Most engineers who configure Nginx have a rough mental model of it: it takes requests in, forwards some to an upstream, and does it fast. That model breaks down the moment you need to tune it seriously, debug a connection spike, or understand why a misconfigured buffer is causing your upstream to get hammered. This article covers how Nginx actually works at the process, event, and request levels, so you have something concrete to reason from.

---

The Master-Worker Process Model

When Nginx starts, it spawns one master process and one or more worker processes. The master process holds the listening sockets and owns the lifecycle: it reads config, binds ports, starts workers, and forwards OS signals to them. It does not handle connections.

Workers are the processes that actually do work. Each worker runs a single-threaded event loop and handles thousands of connections concurrently without spawning threads or additional processes per connection. On Linux, a worker that needs to use thread pools for blocking operations (disk I/O, for example) can dispatch those via aio threads, but the connection-handling logic is always event-driven.

Signal-based lifecycle management is how you interact with the master:

SIGHUP   -> reload config (gracefully restart workers with new config)
SIGUSR1  -> reopen log files (for log rotation)
SIGUSR2  -> upgrade binary in-place (live upgrade)
SIGWINCH -> gracefully shut down workers without stopping master
SIGTERM  -> fast shutdown
SIGQUIT  -> graceful shutdown (drain existing connections)

A config reload via nginx -s reload sends SIGHUP to the master. The master re-reads the config, validates it, then starts new workers using the new config while the old workers continue draining their existing connections. Once old workers hit zero active connections, they exit. This gives you zero-downtime config changes under normal conditions.

The live binary upgrade (SIGUSR2 followed by SIGWINCH on the old master) works by having the new master inherit the listening socket file descriptors from the old one via the environment, so no connections are dropped during the upgrade window.

---

The Event Loop: epoll, kqueue, and the C10K Model

The reason Nginx can handle tens of thousands of concurrent connections per worker with low memory overhead is that it does not block. A thread-per-connection model keeps one OS thread parked for each open connection, which is expensive: each thread carries a stack (typically 8 MB by default), and context switches add up at scale. Nginx inverts this: one worker thread manages all connections, and the OS notifies it when something is ready to be processed.

On Linux, the notification mechanism is epoll. On BSD and macOS, it is kqueue. Both are level-triggered by default but support edge-triggered mode; Nginx uses edge-triggered epoll (EPOLLET) on Linux, which means it gets notified only when a file descriptor transitions from not-ready to ready rather than repeatedly while data remains. This avoids redundant wakeups at high connection counts.

The worker’s event loop at a simplified level:

while (running) {
  n = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);
  for (i = 0; i < n; i++) {
    handler = events[i].data.ptr;  // pointer to connection/request handler
    handler->read_handler() or handler->write_handler();
  }
  process_timers();
}

Each connection is represented by an ngx_connection_t struct. Nginx preallocates a pool of these at startup based on worker_connections. When epoll_wait returns a readable event on the listening socket, Nginx calls accept() to get the new connection fd, initializes an ngx_connection_t from the pool, registers it with epoll, and starts the HTTP processing chain.

Because everything runs on one thread, there are no mutexes protecting per-connection state. The only shared resource between workers is the accept mutex (accept_mutex), which serializes which worker accepts new connections at a given moment. Without it, all workers would wake up for each new connection (the thundering herd problem), and the OS would give the connection to only one of them, wasting the others’ wakeups. accept_mutex is enabled by default; in modern kernels with SO_REUSEPORT, you can disable accept_mutex and let the kernel distribute connections across workers directly, which reduces latency.

---

The Multi-Phase Request Processing Pipeline

Once a connection is accepted, the HTTP stack takes over. Nginx processes an HTTP request through a fixed sequence of phases. Handlers can be registered at each phase; when a phase has no handlers registered, it is skipped.

The eleven phases in order:

NGX_HTTP_POST_READ_PHASE        - runs after reading request header
NGX_HTTP_SERVER_REWRITE_PHASE   - server-level rewrite rules
NGX_HTTP_FIND_CONFIG_PHASE      - location block matching (not user-hookable)
NGX_HTTP_REWRITE_PHASE          - location-level rewrite rules
NGX_HTTP_POST_REWRITE_PHASE     - internal rewrite loop control
NGX_HTTP_PREACCESS_PHASE        - runs before access checks (e.g., limit_req)
NGX_HTTP_ACCESS_PHASE           - access control (e.g., allow/deny, auth_basic)
NGX_HTTP_POST_ACCESS_PHASE      - finalize access results
NGX_HTTP_PRECONTENT_PHASE       - try_files is here
NGX_HTTP_CONTENT_PHASE          - content handlers (proxy_pass, static, fastcgi)
NGX_HTTP_LOG_PHASE              - logging

Location matching at NGX_HTTP_FIND_CONFIG_PHASE deserves attention. Nginx evaluates location blocks in this order:

1. Exact match (= /path) — if found, stops immediately.
2. Longest prefix match, noted but not final.
3. Regular expression matches in config order (~ case-sensitive, ~* case-insensitive). First match wins.
4. If no regex matched, use the noted longest prefix match.
5. A prefix ending with ^~ disables regex matching if it is the longest prefix.

The matching result determines which location block’s configuration applies to the rest of the request lifecycle.

---

HTTP Parsing

Nginx’s HTTP parser is hand-written and does not rely on a general-purpose parser library. It processes request bytes incrementally as they arrive from the socket buffer, advancing through a state machine per byte. This means Nginx can start processing a request before the full body has been received, which matters for large uploads.

Request headers are stored in a singly-linked list of ngx_table_elt_t nodes allocated from a per-request pool. Nginx does not copy header values unless it needs to modify them; it stores offsets into the receive buffer. This avoids allocations on the hot path.

The request body is handled separately. Nginx can:

• Buffer it entirely in memory (up to client_body_buffer_size, default 8 KB or 16 KB depending on platform).
• Spill it to a temp file when the body exceeds the buffer.
• Stream it directly to the upstream without full buffering (with proxy_request_buffering off).

The proxy_request_buffering off path matters for large uploads: Nginx starts forwarding bytes to the upstream as they arrive rather than waiting for the full body. The tradeoff is that if the upstream is slow and the client is fast, you are holding an upstream connection open longer while the client streams.

---

Upstream Proxy: Connection Pooling and Keepalive

When the content phase handler is proxy_pass, Nginx needs to open a connection to the upstream server. The upstream module resolves the address (or picks one from a load balancing pool), then either opens a new TCP connection or reuses one from a keepalive pool.

Upstream keepalive is configured per upstream block:

upstream backend {
  server 10.0.1.10:8080;
  server 10.0.1.11:8080;
  keepalive 64;          # max idle connections to keep per worker
  keepalive_requests 1000;
  keepalive_timeout 60s;
}

server {
  location /api/ {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";  # required for keepalive to upstream
  }
}

Without proxy_http_version 1.1 and clearing the Connection header, Nginx sends Connection: close to the upstream by default, meaning every request gets a new TCP connection. At high request rates, this exhausts ephemeral ports and adds latency for the three-way handshake. The keepalive 64 directive says each worker can keep at most 64 idle connections to this upstream pool. Since workers run independently, a system with 4 workers can hold up to 256 idle upstream connections total.

Load balancing within the upstream block defaults to round-robin. Other built-in algorithms:

• least_conn: sends to the upstream with the fewest active connections.
• ip_hash: hashes client IP to a specific upstream for session affinity.
• hash $variable consistent: consistent hash based on any variable, with optional ketama-compatible consistent hashing.

Nginx open source does not include active health checks; it uses passive health checking by marking a peer as unavailable after max_fails failures within fail_timeout. Nginx Plus adds active health checks. In open source, you can approximate active health checks with a side process or by putting a health-check endpoint in the upstream application and using ngx_http_healthcheck_module (if compiled in).

---

Upstream Response Buffering

By default, Nginx buffers the upstream response. It reads the response from the upstream into memory buffers, then sends it to the client. This decouples upstream response speed from client receive speed, which protects upstream workers from slow clients.

proxy_buffering on;            # default: on
proxy_buffer_size 4k;          # size of buffer for first part of response (headers)
proxy_buffers 8 16k;           # number and size of buffers for response body
proxy_busy_buffers_size 24k;   # max buffers in active use while sending to client
proxy_max_temp_file_size 1024m; # max size of temp file for overflow buffering

When buffering is on and the response fits in memory, the upstream connection is freed quickly and the worker sends to the client from buffer. When the response overflows buffers, Nginx spills to a temp file under proxy_temp_path.

With proxy_buffering off, Nginx forwards bytes from the upstream to the client as they arrive. The upstream connection stays open until the client finishes receiving. This is the right setting for Server-Sent Events (SSE) and streaming responses where you need low latency delivery.

---

Shared Memory Zones: Rate Limiting and Caching Across Workers

Workers run in separate processes with no shared heap. For state that needs to be consistent across all workers, Nginx uses shared memory zones backed by OS shared memory (shmget/mmap). These zones hold:

• Rate limiting counters (used by limit_req_zone and limit_conn_zone).
• Upstream peer state (active connections, failure counts).
• The proxy cache index and metadata.
• Lua shared dicts (if using OpenResty).

http {
  limit_req_zone $binary_remote_addr zone=api_limit:10m rate=100r/s;

  server {
    location /api/ {
      limit_req zone=api_limit burst=50 nodelay;
      proxy_pass http://backend;
    }
  }
}

The 10m argument allocates a 10 MB shared memory zone named api_limit. Nginx stores per-key state (the client IP in $binary_remote_addr) using a red-black tree inside this zone, with an LRU list for eviction when the zone fills. Each entry is approximately 64 bytes, so 10 MB holds roughly 160,000 unique keys.

The rate limiting algorithm is leaky bucket (specifically, the GCRA variant). burst=50 nodelay means requests exceeding the rate are allowed up to a burst of 50 without delay, but once the burst is consumed, requests above the rate are rejected with 503. Without nodelay, excess burst requests are queued, which adds latency.

For the proxy cache, the shared memory zone (proxy_cache_path ... keys_zone=cache_name:size) holds the cache key index in shared memory, while the actual response data lives on disk. Workers coordinate cache reads and writes through this index. When a cached item is stale, one worker fetches from the upstream (under a mutex held by the cache key) while others serve the stale content, controlled by proxy_cache_lock and proxy_cache_use_stale.

---

Production Considerations

worker_processes: Set to auto in most cases, which matches the number of CPU cores. The rationale is that each worker is single-threaded and CPU-bound work is minimal (mostly memory copies and epoll bookkeeping), so one worker per core avoids context switch overhead while keeping all cores utilized. If you are doing SSL termination at high volume, CPU usage per worker goes up and you may want to profile before assuming auto is optimal.

worker_connections: The maximum number of simultaneous connections per worker. The actual connection limit for the process is also gated by nofile (open file descriptors per process). Set worker_rlimit_nofile to at least worker_connections * 2 (each proxied request uses two connections: client-side and upstream-side). A common production setting:

worker_processes auto;
worker_rlimit_nofile 65536;

events {
  worker_connections 16384;
  use epoll;           # explicit on Linux, though auto-detected
  multi_accept on;     # accept as many connections as possible per epoll wakeup
}

With 4 workers and worker_connections 16384, you have headroom for roughly 32,000 proxied connections in flight (each uses two connections).

Buffer sizing: Mismatched buffer sizes are a common source of production issues. If proxy_buffer_size is too small for upstream response headers, Nginx falls back to temp file I/O for every response, adding disk latency. Check for warnings in the error log: upstream sent too big header indicates the header response exceeded proxy_buffer_size. For upstreams that send large cookies or JWT tokens in headers, set proxy_buffer_size 16k.

Keepalive to upstream: Always configure keepalive for high-throughput proxy scenarios. Without it, you will see TIME_WAIT socket exhaustion under load. Monitor /proc/net/sockstat for TIME_WAIT counts; if they are consistently above a few thousand, keepalive is not working or the pool size is too small.

SSL termination: ssl_session_cache shared:SSL:50m shares the SSL session cache across workers. Without this, a client resuming a TLS session that was originally handled by a different worker gets a full handshake instead of a session resumption. Set ssl_session_timeout 1d to match the cache lifetime. TLS 1.3 session tickets eliminate this cross-worker session cache concern, but TLS 1.2 clients still benefit from the shared cache.

Logging: Access log writes are buffered by default only when you specify a buffer size. High-volume access logging without buffering does a write syscall per request. Add buffer=32k flush=1s to the log path to reduce syscall overhead:

access_log /var/log/nginx/access.log main buffer=32k flush=1s;

---

Tradeoffs Comparison

Dimension	Nginx	HAProxy	Caddy	Envoy	Traefik
Architecture	Event-driven, master-worker	Event-driven, single process or multi-thread	Event-driven, Go runtime	Event-driven, C++, multi-thread	Event-driven, Go runtime
Primary use case	Web server + reverse proxy + cache	TCP/HTTP load balancer	HTTPS-first reverse proxy with auto TLS	Service mesh data plane / edge proxy	Cloud-native reverse proxy with service discovery
Config model	Declarative config file	Declarative config file	Declarative Caddyfile or JSON	xDS gRPC API (dynamic) or static YAML	Static file + dynamic discovery (Consul, K8s, Docker)
TLS automation	Manual (certbot integration)	Manual	Built-in ACME/Let’s Encrypt	Manual or via control plane	Built-in ACME/Let’s Encrypt
Active health checks	Nginx Plus only	Yes, built-in	Yes	Yes	Yes
Dynamic config reload	SIGHUP (graceful)	Runtime API (hitless)	Runtime API (hitless)	xDS, hitless	API or file watch
HTTP/3 (QUIC)	Experimental (1.25+)	No (1.x)	Yes	Yes	No
gRPC support	Yes (proxy_pass + http2)	Yes	Yes	Native, first-class	Yes
Observability	Basic access log, stub_status	Stats socket, Prometheus exporter	Prometheus metrics	Native Prometheus + tracing	Native Prometheus + tracing
Memory footprint	Very low (C, no GC)	Very low (C, no GC)	Moderate (Go GC)	Moderate to high (C++, depends on config)	Moderate (Go GC)
Plugin/extension model	C modules (compile-time) or Lua/njs	Lua (HAProxy 2.x)	Plugins via Go	Custom filters in C++/Wasm/Go	Middleware plugins in Go
Production maturity	Very high (20+ years)	Very high (20+ years)	High	High (cloud-native standard)	High
Sweet spot	High-traffic web serving, reverse proxy, static files, SSL offload	Pure TCP/HTTP load balancing, connection-level control	Developer-friendly HTTPS proxying, small teams	Kubernetes sidecar proxy, complex routing, progressive delivery	Kubernetes ingress, Docker-native, fast setup

---

Nginx’s design is a direct response to the C10K problem and it shows in every architectural decision: no threads per connection, preallocated pools, shared memory for cross-worker state, and a phase-based pipeline that keeps the hot path minimal. Understanding these internals makes the difference between tuning Nginx with intent and adjusting knobs until the symptoms go away.