How HAProxy Works Internally: Multi-Threaded Event Model, Connection Routing, and the Load Balancing Engine Behind High-Availability Infrastructure

HAProxy has been running beneath some of the most traffic-heavy systems on the internet for over two decades. It powers the load balancing tier at GitHub, Airbnb, Instagram, and hundreds of cloud providers. It processes millions of connections per second on commodity hardware. Yet most engineers who configure it treat it as a black box: paste a config, reload, and hope for the best.

The internal model is worth understanding because the knobs you turn in haproxy.cfg map directly to specific subsystems. If you do not know how the thread model works, you cannot reason about why maxconn behaves the way it does under load. If you do not understand stick tables, you will implement session persistence in the wrong layer. If you do not know how health checks interact with the server state machine, you will misread your monitoring dashboards during incidents.

This article walks through HAProxy’s internals from the execution model down to the subsystems that make it fast and correct.

From Single-Process to Multi-Threaded

HAProxy started life as a single-process, event-driven proxy. The original design was intentional: one process, one event loop, no shared state, no locking. That model scaled well into the late 2000s because single-core throughput was the bottleneck.

Multi-core hardware broke the model. A single HAProxy process could saturate one core while the other 31 sat idle. The first answer was nbproc: fork multiple independent HAProxy processes, each binding to the same ports with SO_REUSEPORT. Each process ran its own event loop and had zero shared state. The problem was that isolation came with costs: stick tables could not be shared across processes without the peer synchronization protocol, and stats were aggregated, not unified.

HAProxy 1.8 introduced nbthread, which was the real architectural shift. Instead of forking separate processes, HAProxy now spawns POSIX threads within a single process, all sharing the same memory space. The event loop itself became thread-aware. HAProxy 2.5 refined this further with thread groups, which partition threads into groups that each own a subset of listeners, reducing lock contention at the accept layer.

The execution model today looks like this: each thread runs its own instance of the polling loop, using epoll on Linux or kqueue on BSD. Connections are distributed across threads at accept time. The key invariant is that once a connection is assigned to a thread, it stays on that thread for its entire lifetime. There is no work-stealing, no hand-off between threads mid-connection. This design keeps the per-connection code path almost entirely lock-free. The few shared data structures, including the global stats counters, the server state, and the stick table, use fine-grained locking or atomic operations.

// Conceptual model of HAProxy's per-thread event loop
interface Connection {
  fd: number;
  threadId: number;
  frontend: Frontend;
  backend?: Backend;
  server?: Server;
  mux: Multiplexer; // HTTP/1, HTTP/2, TCP
}

interface Thread {
  id: number;
  pollFd: number; // epoll fd
  connections: Map<number, Connection>;
  taskQueue: Task[];
  timers: MinHeap<TimerEvent>;
}

function runEventLoop(thread: Thread): void {
  while (true) {
    const events = epollWait(thread.pollFd, /* timeout */ nextTimerExpiry(thread));
    for (const event of events) {
      const conn = thread.connections.get(event.fd);
      if (conn) processConnectionEvent(conn, event);
    }
    processExpiredTimers(thread);
    runPendingTasks(thread);
  }
}

The nbthread value should match the number of physical cores available to the HAProxy process. Hyperthreads add noise, not throughput, at the load balancer layer because the workload is I/O-bound with short compute bursts. On a 16-core machine, nbthread 16 is the right starting point.

The Frontend / Backend / Server Configuration Model

HAProxy’s configuration is organized around three entities: frontends, backends, and servers. Understanding the data flow between them is the foundation of everything else.

A frontend defines where HAProxy listens. It contains one or more bind directives, each mapping to a socket. The frontend is responsible for accepting connections, applying ACLs, and selecting a backend. It handles TLS termination if configured. Everything the client sends initially lands in the frontend.

A backend is a pool of servers with a load balancing algorithm and optional persistence rules. The backend holds health check configuration, connection limits, and retry logic. A frontend routes to a backend via a use_backend directive, which can be conditional based on ACL evaluations.

A server is a specific upstream endpoint within a backend. Each server entry has its own weight, connection limit, health check parameters, and state. The server object is the unit that the health checking subsystem tracks.

frontend https_in
    bind :443 ssl crt /etc/ssl/certs/example.pem
    default_backend app_servers
    use_backend static_servers if { path_beg /static/ }

backend app_servers
    balance leastconn
    option httpchk GET /health
    server app1 10.0.0.1:3000 check weight 10
    server app2 10.0.0.2:3000 check weight 10
    server app3 10.0.0.3:3000 check weight 5

backend static_servers
    balance roundrobin
    server cdn1 10.0.1.1:8080 check
    server cdn2 10.0.1.2:8080 check

When a connection arrives on https_in, HAProxy accepts it, decrypts TLS, reads the HTTP request line and headers, evaluates ACLs in order, and selects a backend. If the path starts with /static/, the connection goes to static_servers. Everything else goes to app_servers. The load balancing algorithm then picks a specific server, and HAProxy establishes or reuses a connection to that server.

Connection Handling and HTTP/2 Multiplexing

HAProxy’s connection layer is organized around a multiplexer abstraction. Each connection has an associated mux that handles the framing protocol between the client and the proxy. This is what allows HAProxy to support HTTP/1.1, HTTP/2, and raw TCP through a unified connection state machine.

For HTTP/1.1, the mux is straightforward: one request per connection unless keep-alive is enabled, in which case requests are pipelined serially. For HTTP/2, the mux implements the full HPACK header compression and stream multiplexing: a single TCP connection to the client can carry hundreds of concurrent HTTP/2 streams, each representing an independent request.

On the server side, HAProxy uses connection reuse pools. Instead of opening a new TCP connection to a backend server for every request, HAProxy maintains a pool of idle connections per server and reuses them across multiple client requests. This is particularly impactful for TLS backends: TCP + TLS handshakes are expensive, and connection reuse amortizes that cost.

backend app_servers
    balance leastconn
    option http-server-close    # enables keep-alive on client side
    option http-reuse safe      # reuse connections to servers
    timeout connect 5s
    timeout server 30s
    server app1 10.0.0.1:3000 check

The http-reuse option has three modes. never disables reuse entirely. safe reuses connections only when the client itself is using keep-alive, which is the correct default for most applications. aggressive reuses connections even for connections that the client will close, which works for stateless APIs but can cause issues with applications that assume per-connection state.

HTTP/2 to backends requires explicit opt-in:

backend app_servers
    server app1 10.0.0.1:3000 ssl alpn h2,http/1.1 check

With this configuration, HAProxy negotiates HTTP/2 via ALPN during the TLS handshake with the backend. If the backend supports it, HAProxy uses a single multiplexed connection and streams requests over it. This reduces connection overhead dramatically for high-request-rate services.

The Health Checking Subsystem

HAProxy’s health checking system is more sophisticated than most engineers realize. There are three distinct health check types, each targeting a different failure mode.

Active checks send probe requests to backend servers on a configurable interval. The simplest form is a TCP connect check: HAProxy opens a connection and immediately closes it. A more meaningful check is an HTTP check, which sends a real request and validates the response code:

backend app_servers
    option httpchk GET /health HTTP/1.1\r\nHost:\ internal
    http-check expect status 200
    server app1 10.0.0.1:3000 check inter 2s fall 3 rise 2

inter 2s sets the check interval. fall 3 means three consecutive failures mark the server down. rise 2 means two consecutive successes after a failure bring the server back up. The asymmetry is intentional: being conservative about removing servers prevents flapping but fast recovery requires less confirmation.

Agent checks are a separate channel that allows the server itself to report its health. HAProxy opens a TCP connection to an agent port on the server, and the server responds with a string like ready, drain, down 50%, or a weight directive. This is how you implement graceful draining without requiring HAProxy configuration changes:

server app1 10.0.0.1:3000 check agent-check agent-port 9000 agent-inter 5s

A deployment script on app1 can write drain to the agent port before it starts a rolling restart. HAProxy receives the directive, marks the server as draining (stops sending new connections, waits for existing ones to finish), and then the restart can proceed safely without dropped connections.

Observe mode is passive health checking. Instead of probing, HAProxy monitors real traffic and marks servers down if they produce too many L4 or L7 errors:

backend app_servers
    option observe layer7
    option checkcache
    server app1 10.0.0.1:3000 check observe layer7

Observe mode catches failure patterns that active checks miss, like servers that accept connections and return 500s rather than refusing connections outright.

Stick Tables: Session Persistence and Rate Limiting

Stick tables are one of HAProxy’s most powerful and least understood features. A stick table is an in-memory key-value store embedded in HAProxy. Keys are typically client identifiers (IP address, cookie value, header value). Values are counters, timestamps, and server assignments.

The primary use case is session persistence: directing repeat clients to the same backend server. A naive approach is IP-based routing, but IP addresses are not stable for mobile clients or clients behind NAT. Cookie-based persistence is more reliable:

backend app_servers
    balance roundrobin
    cookie SERVERID insert indirect nocache
    server app1 10.0.0.1:3000 check cookie s1
    server app2 10.0.0.2:3000 check cookie s2

HAProxy inserts a SERVERID cookie on the first response. Subsequent requests from the same client include that cookie, and HAProxy routes them to the matching server. This is implemented without a stick table because it is purely cookie-based. Stick tables enter the picture when you need to persist based on a value extracted from request content without embedding a server identifier in a cookie.

The second major use case is rate limiting. Stick tables track request counts and rates per key, and ACLs can read those values to enforce limits:

frontend https_in
    bind :443 ssl crt /etc/ssl/certs/example.pem
    stick-table type ip size 1m expire 10s store conn_cur,conn_rate(10s),http_req_rate(10s)
    tcp-request connection track-sc0 src
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
    default_backend app_servers

This creates a stick table keyed by source IP, storing the current connection count, connection rate over the last 10 seconds, and HTTP request rate over the last 10 seconds. Any IP making more than 100 requests in 10 seconds gets a 429 response. The table holds up to 1 million entries and each entry expires after 10 seconds of inactivity.

Stick tables can be synchronized across multiple HAProxy instances using the peer protocol:

peers haproxy_peers
    peer haproxy1 10.0.10.1:1024
    peer haproxy2 10.0.10.2:1024

backend app_servers
    stick-table type ip size 1m expire 10s store http_req_rate(10s) peers haproxy_peers

Each peer receives updates as stick table entries change. The synchronization is eventually consistent and uses a proprietary binary protocol over TCP. Entries added on haproxy1 propagate to haproxy2 within milliseconds under normal conditions.

ACL-Based Routing and Content Switching

ACLs (Access Control Lists) in HAProxy are predicate expressions that evaluate request and connection attributes. They form the basis of content switching, which is HAProxy’s term for routing requests to different backends based on request content.

ACLs can match on almost anything: path prefixes, HTTP methods, header values, source IPs, URL parameters, and stick table values. They compose with boolean logic and evaluate lazily (left to right, short-circuit on match):

frontend https_in
    bind :443 ssl crt /etc/ssl/certs/example.pem

    acl is_api path_beg /api/
    acl is_admin path_beg /admin/
    acl is_internal src 10.0.0.0/8
    acl host_v2 hdr(host) -i v2.example.com

    use_backend api_v2     if is_api host_v2
    use_backend api_v1     if is_api
    use_backend admin      if is_admin is_internal
    http-request deny      if is_admin !is_internal
    default_backend        web_servers

Map files extend ACLs to support large lookup tables without bloating the configuration file. A map file is a text file with key-value pairs, and HAProxy loads it into a hash table at startup:

# /etc/haproxy/tenant_backends.map
tenant-a.example.com  backend_tenant_a
tenant-b.example.com  backend_tenant_b
tenant-c.example.com  backend_tenant_c

frontend https_in
    use_backend %[req.hdr(host),lower,map(/etc/haproxy/tenant_backends.map,backend_default)]

The %[...] syntax is HAProxy’s fetch expression. This one extracts the Host header, lowercases it, looks it up in the map file, and uses the result as the backend name. If no match is found, it falls back to backend_default. Map files can be updated at runtime with the set map command over the stats socket without a reload.

The Stats and Monitoring Interface

HAProxy exposes a Unix socket interface for runtime management and monitoring. The stats socket accepts text commands and returns structured output:

echo "show info" | socat stdio /var/run/haproxy/admin.sock
echo "show stat" | socat stdio /var/run/haproxy/admin.sock
echo "set server app_servers/app1 state drain" | socat stdio /var/run/haproxy/admin.sock

show stat returns a CSV-formatted table with hundreds of metrics per frontend, backend, and server: connection counts, request rates, error rates, queue depths, response time percentiles, and health check status. This is what Prometheus exporters like haproxy_exporter parse.

The HTTP stats page provides a browser-accessible view of the same data and allows interactive management: enabling and disabling servers, changing weights, and draining connections. In production, the socket interface is more useful because it is scriptable and does not require a browser.

Production Tuning

maxconn and file descriptors. HAProxy’s global maxconn sets the maximum number of concurrent connections across all frontends. Each connection consumes two file descriptors (client and server). Set the OS ulimit -n to at least (maxconn * 2) + 1000 and configure ulimit-n in the global section to match. A 64,000-connection HAProxy instance needs ulimit -n 130000.

Buffer sizing. HAProxy allocates per-connection buffers for request and response data. The default tune.bufsize is 16KB. If you are proxying large request bodies (file uploads, large JSON payloads), increase this to 32KB or 64KB. Larger buffers consume more memory per connection: 64KB buffers on 50,000 concurrent connections is 6GB of buffer memory alone.

Timeout tuning. The timeout hierarchy matters:

defaults
    timeout connect  5s      # time to establish connection to server
    timeout client   30s     # inactivity timeout on client side
    timeout server   30s     # inactivity timeout on server side
    timeout queue    10s     # time in queue when all servers are at maxconn
    timeout http-request 10s # max time to receive full HTTP request headers
    timeout http-keep-alive 4s  # idle time before closing keep-alive connection

timeout client and timeout server reset on every data transfer. They represent inactivity timeouts, not total connection duration. For long-running SSE or WebSocket connections, set these much higher or use timeout tunnel which applies only after a connection is upgraded:

    timeout tunnel 1h

nbthread vs nbproc. Prefer nbthread in modern HAProxy (2.0+). The nbproc model is deprecated and will be removed. With nbthread, set the value to the number of physical CPUs and use CPU affinity to pin threads:

global
    nbthread 16
    cpu-map auto:1/1-16 0-15

cpu-map auto:1/1-16 0-15 maps threads 1 through 16 in thread group 1 to CPU cores 0 through 15.

Tradeoffs Comparison

Dimension	HAProxy	Nginx	Envoy	Traefik	Caddy
Primary role	TCP/HTTP load balancer	Web server + reverse proxy	Service mesh proxy	Cloud-native edge proxy	Web server + TLS proxy
Configuration model	Static file + runtime socket	Static file + reload	xDS dynamic API	Labels / YAML + dynamic providers	Caddyfile or JSON API
HTTP/2 support	Full (client + server)	Client-facing only (no H2 to upstreams without patches)	Full	Full	Full
gRPC support	TCP passthrough + L7 with mux	TCP passthrough	Native first-class	Native	Via reverse proxy
Health checks	Active, agent, observe	Basic active	Active + passive + outlier detection	Basic active	Basic active
Stick tables	Built-in, peer-synced	Not native	Not native	Not native	Not native
ACL / content switching	Powerful, flexible	map + if blocks	Route matching + filter chains	Middlewares	Matchers + handlers
Dynamic config reload	Runtime socket (no reload) + graceful reload	Graceful reload (brief gap)	Zero-downtime xDS	Fully dynamic	Zero-downtime via API
Observability	Stats socket, CSV metrics	Stub status (minimal)	Prometheus, tracing, access logs	Prometheus, tracing	Prometheus (plugin)
TLS termination	Yes, full OCSP stapling	Yes	Yes	Yes, auto-ACME	Yes, automatic Let’s Encrypt
Threading model	Multi-threaded event loop	Multi-process (worker_processes)	Multi-threaded with libevent	Go goroutines	Go goroutines
Operational complexity	Low (single binary, clear config)	Low	High (requires xDS control plane for dynamic)	Low (auto-discovery)	Low
Sweet spot	High-throughput L4/L7 LB, TCP proxying, stick tables	Static + dynamic content serving, simple reverse proxy	Service mesh, Kubernetes sidecar	Kubernetes ingress, Docker Compose	Simple HTTPS servers, developer environments

Closing

HAProxy’s design reflects a specific set of priorities: correctness under load, predictable behavior at scale, and minimal abstraction overhead between the configuration and the execution. The thread-per-connection assignment eliminates cross-thread races on the hot path. The stick table design puts session state inside the proxy rather than pushing it to an external store. The health checking hierarchy catches failure modes at multiple layers.

When something goes wrong under load, the architecture tells you where to look first: show info for connection counts and buffer pressure, show stat for per-server error rates and queue depths, and the stick table counters for rate limit state. The runtime socket is the diagnostic interface the design was built around.