How TCP Works: The Three-Way Handshake, Flow Control, Congestion Avoidance, and Reliable Delivery From SYN to FIN

Most engineers treat TCP as a reliable pipe. Send bytes in, receive bytes out, in order, without loss. That mental model works until you need to debug a latency spike, tune a high-throughput server, or explain why HTTP/3 exists. TCP is not magic. It is a carefully designed state machine with specific behaviors for connection setup, acknowledgment, flow control, and congestion detection. Understanding those behaviors tells you exactly when TCP helps you and when it gets in your way.

This article covers the full TCP lifecycle: how connections open and close, how sequence numbers enable reliable delivery, how the sliding window controls flow, how congestion control adapts to network conditions, and where TCP’s design creates unavoidable costs. It ends with concrete tuning parameters for Linux servers.

The Connection Lifecycle

The Three-Way Handshake

Before a single byte of application data moves, TCP performs a three-way handshake to establish synchronized sequence numbers on both sides.

Client                          Server
  |                               |
  |-- SYN (seq=1000) ------------>|  Client picks ISN, enters SYN_SENT
  |                               |  Server enters SYN_RECEIVED
  |<-- SYN-ACK (seq=5000,        |
  |            ack=1001) ---------|  Server picks its own ISN, acks client ISN+1
  |                               |
  |-- ACK (seq=1001, ack=5001) -->|  Client enters ESTABLISHED
  |                               |  Server enters ESTABLISHED

Each side picks an Initial Sequence Number (ISN). The ISN is not zero. It is randomized to prevent sequence number collisions across connections reusing the same four-tuple (src IP, src port, dst IP, dst port) and to mitigate TCP hijacking attacks.

The SYN-ACK serves two purposes in a single segment: it acknowledges the client’s SYN (ack=1001 means “I received everything up to 1000, expecting 1001 next”) and announces the server’s own ISN. The third ACK completes the exchange.

The cost here is one full round-trip before data can flow. This is not an accident. Both sides need to confirm the other can send and receive before committing to the connection. The ISN exchange also protects against stale segments from a previous connection confusing the new one.

The FIN Handshake

Closing is a four-way exchange because TCP connections are full-duplex. Either side can independently close its send direction while the other side continues sending.

Client                          Server
  |                               |
  |-- FIN (seq=2000) ------------>|  Client enters FIN_WAIT_1
  |<-- ACK (ack=2001) ------------|  Server enters CLOSE_WAIT
  |                               |  Server still has data to send
  |<-- FIN (seq=6000) ------------|  Server done sending, enters LAST_ACK
  |-- ACK (ack=6001) ------------>|  Client enters TIME_WAIT (2*MSL)
  |                               |  Server enters CLOSED

The TIME_WAIT state on the client side lasts 2 * MSL (Maximum Segment Lifetime, typically 60 seconds). This prevents a stale ACK from a prior connection from disrupting a new connection on the same four-tuple. It is a frequent pain point on servers making many short-lived outbound connections: ports exhaust before TIME_WAIT expires.

Reliable Delivery: Sequence Numbers and Acknowledgments

Every TCP segment carries a sequence number identifying the position of its first byte in the byte stream. The receiver sends cumulative acknowledgments: ack=N means “I have received all bytes up to N-1, send me N next.”

// Simplified segment tracking on the sender side
interface Segment {
  seqStart: number;
  seqEnd: number;   // exclusive
  data: Buffer;
  sentAt: number;   // timestamp for RTT estimation
  acked: boolean;
}

class TcpSendBuffer {
  private segments: Segment[] = [];
  private nextSeq = 0;
  private unackedBase = 0;

  enqueue(data: Buffer): void {
    const seg: Segment = {
      seqStart: this.nextSeq,
      seqEnd: this.nextSeq + data.length,
      data,
      sentAt: Date.now(),
      acked: false,
    };
    this.segments.push(seg);
    this.nextSeq += data.length;
  }

  acknowledge(ackNum: number): void {
    // Cumulative: everything before ackNum is acknowledged
    this.segments = this.segments.filter(s => s.seqEnd > ackNum || !s.acked);
    this.unackedBase = ackNum;
  }

  getUnacked(): Segment[] {
    return this.segments.filter(s => s.seqStart >= this.unackedBase && !s.acked);
  }
}

If an acknowledgment does not arrive within the retransmission timeout (RTO), the sender retransmits the oldest unacknowledged segment. RTO is computed from RTT measurements using an exponential weighted moving average (Karn’s algorithm excludes retransmitted segments from RTT samples to avoid ambiguity).

Selective Acknowledgments (SACK)

The base cumulative ACK scheme has a problem: if segment 3 is lost but segments 4 and 5 arrive, the receiver can only say “send me 3.” The sender does not know 4 and 5 arrived and may retransmit them unnecessarily.

SACK (RFC 2018) adds an option block that reports which non-contiguous ranges the receiver has buffered. The sender uses SACK information to retransmit only the specific missing segments. Modern Linux kernels use SACK by default (net.ipv4.tcp_sack = 1).

Flow Control: The Sliding Window

Flow control prevents the sender from overwhelming the receiver’s buffer. The receiver advertises a receive window (rwnd) in every ACK: the number of bytes it can currently buffer beyond what it has acknowledged.

Sender can have at most rwnd bytes in flight (sent but not yet acked).

[--- acknowledged ---][--- in flight ---][--- can send ---][--- blocked ---]
                      <---   rwnd    --->

As the receiver drains its buffer (application reads data), it advertises a larger rwnd, allowing the sender to push more data. If the application is slow to read, rwnd shrinks. At zero, the sender pauses and probes periodically with zero-window probes to detect when the window re-opens.

The receive window size is a 16-bit field in the TCP header, capped at 65,535 bytes. The window scaling option (RFC 1323) lets both sides negotiate a shift factor at connection setup, scaling rwnd up to 1 GB. Without window scaling, a connection with 100ms RTT is bandwidth-limited to about 5 Mbps regardless of available network capacity.

Congestion Control

Flow control handles the receiver’s buffer. Congestion control handles the network between sender and receiver. TCP infers network congestion from packet loss (timeout or duplicate ACKs) and uses that signal to adjust how aggressively it sends.

The sender maintains a congestion window (cwnd) separate from rwnd. Effective window = min(cwnd, rwnd). The sender never has more than this many bytes in flight.

Slow Start

At connection open, cwnd starts at a small value (10 segments by default since RFC 6928). For each ACK received, cwnd increases by one segment. This produces exponential growth: cwnd doubles each RTT.

RTT 1: cwnd = 10 segments, 10 ACKs received -> cwnd = 20
RTT 2: cwnd = 20 segments, 20 ACKs received -> cwnd = 40
RTT 3: cwnd = 40 segments ...

Slow start runs until cwnd reaches the slow start threshold (ssthresh) or a loss event occurs.

Congestion Avoidance

Once cwnd >= ssthresh, TCP switches from exponential to linear growth: cwnd increases by one segment per RTT regardless of how many ACKs arrive. This probes for additional capacity cautiously.

Fast Retransmit and Fast Recovery

When the receiver gets an out-of-order segment, it sends an immediate duplicate ACK for the last in-order byte. Three duplicate ACKs tell the sender “one segment is missing, but data is still flowing.” Rather than waiting for an RTO, the sender immediately retransmits the missing segment (fast retransmit).

After fast retransmit, TCP Reno halves cwnd and ssthresh (multiplicative decrease) and enters congestion avoidance from the new cwnd. TCP CUBIC (the Linux default since kernel 2.6.19) uses a cubic function of elapsed time since the last congestion event to compute cwnd, which recovers faster on high-bandwidth-delay-product paths without being overly aggressive on short-RTT paths.

// Simplified TCP Reno congestion state machine
type CongestionState = "slow_start" | "congestion_avoidance" | "fast_recovery";

class CongestionController {
  cwnd: number = 10;         // segments
  ssthresh: number = 65535;  // segments (effectively unlimited initially)
  state: CongestionState = "slow_start";
  private dupAckCount = 0;

  onAck(): void {
    if (this.state === "slow_start") {
      this.cwnd += 1;
      if (this.cwnd >= this.ssthresh) {
        this.state = "congestion_avoidance";
      }
    } else if (this.state === "congestion_avoidance") {
      this.cwnd += 1 / this.cwnd; // one segment per RTT
    } else if (this.state === "fast_recovery") {
      this.cwnd = this.ssthresh;
      this.state = "congestion_avoidance";
      this.dupAckCount = 0;
    }
  }

  onDuplicateAck(): void {
    this.dupAckCount += 1;
    if (this.dupAckCount === 3) {
      // Fast retransmit
      this.ssthresh = Math.max(this.cwnd / 2, 2);
      this.cwnd = this.ssthresh + 3;
      this.state = "fast_recovery";
    } else if (this.state === "fast_recovery") {
      this.cwnd += 1; // inflate during recovery
    }
  }

  onTimeout(): void {
    this.ssthresh = Math.max(this.cwnd / 2, 2);
    this.cwnd = 1; // back to slow start
    this.state = "slow_start";
    this.dupAckCount = 0;
  }
}

Nagle’s Algorithm and Delayed ACKs

Nagle’s algorithm (RFC 896) addresses a specific problem: small writes from the application (say, a character-by-character terminal session) generate a stream of tiny segments, each carrying 40 bytes of TCP/IP header for 1 byte of payload. Nagle says: if there is unacknowledged data in flight, buffer small writes and send them as one segment when the ACK arrives or when the buffer reaches MSS.

Delayed ACKs (RFC 1122) are a receiver optimization: instead of sending an ACK for every segment, wait up to 40ms and batch ACKs together, or piggyback the ACK on outbound data. This halves ACK traffic on bidirectional connections.

These two optimizations interact badly. With Nagle on and delayed ACKs on, a write-write-read sequence can produce a 40ms stall:

1. Client sends request part 1 (< MSS)
2. Client sends request part 2 (< MSS). Nagle buffers it because part 1 is unacked.
3. Server receives part 1, waits up to 40ms to ACK (delayed ACK).
4. After 40ms, server sends ACK. Nagle releases part 2.
5. Server receives part 2, sends response.

Total delay: up to 40ms added to the RTT for no useful reason.

The fix is TCP_NODELAY. Disabling Nagle sends every write immediately.

import * as net from "net";

const server = net.createServer((socket) => {
  // Disable Nagle for latency-sensitive connections (RPC, real-time APIs)
  socket.setNoDelay(true);

  socket.on("data", (data) => {
    // Process and respond; response is sent immediately, no buffering
    const response = processRequest(data);
    socket.write(response);
  });
});

When to keep Nagle on: bulk data transfers (file sync, database dumps) where throughput matters more than latency and small writes are not in the hot path.

When to set TCP_NODELAY: any request-response protocol, RPC, database wire protocols, real-time telemetry, or interactive SSH sessions.

Head-of-Line Blocking

TCP delivers bytes in order. If segment N is lost, segments N+1, N+2, … sit in the receive buffer, ready, but the application cannot read them until N is retransmitted and delivered.

For HTTP/1.1 pipelining and HTTP/2, this is a structural problem. HTTP/2 multiplexes multiple logical streams over one TCP connection. A single lost segment can block all streams simultaneously. A video stream, an API call, and a static asset fetch all pause waiting for one retransmission, even if the lost segment belonged to only one of them.

This is TCP head-of-line blocking. It is not a bug in the protocol design (TCP’s ordering guarantee is the point), but it is an inherent cost of multiplexing at the application layer over a transport that serializes delivery.

QUIC and HTTP/3

QUIC (RFC 9000) reimplements the transport layer on top of UDP, designed from the ground up to solve TCP’s structural limitations for modern HTTP workloads.

1. Connection setup without RTT cost

TCP + TLS 1.3 requires 1 RTT for TCP handshake and 1 RTT for TLS handshake (or 0.5 RTT with TLS 1.3 combined with the TCP handshake). QUIC combines transport and cryptographic handshake into a single 1-RTT exchange on first connection. Subsequent connections to the same server can use 0-RTT resumption from a prior session token, sending application data with zero round trips.

2. Stream-level multiplexing without head-of-line blocking

QUIC implements multiplexed streams natively. Each stream has its own independent ordering guarantee. A lost packet affects only the stream(s) whose data it carries. Other streams continue to deliver without waiting.

3. Connection migration

QUIC connections are identified by a connection ID, not the four-tuple. When a mobile client switches from Wi-Fi to LTE (IP changes, port changes), the QUIC connection survives. TCP connections break and require full re-establishment.

4. Encryption is mandatory

QUIC encrypts all payload and most headers with TLS 1.3 as a non-optional component. There is no plaintext QUIC. This eliminates the class of middlebox interference that has historically made deploying new TCP options difficult.

The cost: QUIC’s complexity lives in userspace, not kernel code. CPU overhead per connection is higher than kernel TCP. Servers with very high connection density and commodity CPUs may see CPU before bandwidth as the bottleneck. UDP also shares limited OS send/receive buffer with less mature kernel optimization than TCP paths.

Tradeoffs: TCP vs UDP vs QUIC

Property	TCP	UDP	QUIC
Reliable delivery	Yes, built-in	No, application must handle	Yes, per stream
Ordering	Strict, per connection	None	Per stream, not across streams
Latency (new connection)	1 RTT + TLS (1-2 RTT)	0	1 RTT (0-RTT resume)
Head-of-line blocking	Yes, connection-level	No	No (stream-level only)
Multiplexing	No native (HTTP/2 adds it)	No	Yes, native
Connection migration	No	No	Yes (connection ID)
Congestion control	Kernel, mature	None (implement yourself)	Userspace, improving
Middlebox traversal	Near-universal	Firewalls may block	Better than UDP alone
CPU overhead	Low (kernel path)	Low	Higher (userspace)
Encryption	Optional (TLS)	Optional (DTLS)	Mandatory (TLS 1.3)
Best for	General-purpose, file transfer, databases	Real-time media, game state	HTTP/3, mobile, high-latency paths

Production Considerations

Increase the receive and send buffers

The kernel defaults are conservative for modern networks. On a server with 100ms paths to clients, the default 87KB socket buffer limits throughput to under 7 Mbps per connection.

# Add to /etc/sysctl.conf
net.core.rmem_max = 134217728        # 128 MB max receive buffer
net.core.wmem_max = 134217728        # 128 MB max send buffer
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
net.ipv4.tcp_mem = 786432 1048576 26777216

Enable window scaling and timestamps

These should already be on in modern kernels but verify:

net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1    # Required for PAWS and RTT measurement
net.ipv4.tcp_sack = 1

Handle TIME_WAIT exhaustion on outbound connection servers

Servers that make many short-lived outbound connections (service mesh sidecars, API gateways, scrapers) exhaust ports before TIME_WAIT clears.

net.ipv4.tcp_tw_reuse = 1      # Reuse TIME_WAIT sockets for new outbound connections
net.ipv4.ip_local_port_range = 1024 65535   # Expand ephemeral port range

Never enable tcp_tw_recycle on kernels older than 4.12. It was removed in 4.12 for good reason: it breaks connections from clients behind NAT.

Tune the SYN backlog for bursty accept patterns

net.ipv4.tcp_max_syn_backlog = 4096
net.core.somaxconn = 4096

Match somaxconn in your application’s listen() call. A mismatch silently caps the queue at the lower of the two values.

Set TCP_NODELAY on all RPC and API sockets

This is the single highest-impact TCP setting for latency-sensitive workloads. Nagle plus delayed ACKs can add 40ms to every request-response on connections with small payloads. Disable it at the socket level or at the framework level (most HTTP servers expose this as a configuration flag).

Monitor retransmission rates

High retransmission rates indicate network congestion, packet loss, or buffer bloat between your server and clients. Watch ss -s for retransmit counts or instrument with:

netstat -s | grep -i retransmit
# Or with ss:
ss -ti | grep retrans

A retransmission rate above 1-2% under normal load is a signal worth investigating before it becomes a user-facing latency problem.

Choose TCP_CORK for bulk streaming

TCP_CORK (Linux-specific) suppresses segment transmission until the cork is removed or the segment fills to MSS. Use it when you are building up a response from multiple write() calls and want to ensure they flush as a single segment.

import * as net from "net";

// Pseudocode: Linux-specific socket option not directly in Node.js net module
// Use at the native addon layer or configure via SO_SNDBUF behavior
// Most frameworks (http2, fastify) handle this correctly by default

For HTTP/2 servers, trust the framework’s write coalescing. For raw TCP servers, consider whether you need Nagle (batch small writes) or TCP_NODELAY (minimize latency) and set accordingly. They are mutually exclusive choices based on your workload shape.

TCP’s design is four decades old and still moves the majority of internet traffic. Understanding how the handshake, sequence numbers, flow control, and congestion algorithms interact is the foundation for diagnosing latency problems, reasoning about QUIC’s design decisions, and making informed tuning choices in production. Most of what goes wrong with TCP in real deployments comes down to a handful of mismatched defaults: buffer sizes too small for the bandwidth-delay product, Nagle on when you need low latency, TIME_WAIT exhaustion from connection churn, and untuned backlog limits under bursty load.