TCP: Explanation & Insights

Transmission Control Protocol: the core internet protocol that delivers data reliably and in order between two computers over an unreliable network.

What It Is

TCP — the Transmission Control Protocol — is the thing that turns the internet's chaos into something you can actually program against.

Here's the chaos it's hiding. The network underneath, run by IP (the Internet Protocol), makes no promises at all. It takes a chunk of your data, slaps an address on it, and flings it toward the destination — and that's the whole contract. The chunk might arrive. It might arrive twice. It might arrive after the chunk you sent after it. It might quietly vanish in a router somewhere in Frankfurt and nobody will ever tell you. IP is a postcard tossed into a storm: cheap, fast, and entirely willing to lose your mail.

And yet when you open a webpage, every byte arrives, in order, exactly once. When you git push, the commit on the far end is bit-for-bit the one you sent. That gap — between a network that guarantees nothing and an application that expects everything — is the gap TCP fills. It is a reliable, ordered, error-checked stream of bytes built on top of a medium that offers none of those three.

The official name for the bytes, in RFC 793 — the 1981 spec that still governs how this works — is "octets." Eight bits. In 1981 a "byte" wasn't yet reliably eight bits everywhere, so the standard reached for the word that left no room for argument. Forty years on, every machine agrees a byte is eight bits, and the word "octet" survives only in old network specs, like a fly in amber — a fossil of the brief era when even that was up for debate.

Why It Matters

Nearly everything you think of as "the internet" rides on TCP. The web (HTTP and HTTPS), SSH, email, database connections, git, Docker pulls — all of it is TCP underneath. When you run ss or the older netstat and see a wall of connections, that wall is TCP. When a service "listens on a port," it is a TCP socket waiting for someone to knock.

So the day you run a server, TCP stops being trivia and starts being the thing that's either working or ruining your afternoon. A connection stuck in the wrong state, a port that won't free up after a restart, a transfer that crawls for no reason you can see — every one of those is TCP behaving exactly as designed, faithfully following a rule you didn't know you'd invoked. The machine is never being difficult. It's being obedient — to instructions written into the protocol decades before you met it.

The Three-Way Handshake

Before a single byte of your data moves, the two machines have a little three-message conversation. It looks almost too simple to need explaining, and the reason it has exactly three steps is one of the loveliest things in all of networking.

Client                                 Server
  |                                       |
  |  ──────  SYN, seq=x  ──────────────▶  |   "Let's talk. I'll number my bytes starting at x."
  |                                       |
  |  ◀────  SYN-ACK, seq=y, ack=x+1  ───  |   "Heard you. I'll start at y. Expecting your x+1 next."
  |                                       |
  |  ──────  ACK, ack=y+1  ────────────▶  |   "Heard you too. We're connected."
  |                                       |

SYN means synchronize — it carries each side's opening sequence number, the count that will let every byte be put back in order at the far end. ACK means acknowledge — "I received up to here." That's the whole handshake: SYN, SYN-ACK, ACK. Three messages, and the connection is live.

Now the question almost nobody stops to ask: why three? Why not two — you knock, I answer, done? Why not four, to be safe?

The answer is that two machines trying to agree on something over a channel that can lose messages are stuck inside a problem that was proven impossible to solve — and three is the most you can squeeze out of it before the proof catches up with you.

The Two Generals, and the Problem Nobody Can Solve

Picture two generals on two hills, an enemy city in the valley between them. They can only win if they attack at the same dawn. Their only way to coordinate is to send a messenger down through the valley — and the valley is full of the enemy, so any messenger might be caught. The first general sends word: "Attack at dawn." But did it arrive? He can't know unless a messenger comes back confirming it. So the second general sends a confirmation. But now he can't be sure his messenger made it — so he needs a confirmation of the confirmation. And so on, forever. There is no final message that makes both sides certain, because the very last message sent — whichever one it is — might be the one that's lost, and the sender of it can never know.

This is the Two Generals' Problem, and it holds a particular distinction: it was the first problem in computer communication ever proven to be unsolvable. Not "hard." Not "unsolved." Proven impossible, the way you can prove there's no largest prime — no cleverness, no future technology, no number of round-trips will ever crack it. Perfect agreement over an unreliable channel cannot be had.

And here is the thing that should stop you for a second: your server is fighting the Two Generals' Problem every time someone opens a connection to it. Two machines, a valley of routers between them that eats packets, trying to agree "we are now connected." It is the exact same problem, proven exactly as impossible.

So what does TCP do about an impossibility? The only honest thing anyone can do — it stops trying to win the argument and settles for making the failure rare. Three messages is the practical sweet spot: enough that both sides have heard from each other and heard that they were heard, which is all you can get. The third ACK might still be lost. TCP knows this. If it is, a timer fires and the SYN-ACK is sent again. TCP doesn't defeat the Two Generals — nothing defeats the Two Generals — it just keeps sending messengers until one almost certainly gets through, and gets on with its day. The whole protocol is an engineer's shrug at a mathematical impossibility, and it carries the entire internet.

Backstory

The handshake and the trick of picking a starting sequence number both trace to the original 1974 design by Vint Cerf and Bob Kahn, with Ray Tomlinson — the same man who put the @ in email — credited for the initial-sequence-number idea. They were solving the Two Generals before most people had heard of it.

How TCP Keeps the Stream Honest

Once connected, TCP has to deliver a clean, ordered byte stream over a network still doing its postcards-in-a-storm routine. Four mechanisms do the work, and each is worth knowing because each one, when it misbehaves, looks like a different server problem.

Sequence numbers. Every byte is numbered. The receiver uses the numbers to reassemble the stream in order no matter what jumble the packets arrive in — packet 7 showing up before packet 5 is fine, the numbers sort it out. This is also why a lost packet doesn't corrupt anything; it just leaves a numbered gap the receiver notices.

Acknowledgements and retransmission. The sender keeps a copy of everything it sends and starts a timer. If an ACK for that data comes back, the copy is dropped and life goes on. If the timer expires first — the retransmission timeout — TCP assumes the packet died in the valley and sends it again. It also watches for duplicate ACKs: if the far end keeps saying "still expecting byte 5000" while bytes stream past, that repetition is a flare going up — packet 5000 is missing — and TCP resends it without waiting for the timer. Loss isn't an error to TCP. It's the expected weather, and the protocol is built to sail through it.

The checksum. Every segment carries a 16-bit checksum covering the header and the data. It's a sum the receiver recomputes; if it doesn't match, the bits got mangled in flight and the segment is silently dropped, to be retransmitted like any other loss. It's a weak check by modern standards — a coarse net that catches the cosmic-ray bit-flip and the flaky cable, not a determined adversary — but it's caught an astonishing amount of quietly broken hardware over the decades.

The sliding window — flow control. Here's a problem that isn't about the network at all: what if the sender is faster than the receiver? A beefy server firehosing data at a small device would simply overflow its buffers, and the overflow gets dropped. So in every ACK, the receiver also announces its window: "I have room for this many more bytes right now." The sender never sends more than the window allows. It's the receiver, in effect, holding up a hand — not so fast, let me catch up — and the sender obeying. A window that collapses to zero is a receiver that has run out of buffer, and the whole stream stalls until it drains. More than one mysterious "the network is slow" turns out to be one end simply unable to keep up, politely throttling the other.

Note

A segment is TCP's unit of data — one chunk with a TCP header on the front. The header is 20 bytes at minimum (up to 60 with options). The largest payload a side will accept in one segment is the MSS, the maximum segment size, advertised during the handshake. Set it wrong — common with VPNs and tunnels that add their own headers — and segments get fragmented or dropped, producing connections that open fine but hang the moment real data flows. A baffling failure with a one-line cause.

The States: Reading a Connection's Life

A TCP connection is a little state machine, and when you run ss the state column tells you exactly where in its life each connection is sitting. Learn a handful and the output starts talking to you.

State	What it means
`LISTEN`	A server socket waiting for incoming connections. Your service is up.
`SYN-SENT`	This side fired a SYN and is waiting for the reply. Stuck here = the far end isn't answering (firewall, wrong port, dead host).
`SYN-RECV`	A SYN arrived, SYN-ACK sent, waiting for the final ACK. A pile of these is a sign of trouble (see below).
`ESTABLISHED`	The handshake is done. Real data flows here. This is the healthy, working state.
`FIN-WAIT-1` / `FIN-WAIT-2`	This side started closing and is waiting for the other to agree.
`CLOSE-WAIT`	The other side closed; this side hasn't yet. A heap of `CLOSE-WAIT` is almost always an application bug — your code forgot to `close()` its sockets.
`TIME-WAIT`	The connection is closed, but this side is lingering on purpose. The interesting one.

That CLOSE-WAIT line is worth tattooing somewhere. If ss shows hundreds of connections stuck in CLOSE-WAIT and climbing, the network is fine — your program received the goodbye and never said goodbye back. The kernel is faithfully holding the connection half-open, waiting for a close() call that your code never makes. It's not a leak the OS can fix; the OS is doing precisely what it was told.

TIME-WAIT, or Sitting by the Phone

The closing handshake has the same Two Generals problem as the opening one, and it surfaces in a state that confuses every admin exactly once.

When a connection closes, the side that closes first doesn't get to vanish immediately. It goes into TIME-WAIT and sits there, doing nothing, for a fixed stretch — typically 60 seconds on Linux. Why? Because the final ACK it sent might be lost in the valley. If it is, the other end will give up waiting and retransmit its FIN — "did you hear me? I'm closing" — and someone has to be home to answer. If our side had already torn the connection down, that retransmitted FIN would hit a closed door and bounce back an angry RST (reset), leaving the far end confused about how the conversation ended.

So TIME-WAIT is your machine sitting by the phone for two minutes after the call's over, in case the other person didn't quite hear you hang up. The 60 seconds is two times the MSL, the maximum segment lifetime — long enough for a lost ACK to time out and a retransmitted FIN to make the trip back. It's pure courtesy to a connection that's already dead, and it's the right thing to do.

It also bites. Restart a busy server and you may find the port "already in use" for a minute — a crowd of old connections still politely lingering in TIME-WAIT, holding the door. It looks like a bug. It's the protocol keeping a promise to connections that no longer exist.

Pro Tip

When a service won't rebind its port right after a restart, the fix is usually the SO_REUSEADDR socket option in the application, which tells the kernel "I know there are TIME-WAIT connections here, let me bind anyway." Reach for it before you reach for the kernel knobs that shorten TIME-WAIT system-wide — those trade a real safety margin for a convenience, and the bill comes later.

The Security Reality

Two pieces of TCP's design turned out to matter enormously for security, and both make better sense once you know the story.

The SYN flood. Remember SYN-RECV — the state where a server has answered a SYN and is holding resources while it waits for the final ACK. An attacker can send a flood of SYNs and never send the third message, leaving the server with thousands of half-open connections, each consuming a little memory, until the table for new connections is full and real users can't get in. It's a denial-of-service attack that costs the attacker almost nothing and weaponizes the handshake's own politeness against it. The modern defense, SYN cookies, is wonderfully sneaky: the server refuses to allocate any memory for a half-open connection and instead encodes the connection's state into the sequence number it sends back. If a real ACK returns, the number it carries lets the server reconstruct the state from nothing. It answers without remembering — and an attacker can't flood a table that was never allocated.

The sequence-number guess. A connection is identified by its addresses, ports, and sequence numbers. An attacker who can't see your traffic but can guess the current sequence number can forge packets that the receiver accepts as genuine — injecting data, or tearing the connection down. For TCP's first two decades the initial sequence number was nearly predictable, often just a counter ticking up. In 1995 this stopped being theoretical: the intrusion that made Kevin Mitnick infamous used exactly this trick, forging a trusted connection by predicting the sequence numbers. The fix, eventually standardized, was to make the initial sequence number genuinely hard to guess across the full 32-bit range — turning a one-in-a-handful guess into roughly one in four billion. A whole class of attack closed by the simple act of starting the count somewhere unpredictable.

A Famous Performance Trap

One last thing worth carrying, because it has cost real engineers real days. Two of TCP's own optimizations, each sensible alone, can deadlock against each other.

The first is Nagle's algorithm, born of a real headache: applications that send one byte at a time. A single byte of data wrapped in TCP and IP headers is around 40 bytes of overhead for 1 byte of payload — a postage stamp the size of a parcel. Nagle's fix: hold a small outgoing chunk briefly, see if more data shows up to send along with it, and bundle them. The second is delayed ACK: don't fire off a bare acknowledgement instantly: wait a fraction of a second in case you're about to send a reply you can piggyback the ACK onto.

Now watch them meet. One side is waiting (Nagle) to send more data until it gets an ACK. The other side is waiting (delayed ACK) to send that ACK until it has data to attach it to. Each is politely waiting for the other to go first. Neither does — until the delayed-ACK timer finally gives up and fires, hundreds of milliseconds later. The result is a connection that's correct, reliable, and inexplicably sluggish in fixed little stalls. The usual cure is the TCP_NODELAY socket option, which switches Nagle off for traffic that's latency-sensitive. Two good ideas, each right on its own, jamming because nobody told them about each other — the machine doing flawlessly what each was told, with no idea the other existed.

History and Philosophy

TCP and IP were born joined. The 1974 design by Vint Cerf and Bob Kahn was one protocol; only later was it split into two layers, and the wisdom of that split is the reason the internet could grow the way it did. IP does one job — get a packet toward an address, no promises — and stays gloriously simple. TCP layers reliability on top, at the endpoints, without the network in the middle needing to understand or guarantee anything. The routers between you and a server in Tokyo know nothing about your connection; all the intelligence lives at the two ends. That principle — a dumb, simple network and smart endpoints — is the architecture that let the internet scale to billions of machines nobody coordinated in advance.

Before TCP/IP, the ARPANET ran on NCP, the Network Control Protocol, which assumed a single, trusted, reliable network — it had no notion of stitching separate networks together. As the idea of an inter-network took hold, NCP's assumptions ran out. On January 1, 1983, the ARPANET threw the switch in a coordinated cutover its engineers called a "flag day": NCP off, TCP/IP on, and any machine that hadn't made the change simply lost access. That flag day is as good a birthday as any for the internet we actually use. The protocol carrying this page to you is, in its essentials, the one from that 1981 spec — old enough to draw a pension, and still moving nearly every byte you'll ever send.