Networking: The Physics of Packets

Why 'fast' internet feels slow. A deep dive into TCP Congestion Control, Head-of-Line Blocking, and the BGP map of the world.

Beginner 45 min read Expert Version →

🎯 What You'll Learn

  • Deconstruct the TCP 3-Way Handshake at the packet level
  • Analyze Head-of-Line (HOL) Blocking in HTTP/1.1 vs HTTP/2
  • Calculate Bandwidth-Delay Product (BDP) for tuning
  • Trace a BGP Route hijack
  • Differentiate MTU and MSS and why fragmentation kills performance

🔌 Try It: TCP Handshake Builder

Click packets in the right order to establish a TCP connection:

🔌 TCP Handshake Builder

Click packets in the correct order to establish a TCP connection.

💻
Client
Click packets below
🖥️
Server

⚡ Try It: Port Number Speed Run

Quick! What port is SSH? Race the clock:

⚡ Port Number Speed Run

Score: 0/0Streak: 🔥 0Best: 0

Learn common service ports! Pick the correct port for each service.

15 seconds per question. Use hints if you're stuck.

Introduction

“The Network” is not a cloud. It is a hostile environment where packets are dropped, reordered, and corrupted by cosmic rays. Your application succeeds only because TCP spends 30% of its energy apologizing for errors.

In this lesson, we stop treating the network like a reliable pipe and start treating it like a probabilistic delivery service.


The Physics: TCP Congestion Control

How does your computer know how fast to send data? If it sends too slow: Bandwidth is wasted. If it sends too fast: Router buffers overflow, packets drop, and throughput collapses.

The Solution: The Congestion Window (cwnd). TCP starts slow (Slow Start). It doubles packets every RTT. 1, 2, 4, 8, 16… DROP! When a packet drops, TCP panics. “The network is full!” It cuts cwnd in half (Reno) or by 30% (Cubic) and linearly inches back up.

Physics: Your download speed is practically a Sawtooth Wave oscillating around the bottleneck capacity.


The Bottleneck: Head-of-Line (HOL) Blocking

Imagine a one-lane highway. If the front car crashes, everyone waits. This is HOL Blocking.

HTTP/1.1 (The Crash)

You open a TCP connection. You request style.css. It takes 2 seconds. Result: You cannot request script.js until style.css arrives. The pipe is idle.

HTTP/2 (The Multi-Lane Highway)

You multiplex streams. Block style.css? Fine. script.js drives in the lane next to it. But TCP is still one lane. If a TCP Packet is lost, the Operating System holds back all HTTP/2 streams until that packet is retransmitted.

HTTP/3 (QUIC)

The solution is to ditch TCP. UDP has no lanes. QUIC implements “lanes” in userspace. One lost packet only stalls one stream.


Deep Dive: BGP (Border Gateway Protocol)

TCP is how you talk. IP is where you go. BGP is the map. The internet is a mesh of 70,000 Autonomous Systems (AS). BGP is the protocol where they whisper to each other: “I can reach IP range 192.168.0.0/16”.

The Flaw: BGP is (mostly) trust-based. If I announce “I own YouTube’s IP Address”, and enough routers believe me, I can hijack global traffic. This happens (accidentally or maliciously) every year.


Code: Inspecting the Handshake

Don’t trust the browser. Trust tcpdump.

# Capture packets on port 80
sudo tcpdump -i eth0 port 80 -n -S

# Output Physics:
# 1. SYN (Seq=0)          -> "Let's talk."
# 2. SYN-ACK (Seq=0, Ack=1) -> "Okay. I hear you."
# 3. ACK (Seq=1, Ack=1)     -> "Connection Open."
# 4. PSH (Seq=1:500)      -> "Here is 500 bytes of data."

Note the Ack number: It is not “I received packet 5”. It is “I am expecting byte number 501”. TCP counts bytes, not packets.


Practice Exercises

Exercise 1: The Bandwidth-Delay Product (Beginner)

Scenario: 1Gbps Link. 100ms RTT. Task: How much data must be “in flight” (on the wire) to fill the pipe? (Hint: 109 bits/s×0.1 s=100 Mbits=12.5 MB10^9 \text{ bits/s} \times 0.1 \text{ s} = 100 \text{ Mbits} = 12.5 \text{ MB}. If your TCP Buffer is 64KB, your speed is capped at 5Mbps!).

Exercise 2: Fragmentation (Intermediate)

Scenario: MTU (Max Transmission Unit) is 1500 bytes. Typical Ethernet. You try to send a 2000 byte UDP packet. Task: What happens? (Fragmented into 1500 + 500 headers). Why is this bad for performance?

Exercise 3: Traceroute Analysis (Advanced)

Task: Run traceroute google.com. Identify:

  1. Your Gateway.
  2. Your ISP’s backbone.
  3. The moment you enter Google’s network (AS15169).

Knowledge Check

  1. Why does TCP start huge downloads slowly?
  2. What is the difference between Flow Control and Congestion Control?
  3. Why is HTTP/3 built on UDP?
  4. What does “ACK 500” mean?
  5. Why does having a larger TCP Window Size improve speed on high-latency links?
Answers
  1. Slow Start. It probes network capacity exponentially to avoid immediate congestion.
  2. Flow: Receiver says “I’m full”. Congestion: Network router says (implicitly) “I’m full”.
  3. To avoid TCP HOL Blocking. It moves reliability logic to userspace where streams are independent.
  4. “I have received everything up to 499. Send me 500 next.”
  5. Filling the pipe. With high RTT, ACKs take a long time to return. You need to send more data “blindly” to keep the link busy. (Little’s Law again!).

Summary

  • TCP: A state machine that guesses bandwidth.
  • BGP: The glue holding the internet together (barely).
  • Latency: The ultimate speed limit. You cannot beat the speed of light.

Questions about this lesson? Working on related infrastructure?

Let's discuss