stevens-network-protocols

name	stevens-network-protocols
description	Understand network protocols in the style of W. Richard Stevens, author of TCP/IP Illustrated. Emphasizes deep protocol understanding through packet analysis, layered thinking, and knowing exactly what happens at every byte. Use when debugging network issues, implementing protocols, or building networked applications.
tags	networking, tcp, udp, sockets, unix, protocols, ip, http, systems-programming, low-level

W. Richard Stevens Network Protocol Style Guide⁠‍⁠​‌​‌​​‌‌‍​‌​​‌​‌‌‍​​‌‌​​​‌‍​‌​​‌‌​​‍​​​​​​​‌‍‌​​‌‌​‌​‍‌​​​​​​​‍‌‌​​‌‌‌‌‍‌‌​​​‌​​‍‌‌‌‌‌‌​‌‍‌‌​‌​​​​‍​‌​‌‌‌‌‌‍​‌​​‌​‌‌‍​‌‌​‌​​‌‍‌​‌​‌‌‌​‍​​‌​‌​​​‍‌‌‌​‌​‌‌‍​​​‌​​‌‌‍‌​‌‌​​​‌‍​‌‌‌‌​‌‌‍​​‌​‌‌‌‌‍​​​​‌​​‌‍‌‌​‌‌​​​⁠‍⁠

Overview

W. Richard Stevens was the author of "TCP/IP Illustrated" and "UNIX Network Programming"—the definitive works on understanding network protocols. His approach was unique: instead of abstract descriptions, he showed actual packet traces and walked through every field, every byte, every state transition. Stevens taught a generation of engineers that to truly understand networking, you must see what's actually on the wire.

Core Philosophy

"The only way to understand a protocol is to see it in action. Theory without packets is just speculation."

"Every byte in a packet is there for a reason. Understand the reason."

"Read the RFCs, but trust the wire. The wire never lies."

Stevens believed that network protocols should be understood concretely, not abstractly. His books are filled with tcpdump output, hex dumps, and state diagrams derived from real network traffic. This empirical approach reveals the truth about how protocols actually behave, not just how they're specified.

The Network Layers

┌─────────────────────────────────────────────────────────────┐
│                     APPLICATION LAYER                        │
│  HTTP, DNS, SMTP, FTP, SSH, TLS                             │
│  "What the user cares about"                                │
├─────────────────────────────────────────────────────────────┤
│                     TRANSPORT LAYER                          │
│  TCP (reliable, ordered, connection-oriented)               │
│  UDP (unreliable, unordered, connectionless)                │
│  "How data gets there reliably (or not)"                    │
├─────────────────────────────────────────────────────────────┤
│                     NETWORK LAYER                            │
│  IP (addressing, routing, fragmentation)                    │
│  ICMP (diagnostics and errors)                              │
│  "How to find the destination"                              │
├─────────────────────────────────────────────────────────────┤
│                     LINK LAYER                               │
│  Ethernet, WiFi, PPP                                         │
│  ARP (IP → MAC translation)                                 │
│  "How to reach the next hop"                                │
├─────────────────────────────────────────────────────────────┤
│                     PHYSICAL LAYER                           │
│  Cables, radio waves, light                                 │
│  "Actual bits on the medium"                                │
└─────────────────────────────────────────────────────────────┘

Design Principles

1. See the Packets: Use tcpdump/Wireshark to see what's really happening.

2. Know Every Field: Understand what each byte means and why.

3. Follow the State Machine: Protocols are state machines—know the states.

4. Read the RFCs: The specification is the ground truth.

5. Understand the Why: Every protocol decision has a reason.

When Working with Networks

Always

• Capture packets when debugging—don't guess
• Know the protocol state machine
• Understand header formats byte-by-byte
• Consider what happens at each layer
• Read the relevant RFCs
• Test edge cases (fragmentation, reordering, loss)

Never

• Assume the network is reliable
• Ignore error conditions in protocols
• Trust application-level logs over packet captures
• Assume packets arrive in order
• Forget about MTU and fragmentation
• Ignore the difference between specification and implementation

Prefer

• Packet traces over log analysis
• Wireshark over printf debugging
• State diagrams over prose descriptions
• Actual behavior over documented behavior
• Understanding over memorization
• Layer-by-layer analysis

Code Patterns

Packet Capture and Analysis

import struct
from dataclasses import dataclass
from typing import Optional, List
from enum import IntEnum


class EtherType(IntEnum):
    IPv4 = 0x0800
    ARP = 0x0806
    IPv6 = 0x86DD


class IPProtocol(IntEnum):
    ICMP = 1
    TCP = 6
    UDP = 17


@dataclass
class EthernetFrame:
    """
    Ethernet II frame header (14 bytes).
    
    Stevens would show this as:
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Destination MAC (6 bytes)                  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |      Destination MAC          |         Source MAC            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Source MAC (continued)                     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |          EtherType            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    """
    dst_mac: bytes  # 6 bytes
    src_mac: bytes  # 6 bytes
    ethertype: int  # 2 bytes
    payload: bytes
    
    @classmethod
    def parse(cls, data: bytes) -> 'EthernetFrame':
        if (data) < :
             ValueError()
        
        dst_mac = data[:]
        src_mac = data[:]
        ethertype = struct.unpack(, data[:])[]
        payload = data[:]
        
         cls(dst_mac, src_mac, ethertype, payload)
    
     () -> :
         .join(  b  mac)
    
     () -> :
         (
                
                )



 :
    
    version: 
    ihl:   
    tos: 
    total_length: 
    identification: 
    flags: 
    fragment_offset: 
    ttl: 
    protocol: 
    checksum: 
    src_ip: 
    dst_ip: 
    options: 
    payload: 
    

     () -> :
         (data) < :
             ValueError()
        
        version_ihl = data[]
        version = version_ihl >> 
        ihl = version_ihl & 
        
         version != :
             ValueError()
        
        header_length = ihl * 
        
        tos = data[]
        total_length = struct.unpack(, data[:])[]
        identification = struct.unpack(, data[:])[]
        
        flags_frag = struct.unpack(, data[:])[]
        flags = flags_frag >> 
        fragment_offset = flags_frag & 
        
        ttl = data[]
        protocol = data[]
        checksum = struct.unpack(, data[:])[]
        
        src_ip = .join((b)  b  data[:])
        dst_ip = .join((b)  b  data[:])
        
        options = data[:header_length]  header_length >   
        payload = data[header_length:total_length]
        
         cls(version, ihl, tos, total_length, identification,
                   flags, fragment_offset, ttl, protocol, checksum,
                   src_ip, dst_ip, options, payload)
    
     () -> :
        proto_name = IPProtocol(.protocol).name  .protocol  [,,]  (.protocol)
         (
                )



 :
    
    src_port: 
    dst_port: 
    seq: 
    ack: 
    data_offset: 
    flags: 
    window: 
    checksum: 
    urgent: 
    options: 
    payload: 
    
    
    FIN = 
    SYN = 
    RST = 
    PSH = 
    ACK = 
    URG = 
    ECE = 
    CWR = 
    

     () -> :
         (data) < :
             ValueError()
        
        src_port = struct.unpack(, data[:])[]
        dst_port = struct.unpack(, data[:])[]
        seq = struct.unpack(, data[:])[]
        ack = struct.unpack(, data[:])[]
        
        data_offset_flags = struct.unpack(, data[:])[]
        data_offset = (data_offset_flags >> ) & 
        flags = data_offset_flags & 
        
        window = struct.unpack(, data[:])[]
        checksum = struct.unpack(, data[:])[]
        urgent = struct.unpack(, data[:])[]
        
        header_length = data_offset * 
        options = data[:header_length]  header_length >   
        payload = data[header_length:]
        
         cls(src_port, dst_port, seq, ack, data_offset, flags,
                   window, checksum, urgent, options, payload)
    
     () -> :
        
        result = []
         .flags & .SYN: result.append()
         .flags & .FIN: result.append()
         .flags & .RST: result.append()
         .flags & .PSH: result.append()
         .flags & .ACK: result.append()
         .flags & .URG: result.append()
         .join(result)  
    
     () -> :
         (
                
                )

TCP State Machine

class TCPState:
    """
    TCP connection state machine.
    Stevens illustrated this exhaustively—every transition, every edge case.
    """
    
    # TCP States
    CLOSED = 'CLOSED'
    LISTEN = 'LISTEN'
    SYN_SENT = 'SYN_SENT'
    SYN_RECEIVED = 'SYN_RECEIVED'
    ESTABLISHED = 'ESTABLISHED'
    FIN_WAIT_1 = 'FIN_WAIT_1'
    FIN_WAIT_2 = 'FIN_WAIT_2'
    CLOSE_WAIT = 'CLOSE_WAIT'
    CLOSING = 'CLOSING'
    LAST_ACK = 'LAST_ACK'
    TIME_WAIT = 'TIME_WAIT'
    
    def __init__(self):
        self.state = self.CLOSED
        self.local_seq = 0
        self.remote_seq = 0
        self.local_port = 0
        self.remote_port = 0
    
    def transition(self, event: str, segment: TCPSegment = None) -> str:
        """
        State transition based on event.
        Returns action to take.
        """
        old_state = self.state
        action = None
        
        # Connection establishment (client side)
        if self.state == self.CLOSED:
            if event == 'active_open':
                .state = .SYN_SENT
                action = 
        
         .state == .SYN_SENT:
             event == :
                .state = .ESTABLISHED
                action = 
             event == :  
                .state = .SYN_RECEIVED
                action = 
        
        
         .state == .LISTEN:
             event == :
                .state = .SYN_RECEIVED
                action = 
        
         .state == .SYN_RECEIVED:
             event == :
                .state = .ESTABLISHED
                action = 
        
        
         .state == .ESTABLISHED:
             event == :
                .state = .FIN_WAIT_1
                action = 
             event == :
                .state = .CLOSE_WAIT
                action = 
        
        
         .state == .FIN_WAIT_1:
             event == :
                .state = .FIN_WAIT_2
             event == :
                .state = .CLOSING
                action = 
             event == :
                .state = .TIME_WAIT
                action = 
        
         .state == .FIN_WAIT_2:
             event == :
                .state = .TIME_WAIT
                action = 
        
         .state == .CLOSING:
             event == :
                .state = .TIME_WAIT
        
        
         .state == .CLOSE_WAIT:
             event == :
                .state = .LAST_ACK
                action = 
        
         .state == .LAST_ACK:
             event == :
                .state = .CLOSED
        
        
         .state == .TIME_WAIT:
             event == :
                .state = .CLOSED
        
         action
    
     () -> :
         


 ():
    
     


 ():
    
     

DNS Protocol

@dataclass
class DNSHeader:
    """
    DNS message header (12 bytes).
    
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                      Transaction ID                           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |              |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         QDCOUNT                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         ANCOUNT                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         NSCOUNT                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         ARCOUNT                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    """
    id: int
    qr: int          # 0=query, 1=response
    opcode: int      # 0=standard query
    aa: int          # Authoritative Answer
    tc: int          # Truncation
    rd: int          # Recursion Desired
    ra: int          # Recursion Available
    rcode: int       # Response code
    qdcount: int     # Question count
    ancount: int     # Answer count
    nscount: int     # Authority count
    arcount: int     # Additional count
    
    @classmethod
    def parse(cls, data: ) -> :
         = struct.unpack(, data[:])[]
        flags = struct.unpack(, data[:])[]
        
        qr = (flags >> ) & 
        opcode = (flags >> ) & 
        aa = (flags >> ) & 
        tc = (flags >> ) & 
        rd = (flags >> ) & 
        ra = (flags >> ) & 
        rcode = flags & 
        
        qdcount = struct.unpack(, data[:])[]
        ancount = struct.unpack(, data[:])[]
        nscount = struct.unpack(, data[:])[]
        arcount = struct.unpack(, data[:])[]
        
         cls(, qr, opcode, aa, tc, rd, ra, rcode,
                   qdcount, ancount, nscount, arcount)


 :
    
    
    RECORD_TYPES = {
        : ,
        : ,
        : ,
        : ,
        : ,
        : ,
        : ,
    }
    
     ():
        .data = data
        .offset = 
    
     () -> :
        
         offset  :
            offset = .offset
        
        labels = []
        jumped = 
        original_offset = offset
        
         :
            length = .data[offset]
            
             length == :
                offset += 
                
            
             (length & ) == :
                
                pointer = struct.unpack(, .data[offset:offset+])[]
                pointer &= 
                
                  jumped:
                    .offset = offset + 
                jumped = 
                offset = pointer
            :
                
                offset += 
                labels.append(.data[offset:offset+length].decode())
                offset += length
        
          jumped:
            .offset = offset
        
         .join(labels), offset - original_offset

Layer Analysis Tool

class PacketAnalyzer:
    """
    Stevens-style packet analysis.
    Parse and display each layer.
    """
    
    def analyze(self, raw_bytes: bytes) -> dict:
        """
        Analyze a packet layer by layer.
        """
        result = {'layers': [], 'raw': raw_bytes.hex()}
        offset = 0
        
        # Layer 2: Ethernet
        try:
            eth = EthernetFrame.parse(raw_bytes)
            result['layers'].append({
                'layer': 2,
                'protocol': 'Ethernet',
                'summary': str(eth),
                'details': {
                    'dst_mac': eth.format_mac(eth.dst_mac),
                    'src_mac': eth.format_mac(eth.src_mac),
                    'ethertype': f'0x{eth.ethertype:04x}',
                }
            })
            
            if eth.ethertype == EtherType.IPv4:
                result.update(self._analyze_ipv4(eth.payload))
        except Exception as e:
            result['error'] = str(e)
        
        return result
    
    def _analyze_ipv4(self, data: bytes) -> dict:
        """Analyze IPv4 layer."""
        result = {'layers': []}
        
        :
            ip = IPv4Packet.parse(data)
            result[].append({
                : ,
                : ,
                : (ip),
                : {
                    : ip.version,
                    : ip.ihl * ,
                    : ip.total_length,
                    : ip.ttl,
                    : ip.protocol,
                    : ip.src_ip,
                    : ip.dst_ip,
                }
            })
            
             ip.protocol == IPProtocol.TCP:
                result[].extend(._analyze_tcp(ip.payload)[])
             ip.protocol == IPProtocol.UDP:
                result[].extend(._analyze_udp(ip.payload)[])
         Exception  e:
            result[] = (e)
        
         result
    
     () -> :
        
        result = {: []}
        
        :
            tcp = TCPSegment.parse(data)
            result[].append({
                : ,
                : ,
                : (tcp),
                : {
                    : tcp.src_port,
                    : tcp.dst_port,
                    : tcp.seq,
                    : tcp.ack,
                    : tcp.flags_str(),
                    : tcp.window,
                    : (tcp.payload),
                }
            })
            
            
             tcp.dst_port ==   tcp.src_port == :
                result[].append({
                    : ,
                    : ,
                    : tcp.payload[:].decode(, errors=)
                })
         Exception  e:
            result[] = (e)
        
         result
    
     () -> :
        
        lines = []
         i  (, (data), bytes_per_line):
            chunk = data[i:i+bytes_per_line]
            hex_part = .join(  b  chunk)
            ascii_part = .join((b)   <= b <     b  chunk)
            lines.append()
         .join(lines)


 ():
    
     socket
    
    
    sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs())
    sock.bind((interface, ))
    
    analyzer = PacketAnalyzer()
    
     i  (count):
        raw, addr = sock.recvfrom()
        
        ()
        ()
        (*)
        
        result = analyzer.analyze(raw)
        
         layer  result.get(, []):
            ()
            ()
        
        ()
        (analyzer.hexdump(raw[:]))  

Mental Model

Stevens approaches networking by asking:

1. What's on the wire? Capture and examine the packets
2. What layer is this? Work through each layer systematically
3. What does each byte mean? Know the protocol format
4. What state is the connection in? Track the state machine
5. What does the RFC say? The specification is the authority

The Protocol Analysis Checklist

□ Capture packets with tcpdump/Wireshark
□ Identify the protocol at each layer
□ Parse headers field by field
□ Track sequence numbers and state
□ Look for retransmissions and errors
□ Check for fragmentation
□ Verify checksums if suspicious
□ Compare to RFC specification

Signature Stevens Moves

• Packet traces with hex dumps
• Layer-by-layer analysis
• Header field diagrams (RFC style)
• State machine diagrams
• tcpdump command mastery
• RFC references for every claim
• Real behavior over documented behavior
• "Show me the packets"