osdev-networking

name	osdev-networking
description	Network stack implementation for OS kernels. Ethernet framing, ARP, IPv4, TCP/UDP, DHCP, and NIC drivers (e1000, RTL8139, virtio-net). Use when building a TCP/IP stack, writing network card drivers, or implementing network protocols.
origin	MCC

Network Stack Implementation for OS Kernels

Complete reference for building a TCP/IP stack from NIC driver to socket layer in a bare-metal kernel.

When to Use

• Writing a NIC driver (e1000, RTL8139, virtio-net)
• Implementing Ethernet frame parsing and construction
• Building an ARP resolver (IP-to-MAC translation)
• Implementing IPv4 header parsing and checksum
• Adding UDP or TCP transport protocols
• Debugging packet receive/transmit failures
• Setting up DHCP for automatic IP configuration

Network Stack Layers

┌──────────────────────────────┐
│  Application (DHCP, DNS)     │  ← uses sockets
├──────────────────────────────┤
│  Transport (TCP, UDP)        │  ← port multiplexing, reliability
├──────────────────────────────┤
│  Network (IPv4, ARP, ICMP)   │  ← addressing, routing
├──────────────────────────────┤
│  Data Link (Ethernet)        │  ← framing, MAC addressing
├──────────────────────────────┤
│  Physical (NIC driver)       │  ← DMA, interrupts, hardware
└──────────────────────────────┘

Why layering matters: Each layer has a single responsibility with a clean interface to its neighbors. The NIC driver does not need to understand TCP. The TCP implementation does not need to know whether the physical medium is Ethernet or WiFi. This lets you swap NIC drivers without touching protocol code, and add new protocols without touching drivers.

Build order: Start at the bottom. You cannot test ARP without Ethernet, and you cannot test TCP without IP. A working implementation order is: NIC driver -> Ethernet -> ARP -> IPv4 -> ICMP (ping) -> UDP -> DHCP -> TCP.

NIC Drivers

All three common NIC types use PCI for device discovery. You need a working PCI bus scan first.

RTL8139 (Simple, PIO-based)

The simplest NIC to implement. Good for learning, used in QEMU/KVM. PCI vendor 0x10EC, device 0x8139.

Key registers (I/O port offsets):

Offset	Size	Name	Purpose
0x00	6	MAC0-5	MAC address (read at init)
0x30	4	RBSTART	RX buffer physical address
0x37	1	CMD	Command register (TX/RX enable, reset)
0x38	2	CAPR	Current Address of Packet Read
0x3C	2	IMR	Interrupt Mask Register
0x3E	2	ISR	Interrupt Status Register
0x20	4	TSAD0	TX start address descriptor 0
0x10	4	TSD0	TX status descriptor 0
0x44	4	RCR	Receive Configuration Register

Init sequence: Power on (0x00 to port 0x52) -> Software reset (0x10 to CMD, wait for RST bit clear) -> Set RX buffer address (physical addr to RBSTART) -> Set IMR for TOK+ROK (0x0005) -> Configure RCR (accept broadcast + physical match + WRAP) -> Enable TX+RX (0x0C to CMD).

TX model: 4 TX descriptors that rotate. Write packet physical address to TSAD[n], write size to TSD[n] (clearing OWN bit). NIC DMAs the packet and sets OWN when done.

RX model: Single ring buffer. NIC writes packets sequentially. Each packet is preceded by a 4-byte header (status + length). Update CAPR after reading to tell the NIC you have consumed the data.

e1000 (Intel, QEMU default)

More complex but the default NIC in QEMU. Uses MMIO instead of PIO. PCI vendor 0x8086, device 0x100E (82540EM).

Key features: TX/RX descriptor rings (arrays of descriptors in DMA memory), MMIO register access, hardware checksum offload.

Init sequence: PCI BAR0 gives MMIO base -> Read MAC from EEPROM or RAL/RAH registers -> Allocate TX and RX descriptor rings (aligned, physically contiguous) -> Configure TDBAL/TDBAH, TDLEN, TDH, TDT for TX -> Configure RDBAL/RDBAH, RDLEN, RDH, RDT for RX -> Set RCTL (receiver control) and TCTL (transmit control) -> Enable interrupts.

virtio-net (Paravirtualized)

Fastest in virtual environments because the hypervisor cooperates. Uses virtqueues instead of hardware DMA rings. Most efficient but requires understanding the virtio specification.

See references/e1000-driver.md for complete e1000 register map, descriptor formats, and init code.

Ethernet

Every packet on the wire is wrapped in an Ethernet frame. This is the first thing you parse on receive and the last thing you construct on transmit.

Frame Format

┌──────────┬──────────┬───────────┬────────────────┬─────┐
│ Dst MAC  │ Src MAC  │ EtherType │ Payload        │ FCS │
│ 6 bytes  │ 6 bytes  │ 2 bytes   │ 46-1500 bytes  │ 4B  │
└──────────┴──────────┴───────────┴────────────────┴─────┘

FCS (Frame Check Sequence): CRC32 appended by the NIC hardware on transmit and stripped/verified on receive. You usually do not need to handle this in software.

EtherType Values

Value	Protocol
0x0800	IPv4
0x0806	ARP
0x86DD	IPv6

typedef struct __attribute__((packed)) {
    uint8_t  dst_mac[6];
    uint8_t  src_mac[6];
    uint16_t ethertype;   // big-endian!
    uint8_t  payload[];
} ethernet_frame_t;

Byte order: All multi-byte fields in network protocols are big-endian (network byte order). Your OS is almost certainly little-endian. Use htons()/ntohs() for 16-bit and htonl()/ntohl() for 32-bit conversions. Getting this wrong is the #1 networking bug.

ARP

Why ARP Exists

When your OS wants to send an IP packet to 192.168.1.1, it needs to know the MAC address of that host (or the gateway router) to construct the Ethernet frame. ARP resolves IP addresses to MAC addresses on the local network.

Packet Structure

typedef struct __attribute__((packed)) {
    uint16_t htype;       // hardware type: 1 = Ethernet
    uint16_t ptype;       // protocol type: 0x0800 = IPv4
    uint8_t  hlen;        // hardware address length: 6 (MAC)
    uint8_t  plen;        // protocol address length: 4 (IPv4)
    uint16_t opcode;      // 1 = request, 2 = reply
    uint8_t  sender_mac[6];
    uint32_t sender_ip;
    uint8_t  target_mac[6];  // zeroed in requests
    uint32_t target_ip;
} arp_packet_t;

Request/Reply Flow

1. Need to send to 10.0.0.5: Check ARP cache. Miss.
2. Send ARP request: Broadcast (dst MAC = FF:FF:FF:FF:FF:FF), EtherType 0x0806, opcode 1, target_mac = 00:00:00:00:00:00, target_ip = 10.0.0.5.
3. Target receives request: Recognizes its own IP, updates its own ARP cache with sender's info.
4. Target sends ARP reply: Unicast to requester's MAC, opcode 2, fills in its own MAC as sender_mac.
5. Requester receives reply: Caches the IP-to-MAC mapping. Sends the original IP packet.

ARP Cache

Keep a simple table of (IP, MAC, timestamp) entries. Expire entries after a timeout (typically 60-300 seconds). When an entry is missing, queue the outgoing packet and send an ARP request. When the reply arrives, dequeue and transmit.

See references/ethernet-arp.md for Ethernet frame struct, ARP handling code, and cache implementation.

IPv4

Header Format (20 bytes minimum)

typedef struct __attribute__((packed)) {
    uint8_t  version_ihl;     // version (4) << 4 | IHL (header length in 32-bit words)
    uint8_t  dscp_ecn;        // differentiated services + ECN
    uint16_t total_length;    // entire packet size including header
    uint16_t identification;  // fragment reassembly ID
    uint16_t flags_frag;      // flags (3 bits) + fragment offset (13 bits)
    uint8_t  ttl;             // time to live (hop count)
    uint8_t  protocol;        // upper layer protocol
    uint16_t checksum;        // header checksum
    uint32_t src_ip;
    uint32_t dst_ip;
} ipv4_header_t;

Protocol Numbers

Number	Protocol
1	ICMP
6	TCP
17	UDP

Checksum Calculation

The IPv4 header checksum is a 16-bit one's complement sum of the header (with checksum field set to 0):

uint16_t ip_checksum(void *data, size_t len) {
    uint32_t sum = 0;
    uint16_t *ptr = (uint16_t *)data;

    while (len > 1) {
        sum += *ptr++;
        len -= 2;
    }
    if (len == 1)
        sum += *(uint8_t *)ptr;

    // Fold 32-bit sum to 16-bit
    while (sum >> 16)
        sum = (sum & 0xFFFF) + (sum >> 16);

    return (uint16_t)~sum;
}

Verification: Compute the checksum over the received header including the checksum field. If the result is 0, the header is valid.

TCP/UDP

UDP (Simple)

typedef struct __attribute__((packed)) {
    uint16_t src_port;
    uint16_t dst_port;
    uint16_t length;      // header (8) + data
    uint16_t checksum;    // optional in IPv4 (0 = no checksum)
    uint8_t  data[];
} udp_header_t;

UDP is connectionless and unreliable. No handshake, no retransmission, no ordering. Ideal for DHCP, DNS, and simple protocols. Implement UDP before TCP.

TCP (Connection-oriented)

typedef struct __attribute__((packed)) {
    uint16_t src_port;
    uint16_t dst_port;
    uint32_t seq_num;         // sequence number
    uint32_t ack_num;         // acknowledgment number
    uint8_t  data_offset;     // header length in 32-bit words (top 4 bits)
    uint8_t  flags;           // SYN, ACK, FIN, RST, PSH, URG
    uint16_t window;          // receive window size
    uint16_t checksum;        // mandatory (uses pseudo-header)
    uint16_t urgent_ptr;
} tcp_header_t;

TCP Flags:

Flag	Bit	Purpose
FIN	0x01	Sender is done
SYN	0x02	Synchronize sequence numbers (connection setup)
RST	0x04	Reset the connection
PSH	0x08	Push data to application immediately
ACK	0x10	Acknowledgment field is valid
URG	0x20	Urgent pointer is valid

Three-way handshake:

1. Client sends SYN (seq=X)
2. Server sends SYN+ACK (seq=Y, ack=X+1)
3. Client sends ACK (seq=X+1, ack=Y+1)

State machine overview: CLOSED -> (SYN sent) -> SYN_SENT -> (SYN+ACK received, ACK sent) -> ESTABLISHED -> (FIN sent) -> FIN_WAIT_1 -> ... -> CLOSED

See references/tcp-ip-stack.md for TCP state machine, pseudo-header checksum, and socket abstraction.

Packet Flow

Receive Path

NIC hardware receives frame
  → NIC triggers IRQ
    → ISR reads packet from RX buffer/ring
      → Parse Ethernet header
        → EtherType 0x0806? → ARP handler
        → EtherType 0x0800? → IPv4 handler
          → Parse IPv4 header, verify checksum
            → Protocol 1? → ICMP handler (ping reply)
            → Protocol 17? → UDP handler → deliver to socket by port
            → Protocol 6? → TCP handler → match connection, process state machine

Transmit Path

Application calls send(socket, data)
  → TCP/UDP adds transport header
    → IPv4 adds IP header (src_ip, dst_ip, checksum)
      → ARP lookup for dst_ip (or gateway IP)
        → Cache hit? → Ethernet adds frame header (dst MAC, src MAC, EtherType)
          → NIC driver queues frame in TX descriptor
            → NIC DMAs frame and transmits
        → Cache miss? → Queue packet, send ARP request, wait for reply

Common Pitfalls

Pitfall	Consequence	Fix
Forgetting byte order conversion	All fields are garbage; nothing works	Use htons/ntohs for 16-bit, htonl/ntohl for 32-bit
Using virtual addresses for DMA buffers	NIC reads garbage from wrong physical location	Always pass physical addresses to NIC descriptor rings
Not enabling PCI bus mastering	NIC cannot perform DMA, no packets move	Set bit 2 in PCI command register before using NIC
Forgetting to acknowledge IRQ in ISR	NIC stops generating interrupts	Write to ISR register (RTL8139) or ICR register (e1000) to clear
Not updating CAPR after reading RX (RTL8139)	NIC thinks buffer is full, stops receiving	Update CAPR = current_read_offset - 0x10
Sending ARP request with target_mac filled in	Some hosts ignore non-zero target_mac in requests	Zero target_mac in ARP requests
Wrong IP checksum (including payload)	IP checksum only covers the header	Set checksum field to 0, compute over header bytes only
TCP checksum without pseudo-header	TCP checksum must include a pseudo-header (src_ip, dst_ip, protocol, length)	Prepend 12-byte pseudo-header to checksum calculation
Not handling ARP for gateway	Packets to non-local IPs never resolve	If dst_ip is outside local subnet, ARP the gateway IP instead
TX descriptors not physically contiguous	e1000 cannot walk the ring	Allocate descriptor arrays from physically contiguous memory

Related Skills

• osdev-interrupts -- IRQ handlers for NIC interrupt lines
• osdev-memory -- physical memory allocation for DMA buffers and descriptor rings
• osdev-boot -- PCI bus enumeration to discover NIC devices