networking-transport

name

networking-transport

description

Networking and transport-layer architecture for production services. TCP keepalive, head-of-line blocking, TTFB budget, HTTP/1.1 vs HTTP/2 vs HTTP/3, when to choose gRPC/Protobuf over REST/JSON, connection pooling, long-lived connections (websockets / SSE / gRPC streaming), and CDN-to-origin patterns. Use when designing service-to-service APIs, debugging tail latency, choosing a wire protocol, configuring load balancer idle timeouts, or evaluating HTTP/3 / QUIC for mobile / cross-region workloads.

Networking & Transport - Architecture Patterns

The transport-and-wire-format layer underneath your API design. 320-api-design.mdc covers the API surface (verbs, status codes, versioning). This skill covers everything below it: TCP, HTTP/n, QUIC, serialization, connection lifecycle, head-of-line, TTFB budget.

Get this layer wrong and your P99 latency, mobile UX, edge cold-start cost, and east-west microservice throughput all suffer in ways that profiling your application code can't fix.

When to invoke

• Designing or reviewing a service-to-service API (REST vs gRPC choice)
• Debugging tail latency / TTFB spikes / "fast in isolation, slow in production"
• Configuring load balancer idle timeouts (NLB, ALB, API Gateway, Cloudflare)
• Designing long-lived connections (websockets, SSE, gRPC streaming)
• Evaluating HTTP/3 / QUIC for mobile / cross-region / lossy-network workloads
• Choosing a wire format (JSON vs Protobuf vs MessagePack vs Avro)
• Reviewing a .proto file or a protobuf schema change

Five golden rules

1. TCP keepalive is mandatory on long-lived clients. Linux defaults (tcp_keepalive_time = 7200s = 2 hours) are wrong for cloud. Set explicit per-language values.
2. Reuse HTTP clients. A fresh client per request defeats keepalive and burns ~50-150ms on TCP+TLS handshake. The single most common production performance bug.
3. Choose the wire by callers, not by hype. External / browser / human-debugged -> REST/JSON. Internal east-west / polyglot / streaming -> gRPC/Protobuf. Don't gRPC your public API just because it's faster.
4. HTTP/2 multiplexes streams; HTTP/3 multiplexes connections. HTTP/2 still has TCP-level head-of-line. HTTP/3 fixes it. Matters when packet loss is non-zero (mobile, satellite, cross-region).
5. Idle timeouts kill connections silently. AWS NLB drops at 350s, ALB at 60s, API Gateway REST at 29s. Set keepalive < min(idle timeout) or watch your "random" disconnects.

---

1. Mental model: four layers, four latency budgets

+----------------------------------------+
|  L7 wire format    JSON / Protobuf     |  Serialization CPU + size on wire
+----------------------------------------+
|  L7 protocol       HTTP/1.1 /2 /3      |  Multiplexing + HoL behavior
+----------------------------------------+
|  L6 transport sec  TLS 1.2 /1.3        |  Handshake RTTs (1-RTT, 0-RTT, session resume)
+----------------------------------------+
|  L4 transport      TCP / QUIC (UDP)    |  Handshake, congestion control, HoL
+----------------------------------------+
|  L3 IP             routing / DNS       |  DNS lookup, geo distance, MTU
+----------------------------------------+

Latency tuning at each layer:

Layer	Typical contribution to TTFB	Tuning levers
DNS	0-50ms (cold), 0-1ms (cached)	TTL, edge DNS, prefetch
TCP handshake	1 RTT (~5-150ms by geo)	Connection reuse (keepalive), HTTP/3 (0 RTT)
TLS handshake	1-2 RTT (TLS 1.3 = 1; TLS 1.2 = 2)	TLS 1.3, session resumption, 0-RTT (with replay risk)
Request processing	App-dependent	Profile; out of scope here
First byte	Streaming vs buffering	Stream early; don't await full result before responding

---

2. TCP keepalive

What it is

OS-level periodic probes on idle TCP connections to detect dead peers. Without keepalive, a connection in a half-open state (peer died, NAT timed out, LB recycled) appears alive until the next read/write attempt fails.

Why defaults are wrong

OS / setting	Default	Problem in cloud
Linux tcp_keepalive_time	7200s (2 hours)	LB drops connection at 60-350s; client believes still alive; first send after 6 minutes fails
Linux tcp_keepalive_intvl	75s	OK
Linux tcp_keepalive_probes	9	Total death-detection time = 2h + 9*75s = ~2.2h
macOS net.inet.tcp.keepidle	7200000ms	Same
Windows KeepAliveTime	7200000ms	Same

What to set per environment

Environment	Suggested idle	Why
AWS NLB-fronted (350s idle)	60-120s	Probe twice before LB cuts
AWS ALB-fronted (60s idle, configurable)	30s	Probe before LB cuts
API Gateway REST (29s idle, fixed)	Don't keepalive; reconnect per request	Pool inside a Lambda invocation only
Cross-region (RTT 100-200ms)	30-60s	Detect partition fast
Inside k8s cluster	30-60s	Pod churn, kube-proxy reset

Per-language settings

// Go: customize the Transport's DialContext
tr := &http.Transport{
    DialContext: (&net.Dialer{
        Timeout:   5 * time.Second,
        KeepAlive: 30 * time.Second,    // probe interval
    }).DialContext,
    IdleConnTimeout:       90 * time.Second,
    MaxIdleConnsPerHost:   100,
    DisableKeepAlives:     false,
}
client := &http.Client{Transport: tr}

# Python (requests/httpx) - OS-level keepalive only; tune via Session
import requests
from requests.adapters import HTTPAdapter

session = requests.Session()
session.mount('https://', HTTPAdapter(pool_connections=20, pool_maxsize=100))
# OS-level: sysctl net.ipv4.tcp_keepalive_time=60 (container or host)

// Node: enable keepalive on the Agent (NOT default in older Node versions)
const https = require('https');
const agent = new https.Agent({
  keepAlive: true,
  keepAliveMsecs: 30_000,           // probe interval
  maxSockets: 100,
  timeout: 30_000,
});

Anti-patterns

• Default OS keepalive in cloud (2-hour idle). Connection looks alive; first send fails.
• Creating a new http.Client / requests object per request. No pooled connection, no keepalive, ~50-150ms handshake tax per call.
• Setting keepalive longer than the load balancer's idle timeout. Backwards; LB still wins.

---

3. TCP head-of-line blocking

The problem at three layers

Layer	What blocks	Fixed by
HTTP/1.1	Whole connection blocks until current request completes (serial requests per connection)	Concurrent connections (browsers cap at ~6/host); HTTP/2
HTTP/2 over TCP	Streams are multiplexed in software, but TCP packet loss blocks all streams on that connection until retransmit	HTTP/3 over QUIC
TCP	Lost packet stalls every higher-layer message on the connection	QUIC (per-stream loss handling at transport)

When this matters

• Mobile networks (packet loss 1-5%): HTTP/2 single connection performs worse than HTTP/1.1 with parallel connections because one lost packet stalls everything.
• Cross-region (especially via lossy intermediate networks): same.
• High-fan-out backends: one slow upstream response on an HTTP/2 connection delays unrelated responses sharing the connection.

When it doesn't matter

• LAN / data-center east-west with <0.001% loss: HTTP/2 multiplexing wins; HoL is theoretical.
• HTTP/1.1 with one request at a time (most internal microservices today): never see it.

---

4. TTFB (time to first byte) budget

Component	Cost (typical)	What to do about it
DNS lookup	0-50ms cold, 0-1ms cached	High TTL, edge DNS, DNS prefetch in clients
TCP handshake	1 RTT (5-150ms by geo)	Reuse connections (keepalive); HTTP/3 0-RTT
TLS handshake	1 RTT (TLS 1.3) or 2 RTT (TLS 1.2)	TLS 1.3 mandatory; session resumption / tickets
Server processing	App-dependent	Profile; cache; precompute
First byte to client	1 RTT to client	Stream the response; don't buffer-then-send

The streaming insight: if your handler takes 800ms to compute a 4KB response, TTFB is 800ms + 1 RTT. If you stream incrementally, TTFB drops to whatever the first chunk takes (often <100ms). Big TTFB win, no algorithmic change required.

---

5. HTTP/1.1 vs HTTP/2 vs HTTP/3

Aspect	HTTP/1.1	HTTP/2	HTTP/3
Transport	TCP	TCP	QUIC over UDP
Connection per host	Many (browsers ~6)	One (multiplexed)	One (multiplexed)
Request concurrency on one connection	None (serial)	Many streams	Many streams
TCP HoL	N/A (serial)	Yes (packet loss blocks all streams)	No (per-stream)
Handshake RTT	1 (TCP) + 1-2 (TLS)	1 + 1-2	0-1 (0-RTT with replay caveat)
Header compression	None	HPACK	QPACK
Server push	None	Yes (deprecated 2022)	No (dropped)
Browser/server support	Universal	Universal	Strong but middlebox issues
When it wins	Simple, debuggable	Internal east-west, LAN	Mobile, cross-region, lossy networks

HTTP/3 prerequisites

• UDP port 443 must be reachable end-to-end. Some corporate proxies, old firewalls, and certain ISP middleboxes drop or rate-limit UDP/443.
• Server advertises via alt-svc header in an HTTP/2 or 1.1 response: alt-svc: h3=":443"; ma=86400.
• Client must support and choose to attempt H3. Browsers do this transparently; SDKs / curl need explicit flags.
• TLS 1.3 mandatory (HTTP/3 only runs over TLS 1.3+).

When HTTP/3 is the right choice

• Public-facing edge traffic for mobile users
• Cross-region / cross-cloud / high-RTT paths
• Edge POPs to mobile clients (Cloudflare, Fastly, CloudFront all support)
• Anywhere packet loss meaningfully affects current HTTP/2 P95/P99

When HTTP/2 is still right

• Internal east-west (low loss, low RTT, no UDP-path concerns)
• gRPC (which requires HTTP/2)
• Anywhere your egress path doesn't reliably pass UDP/443

---

6. REST/JSON vs gRPC/Protobuf vs GraphQL vs Connect

Decision matrix

Use case	Choice	Why
External public API	REST + JSON	Browser-debuggable, curl-friendly, broadest tooling, OpenAPI spec mature
Internal east-west microservices (polyglot)	gRPC + Protobuf	Schema-first, code generation, streaming, ~5-10x smaller wire, ~3-5x faster serialization, native HTTP/2
Browser <-> internal services	REST or gRPC-Web (with proxy) or Connect	gRPC doesn't run in browsers natively; gRPC-Web requires an Envoy/Nginx proxy that translates
Client-driven shape (graph fetches)	GraphQL	Single endpoint, client picks fields, batches
Want gRPC's API style without HTTP/2 mandate	Connect (or Twirp)	Protobuf schemas, HTTP/1.1-friendly, browser-callable directly
Mobile <-> backend	REST + Protobuf (or Connect)	gRPC works on mobile but battery / network resilience varies
Event-driven / pub-sub	Kafka / NATS / SQS with Protobuf or Avro schemas	Not a request/response problem; see 483-kafka.mdc

See references/grpc-vs-rest-decision.md for the long version with worked examples.

gRPC's real wins

• Schema-first contract. .proto is the single source of truth; client + server generated from it. No "is this field optional?" debates.
• Streaming. Server, client, and bidirectional streams as first-class concepts. Long-lived RPC over a single connection.
• Polyglot code-gen. Same .proto -> Go, Python, Java, TS, Swift, Kotlin, Ruby, C++, Rust. Type-safe in each.
• Smaller wire, faster parse. Protobuf vs JSON: ~3-10x smaller, ~3-5x faster to (de)serialize.
• HTTP/2 native. Multiplexed streams, no per-request connection.

gRPC's real losses

• Browsers don't speak gRPC. Need gRPC-Web + Envoy translation, or use Connect.
• Debuggable from a CLI? Need grpcurl (not curl) and a reflection server or the .proto file.
• Load balancing. gRPC's persistent HTTP/2 connection breaks naive L4 load balancers; need L7 (Envoy, Linkerd, AWS Application LB with gRPC support).
• No browser-friendly caching. REST Cache-Control and CDN integration is a solved problem; gRPC isn't.
• Tail-latency observability. Per-call metrics need extra instrumentation; less out-of-box than HTTP middleware.

Connect (and Twirp): the middle ground

• Protobuf schemas + code-gen (like gRPC)
• HTTP/1.1-friendly (like REST)
• Browser-callable directly with fetch (unlike gRPC)
• Streaming supported (Connect: yes; Twirp: no)
• Less feature-complete than gRPC but covers the common 80%

Anti-pattern: gRPC for the public API

If your callers are external (third-party developers, mobile teams you don't control, browsers): use REST or Connect. gRPC's operational burden on external callers is real.

---

7. Protobuf design essentials

syntax = "proto3";

package myorg.users.v1;     // semantic-version the package; never break clients of v1

message User {
  string id = 1;            // field numbers are wire identity - never re-number
  string email = 2;
  string display_name = 3;
  reserved 4, 5;            // numbers of removed fields; prevents accidental reuse
  reserved "deprecated_field_name";

  // Presence semantics: in proto3, scalar zero values don't serialize.
  // Use `optional` (proto3 since 2020) or wrapper types for true "absent" semantics.
  optional string phone = 6;  // absent vs empty distinguishable
}

message UpdateUserRequest {
  string id = 1;
  // oneof for mutually-exclusive update modes
  oneof update {
    string new_email = 2;
    string new_display_name = 3;
  }
}

Schema-evolution rules (memorize)

Change	Compatible?
Add a new field with a new number	Yes (forward + backward)
Remove a field, reserve the number	Yes
Re-number a field	NO - changes wire identity; corrupts existing data
Change a field's type	NO (with rare exceptions like int32 <-> int64 same wire kind)
Rename a field (number unchanged)	Yes (wire is by number, not name)
Add value to an enum	Backward-compatible; old code sees UNKNOWN
Remove an enum value	Risky; old data may carry it
Change optional to repeated	NO
Make required field optional (proto2)	Yes (no-op in proto3; everything is optional)

See references/protobuf-schema-evolution.md for the long form with examples.

Wire-format gotchas

• Default values don't appear on the wire (proto3 scalars). Receiver can't distinguish "set to zero" from "absent". Use optional (since 2020) or wrapper types (google.protobuf.StringValue).
• Field numbers 1-15 use 1 byte tag; 16+ use 2. Reserve 1-15 for hot fields.
• Unknown fields are preserved by default in most languages. Don't strip them.
• bytes and string are similar on the wire; bytes skips UTF-8 validation. Use bytes for binary; string for text.

---

8. Connection pooling per language

The #1 production-performance bug: a fresh HTTP client per request.

Go

// BAD: new Client per request -> no keepalive
func bad(url string) {
    resp, _ := http.Get(url)  // uses http.DefaultClient with default Transport
    // but if you do `http.Client{Transport: &http.Transport{...}}` per call, you lose pooling
}

// GOOD: package-level (or DI'd) client, reused
var client = &http.Client{
    Timeout: 10 * time.Second,
    Transport: &http.Transport{
        MaxIdleConnsPerHost: 100,
        IdleConnTimeout:     90 * time.Second,
        DialContext: (&net.Dialer{
            Timeout:   5 * time.Second,
            KeepAlive: 30 * time.Second,
        }).DialContext,
    },
}

Python

# BAD: requests.get() per call - no session reuse
def bad(url):
    return requests.get(url)

# GOOD: Session pools connections, sends keepalive headers
session = requests.Session()
adapter = HTTPAdapter(pool_connections=20, pool_maxsize=100, max_retries=3)
session.mount('https://', adapter)
session.mount('http://', adapter)

Node

// BAD: implicit Agent per call (older Node)
const res = await fetch(url);

// GOOD: shared keepalive agent (Node 19+ defaults to keepAlive: true; older versions don't)
const https = require('https');
const agent = new https.Agent({ keepAlive: true, maxSockets: 100, timeout: 30_000 });
const res = await fetch(url, { agent });

gRPC clients

# Single channel reused across calls (NOT a channel per call)
channel = grpc.insecure_channel('service:50051', options=[
    ('grpc.keepalive_time_ms', 30_000),
    ('grpc.keepalive_timeout_ms', 5_000),
    ('grpc.keepalive_permit_without_calls', True),
    ('grpc.http2.max_pings_without_data', 0),
])
stub = MyServiceStub(channel)

A gRPC channel is itself a connection pool; create one per (service, credentials) pair and reuse for the lifetime of the process.

---

9. Long-lived connections

Pattern	Protocol	Use when	Watch out for
Websocket	WS over HTTP Upgrade	Bidirectional, real-time, browser-friendly	LB idle timeout (NLB 350s; ALB 60s; Cloudflare 100s free / 300s paid); reconnect logic with exponential backoff
Server-Sent Events (SSE)	HTTP/1.1+ text/event-stream	Server-to-client push, browser-friendly, simpler than WS	One-way only; some proxies buffer; need keepalive comments to defeat idle timeouts
gRPC streaming	HTTP/2 streams	Polyglot service-to-service, bidirectional or unidirectional	Browser unsupported (use Connect streams instead); load balancer must understand HTTP/2
Long polling	HTTP/1.1	Legacy clients, last-resort fallback	High per-request overhead; consider SSE first

Idle-timeout cheat sheet

Proxy / LB	Default idle timeout	Configurable
AWS Network Load Balancer (NLB)	350s (TCP), 350s (UDP)	No (fixed)
AWS Application LB (ALB)	60s	Yes, 1-4000s
AWS API Gateway REST	29s	No (fixed; will be 30s+ for HTTP APIs)
AWS API Gateway WebSocket	10 min idle, 2 hr total	No
Cloudflare WebSocket	100s (free), 300s (paid)	No (use tickets/heartbeats)
Cloudflare HTTP/2	100s	No
Nginx (default)	60s (proxy_read_timeout)	Yes
Envoy	1h (stream_idle_timeout)	Yes

Application-level heartbeats (every 30-60s) are the only reliable way to defeat idle-timeout disconnects across this matrix.

---

10. Edge / CDN patterns

• Cold POPs are real: first request to a POP that hasn't seen your origin pays a full handshake. Subsequent requests reuse the warm connection.
• CDN-to-origin keepalive is on by default in Cloudflare, Fastly, CloudFront. Verify and tune origin-side keepalive timeouts to match (origin must not drop before CDN reuses).
• Anycast routing brings the connection to the nearest POP; routes between POPs and origin use the CDN's own backbone (faster than the public internet for cross-region).
• HTTP/3 from edge to client is supported by all major CDNs; origin-to-edge usually remains HTTP/2.
• Regional fall-through: if a POP can't reach origin, CDN may try other regions. Configure origin allow-lists or auth that work from any POP.

---

11. Observability for the transport layer

Metrics to capture per outbound HTTP/gRPC client:

Metric	Why
http_client_requests_total{status, host}	Error budget
http_client_duration_seconds_bucket{host, status}	P50/P95/P99 latency by host
http_client_connection_reuses_total{host} / _creates_total	Connection-pool effectiveness; ratio should be high
tcp_keepalive_probe_failures_total	OS-side dead-peer detection rate
tls_handshake_duration_seconds_bucket	Handshake cost contribution to TTFB
dns_lookup_duration_seconds_bucket	DNS cost (often forgotten)
grpc_client_msg_received_total{rpc, code}	gRPC streaming health

For deeper observability patterns see rules/330-observability.mdc.

---

12. Anti-patterns

1. New HTTP client per request. Defeats connection reuse, burns ~50-150ms / call on handshake. #1 production performance bug.
2. Default OS TCP keepalive in cloud. 2-hour idle means connections look alive long after LB cut them; first write fails.
3. HTTP/2 single connection across mobile / lossy networks. TCP HoL stalls all streams; HTTP/1.1 parallel or HTTP/3 wins.
4. gRPC for a browser-facing API without gRPC-Web/Connect. Doesn't work; teams discover too late.
5. Re-numbering a protobuf field. Wire-incompatible; existing data and clients corrupt.
6. proto3 scalar field for true "optional" semantics. Default-zero-doesn't-serialize means receiver can't distinguish; use optional (since 2020) or wrappers.
7. Buffering then sending. TTFB blows up; stream incrementally where possible.
8. Keepalive longer than LB idle timeout. LB wins; "random" disconnects appear in P99.
9. One TCP connection per RPC in a high-call-rate service. Channel-per-call instead of channel-reuse is gRPC's equivalent of #1.
10. Assuming HTTP/3 works without verifying UDP/443. Corporate proxies and some ISPs filter; fall back gracefully.

---

13. Reviewer checklist

For any PR introducing or modifying an HTTP / gRPC client:

• Client is created once (package-level, DI, singleton) and reused
• Explicit timeout set (overall request, not just connect)
• TCP keepalive interval set < load balancer idle timeout
• MaxIdleConnsPerHost / pool size sized to the workload (default is often 2)
• TLS minimum is 1.2; prefer 1.3
• Retry policy bounded (max attempts, max total time, backoff with jitter)
• Metrics: latency histogram, status counter, connection-reuse ratio
• If gRPC: channel reused; keepalive options set; permit_without_calls if appropriate

For any PR adding or changing a .proto:

• No field re-numbered
• Removed fields have reserved for both number and name
• New fields use unused numbers (never reused)
• Optional semantics: optional keyword (proto3 since 2020) or wrapper types
• Package versioned in the namespace (myorg.users.v1)
• Generated code committed or generated in CI from the .proto

---

References

• references/grpc-vs-rest-decision.md - Decision matrix with worked examples
• references/protobuf-schema-evolution.md - Schema evolution rules, presence semantics, the field-renumber hazard

Related

• Rule: 320-api-design.mdc - API surface (verbs, status, versioning) - the layer above this skill
• Rule: 316-zero-trust.mdc - Network section (mTLS, default-deny egress) - the security model on top of this transport
• Rule: 330-observability.mdc - Logging, metrics, tracing - including the transport-layer metrics above
• Rule: 400-cloudflare.mdc - CDN patterns, Workers, edge-to-origin connection lifecycle
• Rule: 410-aws.mdc - AWS ALB/NLB/API Gateway idle-timeout specifics
• Rule: 483-kafka.mdc - Kafka transport + Avro/Protobuf schema registry
• Rule: 510-mcp-servers.mdc - MCP protocol patterns
• Skill: zero-trust - Network-tier security pairing
• Skill: cloud-platforms - Per-cloud LB and edge specifics