reverse-proxy-for-agents

license	Apache-2.0
name	reverse-proxy-for-agents
description	Reverse proxy architecture for AI agent systems — Nginx, Caddy, Traefik, and Cloudflare. Agent-specific patterns: routing to specialist agents, load balancing, TLS termination, rate limiting, WebSocket proxying, header injection for agent identity, and the "agent gateway" pattern. Covers LLM routers (LiteLLM, OpenRouter) as intelligent reverse proxies, service mesh (Envoy/Istio) for agent fleets, and MCP servers behind reverse proxies. Activate on 'reverse proxy', 'nginx proxy', 'caddy', 'traefik', 'agent gateway', 'LLM router', 'LiteLLM proxy', 'WebSocket proxy', 'service mesh agents'. NOT for: tunnels (use tunnels-for-agents), firewall rules (use agentic-zero-trust-security), container orchestration (use devops-automator), DNS configuration (use infrastructure skills).
allowed-tools	Read,Write,Edit,Bash,Glob,Grep,WebSearch,WebFetch
metadata	{"category":"Infrastructure & Networking","tags":["reverse-proxy","nginx","caddy","traefik","load-balancing","api-gateway","llm-router","websocket","service-mesh","agent-routing","tls-termination","mcp-proxy"],"pairs-with":[{"skill":"tunnels-for-agents","reason":"Tunnels expose local services; reverse proxies route and protect them"},{"skill":"ipc-communication-patterns","reason":"IPC handles inter-process messaging; reverse proxy handles inter-network routing"},{"skill":"agentic-zero-trust-security","reason":"Zero-trust principles govern how the proxy authenticates and authorizes agent traffic"},{"skill":"daemon-development","reason":"Reverse proxies are long-running daemons with lifecycle management needs"},{"skill":"multi-agent-coordination","reason":"Multi-agent systems need intelligent routing to coordinate work across agents"}]}
category	Backend & Infrastructure
tags	["reverse-proxy","agents","routing","load-balancing","api-gateway"]

Reverse Proxy for Agents

Expert in reverse proxy architecture for AI agent systems, specializing in routing, load balancing, TLS termination, and real-time communication patterns.

Decision Points

Given requirements, choose proxy tool:

INPUT: (throughput, k8s_env, auto_reload, simplicity)

IF throughput > 10k_rps:
  → Use Nginx (C implementation, event-driven)
  → Configure keepalive, worker processes
  → Manual config reload with nginx -s reload

ELIF k8s_env == true AND auto_reload == true:
  → Use Traefik (Docker/K8s service discovery)
  → Configure via container labels
  → Automatic routing updates on container changes

ELIF simplicity == true AND throughput < 5k_rps:
  → Use Caddy (automatic HTTPS, zero-config WebSocket)
  → Caddyfile syntax, hot reload via API
  → Built-in Let's Encrypt integration

ELIF edge_protection == true:
  → Use Cloudflare (DDoS, WAF, CDN at edge)
  → Dashboard config, instant propagation
  → DNS-based routing

Routing Strategy Decision Tree

REQUEST ANALYSIS:

IF agent_fleet_size < 5:
  → Path-based routing: /api/code/* → code-agent
  → Manual upstream configuration
  → Health checks every 30s

ELIF agents_are_stateful == true:
  → Cookie-based sticky sessions
  → lb_policy cookie agent_session (Caddy)
  → ip_hash (Nginx) for IP-based affinity

ELIF real_time_required == true:
  → WebSocket proxying configuration
  → proxy_buffering off (Nginx)
  → Long timeouts: proxy_read_timeout 86400s
  → Connection upgrade headers

ELIF content_based_routing == true:
  → Implement lightweight classifier at proxy
  → Keyword matching or fast LLM (Haiku-class)
  → Route based on request body analysis

LLM Router Selection

CONSTRAINTS ANALYSIS:

IF budget_control == critical:
  → LiteLLM (self-hosted, full cost tracking)
  → max_budget configuration
  → Per-user spending limits

ELIF model_variety > 100:
  → OpenRouter (500+ models, provider fallback)
  → Cloud service, per-token markup
  → Built-in latency-based routing

ELIF observability == priority:
  → Helicone (request tracing, analytics)
  → Pass-through proxy model
  → Cost per request tracking

Failure Modes

1. "Dead Backend Broadcasting" Anti-Pattern

Symptom Detection:

• 502 Bad Gateway errors in proxy logs
• Health check endpoint returning failures
• Upstream connect timeouts in metrics

Root Cause: Proxy sending traffic to crashed/unresponsive agents Fix Strategy:

1. Configure health checks: health_uri /health (Caddy) or max_fails=2 fail_timeout=60s (Nginx)
2. Set fail_timeout to remove dead backends from rotation
3. Monitor upstream response codes: 502/503/504 indicate backend issues