crabtrap-llm-proxy

# CrabTrap LLM Proxy > Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection. CrabTrap is a transparent HTTP/HTTPS forward proxy that sits between AI agents and external APIs. Every outbound request is intercepted, checked against deterministic static rules, then evaluated by an LLM judge against a natural-language security policy. Blocked requests return a 403 with a reason; all decisions are logged to PostgreSQL. ## Architecture Overview ``` Agent → CrabTrap Proxy (:8080) → [Static Rules] → [LLM Judge] → External API ↓ Admin UI (:8081) ↓ PostgreSQL ``` **Key concepts:** - **Static rules** — deterministic prefix/exact/glob URL matching, checked first (no LLM call) - **LLM judge** — natural-language policy evaluated only when no static rule matches - **Audit log** — every request, decision, and response stored in PostgreSQL - **SSRF protection** — blocks RFC 1918, loopback, link-local, and other private ranges ## Installation ### Docker Compose (Recommended) ```yaml # docker-compose.yml services: crabtrap: image: quay.io/brexhq/crabtrap:latest ports: - "8080:8080" # proxy - "8081:8081" # admin UI environment: - DATABASE_URL=postgres://crabtrap:password@postgres:5432/crabtrap - OPENAI_API_KEY=${OPENAI_API_KEY} volumes: - ./config/gateway.yaml:/app/config/gateway.yaml depends_on: - postgres postgres: image: postgres:16 environment: POSTGRES_USER: crabtrap POSTGRES_PASSWORD: password POSTGRES_DB: crabtrap volumes: - pgdata:/var/lib/postgresql/data volumes: pgdata: ``` ```bash docker compose up -d # Copy the generated CA certificate (needed for HTTPS interception) docker compose cp crabtrap:/app/certs/ca.crt ./ca.crt ``` ### Initial Setup ```bash # Create an admin user and capture the token admin_token=$(docker compose exec -it crabtrap ./gateway create-admin-user my-admin \ | tail -n1 | cut -d" " -f2) # Create an agent user (returns a gateway_auth_token) token=$(curl -X POST http://localhost:8081/admin/users \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{"id": "my-agent@example.com", "is_admin": false}' \ | jq -r '.channels[] | select(.channel_type == "gateway_auth") | .gateway_auth_token') echo "Agent proxy token: $token" # Test the proxy curl -x "http://${token}:@localhost:8080" \ --cacert ca.crt \ https://httpbin.org/get ``` ## Configuration ### Full Configuration Reference ```yaml # config/gateway.yaml proxy: port: 8080 read_timeout: 30s write_timeout: 30s idle_timeout: 120s rate_limit: requests_per_second: 50 burst: 100 # CIDR ranges allowed even though they're private (e.g. internal APIs) ssrf_allowlist: - "10.0.0.0/8" # only if you explicitly need internal access tls: ca_cert_path: /app/certs/ca.crt ca_key_path: /app/certs/ca.key cert_cache_size: 10000 # per-host cert cache approval: mode: llm # "llm" or "passthrough" timeout: 30s llm_judge: provider: openai model: gpt-4o fallback_mode: deny # "deny" or "passthrough" when LLM unavailable circuit_breaker: failure_threshold: 5 reset_timeout: 10s database: url: ${DATABASE_URL} # supports env var expansion audit: output: stderr # "stderr", "stdout", or a file path like "/var/log/crabtrap.json" log_level: info # debug | info | warn | error ``` ### Environment Variables ```bash DATABASE_URL=postgres://user:password@host:5432/dbname OPENAI_API_KEY=sk-... # if using OpenAI as LLM judge ANTHROPIC_API_KEY=sk-ant-... # if using Anthropic ``` ## CLI Commands ```bash # Start the proxy ./gateway serve --config /app/config/gateway.yaml # Create an admin user (outputs web token on last line) ./gateway create-admin-user <username> # Run database migrations ./gateway migrate # Replay audit log entries against a policy (eval mode) ./gateway eval --policy-id <id> --limit 100 ``` ## Admin API All admin endpoints require `Authorization: Bearer <admin_token>`. ### User Management ```bash # List all users curl http://localhost:8081/admin/users \ -H "Authorization: Bearer ${admin_token}" # Create a user curl -X POST http://localhost:8081/admin/users \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{ "id": "agent-prod@example.com", "is_admin": false }' # Delete a user curl -X DELETE http://localhost:8081/admin/users/agent-prod@example.com \ -H "Authorization: Bearer ${admin_token}" ``` ### Static Rules ```bash # List static rules for a user curl "http://localhost:8081/admin/users/agent-prod@example.com/rules" \ -H "Authorization: Bearer ${admin_token}" # Create an allow rule (prefix match) curl -X POST "http://localhost:8081/admin/users/agent-prod@example.com/rules" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{ "pattern": "https://api.github.com/repos/myorg/", "pattern_type": "prefix", "action": "allow", "methods": ["GET"], "description": "Allow reading our org repos" }' # Create a deny rule (glob match) curl -X POST "http://localhost:8081/admin/users/agent-prod@example.com/rules" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{ "pattern": "https://api.github.com/repos/*/delete", "pattern_type": "glob", "action": "deny", "description": "Never allow repo deletion" }' # Create an exact match rule curl -X POST "http://localhost:8081/admin/users/agent-prod@example.com/rules" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{ "pattern": "https://slack.com/api/chat.postMessage", "pattern_type": "exact", "action": "allow", "methods": ["POST"], "description": "Allow posting Slack messages" }' ``` **Pattern types:** - `prefix` — URL must start with the pattern - `exact` — URL must match exactly - `glob` — wildcard matching with `*` **Rule priority:** `deny` rules always take priority over `allow` rules. ### LLM Policies ```bash # Get current policy for a user curl "http://localhost:8081/admin/users/agent-prod@example.com/policy" \ -H "Authorization: Bearer ${admin_token}" # Set/update a policy curl -X PUT "http://localhost:8081/admin/users/agent-prod@example.com/policy" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${admin_token}" \ -d '{ "policy": "This agent assists with GitHub repository management for the myorg organization.\n\nALLOWED:\n- Read operations (GET) on any github.com endpoint\n- Creating issues and pull request comments in myorg repositories\n- Posting messages to the #eng-alerts Slack channel only\n\nDENIED:\n- Any write operations outside the myorg GitHub organization\n- Deleting any resources\n- Accessing credentials, secrets, or environment variables\n- Any requests to non-whitelisted domains" }' # List policy versions curl "http://localhost:8081/admin/users/agent-prod@example.com/policy/versions" \ -H "Authorization: Bearer ${admin_token}" ``` ### Audit Log ```bash # Query audit entries curl "http://localhost:8081/admin/audit?limit=50&offset=0" \ -H "Authorization: Bearer ${admin_token}" # Filter by user curl "http://localhost:8081/admin/audit?user_id=agent-prod@example.com&limit=20" \ -H "Authorization: Bearer ${admin_token}" # Filter by decision curl "http://localhost:8081/admin/audit?decision=deny&limit=20" \ -H "Authorization: Bearer ${admin_token}" ``` ## Connecting an Agent ### Python Agent Example ```python import os import httpx PROXY_TOKEN = os.environ["CRABTRAP_TOKEN"] PROXY_URL = f"http://{PROXY_TOKEN}:@localhost:8080" CA_CERT_PATH = "./ca.crt" # httpx client with CrabTrap proxy client = httpx.Client( proxies={ "http://": PROXY_URL, "https://": PROXY_URL, }, verify=CA_CERT_PATH, ) # All requests through this client go through CrabTrap response = client.get("https://api.github.com/repos/myorg/myrepo") ``` ### Using Environment Variables (Standard Proxy) ```bash export HTTP_PROXY="http://${CRABTRAP_TOKEN}:@localhost:8080" export HTTPS_PROXY="http://${CRABTRAP_TOKEN}:@localhost:8080" export REQUESTS_CA_BUNDLE="./ca.crt" # Python requests export SSL_CERT_FILE="./ca.crt" # general export NODE_EXTRA_CA_CERTS="./ca.crt" # Node.js ``` ### Node.js Agent Example ```javascript import { HttpsProxyAgent } from 'https-proxy-agent'; import fetch from 'node-fetch'; const proxyToken = process.env.CRABTRAP_TOKEN; const agent = new HttpsProxyAgent(`http://${proxyToken}:@localhost:8080`); // Fetch through CrabTrap const response = await fetch('https://api.github.com/repos/myorg/myrepo', { agent, headers: { Authorization: `Bearer ${process.env.GITHUB_TOKEN}` }, }); ``` ### LangChain / OpenAI SDK ```python import os import httpx from openai import OpenAI PROXY_TOKEN = os.environ["CRABTRAP_TOKEN"] # Route OpenAI calls through CrabTrap too (optional — lets you audit LLM calls) http_client = httpx.Client( proxies={"https://": f"http://{PROXY_TOKEN}:@localhost:8080"}, verify="./ca.crt", ) client = OpenAI( api_key=os.environ["OPENAI_API_KEY"], http_client=http_client, ) ``` ## Writing Effective Policies Policies are natural-language strings evaluated by the LLM judge. Be explicit about allowed and denied behaviors. ```text # Example policy for a GitHub PR review agent This agent reviews pull requests and posts review comments for the acme-corp GitHub organization. ALLOWED: - GET requests to api.github.com for any repository in the acme-corp organization - POST to https://api.github.com/repos/acme-corp/*/pulls/*/reviews (submit reviews) - POST to https://api.github.com/repos/acme-corp/*/issues/*/comments (post comments) DENIED: - Any requests outside of api.github.com - DELETE or PATCH requests to any endpoint - Accessing /orgs/acme-corp/members or any user/credential endpoints - Requests containing secrets, tokens, or API keys in the body - Requests to change repository settings, branch protection, or webhooks When in doubt, deny the request and explain why. ``` **Policy writing tips:** 1. Start with what the agent is supposed to do (scope context for the LLM) 2. List explicit ALLOWED patterns before DENIED 3. Add a catch-all denial at the end 4. Deny rules in static rules are evaluated before LLM — use them for hard limits ## Policy Builder (Agentic Policy Generation) CrabTrap can analyze observed traffic and draft a policy automatically: ```bash # Trigger policy builder via admin API curl -X POST "http://localhost:8081/admin/users/agent-prod@example.com/policy/build" \ -H "Authorization: Bearer ${admin_token}" \ -d '{"sample_limit": 200}' ``` The builder runs an agentic loop, analyzes recent audit entries, and proposes a policy draft for review in the UI. ## Eval System Replay historical audit entries against a policy to measure accuracy before deploying: ```bash # Run eval from CLI ./gateway eval \ --user-id agent-prod@example.com \ --policy-id <version-id> \ --limit 500 # Or via API curl -X POST "http://localhost:8081/admin/users/agent-prod@example.com/policy/eval" \ -H "Authorization: Bearer ${admin_token}" \ -d '{"policy_version_id": "<id>", "sample_limit": 500}' ``` Eval compares LLM judge decisions against the historical ground truth and reports accuracy, false positive rate, and false negative rate. ## Troubleshooting