OpenRouter Production Checklist
Overview
A comprehensive production readiness checklist for OpenRouter integrations covering security, reliability, observability, cost management, and operational procedures. Each item includes the specific API endpoint or configuration needed to verify compliance.
Prerequisites
- An OpenRouter API key (
sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
- A working OpenRouter integration ready to launch — this skill validates an existing integration, it doesn't build one
curl and jq for the pre-launch validation script and the auth/credit-limit checks- Python 3.8+ if you turn the checklist dicts (
SECURITY, RELIABILITY, OBSERVABILITY) into automated CI checks
- A secret scanner in CI (gitleaks or trufflehog) — the security checklist verifies its presence
Instructions
- Work through the Security Checklist: keys in a secrets manager, 90-day rotation, per-key credit limits (
GET /api/v1/auth/key → .data.limit), secret scanning in CI, HTTPS-only endpoints.
- Verify the Reliability Checklist: a fallback chain (
models array + route: "fallback"), SDK max_retries=3 with built-in backoff, timeout=30.0, a circuit breaker on the primary model, and max_tokens on every request.
- Confirm the Observability Checklist: structured logs carrying
generation_id/model/latency/tokens/cost, error-rate alerts (>5% over 5 min), P50/P95 latency per model, daily cost tracking via GET /api/v1/generation?id=, and credit-balance alerts.
- Run the Pre-Launch Validation Script — it exercises auth, credit limit, primary-model availability, a live test request, and a hardcoded-key scan.
- Fix every FAIL and re-run until the script prints
READY FOR PRODUCTION, then wire it into CI as a pre-deploy gate per Enterprise Considerations.
Security Checklist
SECURITY = {
"api_key_storage": {
"check": "API keys stored in secrets manager (not .env files on disk)",
"verify": "grep -r 'sk-or-v1-' --include='*.py' --include='*.ts' . | grep -v node_modules",
"pass": "Zero matches",
},
"key_rotation": {
"check": "Keys rotated on 90-day schedule",
"verify": "Check key creation dates in OpenRouter dashboard",
"api": "GET /api/v1/keys (management key)",
},
"credit_limits": {
"check": "Per-key credit limits set to isolate blast radius",
"verify": "curl -s https://openrouter.ai/api/v1/auth/key -H 'Authorization: Bearer $KEY' | jq '.data.limit'",
"pass": "Non-null limit value",
},
"secret_scanning": {
"check": "CI pipeline includes secret scanning (gitleaks, trufflehog)",
"verify": "Check CI config for secret scanning step",
},
"https_enforced": {
"check": "All requests use https://openrouter.ai/api/v1",
"verify": "Grep codebase for 'http://openrouter' (should be zero)",
},
}
Reliability Checklist
RELIABILITY = {
"fallback_models": {
"check": "Fallback chain configured for critical models",
"config": """extra_body={"models": ["primary", "secondary", "tertiary"], "route": "fallback"}""",
},
"retry_logic": {
"check": "Retry with exponential backoff for 429 and 5xx errors",
"config": "OpenAI SDK max_retries=3 (built-in backoff)",
},
"timeouts": {
"check": "Per-request timeout configured",
"config": "OpenAI(timeout=30.0) # 30s per request",
},
"circuit_breaker": {
"check": "Circuit breaker on primary model (3 failures → fallback)",
"verify": "Review client wrapper for circuit breaker pattern",
},
"max_tokens": {
"check": "max_tokens set on EVERY request",
"verify": "Grep codebase for .create( calls without max_tokens",
},
}
Observability Checklist
OBSERVABILITY = {
"structured_logging": {
"check": "Every API call logged with generation_id, model, latency, tokens, cost",
"fields": ["timestamp", "generation_id", "model", "latency_ms", "prompt_tokens",
"completion_tokens", "cost", "status", "user_id"],
},
"error_alerting": {
"check": "Alerts on error rate spikes (>5% over 5 min window)",
"metric": "count(status=error) / count(*) over sliding 5min window",
},
"latency_monitoring": {
"check": "P50 and P95 latency tracked per model",
"threshold": "P95 < 10s for standard models, P95 < 30s for reasoning models",
},
"cost_tracking": {
"check": "Daily cost tracked and compared to budget",
"api": "GET /api/v1/generation?id={gen_id} for exact per-request cost",
},
"credit_balance_alert": {
"check": "Alert when credits drop below threshold",
"api": "GET /api/v1/auth/key → .data.usage vs .data.limit",
},
}
Pre-Launch Validation Script
#!/bin/bash
echo "=== OpenRouter Production Readiness ==="
PASS=0; FAIL=0
echo -n "1. API Authentication: "
AUTH=$(curl -s https://openrouter.ai/api/v1/auth/key \
-H "Authorization: Bearer $OPENROUTER_API_KEY" | jq -r '.data.label // "FAIL"')
if [ "$AUTH" != "FAIL" ]; then echo "PASS ($AUTH)"; ((PASS++)); else echo "FAIL"; ((FAIL++)); fi
echo -n "2. Credit Limit: "
LIMIT=$(curl -s https://openrouter.ai/api/v1/auth/key \
-H "Authorization: Bearer $OPENROUTER_API_KEY" | jq -r '.data.limit // "NONE"')
if [ "$LIMIT" != "NONE" ] && [ "$LIMIT" != "null" ]; then
echo "PASS (\$$LIMIT)"; ((PASS++))
else echo "WARN (no limit set)"; ((FAIL++)); fi
echo -n "3. Primary Model Available: "
MODEL="anthropic/claude-3.5-sonnet"
EXISTS=$(curl -s https://openrouter.ai/api/v1/models | jq --arg m "$MODEL" )
[ -gt 0 ]; ; ((PASS++)); ; ((FAIL++));
-n
TEST=$(curl -s https://openrouter.ai/api/v1/chat/completions \
-H \
-H \
-d \
| jq -r )
[ != ]; ; ((PASS++)); ; ((FAIL++));
-n
KEYS=$(grep -r --include= --include= --include= . 2>/dev/null | grep -v node_modules | grep -v | -l)
[ -eq 0 ]; ; ((PASS++)); ; ((FAIL++));
[ -eq 0 ] && ||
Output
- A pass/fail pre-launch validation report: five numbered checks, a
Results: N passed, M failed summary, and a final READY FOR PRODUCTION / FIX FAILURES BEFORE LAUNCH verdict
- A completed three-part readiness checklist (security, reliability, observability) with the verify command or API endpoint recorded per item
- A hardcoded-key scan result — zero
sk-or-v1- matches in source outside .env and node_modules
Examples
A correctly configured production key produces a clean run of the Pre-Launch Validation Script:
=== OpenRouter Production Readiness ===
1. API Authentication: PASS (my-app-prod)
2. Credit Limit: PASS ($100)
3. Primary Model Available: PASS (anthropic/claude-3.5-sonnet)
4. Test Request: PASS
5. No Hardcoded Keys: PASS
Results: 5 passed, 0 failed
READY FOR PRODUCTION
A key with no credit limit instead shows WARN (no limit set) on check 2 and counts as a failure — set a per-key limit to bound blast radius before launch. More worked examples: references/examples.md.
Error Handling
| Error | Cause | Fix |
|---|
| Production key exposed | Key logged or committed | Rotate immediately; deploy from secrets manager |
| No fallback configured | Primary model goes down | Add models array with route: "fallback" |
| Missing monitoring | Errors go undetected | Set up alerting before launch |
| No max_tokens | Runaway completion costs | Add max_tokens to every request |
Enterprise Considerations
- Run the validation script in CI as a pre-deploy gate
- Set up runbooks for common failure scenarios: rate limiting, credit exhaustion, provider outage
- Load test at 2x expected peak traffic to validate rate limits and fallback behavior
- Document escalation paths: when to contact OpenRouter support vs handle internally
- Review and update this checklist quarterly as OpenRouter adds features
- Keep a "break glass" procedure for emergency key rotation
References
