OpenRouter Known Pitfalls
Overview
A curated list of real-world mistakes developers make when integrating OpenRouter, each with the specific API behavior that causes the problem and the exact fix. These are not theoretical -- they come from production incidents and support requests.
Prerequisites
- An existing (or in-progress) OpenRouter integration to audit against the 10 pitfalls below
- An OpenRouter API key (
sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
- Python 3.8+ with the OpenAI SDK (
pip install openai) to run the validation snippets (e.g., the startup check against /api/v1/models)
- Grep access to the codebase to hunt hardcoded
sk-or-v1- keys and scattered model IDs
Instructions
- Audit request format first: every model ID uses the
provider/model form (Pitfall 1), and model IDs live in one MODELS config validated against /api/v1/models at startup instead of being scattered through the code (Pitfall 3).
- Check cost controls:
max_tokens is set on every request (Pitfall 2) and no :free models are used in production, where the 50-1000 req/day limits will 429 you (Pitfall 5).
- Review routing: sensitive-data requests pin
provider.order with allow_fallbacks: False (Pitfall 4), and response.model is logged on every call to catch unexpected fallbacks (Pitfall 6).
- Inspect client hygiene: one shared client instance with connection pooling (Pitfall 7) configured with
timeout and max_retries (Pitfall 9).
- Sweep for secrets: grep for hardcoded
sk-or-v1- strings and move any hits to env vars or a secrets manager, rotating the exposed keys (Pitfall 8).
- Verify caching only stores deterministic
temperature=0 responses (Pitfall 10).
- Finish by walking the Quick Checklist (
PITFALL_CHECKLIST) top to bottom — it condenses all 10 pitfalls into a code-review pass.
Pitfall 1: Missing Provider Prefix on Model ID
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
)
response = client.chat.completions.create(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
)
Pitfall 2: No max_tokens = Runaway Costs
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Write a story"}],
)
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Write a story"}],
max_tokens=500,
)
Pitfall 3: Hardcoded Model IDs Break When Models Are Renamed
MODELS = {
"primary": "anthropic/claude-3.5-sonnet",
"budget": "openai/gpt-4o-mini",
"free": "google/gemma-2-9b-it:free",
}
import requests
available = {m["id"] for m in requests.get("https://openrouter.ai/api/v1/models").json()["data"]}
for name, model_id in MODELS.items():
if model_id not in available:
print(f"WARNING: {name} model '{model_id}' not available!")
Pitfall 4: Fallbacks Route to Unexpected Providers
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": sensitive_data}],
)
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": sensitive_data}],
extra_body={
"provider": {
"order": ["Anthropic"],
"allow_fallbacks": False,
},
},
)
Pitfall 5: Ignoring the Free Model Daily Limit
response = client.chat.completions.create(
model="google/gemma-2-9b-it:free",
messages=[{"role": "user", "content": "Hello"}],
)
Pitfall 6: Not Checking Which Model Actually Served the Request
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Hello"}],
)
print(f"Served by: {response.model}")
if response.model != "anthropic/claude-3.5-sonnet":
log.warning(f"Fallback triggered: requested claude-3.5-sonnet, got {response.model}")
Pitfall 7: Creating New Client Instance Per Request
for prompt in prompts:
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=key)
client.chat.completions.create(...)
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
default_headers={"HTTP-Referer": "https://my-app.com", "X-Title": "my-app"},
)
for prompt in prompts:
client.chat.completions.create(...)
Pitfall 8: Storing API Keys in Source Code
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key="sk-or-v1-abc123...",
)
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
)
Pitfall 9: Not Setting Timeouts
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=key)
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
timeout=30.0,
max_retries=3,
)
Pitfall 10: Caching Non-Deterministic Responses
cache[key] = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=msgs,
temperature=0.7,
)
if temperature == 0:
cache[key] = response
Quick Checklist
PITFALL_CHECKLIST = [
"Model IDs use provider/model format (e.g., openai/gpt-4o)",
"max_tokens set on every request",
"API keys in env vars or secrets manager, never in code",
"Single client instance reused (not created per request)",
"Timeout and max_retries configured",
"response.model checked (may differ from requested model)",
"Free models NOT used in production",
"Fallback behavior explicitly controlled for sensitive data",
"Model IDs centralized in config (not scattered in code)",
"Only deterministic responses (temp=0) are cached",
]
Output
An audit pass with this skill produces:
- A pitfall-by-pitfall verdict on your integration — each of the 10 items either confirmed clean or flagged with the exact fix from its section
- Startup validation output from the Pitfall 3 snippet:
WARNING: primary model 'anthropic/claude-3.5-sonnet' not available! for any config entry missing from /api/v1/models
- Fallback-detection log lines from Pitfall 6:
Fallback triggered: requested claude-3.5-sonnet, got <response.model>
- The completed
PITFALL_CHECKLIST — a 10-line review artifact to attach to the integration PR
Examples
Validating your centralized model config at startup (Pitfall 3):
available = {m["id"] for m in requests.get("https://openrouter.ai/api/v1/models").json()["data"]}
for name, model_id in MODELS.items():
if model_id not in available:
print(f"WARNING: {name} model '{model_id}' not available!")
A warning here means a rename or removal upstream — update the one MODELS entry instead of chasing hardcoded IDs across the codebase. More worked examples: references/examples.md.
Error Handling
| Pitfall | Symptom | Quick Fix |
|---|
| Missing provider prefix | 400 model not found | Add openai/, anthropic/, etc. |
| No max_tokens | Unexpected high costs | Add max_tokens to every call |
| Hardcoded API key | Key exposed in git history | Rotate key; use env vars |
| No timeout | Hanging requests | Set timeout=30.0 |
| Free model in prod | 429 after 50-1000 requests | Use paid models |
Enterprise Considerations
- Run the pitfall checklist during code review for any OpenRouter integration PR
- Add pre-commit hooks that scan for hardcoded
sk-or-v1- patterns
- Centralize model IDs in a config file and validate against
/api/v1/models at startup
- Log
response.model on every request to catch unexpected fallbacks
- Set
max_tokens as a team-wide policy enforced in your client wrapper
References
