openrouter-api-best-practices

name	openrouter-api-best-practices
description	Use when integrating OpenRouter API, optimizing LLM routing, managing API costs, or troubleshooting OpenRouter requests.
version	1.0.0
author	Hermes Agent
license	MIT
platforms	["linux","macos","windows"]
metadata	{"hermes":{"tags":["openrouter","api","llm","routing","cost-optimization","best-practices"],"related_skills":["systematic-debugging","subagent-driven-development","test-driven-development"]}}

OpenRouter API Best Practices

Overview

OpenRouter is a unified API gateway that provides access to multiple LLM providers (OpenAI, Anthropic, Google, Meta, Mistral, etc.) through a single API endpoint. It handles provider abstraction, rate limiting, and billing so you don't need to manage multiple API keys.

Key advantages:

• Single API key for 100+ models
• Automatic provider fallback
• Built-in usage tracking and cost management
• Standard OpenAI-compatible API format

When to Use

Use this skill when:

• Integrating OpenRouter API into an application
• Selecting appropriate models for tasks
• Debugging API errors or rate limit issues
• Optimizing API costs
• Setting up fallback strategies
• Configuring request/response handling

Don't use for:

• Non-API issues (UI, infrastructure)
• Provider-specific features beyond OpenRouter's abstraction

Core Configuration

API Key Management

Store your OpenRouter API key securely — never hardcode:

# Environment variable (recommended)
import os
api_key = os.environ.get("OPENROUTER_API_KEY")

# Or use a secrets manager
from keyring import get_password
api_key = get_password("openrouter", "api_key")

Key rotation: OpenRouter keys can be regenerated from the dashboard. Plan for key rotation if using service accounts.

Base URL and Endpoint

# OpenRouter API base
BASE_URL = "https://openrouter.ai/api/v1"

# Chat completions endpoint
CHAT_COMPLETIONS = f"{BASE_URL}/chat/completions"

Model Selection

Model ID Format

OpenRouter uses a namespace format: provider/model-name

# Examples
MODELS = {
    "gpt4o": "openai/gpt-4o",
    "gpt4o_mini": "openai/gpt-4o-mini",
    "claude_sonnet": "anthropic/claude-sonnet-4-20250514",
    "claude_opus": "anthropic/claude-opus-4-20250514",
    "gemini_pro": "google/gemini-pro",
    "deepseek": "deepseek/deepseek-chat-v3-0324",
    "llama": "meta-llama/llama-3-70b-instruct",
    "mixtral": "mistralai/mixtral-8x22b-instruct",
}

Model Selection Criteria

Criterion	Low-latency Tasks	High-quality Tasks	Cost-sensitive
Recommended	gpt-4o-mini, gemini-flash	claude-opus-4, gpt-4o	llama-3-70b, deepseek-v3
Context window	128k	200k	128k-1M
Price tier	$0.15/1M tokens	$15/1M tokens	$0.50-2/1M tokens

Provider Priority

When you don't care about specific models, use OpenRouter's ranking endpoint:

# Get best available model for a prompt
import requests

response = requests.post(
    f"{BASE_URL}/chat/completions",
    headers={
        "Authorization": f"Bearer {api_key}",
        "HTTP-Referer": "https://your-app.com",
        "X-Title": "Your App Name",
    },
    json={
        "model": "openrouter/auto",  # Let OpenRouter choose
        "messages": [{"role": "user", "content": "What's the weather?"}],
        "route": "fallback",  # Try alternatives on failure
    }
)

Request Handling

Standard Chat Completion Request

import requests

def chat_completion(messages, model="openai/gpt-4o-mini", **kwargs):
    response = requests.post(
        CHAT_COMPLETIONS,
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
            "HTTP-Referer": "https://your-app.com",
            "X-Title": "Your App Name",
        },
        json={
            "model": model,
            "messages": messages,
            # Common options
            "temperature": kwargs.get("temperature", 0.7),
            "max_tokens": kwargs.get("max_tokens", 1024),
            "stream": kwargs.get("stream", False),
            "timeout": kwargs.get("timeout", 60),
        },
        timeout=kwargs.get("timeout", 60)
    )
    
    if response.status_code != 200:
        raise APIError(f"OpenRouter error: {response.status_code} - {response.text}")
    
    return response.json()

Streaming Responses

def stream_chat(messages, model="openai/gpt-4o-mini"):
    import requests
    
    response = requests.post(
        CHAT_COMPLETIONS,
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
        json={
            "model": model,
            "messages": messages,
            "stream": True,
        },
        stream=True
    )
    
    for line in response.iter_lines():
        if line:
            # Parse SSE format: data: {"choices":[...]}
            if line.startswith("data: "):
                data = json.loads(line[6:])
                if data.get("choices"):
                    delta = data["choices"][0].get("delta", {})
                    if delta.get("content"):
                        yield delta["content"]

Structured Outputs

from pydantic import BaseModel

class Recipe(BaseModel):
    title: str
    ingredients: list[str]
    instructions: list[str]

def get_structured_output(messages, output_model=Recipe):
    response = requests.post(
        CHAT_COMPLETIONS,
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
        json={
            "model": "openai/gpt-4o-2024-08-06",  # Requires structured output support
            "messages": messages,
            "response_format": {
                "type": "json_schema",
                "json_schema": output_model.model_json_schema()
            }
        }
    )
    
    result = response.json()
    content = result["choices"][0]["message"]["content"]
    return output_model.model_validate_json(content)

Error Handling

Error Response Format

OpenRouter returns errors in this format:

{
    "error": {
        "message": "Invalid API key",
        "type": "auth_error",
        "code": 401
    }
}

Retry Logic with Exponential Backoff

import time
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def create_session():
    session = requests.Session()
    retry = Retry(
        total=3,
        backoff_factor=1,
        status_forcelist=[429, 500, 502, 503, 504],
        allowed_methods=["POST"]
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount("https://", adapter)
    return session

def chat_with_retry(messages, model, max_retries=3):
    session = create_session()
    
    for attempt in range(max_retries):
        try:
            response = session.post(
                CHAT_COMPLETIONS,
                headers={
                    "Authorization": f"Bearer {api_key}",
                    "Content-Type": "application/json",
                },
                json={"model": model, "messages": messages},
                timeout=60
            )
            
            if response.status_code == 200:
                return response.json()
            
            error = response.json().get("error", {})
            
            # Rate limit - wait and retry
            if response.status_code == 429:
                wait_time = int(response.headers.get(, ))
                ()
                time.sleep(wait_time)
                
            
            
             response.status_code >= :
                wait_time =  ** attempt
                time.sleep(wait_time)
                
            
            
             APIError()
            
         requests.exceptions.Timeout:
             attempt < max_retries - :
                time.sleep( ** attempt)
                
            

     APIError()

Error Types and Handling

Error Code	Type	Action
401	Auth error	Check API key validity
403	Forbidden	Check key permissions, account status
429	Rate limited	Wait for Retry-After, implement backoff
500-504	Server error	Retry with backoff
Circuit open	Provider down	Use fallback model

Rate Limits

Understanding Limits

OpenRouter has two rate limit layers:

1. OpenRouter global limits — per-key limits
2. Provider limits — underlying model provider limits

# Check rate limit headers in response
headers = response.headers
print(f"Remaining: {headers.get('X-RateLimit-Remaining')}")
print(f"Reset: {headers.get('X-RateLimit-Reset')}")

Rate Limit Strategy

from collections import defaultdict
import time

class RateLimiter:
    def __init__(self, calls_per_minute=60):
        self.calls_per_minute = calls_per_minute
        self.calls = defaultdict(list)
    
    def wait_if_needed(self, model=None):
        now = time.time()
        key = model or "default"
        
        # Remove calls older than 1 minute
        self.calls[key] = [t for t in self.calls[key] if now - t < 60]
        
        if len(self.calls[key]) >= self.calls_per_minute:
            oldest = self.calls[key][0]
            wait_time = 60 - (now - oldest)
            if wait_time > 0:
                time.sleep(wait_time)
                self.calls[key] = [t for t in self.calls[key] if time.time() - t < 60]
        
        self.calls[key].append(time.time())

Cost Optimization

Token Tracking

def estimate_cost(usage, model):
    """Calculate cost from usage dict."""
    # Prices per million tokens (approximate)
    PRICES = {
        "openai/gpt-4o": {"input": 2.50, "output": 10.00},
        "openai/gpt-4o-mini": {"input": 0.15, "output": 0.60},
        "anthropic/claude-sonnet-4-20250514": {"input": 3.00, "output": 15.00},
        "deepseek/deepseek-chat-v3-0324": {"input": 0.14, "output": 0.28},
        "meta-llama/llama-3-70b-instruct": {"input": 0.50, "output": 0.80},
    }
    
    model_prices = PRICES.get(model, {"input": 1.0, "output": 2.0})
    
    input_cost = (usage["prompt_tokens"] / 1_000_000) * model_prices["input"]
    output_cost = (usage["completion_tokens"] / 1_000_000) * model_prices["output"]
    
    return {
        "input_tokens": usage["prompt_tokens"],
        "output_tokens": usage["completion_tokens"],
        "total_tokens": usage["total_tokens"],
        "estimated_cost_usd": (input_cost + output_cost, )
    }


response = chat_completion(messages)
usage = response.get(, {})
cost = estimate_cost(usage, response[])
()

Cost-Saving Strategies

1. Use cheaper models for simple tasks

   def route_model(task_complexity):
       if task_complexity == "simple":
           return "openai/gpt-4o-mini"  # $0.15/1M input
       elif task_complexity == "medium":
           return "deepseek/deepseek-chat-v3-0324"  # $0.14/1M input
       else:
           return "openai/gpt-4o"  # $2.50/1M input
   

2. Enable prompt caching (where supported)

   # Messages with repeated system prompts benefit from caching
   messages = [
       {"role": "system", "content": SYSTEM_PROMPT},  # Cached
       {"role": "user", "content": user_input},       # Not cached
   ]
   

3. Truncate context intelligently

   def truncate_to_token_limit(messages, max_tokens=100000):
       """Preserve recent messages while staying under limit."""
       # Sort by recency, keep most recent
       while count_tokens(messages) > max_tokens:
           # Remove oldest non-system message
           for i, msg in enumerate(messages):
               if msg["role"] != "system":
                   messages.pop(i)
                   
   

Fallback and Resilience

Multi-Model Fallback

def chat_with_fallback(messages, models=None):
    """
    Try models in order until one succeeds.
    models: list of (model_id, priority)
    """
    if models is None:
        models = [
            ("openai/gpt-4o-mini", 1),
            ("deepseek/deepseek-chat-v3-0324", 2),
            ("meta-llama/llama-3-70b-instruct", 3),
        ]
    
    errors = []
    
    for model, _ in sorted(models, key=lambda x: x[1]):
        try:
            response = chat_completion(messages, model=model)
            return {"response": response, "model_used": model, "error": None}
        except Exception as e:
            errors.append({"model": model, "error": str(e)})
            continue
    
    return {"response": None, "model_used": None, "error": errors}

Circuit Breaker Pattern

from collections import deque
import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.failures = deque()
        self.state = "closed"  # closed, open, half-open
    
    def call(self, func, *args, **kwargs):
        if self.state == "open":
            if time.time() - self.failures[0] > self.timeout:
                self.state = "half-open"
            else:
                raise CircuitOpenError("Circuit is open")
        
        try:
            result = func(*args, **kwargs)
            if self.state == "half-open":
                self.state = "closed"
                self.failures.clear()
            return result
        except Exception as e:
            self.failures.append(time.time())
            if len(self.failures) >= .failure_threshold:
                .state = 
            

API Reference Headers

OpenRouter requires specific headers for analytics and attribution:

REQUIRED_HEADERS = {
    "Authorization": f"Bearer {api_key}",
    "Content-Type": "application/json",
}

RECOMMENDED_HEADERS = {
    "HTTP-Referer": "https://your-app.com",  # Your website
    "X-Title": "Your App Name",               # App identifier
}

These headers don't affect API functionality but help with:

• Usage analytics in OpenRouter dashboard
• Potential rate limit increases
• Compliance with provider terms

Common Pitfalls

1. Not Handling Rate Limits

Problem: Ignoring 429 responses causes request failures and potential key suspension.

Fix:

# Always check for rate limit headers
if response.status_code == 429:
    retry_after = int(response.headers.get("Retry-After", 60))
    time.sleep(retry_after)
    # Retry the request

2. Hardcoding Model Names

Problem: Provider changes model IDs, breaking production.

Fix: Use constants or configuration:

# Bad
model = "openai/gpt-4o-2024-05-13"

# Good
from config import MODELS
model = MODELS["gpt4o"]  # Centralized, version-agnostic

3. Not Using Structured Outputs for Reliability

Problem: Parsing raw text output is fragile.

Fix: Use response_format for JSON output:

json_resp = requests.post(CHAT_COMPLETIONS, json={
    "model": "openai/gpt-4o-2024-08-06",
    "messages": messages,
    "response_format": {"type": "json_schema", "json_schema": {...}}
})

4. Ignoring Token Usage

Problem: Unexpected costs from large contexts.

Fix: Always log and monitor usage:

usage = response.get("usage", {})
print(f"Tokens: {usage['total_tokens']} | "
      f"Prompt: {usage['prompt_tokens']} | "
      f"Completion: {usage['completion_tokens']}")

5. No Fallback for Provider Outages

Problem: Single model dependency = single point of failure.

Fix: Implement fallback chain:

PRIMARY_MODEL = "openai/gpt-4o"
FALLBACK_MODELS = [
    "anthropic/claude-sonnet-4-20250514",
    "deepseek/deepseek-chat-v3-0324",
]

6. Storing API Keys in Code

Problem: Keys in source code can be leaked via version control.

Fix: Use environment variables or secrets management:

# Environment variable
api_key = os.environ["OPENROUTER_API_KEY"]

# Or secret manager
api_key = get_secrets("openrouter-api-key")

Verification Checklist

• API key stored in environment variable, not hardcoded
• Rate limit errors handled with backoff
• Usage/cost tracking implemented
• At least one fallback model configured
• Proper error handling for all error codes
• Recommended headers (HTTP-Referer, X-Title) included
• Request timeout configured (recommended: 60s)
• Streaming properly handles SSE parsing
• Token counting implemented for cost estimation
• Circuit breaker for resilience against provider outages

Further Reading

• OpenRouter Documentation
• API Reference
• Model Rankings
• Provider Status