Optimize context window usage for OpenRouter models to reduce cost and improve quality. Use when hitting context limits, managing long conversations, or building RAG systems. Triggers: 'openrouter context', 'context window', 'openrouter token limit', 'reduce tokens openrouter'.
Optimize context window usage for OpenRouter models to reduce cost and improve quality. Use when hitting context limits, managing long conversations, or building RAG systems. Triggers: 'openrouter context', 'context window', 'openrouter token limit', 'reduce tokens openrouter'.
OpenRouter models have varying context windows (4K to 1M+ tokens). Since pricing is per-token, stuffing unnecessary context wastes money and can degrade output quality. This skill covers context window lookup, token estimation, conversation trimming, chunking strategies, and Anthropic prompt caching for large contexts.
Prerequisites
An OpenRouter API key (sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
Python 3.8+ with the OpenAI SDK and requests for model-metadata lookup; tiktoken for exact token counting per the references
curl and jq to query context windows and pricing from /api/v1/models
Node.js 18+ if you use the TypeScript context-budget calculator in the references
Instructions
Run the Query Context Limits one-liner — it returns context_length and prompt price per 1M tokens for each candidate model, so you know the real budget before writing code.
Estimate input size (~4 characters per token, or exactly with tiktoken per the references) and pick a model with select_model_for_context() from Context-Aware Model Selection — it applies an 80% safety margin and falls back through gpt-4o-mini (128K) → Claude 3.5 Sonnet (200K) → Gemini 2.0 Flash (1M).
Keep long conversations inside budget with trim_conversation() per Conversation Trimming: system prompt plus the last N messages, with a trim-marker note injected where history was dropped.
For documents that exceed any window, use chunk_and_process() per Chunking for Large Documents — 8,000-char chunks with 500-char overlap, analyzed independently at temperature=0 and then synthesized.
Mark large static blocks with cache_control: {"type": "ephemeral"} per Prompt Caching for Repeated Context to cut repeated input cost by 90% on Anthropic models.
Monitor prompt_tokens on every response (Enterprise Considerations) to catch context bloat before it becomes a 400 context_length_exceeded.
Query Context Limits
# Check context window for specific models
curl -s https://openrouter.ai/api/v1/models | jq '[.data[] | select(
.id == "anthropic/claude-3.5-sonnet" or
.id == "openai/gpt-4o" or
.id == "google/gemini-2.0-flash-001" or
.id == "meta-llama/llama-3.1-70b-instruct"
) | {id, context_length, prompt_per_M: ((.pricing.prompt|tonumber)*1000000)}]'
Context-Aware Model Selection
import os, requests
from openai import OpenAI
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
default_headers={"HTTP-Referer": "https://my-app.com", "X-Title": "my-app"},
)
# Cache model metadata at startup
MODELS = {m["id"]: m for m in requests.get("https://openrouter.ai/api/v1/models").json()["data"]}
defestimate_tokens(text: str) -> int:
"""Rough estimate: 1 token ~ 4 characters for English text."""returnlen(text) // 4defselect_model_for_context(messages: list, preferred: str = "anthropic/claude-3.5-sonnet") -> str:
"""Pick a model that fits the context, falling back to larger windows."""
estimated_tokens = sum(len(m.get("content", "")) for m in messages) // 4
FALLBACK_CHAIN = [
("openai/gpt-4o-mini", 128_000),
("anthropic/claude-3.5-sonnet", 200_000),
("google/gemini-2.0-flash-001", 1_000_000),
]
# Try preferred model first
preferred_ctx = MODELS.get(preferred, {}).get("context_length", 0)
if estimated_tokens < preferred_ctx * 0.8: # 80% safety marginreturn preferred
for model_id, ctx in FALLBACK_CHAIN:
if estimated_tokens < ctx * 0.8:
return model_id
raise ValueError(f"Content too large ({estimated_tokens} est. tokens)")
Conversation Trimming
deftrim_conversation(
messages: list[dict],
max_tokens: int = 100_000,
keep_system: bool = True,
keep_last_n: int = 4,
) -> list[dict]:
"""Trim conversation history to fit context window.
Strategy: Keep system prompt + last N messages.
If still too large, reduce to last 2 messages.
"""
system = [m for m in messages if m["role"] == "system"] if keep_system else []
non_system = [m for m in messages if m["role"] != "system"]
kept = non_system[-keep_last_n:]
trimmed = non_system[:-keep_last_n] iflen(non_system) > keep_last_n else []
total_est = sum(estimate_tokens(m.get("content", "")) for m in system + kept)
if total_est > max_tokens and keep_last_n > 2:
kept = non_system[-2:]
result = system + kept
if trimmed:
summary_note = {
"role": "system",
"content": f"[Previous {len(trimmed)} messages trimmed for context limits]",
}
result = system + [summary_note] + kept
return result
