openrouter-performance-tuning

name	openrouter-performance-tuning
description	Optimize OpenRouter request latency and throughput. Use when building real-time applications, reducing TTFT, or scaling request volume. Triggers: 'openrouter performance', 'openrouter latency', 'openrouter speed', 'optimize openrouter throughput'.
allowed-tools	Read, Write, Edit, Grep, Bash(python3:*)
version	1.20.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","openrouter","performance","latency","optimization"]
compatibility	Designed for Claude Code

OpenRouter Performance Tuning

Overview

OpenRouter adds minimal overhead (~50-100ms) to direct provider calls. Most latency comes from the upstream model. Key levers: model selection (smaller = faster), streaming (lower TTFT), parallel requests, prompt size reduction, and provider routing to faster infrastructure. This skill covers benchmarking, streaming optimization, concurrent processing, and connection tuning.

Prerequisites

• An OpenRouter API key (sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
• Python 3.8+ with the OpenAI SDK (openai package) — the examples use both the sync OpenAI client and AsyncOpenAI for parallel processing
• Credits on the key if you benchmark paid models like anthropic/claude-3.5-sonnet; a :free model is enough to validate the benchmark harness itself
• HTTP-Referer / X-Title header values for your app (set in every client constructor here)

Instructions

1. Establish a baseline: run benchmark_model() from Benchmark Latency against your candidate models (e.g. openai/gpt-4o-mini vs anthropic/claude-3.5-sonnet) and record p50/p95.
2. Check the results against the Model Speed Tiers table to confirm each candidate sits in the right tier for your latency budget (200-500ms TTFT fastest tier; 5-30s for reasoning models).
3. Switch user-facing paths to stream_completion() per Streaming for Lower TTFT and verify ttft_ms drops (typically 2-10x).
4. Move batch workloads to parallel_completions() per Parallel Request Processing, capping concurrency with asyncio.Semaphore (max_concurrent=5-10).
5. Apply Connection Optimization — one shared client with timeout=30.0 and max_retries=2 instead of a new client per request.
6. Work through the Performance Optimization Checklist (set max_tokens, shrink prompts, consider :nitro variants and provider routing), then re-run the benchmark to quantify each change.

Benchmark Latency

import os, time, statistics
from openai import OpenAI

client = OpenAI(
    base_url=,
    api_key=os.environ[],
    default_headers={: , : },
)

 () -> :
    
    latencies = []
     _  (n):
        start = time.monotonic()
        response = client.chat.completions.create(
            model=model,
            messages=[{: , : prompt}],
            max_tokens=,
        )
        latencies.append((time.monotonic() - start) * )

     {
        : model,
        : (statistics.median(latencies)),
        : ((latencies)[((latencies) * )]),
        : (statistics.mean(latencies)),
        : ((latencies)),
        : ((latencies)),
    }


 model  [, , ]:
    result = benchmark_model(model)
    ()