openrouter-reference-architecture

name	openrouter-reference-architecture
description	Design production architectures using OpenRouter as the LLM gateway. Use when planning system design, reviewing architecture, or scaling AI applications. Triggers: 'openrouter architecture', 'openrouter system design', 'openrouter at scale', 'llm gateway architecture'.
allowed-tools	Read, Write, Edit, Grep, Bash(python3:*)
version	1.20.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","openrouter","architecture","system-design","scaling"]
compatibility	Designed for Claude Code

OpenRouter Reference Architecture

Overview

OpenRouter serves as a unified LLM gateway, abstracting provider complexity. A production architecture wraps it with caching, rate limiting, cost controls, observability, and async processing. This skill provides three reference architectures: simple (single service), standard (microservice), and enterprise (event-driven).

Prerequisites

• An OpenRouter API key (sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
• Python 3.8+ with the OpenAI SDK; FastAPI + Pydantic for Architecture 2's AI service, and a Redis instance (with the redis package) for Architecture 2's cache and Architecture 3's queue/results store
• SQLite or Postgres if you implement Architecture 2's budget enforcer
• Your scale numbers — team size, requests/day, and latency needs drive the decision in Choosing an Architecture

Instructions

1. Score your system against the Choosing an Architecture table: team size, requests/day, latency needs, budget-tracking granularity, failure handling, observability.
2. Start with Architecture 1 (Simple): one shared client (max_retries=3, timeout=30.0) behind the logging complete() wrapper.
3. When you need task routing, caching, and per-user budgets, move to Architecture 2 (Standard): a FastAPI /v1/complete endpoint with the ROUTING_TABLE, cache-first lookup, budget check, and a fallback chain (models + route: "fallback").
4. At 100K+ requests/day or mixed sync/async workloads, adopt Architecture 3 (Enterprise): queue (Redis/SQS) → auto-scaling workers running worker_loop() → results store, with OTEL metrics feeding dashboards and alerts.
5. Whichever tier you land on, route every call through the same OpenRouter client wrapper per Enterprise Considerations — consistent logging, cost tracking, and no budget bypass.

Architecture 1: Simple (Single Service)

┌─────────────┐     ┌──────────────────────────┐     ┌──────────────┐
│  Your App   │────▶│  OpenRouter Client        │────▶│  OpenRouter  │
│             │     │  - Retry (SDK built-in)   │     │  /api/v1     │
│             │◀────│  - Cost tracking          │◀────│              │
│             │     │  - Structured logging     │     └──────────────┘
└─────────────┘     └──────────────────────────┘

import os, logging
from openai  OpenAI

log = logging.getLogger()

client = OpenAI(
    base_url=,
    api_key=os.environ[],
    max_retries=,
    timeout=,
    default_headers={: , : },
)

 ():
    kwargs.setdefault(, )
    response = client.chat.completions.create(
        model=model,
        messages=[{: , : prompt}],
        **kwargs,
    )
    log.info()
     response.choices[].message.content