openrouter-gateway

name	openrouter-gateway
description	Route every LLM call in an application through one OpenRouter gateway — per-tenant API keys, per-function model configuration with automatic fallback, retry with backoff, a stable error taxonomy, and usage/cost logging. Use when adding AI to a project, when LLM calls are scattered across features with hardcoded models and keys, when migrating off a provider or another gateway, when a model gets deprecated, or when AI spend needs to become visible per customer and per feature.
license	MIT
compatibility	Built for Supabase Edge Functions (Deno) with Postgres and RLS, but the gateway is plain TypeScript and the design ports to any serverless runtime. Requires an OpenRouter account with credit. Optional - a multi-tenant schema; single-tenant projects keep one fixed tenant row.
metadata	{"version":"1.0.0","provider":"openrouter","body-language":"en","reference-language":"pt-BR","source":"extracted from a production SaaS with 27 AI functions across three providers"}

OpenRouter as the single AI gateway

One place where the application talks to a model. Everything else — which model, whose key, what it cost, what to do when it fails — is configuration, not code scattered across features.

Ships a production gateway (~530 lines), the schema, the migration recipe and a five-sprint plan with acceptance gates, extracted from a SaaS that had 27 AI functions across three different providers.

Reference docs are in Brazilian Portuguese (references/). The skill bodies and code comments are mixed pt-BR/English. Models read both.

---

0. Orient yourself first

# How many ways does this project call an LLM today?
grep -rl "openai.com\|anthropic.com\|googleapis.com/v1beta\|openrouter.ai\|gateway" \
  supabase/functions src 2>/dev/null | head -20

# Is a gateway already here?
ls supabase/functions/_shared/ai-gateway.ts 2>/dev/null

• No AI yet, or one or two direct calls → install the gateway now, before the third: use ai-gateway-setup.
• Several scattered calls → this is a migration. Use ai-migrate: it inventories first, then moves them one at a time behind gates.
• Gateway already here → you are maintaining. Read §2 (contract) and §5 (conventions); open a reference only for the detail.

The single most valuable thing this package does is stop the third way of calling an LLM from ever existing.

---

1. What the gateway buys you

Without it	With it
Model name hardcoded in 27 files	One map, changed in one place
Provider deprecates a model → 27 edits	One line
Key in a global env secret	Per-tenant key, with a platform fallback
429 handled (differently) in each function	Retry with backoff, then a fallback model
Each function invents its own error handling	One taxonomy with stable codes
"What did AI cost us?" → the invoice	Per call: function, model, tokens, cost, latency
Customer wants a better model	Config row, no deploy

2. OpenRouter contract — what matters

• Endpoint: POST https://openrouter.ai/api/v1/chat/completions — OpenAI-compatible. This is why migrating off OpenAI or an OpenAI-shaped gateway is mostly mechanical.
• Auth: Authorization: Bearer <key>.
• Attribution: HTTP-Referer and X-Title headers identify your app in OpenRouter's rankings. Optional, free, and you want it.
• Catalog: GET https://openrouter.ai/api/v1/models is public — no key required. Feed the model picker from it instead of hardcoding a list.
• Model ids are provider/model: openai/gpt-4.1, anthropic/claude-3.5-sonnet, google/gemini-2.5-flash. A dated OpenAI id like gpt-4.1-2025-04-14 does not exist here — drop the date and prefix the provider.

Body field	Purpose
messages	Same shape as OpenAI
max_tokens, temperature	Same
response_format: { type: "json_object" }	Structured output
modalities: ["image"] + image_config	Image generation
messages[].content[].image_url	Image input (vision)
messages[].content[].input_audio	Audio input
usage.cost	Real cost of the call, when the provider reports it

3. Using it — the whole API is two functions

import { callAI, callAIVision, AIGatewayError } from "../_shared/ai-gateway.ts";

const { content, model, usage } = await callAI({
  functionName: "generate-summary",   // = the function's FOLDER name
  orgId,                              // tenant; omit only on public routes
  systemPrompt,
  userPrompt,
  maxTokens: 1500,
  temperature: 0.7,
  responseFormat: { type: "json_object" },   // optional
});

Vision:

const { content } = await callAIVision({
  functionName: "analyze-screenshot",
  orgId, systemPrompt, userPrompt,
  images: [imageUrl],
});

Conversation history instead of system+user — pass messages and the gateway uses it verbatim:

await callAI({ functionName, orgId, systemPrompt: "", userPrompt: "", messages: history });

How a model gets chosen

ai_model_configs (tenant + function, is_active)   ← set through your admin UI
   ↓ none?
FUNCTION_DEFAULTS[functionName]                    ← in the gateway source
   ↓ none?
DEFAULT_MODEL                                      ← global, no fallback

Resolution is cached 60s per isolate per tenant:function. After changing a model in the UI, the next call may still use the old one for up to a minute — say so in the UI or you will get a bug report.

How a key gets chosen

ai_api_keys[tenant].openrouter_api_key   ← tenant brings its own, pays its own
   ↓ none?
OPENROUTER_API_KEY (platform secret)     ← your safety net
   ↓ none?
throw AIGatewayError("NO_API_KEY")

The platform key is not optional if you are migrating. Existing features used a global secret and no tenant ever configured anything; without the fallback, migration breaks AI for every customer at once.

4. Error taxonomy — handle the code, never the message

code	HTTP	Gateway behaviour	What the user should see
NO_API_KEY	—	fails immediately	"Configure the OpenRouter key in Settings → AI"
INVALID_KEY	401	fails immediately, no fallback	"Invalid key"
NO_CREDIT	402	fails immediately, no fallback	"Add credit to your OpenRouter account"
RATE_LIMITED	429	3 attempts w/ backoff → fallback model	"Try again in a moment"
UPSTREAM	5xx	3 attempts w/ backoff → fallback model	"The AI service is unstable"
TIMEOUT	—	retry → fallback (30s text / 45s image)	"Generation took too long"
BAD_REQUEST	400/404	fails immediately → fallback model	generic error

NO_CREDIT and INVALID_KEY skip the fallback on purpose: the problem is the account, not the model. Trying another model with the same key just fails slower.

Full handler code, frontend treatment and diagnostic SQL: assets/examples/erros-e-taxonomia.md.

5. Conventions that keep it working

1. No function fetches an LLM directly. callAI/callAIVision is the only exit. A fetch to openai.com, anthropic.com or any gateway inside your functions directory is a review blocker.
2. functionName = the function's folder name. It is the key for config, defaults and logging at once. Diverging breaks per-tenant configuration silently — the UI saves and nothing changes.
3. Pass orgId whenever a session exists. Without it the call uses global defaults, the platform key, and logs with a null tenant. Public routes are the conscious exception.
4. Every AI function has a FUNCTION_DEFAULTS entry, with a fallback from a different provider — a fallback on the same provider does not survive that provider's outage.
5. The key never reaches the browser. Written through the UI, read only by service role. Show the last 4 characters, never the value.
6. Never log prompts or responses. Log functionName, tenant, model, tokens, cost, latency and the error code. Prompt content is customer data.
7. maxTokens and temperature explicit on every call.
8. Migrate one function per commit. Rollback is reverting one commit.

6. Adapter points

The gateway is self-contained except for four things:

Artifact expects	Adapt to
org_id column + public.organizations(id)	Your tenant column and table (company_id, workspace_id, account_id). Single-tenant: keep the column with one fixed row — removing it spreads exceptions through the whole gateway
user_is_org_member(uuid, uuid)	Your SECURITY DEFINER membership helper. Never inline a subquery over an RLS table inside a policy — it fails silently
user_has_admin_role(uuid, uuid)	Who may write the API key
APP_URL / APP_TITLE	Your domain and product name, for OpenRouter attribution

Runtime: the gateway uses Deno.env.get and @supabase/supabase-js. On another runtime, swap those two and the rest is portable TypeScript.

7. Schema

Three tables, plus the log:

Table	Holds
ai_api_keys	One OpenRouter key per tenant (+ optional monthly spend cap)
ai_model_configs	Primary/fallback model per (tenant, function) — UNIQUE(tenant, function)
ai_favorite_models	Shortcuts for the model picker
ai_usage_log	One row per call: model, tokens, cost, latency, success, error code

The UNIQUE is not decoration: the gateway uses .maybeSingle(), which errors when more than one row comes back. Two active configs for the same pair take that tenant's AI down with a message that never mentions configuration.

Migrations: assets/migrations/ (00 core tables → 01 usage log → 02 verification for projects that already had the tables).

8. Implementation plan (five sprints, each with a gate)

Sprint	Delivers	Gate
0 Foundation	Inventory frozen; platform key validated; ai_usage_log	A real curl to OpenRouter returns 200; SELECT on the log returns empty, not a permission error
1 Gateway	Retry, error taxonomy, usage logging, your FUNCTION_DEFAULTS	With an invalid primary model, the fallback takes over and the call completes
2 Migrate gateway/provider #1	Pilot function, then the rest in batches	No occurrence of the old gateway's URL remains
3 Migrate the direct provider calls	Model names translated	No direct provider fetch remains
4 Administration	Config UI + cost dashboard + per-tenant cap	Tenant A cannot see tenant B's spend

Measuring comes before migrating: the usage log lands in sprint 0 so there is a baseline when features change providers.

Full plan: references/ORQUESTRADOR.md and references/sprint-0 … sprint-4. To execute it, use ai-gateway-setup (sprints 0–1) then ai-migrate (sprints 2–3).

9. Failure modes worth memorizing

Symptom	Almost always	Fix
UI saves a model, nothing changes	function_name ≠ folder name, or the 60s cache	Align the name; wait a minute
Everything returns 402	No credit on the OpenRouter account	Add credit; NO_CREDIT skips fallback by design
One tenant lost AI, others fine	Two active config rows → maybeSingle() errors	The UNIQUE constraint (assets/migrations/02)
A function silently got slower and pricier	It is living on the fallback model	Query is_fallback in the log; the primary is broken
Cost is NULL in the log	OpenRouter did not report usage.cost for that model	Estimate from the catalog price; render —, never 0
Works locally, NO_API_KEY in production	Secret not set in the deployed project	supabase secrets set OPENROUTER_API_KEY=…
JSON parsing breaks after a model change	Model wrapped the JSON in a code fence	Keep JSON.parse in a try/catch and strip fences

10. File map

SKILL.md                       this file
references/
  ORQUESTRADOR.md              master plan: scope, architecture, contract, gates
  sprint-0…4/                  one document per phase — tasks, artifacts, gate
assets/
  ai-gateway.ts                the gateway, ready to drop into _shared/
  migrations/                  00 core tables · 01 usage log · 02 verification
  examples/
    antes-depois.md            migration recipe — real diffs for 3 call patterns
    erros-e-taxonomia.md       error handling, backend and frontend, + diagnostic SQL
    inventario-funcoes.md      a real 27-function inventory as a worked example
    function-defaults-exemplo.ts  a real FUNCTION_DEFAULTS map, with the criteria

Companion skills:

• ai-gateway-setup — installs the gateway, schema and observability.
• ai-migrate — inventories existing LLM calls and moves them in, one per commit.

11. Security

• The OpenRouter key is a credential: per tenant, in the database, never in the browser, never in a log, never in a commit.
• A leaked key is someone else spending your credit. Rotate through the UI and set monthly_limit_usd as a blast radius.
• Prompts and completions are customer data. They do not belong in console.log, in the usage log, or in an error payload returned to the client.
• The usage log stores metadata only — that is a deliberate design choice, not an omission.