request-routing

name	request-routing
description	Configure and manage LLM request routing rules in ai-gateway. Use when you need to: direct specific models to specific providers, set up fallback chains, create routing rules by model pattern, or change provider priority. Triggers include "route gpt-4 to OpenAI", "set fallback provider", "routing rule", "model-to-provider mapping", or any configuration of which provider handles which model.

request-routing

Configure routing rules in ai-gateway to control which LLM provider handles each request.

Routing Rule Format

Each rule has:

• A model glob pattern (e.g., gpt-4*, claude-*, command-r*)
• A primary provider
• An optional ordered fallback chain
• An active/disabled status

Rules are evaluated in database insertion order. First match wins.

Admin API

List rules

curl http://localhost:4081/admin/routes \
  -H "Authorization: Bearer gw_your_admin_key"

Create a rule

curl -X POST http://localhost:4081/admin/routes \
  -H "Authorization: Bearer gw_your_admin_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "GPT-4 to OpenAI with Anthropic fallback",
    "model_pattern": "gpt-4*",
    "provider_id": "openai-provider-id",
    "fallback_ids": ["anthropic-provider-id"],
    "is_active": 1
  }'

Update a rule

curl -X PATCH http://localhost:4081/admin/routes/<rule-id> \
  -H "Authorization: Bearer gw_your_admin_key" \
  -H "Content-Type: application/json" \
  -d '{"fallback_ids": ["anthropic-id", "cohere-id"]}'

Delete a rule

curl -X DELETE http://localhost:4081/admin/routes/<rule-id> \
  -H "Authorization: Bearer gw_your_admin_key"

Pattern Matching

Patterns use glob syntax:

• * matches any sequence of characters
• ? matches a single character
• Literal strings match exactly

Examples:

• gpt-4* matches gpt-4o, gpt-4-turbo, gpt-4-vision-preview
• claude-* matches all Claude models
• gpt-3.5-turbo matches only that exact model
• * matches all models (useful as a catch-all last rule)

Fallback Behavior

The gateway tries providers in order:

1. Primary provider
2. Fallback 1 (if primary fails)
3. Fallback 2 (if fallback 1 fails)
4. ... until all exhausted, then returns 502

Failure triggers:

• HTTP 5xx from provider
• Connection timeout (default 30s)
• Connection refused
• Provider 429 (provider-side rate limit, not gateway rate limit)

Non-retriable (no fallback):

• HTTP 400 invalid request
• HTTP 401 auth error
• HTTP 403 forbidden

Common Patterns

Route by cost tier

[
  { "pattern": "gpt-4o",         "provider": "openai",    "comment": "premium" },
  { "pattern": "gpt-3.5-turbo",  "provider": "openai",    "comment": "budget"  },
  { "pattern": "claude-3-haiku", "provider": "anthropic", "comment": "cheap"   }
]

High availability for critical model

{
  "pattern": "gpt-4o",
  "primary": "openai",
  "fallbacks": ["anthropic-claude-3-5-sonnet", "cohere-command-r-plus"]
}

Shadow routing (A/B test)

Create two rules for the same pattern with different providers, then toggle active state to switch traffic.