running-the-proxy

name	running-the-proxy
description	Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.

Running the Proxy

liter-llm api is a drop-in OpenAI-compatible gateway: 22 REST endpoints that route to 165 providers, with multi-tenant virtual keys, rate limits, budgets, and cost tracking.

Start it

liter-llm api --config liter-llm-proxy.toml

The proxy auto-discovers liter-llm-proxy.toml in the current directory.

Configuration

[server]
host = "0.0.0.0"
port = 4000

[general]
master_key = "${LITER_LLM_MASTER_KEY}"

# Each [[models]] entry maps a routable name to a provider/model and its key.
[[models]]
name = "gpt-4o"
provider_model = "openai/gpt-4o"
api_key = "${OPENAI_API_KEY}"

[[models]]
name = "claude-sonnet"
provider_model = "anthropic/claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"

# Virtual keys scope which configured model names a caller may use.

 = 
 = [, ]
 = 
 = 
 =