running-the-proxy
Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.
ソース情報
- リポジトリ
- xberg-io/liter-llm
- ソースの最終更新活動
- 2026年10月10日 07:47
- 検出された SKILL.md の言語
- 英語
- スター
- 262
- フォーク
- 23
インストール方法
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
ソースファイルを確認
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
SKILL.md を表示中
SKILL.md
ソースの指示 · 読み取り専用プレビュー- name
- running-the-proxy
- description
- Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.
<!--
AI-RULEZ :: GENERATED FILE — DO NOT EDIT
Content-Hash: blake3:f64a695a6f03738acb7f80e2d91d2446346006e3a2c4858b5fe87f88f14c86f5
Source-Hash: blake3:3cb955aedbbbd51d9b5f60a411d90b9386534c5f14bf20a056bef60f7dace49e
Schema-Version: v1
-->
# Running the Proxy
`liter-llm api` is a drop-in OpenAI-compatible gateway: 22 REST endpoints that
route to 174 providers, with multi-tenant virtual keys, rate limits, budgets,
and cost tracking.
## Start it
```bash
liter-llm api --config liter-llm-proxy.toml
```
The proxy auto-discovers `liter-llm-proxy.toml` in the current directory.
## Configuration
```toml
[server]
host = "0.0.0.0"
port = 4000
[general]
master_key = "${LITER_LLM_MASTER_KEY}"
# Each [[models]] entry maps a routable name to a provider/model and its key.
[[models]]
name = "gpt-4o"
provider_model = "openai/gpt-4o"
api_key = "${OPENAI_API_KEY}"
[[models]]
name = "claude-sonnet"
provider_model = "anthropic/claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"
# Virtual keys scope which configured model names a caller may use.
[[keys]]
key = "sk-team-frontend"
models = ["gpt-4o", "claude-sonnet"]
rpm = 60
tpm = 100000
budget_limit = 50.0
```
`${ENV_VAR}` interpolation keeps secrets out of the file.
## Call it like OpenAI
```bash
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-team-frontend" \
-d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
```
## Notes
- Virtual keys scope which models a caller may use and carry their own RPM/TPM
and budget limits; the `master_key` administers them.
- The OpenAPI 3.1 spec is served at `/openapi.json`.
- Endpoints cover chat, embeddings, images, audio, moderations, files, batches,
responses, and model listing.
- Docker: `docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-key ghcr.io/xberg-io/liter-llm`.
GitHubで見る