calling-llms
Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.
소스 정보
- 저장소
- xberg-io/liter-llm
- 최근 소스 활동
- 2026년 10월 10일 07:47
- 감지된 SKILL.md 언어
- 영어
- 스타
- 262
- 포크
- 23
설치 방법
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
소스 파일 검토
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
SKILL.md 표시 중
SKILL.md
소스 지침 · 읽기 전용 미리보기- name
- calling-llms
- description
- Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.
<!--
AI-RULEZ :: GENERATED FILE — DO NOT EDIT
Content-Hash: blake3:116e9240923b4f529f06471698b0e15ce1a14688a5beb1cae28e0b62e3a2837f
Source-Hash: blake3:3cb955aedbbbd51d9b5f60a411d90b9386534c5f14bf20a056bef60f7dace49e
Schema-Version: v1
-->
# Calling LLMs
Build a `ChatCompletionRequest` and send it with `client.chat(request)`. Create
the client with `create_client(...)`. The model string is `provider/model`; the
prefix selects the backend.
```python
import asyncio, json, os
from liter_llm import create_client
from liter_llm._internal_bindings import ChatCompletionRequest
async def main() -> None:
client = create_client(api_key=os.environ["OPENAI_API_KEY"])
request = ChatCompletionRequest.from_json(json.dumps({
"model": "openai/gpt-4o",
"messages": [
{"role": "system", "content": "You are concise."},
{"role": "user", "content": "Name three Rust crates for HTTP."},
],
}))
response = await client.chat(request)
print(response.choices[0].message.content)
asyncio.run(main())
```
## Provider routing
The model string's prefix selects the provider; build a request per backend:
```python
ChatCompletionRequest.from_json('{"model":"anthropic/claude-sonnet-4-20250514","messages":[...]}')
ChatCompletionRequest.from_json('{"model":"google/gemini-2.0-flash","messages":[...]}')
ChatCompletionRequest.from_json('{"model":"groq/llama3-70b","messages":[...]}')
ChatCompletionRequest.from_json('{"model":"mistral/mistral-large-latest","messages":[...]}')
ChatCompletionRequest.from_json('{"model":"bedrock/anthropic.claude-v2","messages":[...]}')
```
Set `model_hint` at construction to drop the prefix on every call:
```python
client = create_client(api_key="sk-...", model_hint="openai")
# the request model can now omit the provider prefix:
request = ChatCompletionRequest.from_json('{"model":"gpt-4o","messages":[...]}')
await client.chat(request) # routes to OpenAI
```
## Notes
- Keys come from env vars (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, …); never
hardcode them.
- Without a prefix and without `model_hint`, routing fails.
- Python errors are typed exceptions exported from `liter_llm`:
`AuthenticationError`, `RateLimitedError`, `BadRequestError`,
`ContextWindowExceededError`, `ContentPolicyError`, `NotFoundError`,
`ServerError`, `ServiceUnavailableError`, `LiterLlmTimeoutError`,
`BudgetExceededError` — all subclasses of `LiterLlmError`.
GitHub에서 보기