| name | add-model |
| description | Add a new model to a live LiteLLM proxy. Walks the user through picking a provider, entering the deployment name and credentials, calls POST /model/new, then test-calls the model to confirm it routes correctly. Use when the user wants to add, register, deploy, or configure a new model on a LiteLLM proxy instance. |
| license | MIT |
| compatibility | Requires curl. |
| metadata | {"author":"BerriAI","version":"1.0"} |
| allowed-tools | Bash(curl:*) |
Add Model
Add a new LLM to a live LiteLLM proxy.
Setup
Ask for these if not already known:
LITELLM_BASE_URL — e.g. https://my-proxy.example.com
LITELLM_API_KEY — proxy admin key
API reference: https://litellm.vercel.app/docs/proxy/model_management
Ask the user
- Public model name — what callers will send in
"model": "..." (e.g. gpt-4o, my-claude, llama3)
- Provider — pick from the table below
- Credentials — whatever that provider needs
Provider table
| Provider | litellm_params.model | Extra params |
|---|
| OpenAI | openai/gpt-4o | api_key |
| Azure OpenAI | azure/<deployment-name> | api_key, api_base, api_version |
| Anthropic | anthropic/claude-3-5-sonnet-20241022 | api_key |
| AWS Bedrock | bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0 | AWS creds via env |
| Google Vertex | vertex_ai/gemini-1.5-pro | vertex_project, vertex_location |
| Ollama | ollama/llama3 | api_base (e.g. http://localhost:11434) |
| Groq | groq/llama-3.3-70b-versatile | api_key |
| Together AI | together_ai/meta-llama/Llama-3-70b | api_key |
| Mistral | mistral/mistral-large-latest | api_key |
Full list: https://docs.litellm.ai/docs/providers
Run
curl -s -X POST "$BASE/model/new" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-d '{
"model_name": "<public-name>",
"litellm_params": {
"model": "<provider/deployment>",
"api_key": "<key>",
"api_base": "<base_if_needed>",
"api_version": "<version_if_azure>"
}
}'
Test it
After adding, verify it routes:
curl -s -X POST "$BASE/chat/completions" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "<public-name>",
"messages": [{"role": "user", "content": "say hi"}],
"max_tokens": 10
}'
Output
Show model_id from the response — needed to update or delete the model later.
Report pass/fail from the test call.
