custom-openai-provider

name	custom-openai-provider
description	Configure any OpenAI-compatible API as a Hermes model provider — NVIDIA NIM, Together AI, Fireworks, Groq, local vLLM/Ollama, or any custom endpoint. Covers the critical provider-clearing pitfall.
version	1.0.0
metadata	{"hermes":{"tags":["models","providers","nvidia-nim","custom-endpoint","openai-compatible"]}}

Custom OpenAI-Compatible Provider Setup

Configure Hermes to use any OpenAI-compatible API endpoint as its model provider.

When to Use

• NVIDIA NIM (https://integrate.api.nvidia.com/v1)
• Together AI, Fireworks AI, Groq
• Local vLLM, Ollama, LM Studio
• Any self-hosted OpenAI-compatible server

Key Steps

1. Add API Key

Use hermes config env-path to find the .env location. Add your key there (use terminal + sed for the protected file).

2. Configure Model Settings

hermes config set model.default "provider/model-name"
hermes config set model.base_url "https://api.example.com/v1"
hermes config set model.api_key "env:YOUR_ENV_VAR_NAME"

3. CRITICAL: Clear Provider

PITFALL: If model.provider is set to a previous provider (e.g., "xiaomi", "openrouter"), Hermes routes requests incorrectly. MUST clear it:

hermes config set model.provider ""

This is the #1 mistake when switching from a named provider to a custom endpoint.

4. Restart

Exit CLI and restart hermes. For gateway: hermes gateway restart.

Switching Between Named Providers (e.g., → mimo)

When the target provider is already defined in config.yaml under providers, do NOT set model.base_url or model.api_key — clear them so Hermes inherits from the provider block. Set model.provider to the provider name:

model:
  api_mode: chat_completions
  provider: mimo       # provider name from config.yaml providers section
  api_key: ''          # MUST be empty — inherits from providers.mimo.api_key
  base_url: ''         # MUST be empty — inherits from providers.mimo.base_url
  default: mimo-v2.5-pro

Then update fallback:

fallback_providers:
- mimo
- xiaomi_mimo
- openrouter

Pitfall: Some API keys need a provider-specific prefix (e.g., mimo uses tp-). Test with curl first:

curl -s -X POST "https://token-plan-sgp.xiaomimimo.com/v1/chat/completions" \
  -H "Authorization: Bearer tp-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"mimo-v2.5-pro","messages":[{"role":"user","content":"Hi"}],"max_tokens":10}'

If 401 "Invalid API Key" — try adding the provider's required prefix.

Mimo Provider Details:

Item	Value
Base URL	https://token-plan-sgp.xiaomimimo.com/v1
API Key Format	tp- prefix required (e.g., tp-sbcomkklr...)
Models	mimo-v2.5-pro, mimo-v2.5, mimo-v2-pro, mimo-v2-omni, mimo-v2-flash
Config in	providers.mimo in config.yaml

NVIDIA NIM

• Base URL: https://integrate.api.nvidia.com/v1
• Get key: https://build.nvidia.com
• Models: deepseek-ai/deepseek-v4-pro, meta/llama-3.1-405b-instruct, etc.

List Available Models

curl -s https://integrate.api.nvidia.com/v1/models \
  -H "Authorization: Bearer nvapi-YOUR_KEY" | jq -r '.data[].id'

Test Model Availability

curl -s https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer nvapi-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "MODEL_NAME", "messages": [{"role": "user", "content": "Hi"}], "max_tokens": 5}'

Diagnose 403 Errors

If /v1/models works (returns model list) but /v1/chat/completions returns 403:

1. Check account credits - https://build.nvidia.com → Billing/Credits
2. Free tier exhausted - New accounts have limited free credits
3. Key permissions - May need to regenerate key with inference permissions
4. Model not enabled - Some models require manual activation in console

Test Multiple Models

When unsure which models are available/working, batch test:

for model in model1 model2 model3; do
  echo -n "$model: "
  curl -s "https://integrate.api.nvidia.com/v1/chat/completions" \
    -H "Authorization: Bearer nvapi-YOUR_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"model\": \"$model\", \"messages\": [{\"role\": \"user\", \"content\": \"Hi\"}], \"max_tokens\": 10}" \
    | grep -q '"choices"' && echo "✅" || echo "❌"
done

Reasoning Models

Some models (e.g., minimaxai/minimax-m2.7) return reasoning_content field instead of content when max_tokens is too low. Use max_tokens: 100+ to get actual responses.

Troubleshooting

Symptom	Fix
Requests go to wrong provider	hermes config set model.provider ""
401 Unauthorized	Verify key with curl test
403 Forbidden (can list models)	Account has no credits or free tier exhausted
403 Forbidden (cannot list models)	Invalid key or key format wrong
Model not found	List models via /v1/models endpoint
Changes not生效	Restart CLI session