| name | llm-provider-apis |
| description | Call configured LLM providers directly over HTTP (OpenAI-compatible endpoints) — load credentials from the Ion .env, smoke-test, and handle provider quirks like reasoning-token budgets. |
LLM Provider APIs (Direct HTTP)
When no dedicated tool exists for a provider, call it over HTTP directly. All
configured providers are stored in /data/.ion/.env as *_API_KEY /
*_BASE_URL pairs, with model mappings in _MODEL_<KEYNAME> entries.
Reading the credential store
The read_file tool REFUSES /data/.ion/.env (credential-store guard). Use
terminal (cat / parse in Python) instead — that is the intended path, not a
bypass to feel guilty about. Never print full keys into replies; confirm
presence (key set: True) and masked prefixes only.
OpenAI-compatible smoke test (stdlib only)
import json, urllib.request
req = urllib.request.Request(
f"{base}/chat/completions",
data=json.dumps({"model": model,
"messages": [{"role": "user", "content": "Say OK"}],
"max_tokens": 512}).encode(),
headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
)
with urllib.request.urlopen(req, timeout=60) as r:
body = json.loads(r.read())
content = body["choices"][0]["message"]["content"]
No SDK needed. This works for MiMo (Xiaomi), DashScope, Kimi, xAI, and any custom provider in the .env.
Reasoning models: max_tokens is shared with thinking
Reasoning models (e.g. MiMo mimo-v2.5-pro-ultraspeed, which returns a separate reasoning_content field) spend reasoning tokens OUT OF the max_tokens budget. A small cap (10–50) yields HTTP 200 with EMPTY content — looks like a broken API, is actually token starvation. Use max_tokens >= 512 for smoke tests and check usage.completion_tokens_details.reasoning_tokens when output is suspiciously short.
Structured-decision pattern (LLM as a component)
When a model drives a deterministic system (agent loop, router, planner), do not let free text leak into the system:
- Prompt for exactly one JSON object (e.g.
{"kind": "done"|"act"|"decompose", ...}) with the available actions enumerated in the prompt.
- Parse defensively: strip markdown fences, extract the first
{...} span, fall back to a safe default on garbage.
- Inject the transport behind an interface so tests use a scripted fake and only a thin marked layer hits the live API.
See references/mimo.md for the MiMo endpoint specifics and a full response-shape example.
