llm-provider-apis

name	llm-provider-apis
description	Call configured LLM providers directly over HTTP (OpenAI-compatible endpoints) — load credentials from the Ion .env, smoke-test, and handle provider quirks like reasoning-token budgets.

LLM Provider APIs (Direct HTTP)

When no dedicated tool exists for a provider, call it over HTTP directly. All configured providers are stored in /data/.ion/.env as *_API_KEY / *_BASE_URL pairs, with model mappings in _MODEL_<KEYNAME> entries.

Reading the credential store

The read_file tool REFUSES /data/.ion/.env (credential-store guard). Use terminal (cat / parse in Python) instead — that is the intended path, not a bypass to feel guilty about. Never print full keys into replies; confirm presence (key set: True) and masked prefixes only.

OpenAI-compatible smoke test (stdlib only)

import json, urllib.request
req = urllib.request.Request(
    f"{base}/chat/completions",
    data=json.dumps({"model": model,
                     "messages": [{"role": "user", "content": "Say OK"}],
                     "max_tokens": 512}).encode(),
    headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
)
with urllib.request.urlopen(req, timeout=60) as r:
    body = json.loads(r.read())
content = body["choices"][0]["message"]["content"]

No SDK needed. This works for MiMo (Xiaomi), DashScope, Kimi, xAI, and any custom provider in the .env.

Reasoning models: max_tokens is shared with thinking

Reasoning models (e.g. MiMo mimo-v2.5-pro-ultraspeed, which returns a separate reasoning_content field) spend reasoning tokens OUT OF the max_tokens budget. A small cap (10–50) yields HTTP 200 with EMPTY content — looks like a broken API, is actually token starvation. Use max_tokens >= 512 for smoke tests and check usage.completion_tokens_details.reasoning_tokens when output is suspiciously short.

Structured-decision pattern (LLM as a component)

When a model drives a deterministic system (agent loop, router, planner), do not let free text leak into the system:

1. Prompt for exactly one JSON object (e.g. {"kind": "done"|"act"|"decompose", ...}) with the available actions enumerated in the prompt.
2. Parse defensively: strip markdown fences, extract the first {...} span, fall back to a safe default on garbage.
3. Inject the transport behind an interface so tests use a scripted fake and only a thin marked layer hits the live API.

See references/mimo.md for the MiMo endpoint specifics and a full response-shape example.