agent-engines

# Agent Engines ## Overview The framework supports pluggable AI engines beneath the agent loop. The **Anthropic engine** is the default and best-in-class path (Claude models). Additional engines can be added via the Vercel AI SDK (OpenAI, Google Gemini, Groq, Mistral, Cohere, Ollama). ## Available Tools | Tool | Purpose | |---|---| | `list-agent-engines` | List all registered engines, their capabilities, the current selection, and whether you can change the organization default (`canUpdateDefault`) | | `set-agent-engine` | Set the organization's default engine and model (owners and admins only) | | `test-agent-engine` | Send a trivial prompt to verify the engine works (connectivity + API key) | | `check-provider-key` | Check a provider API key by listing the models it reaches, before saving it | | `get-provider-models` | Read which models each provider shows in the model picker and default-model select | | `manage-provider-models` | Choose those models (`set`) or go back to the recommended ones (`reset`) | | `list-model-providers` | What Settings › Model shows: each provider's organization and personal key (masked), rejections, the restriction, and the stored default | ## Checking the Current Engine ``` list-agent-engines ``` Returns the registry of all engines (name, label, capabilities, supported models) plus the currently active engine and model. `supportedModels` is what the model picker offers: the provider's checked models when `modelSelection.state` is `"selected"`, otherwise its `recommendedModels`. `credentialRejected: true` means the provider rejected that engine's saved key (`credentialRejectedAt` says when); chats with it stop until the key is replaced, so tell the user. ## Switching Engines ``` set-agent-engine --engine "ai-sdk:openai" --model "gpt-4o" ``` Changes take effect on the next conversation. The default belongs to the organization (the `agent-engine` org setting), and only owners and admins can change it. A member's call is refused with an error to relay: ask an owner or admin, or pick a model for this chat in the model picker. A user with no organization sets their own default. Resolution order (highest priority first): 1. Explicit `engine` option passed to `createAgentChatPlugin()` in the server plugin 2. The organization's default (`agent-engine` org setting) 3. `AGENT_ENGINE` environment variable 4. Default: `"anthropic"` (requires `ANTHROPIC_API_KEY`) ## Settings › Model The Model page (`/settings/model`, page id `model`) has three groups: - **Organization providers:** the organization's Builder.io connection and each organization key. Owners and admins manage them; members see only "N models". - **Personal providers:** a member's own Builder.io connection, their own keys, and the ChatGPT subscription while its lab is on. A personal key is used before the organization's, for that person only. - **Organization settings:** Default model (`manage-agent-engine` `set`, organization models only), Restrict personal API keys (`manage-provider-key-policy`), and Max iterations (`manage-agent-loop-settings`, 1 to 1000). Owners and admins change them; members see the values. Read the page's state with `list-model-providers`, and the models each provider shows with `get-provider-models`. Keys are added and replaced in the provider dialog, which checks a pasted key by listing its models; you can't save a key for the user, so check one with `check-provider-key` and send them to Model (`open-settings-page`) to paste it. Before they remove a provider, run `preview-secret-removal` with the provider's key name and scope and tell them what stops. ## Restricting Personal API Keys `manage-provider-key-policy` reads or changes the organization's "Restrict personal API keys" setting. While it is on, members (not owners or admins) can't use or save their own provider keys or a personal Builder.io connection, so their chats use organization providers; with none, a chat fails with "Owners and admins restricted personal API keys." Nothing is deleted, and turning it off restores their keys. ``` manage-provider-key-policy # read; owners/admins also get affectedMembers manage-provider-key-policy --set true # owners and admins only ``` Read it first and tell the user which members lose which providers before turning it on. ## Testing a New Engine Before switching, verify the engine is working: ``` test-agent-engine --engine "ai-sdk:openai" --model "gpt-4o" ``` Returns `{ ok, latencyMs, response, capabilities }`. If `ok: false`, the error message explains what's wrong (missing API key, package not installed, etc.). To check a key without sending a prompt, for example one the user just gave you: ``` check-provider-key --provider "anthropic" --key "<the key>" ``` Returns `{ ok: true, models }` or `{ ok: false, code, reason }` (`rejected`, `wrong-provider`, `missing-key`, `invalid-endpoint`, `unreachable`, `provider-error`). Relay the reason. Omit `--key` to re-check the saved key; it is only checked against its saved endpoint, so `--baseUrl` needs `--key` (except for Ollama). `--scope org` is for owners and admins. ## Choosing Which Models Show Each provider (Builder.io included) keeps the models its owner checked. Only checked models appear in the chat model picker and the default-model select; Builder.io and providers with their own key show side by side. ``` get-provider-models --provider "openai" manage-provider-models --action "set" --provider "openai" --models '["gpt-6-sol"]' manage-provider-models --action "reset" --provider "openai" ``` - A selection lives at the same scope as the provider's key: `--scope user` is the caller's own, `--scope org` is the organization's (owners and admins only; members get an error to relay). Without `--scope`, the call changes the selection in effect, which follows the key in effect (a member's personal key uses their personal selection). - A member's personal selection never changes the organization's. The default-model select always offers the organization's checked models. - `set` with an empty list hides the provider's chat models, for a key used only by services. Builder.io accepts only its own catalog. - A chat already on an unchecked model keeps running on it. When nothing picked a model and the engine default is unchecked, chats use the first checked model. - A row with `models: null` has nothing chosen and shows the recommended models. `state: "unreadable"` means the selection couldn't be read, not that it is empty. ## Built-in Engines | Engine Name | Provider | Requires | |---|---|---| | `anthropic` | Anthropic Claude SDK | `ANTHROPIC_API_KEY` | | `ai-sdk:anthropic` | Claude via Vercel AI SDK | `ANTHROPIC_API_KEY` | | `ai-sdk:openai` | OpenAI via Vercel AI SDK | `OPENAI_API_KEY` | | `ai-sdk:openrouter` | 300+ models (Anthropic, OpenAI, Google, Meta, …) routed through OpenRouter | `OPENROUTER_API_KEY` | | `ai-sdk:google` | Google Gemini via Vercel AI SDK | `GOOGLE_GENERATIVE_AI_API_KEY` | | `ai-sdk:groq` | Groq LPU via Vercel AI SDK | `GROQ_API_KEY` | | `ai-sdk:mistral` | Mistral via Vercel AI SDK | `MISTRAL_API_KEY` | | `ai-sdk:cohere` | Cohere via Vercel AI SDK | `COHERE_API_KEY` | | `ai-sdk:ollama` | Local Ollama via Vercel AI SDK | None (local) | ## Engine Capabilities Each engine advertises its capabilities: | Capability | Anthropic | AI SDK: Anthropic | AI SDK: OpenAI | AI SDK: Google | |---|---|---|---|---| | `thinking` | ✓ | ✓ | ✗ | ✓ | | `promptCaching` | ✓ | ✓ | ✗ | ✗ | | `vision` | ✓ | ✓ | ✓ | ✓ | | `computerUse` | ✓ | ✗ | ✗ | ✗ | | `parallelToolCalls` | ✓ | ✓ | ✓ | ✓ | ## Anthropic-Exclusive Features When using the `anthropic` engine (or `ai-sdk:anthropic`): - **Prompt caching** is applied automatically to the system prompt — cutting latency and cost on repeated turns. - **Extended thinking** can be enabled via `providerOptions.anthropic.thinking` — the agent reasons longer before responding. These features are silently ignored when a non-Anthropic engine is active (capability-gated, no breakage). ## Using OpenRouter `ai-sdk:openrouter` gives access to 300+ models from many providers through a single API. Model IDs use the `vendor/model` form: ``` set-agent-engine --engine "ai-sdk:openrouter" --model "anthropic/claude-sonnet-4.5" set-agent-engine --engine "ai-sdk:openrouter" --model "openai/gpt-4o" set-agent-engine --engine "ai-sdk:openrouter" --model "google/gemini-2.5-pro" ``` Any `vendor/model` string from [openrouter.ai/models](https://openrouter.ai/models) works — the `supportedModels` list in the registry is a UI hint, not an allow-list. **App attribution** (optional): pass `appName` / `appUrl` in the engine config to set the `X-OpenRouter-Title` / `HTTP-Referer` headers — useful to see your app on the OpenRouter dashboard and leaderboards: ```ts createAISDKEngine("openrouter", { apiKey: process.env.OPENROUTER_API_KEY, appName: "My App", appUrl: "https://myapp.example", }); ``` ## Registering a Custom Engine Register custom engines in a server plugin at startup. Import from the `@agent-native/core/agent/engine` subpath: ```ts // server/plugins/my-engine.ts import { registerAgentEngine, type AgentEngine, type EngineEvent, type EngineStreamOptions, } from "@agent-native/core/agent/engine"; registerAgentEngine({ name: "my-engine", label: "My Custom Engine", description: "...", capabilities: { thinking: false, promptCaching: false, vision: false, computerUse: false, parallelToolCalls: true, }, defaultModel: "my-model-v1", supportedModels: ["my-model-v1", "my-model-v2"], requiredEnvVars: ["MY_ENGINE_API_KEY"], create: (config): AgentEngine => ({ name: "my-engine", label: "My Custom Engine", defaultModel: "my-model-v1", supportedModels: ["my-model-v1", "my-model-v2"], capabilities: { /* same shape as above */ } as any, async *stream(opts: EngineStreamOptions): AsyncIterable<EngineEvent> { // yield text-delta / thinking-delta / tool-call / usage events // as they arrive, then: yield { type: "assistant-content", parts: /* final content parts */ [] }; yield { type: "stop", reason: "end_turn" }; }, }), }); ``` ### Engine stream contract Every engine's `stream(opts)` MUST emit, in order: 1. Zero or more `text-delta`, `thinking-delta`, `tool-call`, and `usage` events as they arrive from the model. 2. Exactly one `{ type: "assistant-content", parts }` event with the structured content for the turn. `runAgentLoop` reads this to reconstruct the assistant message for the next turn. 3. Exactly one terminal `{ type: "stop", reason }` event. After registering, the engine appears in `list-agent-engines` output and can be selected via `set-agent-engine`. ## Env Vars Reference | Variable | Purpose | |---|---| | `ANTHROPIC_API_KEY` | Required for `anthropic` and `ai-sdk:anthropic` engines | | `OPENAI_API_KEY` | Required for `ai-sdk:openai` | | `OPENROUTER_API_KEY` | Required for `ai-sdk:openrouter` | | `GOOGLE_GENERATIVE_AI_API_KEY` | Required for `ai-sdk:google` | | `GROQ_API_KEY` | Required for `ai-sdk:groq` | | `MISTRAL_API_KEY` | Required for `ai-sdk:mistral` | | `COHERE_API_KEY` | Required for `ai-sdk:cohere` | | `AGENT_ENGINE` | Default engine name (overridden by settings store) |