integrate-local-llm

name	integrate-local-llm
description	Maintain Voice Layer's local assistant boundary across Tauri TypeScript, Rust Rig, and a user-managed llama.cpp server. Use when changing LLM chat streaming, model or server configuration, OpenAI-compatible request fields, hidden reasoning handling, reasoning or thinking effort controls, future user settings, or local LLM tests and documentation.

Integrate the Local LLM

Keep inference external to Voice Layer while making the in-app assistant feel native. Rig is the Rust client and orchestration boundary; it does not load GGUF files or replace llama-server.

Preserve the architecture

Use these ownership boundaries:

• llama-server owns model loading, context, GPU offload, token generation, and its OpenAI-compatible HTTP API. It is a separate user-managed process.
• src-tauri/src/llm/mod.rs owns the local endpoint/model policy, message validation, Rig request construction, visible-text filtering, and the local text-protocol tool loop. src-tauri/src/llm/provider_execution.rs owns shared admission, command/terminal lifecycle, request validation, and cancellation.
• src/llm.ts owns typed Tauri IPC only.
• src/main.ts owns visible generation state, session persistence, rendering, focus, and reply-only TTS.

Do not move the LLM HTTP client into the WebView, start or stop the user's server from the app, add a silent cloud fallback, or pass arbitrary provider JSON from TypeScript into Rig.

Start from current evidence

Before editing:

1. Read AGENTS.md, src-tauri/src/llm/mod.rs, src/llm.ts, the submission path in src/main.ts, src-tauri/Cargo.toml, and the local-LLM section of README.md.
2. Record git status --short --branch and preserve unrelated changes.
3. Inspect the compiled rig-core version and the installed llama-server version. Do not assume APIs from latest documentation match the checked-out dependency or local binary.
4. If reasoning controls or a user setting are in scope, read references/reasoning-controls.md completely and refresh its primary-source claims when versions have changed.

Keep the endpoint, alias, timeout, and request defaults in Rust. Read their current values from source rather than copying historical values into new code.

Build requests through Rig

Continue to use Rig's OpenAI-compatible completion client with the localhost base URL. Map prior user and assistant turns to Rig Message values, require the newest non-empty turn to be from the user, and stream the final assistant text through a typed Tauri Channel.

Provider-specific fields belong in Rig's request builder:

let mut request_builder = model
    .completion_request(prompt)
    .messages(history);

if let Some(additional_params) = validated_additional_params {
    request_builder = request_builder.additional_params(additional_params);
}

let request = request_builder.build();

In Rig 0.40.0, additional_params is flattened into the OpenAI chat-completions request body. Recheck that behavior on every Rig upgrade. Construct the JSON from a small validated Rust enum; never accept an unbounded JSON object over Tauri IPC.

Keep reasoning private

Only StreamedAssistantContent::Text may cross into the WebView. Continue to discard reasoning, reasoning deltas, tool calls, unknown content, and final metadata unless a separate product decision explicitly changes that boundary.

Do not persist hidden reasoning or send it back as chat history. GLM preserved-thinking workflows require exact reasoning replay, which conflicts with the current privacy and storage contract; adding that mode is a separate architectural change.

Preserve the app-defined thinking budget

Voice Layer deliberately exposes four product levels backed by explicit token budgets rather than OpenAI-style semantic effort:

App value	thinking_budget_tokens
none	0
low	512
medium	2048
high	-1 (unrestricted)

Keep High as the default so the ordinary unrestricted behavior is preserved. Rust must construct the top-level field through Rig additional_params from the closed enum. Do not substitute reasoning_effort, send raw JSON from TypeScript, or imply that the labels are model-trained semantic effort tiers. Apply a snapshotted value only to requests started after the change; never mutate an in-flight generation.

Make a future setting explicit

When implementing configuration:

• define the same closed preference enum in TypeScript and Rust;
• validate it in the Tauri command before constructing provider parameters;
• persist it under a versioned app-settings key, separate from chat-message storage;
• disable or snapshot the control during an active request;
• show the exact app-defined budget meaning and avoid unsupported quality claims;
• reject unknown choices instead of silently falling back;
• preserve reasoning filtering and reply-only TTS for every choice.

Keep this app-level configuration outside the composer itself so the composer continues to contain only the input and microphone.

Validate in layers

Run the narrowest useful checks first:

cargo test --manifest-path src-tauri/Cargo.toml llm::
just check

When Rust, dependencies, or native behavior changed, also run:

just bundle

Keep normal tests offline. Unit-test message mapping, exact additional-parameter JSON, default omission, invalid preference rejection, reasoning filtering, terminal events, and timeout/error behavior.

Use the ignored live server test only when the user-managed server is already available. Check /health and /v1/models, confirm the advertised alias, and test each supported reasoning choice separately. A successful HTTP response does not prove that the model honored an effort setting.

For packaged UI acceptance, invoke $test-voice-layer-app. Verify streaming, history, focus, persistence, hidden-reasoning filtering, user-input silence, completed-reply TTS, and any new setting at 1200x900.

Completion bar

Finish only when:

• the external-server/Rig/Tauri boundary remains intact;
• localhost and alias failures are actionable;
• no hidden reasoning reaches UI or storage;
• all four values serialize to their exact app-defined top-level token budgets;
• offline tests and just check pass;
• native changes have a fresh bundle;
• any claimed model behavior has live evidence from the exact local server and model.