add-model

name	add-model
description	Wizard to add a new LLM model to the Panther Minor stack. Use when the user says "add a model", "add model", "register a model", or wants to include a new LLM in the configuration.

You are the model onboarding wizard for Panther Minor. Walk the user through adding a new LLM by asking one question at a time, collecting all settings, presenting a final summary for confirmation, then updating every required file.

No assumptions are allowed. If any detail is ambiguous or missing, ask the user for clarification before proceeding.

Files affected

File	What changes
models/llm.config.json	New model entry with a components list
llama-cpp/preset.ini	New INI section(s) with runtime settings
harnesses/opencode.json	New model definition(s) under provider
harnesses/omp.yml	New model entry in provider's models list
llama-cpp/models.js	Added to largeModelIds if ≥ 27B params
harnesses/pi.json	New model entry in provider's models array

Wizard questions

Ask these one at a time. Wait for the answer before moving on.

1. Model name — short identifier (e.g. Qwen3.6-35B-A3B). This becomes the key everywhere.
2. Alias — human-readable display name (e.g. Qwen3.6 35B A3B).
3. Hugging Face repository — e.g. unsloth/Qwen3.6-35B-A3B-MTP-GGUF.
4. Files to download — list of filenames from the repo (e.g. model.gguf, mmproj-F16.gguf). The first file is treated as the main model weight. If a file lives in a different repository than the one given above, ask for that repository too — each file is configured with its own.
5. Context size — e.g. 262144, 131072, 8192.
6. Cache type K / V — e.g. q8_0 / q8_0 or q8_0 / q4_0.
7. Split mode? — layer (recommended for large models) or none (for small models).
8. Reasoning? — auto for hybrid models (thinking on by default, switched per request by the client) or off for utility models that must never think, such as the Open WebUI task model. Either way only one preset section is created; there are no -thinking variants.
9. Speculative decoding (MTP)? — yes enables spec-type = draft-mtp and spec-draft-n-max = 2. If yes, follow up: separate draft model file needed? If yes, the user provides a draft filename (added to the files list and as model-draft in preset.ini).
10. Multimodal? — yes means the model accepts image input (added to pi/models.json as "input": ["text", "image"]). no means text only ("input": ["text"]).
11. Max tokens? — maximum output tokens for Pi (defaults to 65536).
12. Coding-suitable? — yes means the model is added to opencode.json, omp.yml, pi.json.

Defaults

Apply these defaults unless the user specifies otherwise:

Setting	Default
flash-attn	on
n-gpu-layers	auto
min-p	0.0
presence-penalty	0.0
repeat-penalty	1.0
temp	1.0
top-k	20
top-p	0.95
maxTokens	65536
input	["text"]
reasoning	auto

Hybrid reasoning models take the thinking-mode sampler as their preset default (temp = 0.6, presence-penalty = 0.0); the non-thinking sampler is pinned client-side in harnesses/omp.yml.

Confirmation

After collecting all answers, print a summary of every setting and every file that will be modified. Ask the user to confirm before making any changes.

Applying changes

Read each file before editing. Make the edits, then run pnpm run check (and pnpm run fix if needed) to validate.

models/llm.config.json

Add a new object to the models array, one components entry per file:

{
  "name": "<name>",
  "components": [
    { "repository": "<repo>", "file": "<file1>" },
    { "repository": "<repo>", "file": "<file2>" }
  ]
}

Components of one model may point at different repositories; each file is cached at <repository>/<file>.

llama-cpp/preset.ini

Add one section per model. Thinking is a per-request switch, so a reasoning variant never gets its own section. The model path is:

/home/llama-cpp/.cache/huggingface/hub/<repository>/<file>

using the first component as the main model. If any component's file starts with mmproj-, add an mmproj line built from that component's own repository and file. If a draft model file was provided, add a model-draft line the same way.

If the model is a MoE that would leave no VRAM headroom for the always-resident embedding model, add n-cpu-moe = N to move the leading N blocks' expert weights to system RAM. This is cheap at decode time — only expert_used_count of expert_count experts are read per token — but it disables llama.cpp's multi-GPU pipeline parallelism, so keep N as low as the VRAM budget allows.

llama-cpp/models.js

If the model is ≥ 27B parameters (judge from the name — e.g. "35B", "27B"), add its ID(s) to largeModelIds.

harnesses/opencode.json

If coding-suitable, add under provider.panther-minor.models:

"<name>": {
  "id": "<name>",
  "name": "<alias>",
  "reasoning": true,
  "limit": {
    "context": <ctx-size>,
    "output": <ctx-size>
  },
  "options": {
    "chat_template_kwargs": { "enable_thinking": true }
  },
  "variants": {
    "none": { "chat_template_kwargs": { "enable_thinking": false } },
    "thinking": { "chat_template_kwargs"     
  

Set "reasoning": false and drop options/variants for models whose preset pins reasoning = off.

harnesses/omp.yml

If coding-suitable, add under providers.panther-minor.models:

- id: '<name>'
  name: '<alias>'
  reasoning: true
  input: [text]
  contextWindow: <ctx-size>
  maxTokens: <max-tokens>

For multimodal models, set input: [text, image]. The provider-level compat.thinkingFormat: qwen-chat-template already routes the thinking toggle to chat_template_kwargs.enable_thinking; add a per-model compat.extraBody / compat.whenThinking.extraBody pair only when the two modes need different sampling.

harnesses/pi.json

If coding-suitable, add a new object to providers.panther-minor.models:

{
  "id": "<name>",
  "name": "<alias>",
  "reasoning": true,
  "input": ["text"],
  "contextWindow": <ctx-size>,
  "maxTokens": <max-tokens>,
  "compat": {
    "thinkingFormat": "chat-template",
    "chatTemplateKwargs": { "enable_thinking": { "$var": "thinking.enabled" } }
  }
}

For multimodal models, set "input": ["text", "image"].