right-sizing-model-selection

name	right-sizing-model-selection
description	Use when delegating work to other models or subagents, batch-processing many items (including when tempted to process a batch in your own context), choosing between Claude tiers (haiku/sonnet/opus/fable) or local LM Studio models, or when token cost, model overkill, or model underkill is a concern. Invoke BEFORE deciding the execution approach, not as a later confirmation step.

Right-Sizing Model Selection

Overview

Route every task to the cheapest model that meets the quality bar — decided by recorded evidence, not vibes. The strong model analyzes and orchestrates; cheaper models execute. You already know haiku handles simple tasks; what you systematically get wrong is everything that needs memory: which local model actually passed translation last month, what prompt style made it pass, and how much verification that evidence still requires.

Core rule: never assert a model is too weak (or good enough) for a task type without a KB record or a probe result. "Uncalibrated" means probe it, not exclude it.

Measured counter-examples to routing by intuition (2026-06): hand-written difficulty priors were wrong on 10 of 16 benchmark tasks — 9 paid for capability the task didn't need, 1 sent a task to models that fail it (proofreading: small local models overcorrect; haiku 9.0 vs gpt-oss 3.0). In the first probe, local openai/gpt-oss-20b beat Haiku on translation (9.7 vs 9.3, ~5× faster, $0); in calibration, haiku 9.0 beat opus 6.0 on a grounded-analysis task where opus invented a fact (n=1, different judges — directional, not settled).

When to use

• Delegating to subagents (model: option), claude -p, LM Studio, or OpenRouter (or: ids)
• Batches: N similar items where per-item cost × N matters
• Any "which model should do this?" moment, including inside plans

When NOT to delegate at all: one-off tasks on content you have already read (re-reading costs more than doing), tasks needing your full conversation state, or N so small that dispatch overhead exceeds the work (see Economics).

Doing it yourself is also a model selection. Reading a 400-item file into your own context to process it routes the batch to the most expensive model in the building — that choice needs the same KB/probe evidence as any other. The "already in context" exemption covers content you've read for other reasons; it is not a license to ingest batches.

The procedure

digraph route {
    "Task to delegate" [shape=doublecircle];
    "kb query --task-type X" [shape=box];
    "Confident rec? (n>=2, passing)" [shape=diamond];
    "Route to recommended model" [shape=box];
    "probe 3-5 items on cheap candidates" [shape=box];
    "Any candidate passes?" [shape=diamond];
    "Route batch to cheapest passing" [shape=box];
    "Escalate tier or fix prompt, re-probe" [shape=box];
    "Validate mechanically + spot-check" [shape=box];
    "kb record + distill" [shape=doublecircle];

    "Task to delegate" -> "kb query --task-type X";
    "kb query --task-type X" -> "Confident rec? (n>=2, passing)";
    "Confident rec? (n>=2, passing)" -> "Route to recommended model" [label="yes"];
    "Confident rec? (n>=2, passing)" -> "probe 3-5 items on cheap candidates" [label="no"];
    "probe 3-5 items on cheap candidates" -> "Any candidate passes?";
    "Any candidate passes?" -> "Route batch to cheapest passing" [label="yes"];
    "Any candidate passes?" -> "Escalate tier or fix prompt, re-probe" [label="no"];
    "Escalate tier or fix prompt, re-probe" -> "probe 3-5 items on cheap candidates";
    "Route to recommended model" -> "Validate mechanically + spot-check";
    "Route batch to cheapest passing" -> "Validate mechanically + spot-check";
    "Validate mechanically + spot-check" -> "kb record + distill";
}

Difficulty calibration for the first guess (probe confirms): trivial/mechanical (classify, extract with schema) → local or haiku; templated transforms (translate, summarize, simple codegen) → local/haiku, sonnet if nuanced; synthesis, creative quality, multi-step reasoning → sonnet/opus; subtle correctness (concurrency review, security, architectural judgment) → default opus/fable, downgradable only on probe/KB evidence for the specific shape (short contained snippets measured fine on cheap tiers; 800-line modules did not) — cost pressure alone never downgrades these: a confident false negative looks identical to a clean bill of health. Beware look-trivial restraint tasks: proofreading inverted the ladder entirely (small models overcorrect; haiku 9.0 vs gpt-oss 3.0).

Evidence hierarchy (use the strongest available; never route a batch on the bottom rungs alone):

local KB record (n>=2) > fresh probe > family-transfer from KB > benchmark prior > provenance/load-state

When the KB has nothing for a task type, order probe candidates with kb prior --task-type X — public benchmark rankings (SWE-bench, LiveCodeBench, MMLU-Pro, LMArena, …) mapped to our inventory, with per-benchmark validity notes in references/benchmarks.md. Family transfer counts too: a sibling model's KB record (same family/size class) is a stronger prior than any leaderboard. Divergence detector: a model scoring far below its public benchmark profile locally means a broken integration (chat template, max_tokens eaten by reasoning, params) — fix the path before writing off the model; GLM-4.7-Flash benches SWE 59 publicly and scored 0.0 here because of a template leak.

Scripts (in scripts/)

Script	Job	Typical call
llm	Dispatch to any model, one interface	echo "$P" | scripts/llm -m haiku --json; -m or:z-ai/glm-4.7-flash for OpenRouter (explicit or: prefix = paid opt-in, real cost in the envelope); scripts/llm --list shows local models + load state
probe	Empirical fit test: candidates × sample items, judged, ranked	scripts/probe --task-type translation --task "..." --criteria "..." --models openai/gpt-oss-20b,haiku --items-file s.txt --record
kb	Persistent calibration: record/query/prior/distill	scripts/kb query --task-type extraction; scripts/kb record --task-type extraction --model haiku --quality 8.5 --prompt-notes "strict schema + example"; scripts/kb prior --task-type codegen-complex (benchmark-derived ordering when KB is empty); scripts/kb distill

Run --help on each for full flags. Do not write ad-hoc dispatch scripts — extend these. Quality is 0–10 (judged against acceptance criteria), not pass/fail. Task types are the canonical set from kb types — inventing variants (sentiment-classification for classification) fragments the KB and was observed in testing.

Probing protocol

• 3–5 representative items, deliberately including the hardest (longest, noisiest, most placeholders).
• Criteria must name hard constraints (placeholders byte-identical, valid JSON, compiles) — the judge caps violations at 4/10.
• Judge with sonnet (default). Never hand-judge as orchestrator unless adjudicating disagreements — that's where your tokens have leverage.
• --record always: a probe that isn't recorded is money burned twice.

Prompt adaptation per tier

What you'd tell sonnet is not what you send down-ladder. Weaker model ⇒ stricter prompt:

Tier	Add to the prompt
sonnet	Clear instruction + output format
haiku	+ exact schema, decision rules for edge cases, "output nothing but X"
local	+ one worked example (anchors format), simpler sentences, no implicit context, low temperature; re-state constraints at the end

Record what worked via kb record --prompt-notes — prompt style is part of the fit. Per-model quirks: references/models.md.

Verification proportional to evidence

Evidence for the model on this task type	Verification
KB: n≥5 passing	Mechanical validators (row counts, schema, regex) + spot-check ~5%
KB: n=2–4 passing, or fresh probe pass	Mechanical validators on 100% + spot-check ~10%
None (don't do this)	— probe first

A second full pass on another model is for high-stakes output only — name the stake before paying double. Mechanical validation is free; write the validator before the batch runs.

Economics (measured 2026-06)

• claude -p carries ~9–27K input tokens of fixed overhead per invocation (system prompt; varies with cache state — measured ≈$0.013 haiku for a one-word reply). Batch 20–50 items per call; never one item per call.
• Subagents (Agent tool) carry similar per-spawn overhead — fan out for parallelism or isolation, not for tiny items.
• LM Studio: $0, ~1–15s/item, but check llm --list — not loaded models JIT-load (slow) or fail memory guardrails (a 27B+ model may need >40GB). Prefer loaded models; one model loads at a time.
• OpenRouter (or: ids): per-token market prices, no fixed dispatch overhead (direct HTTP), no memory constraints, parallel calls fine (no one-model-at-a-time limit). The price band between free-local and haiku is densely populated (curated shortlist with prices: references/openrouter-models.json); kb/probe rank or: models by real blended $/MTok. Three patterns where it wins: a hosted twin of a locally broken or unloadable model (glm-4.7-flash: 0.0 local, flawless hosted at $0.06/$0.4); scale-out past LM Studio's serial constraint; 1M-context models for inputs no local model holds.
• Privacy is a routing criterion: LM Studio keeps data on the machine; or: sends it to third-party providers, and :free variants are typically free because prompts may be used for training (account data-policy settings can block them — observed as HTTP 404 "guardrail restrictions"). Sensitive data routes local or Claude, not to the cheapest endpoint.

Rationalizations (all observed in baseline testing)

Excuse	Reality
"Local model quality is uncalibrated, exclude it"	Probing takes 2 minutes. gpt-oss-20b beat haiku on translation. Calibrate, then decide.
"I'll hand-label a gold set myself"	You're the most expensive model in the building. Sonnet judges; you adjudicate disagreements.
"Dual full pass for accuracy"	Verification proportional to evidence. Mechanical validators first — they're free.
"No data on this, I'll just use haiku/sonnet to be safe"	That guess unrecorded is the same guess next month. Probe once, record, stop guessing.
"I'll write a quick dispatch script"	scripts/llm exists. Reinvented plumbing is how results stop being comparable.
"Probing is overhead, the batch is only $3"	The KB record outlives this batch. You're buying calibration for every future session.
"Cost mandate says minimize spend, so haiku reviews the concurrency code"	Underkill on subtle-correctness work produces confident false negatives. The mandate is satisfied by right-sizing the other 90%.
"Accuracy matters and it's only ~10K tokens — I'll classify them myself in-context"	You just routed the batch to the priciest model with zero evidence it's needed. Calibrated cheap models score 10/10 on this task type; your accuracy leverage is the spot-check and adjudication, not the bulk pass.
"I'll invoke the skill, expected outcome: my current plan"	Deciding first and consulting second is not consulting. Run kb query before forming the plan, not after.
"It tops SWE-bench, skip the probe"	Benchmarks rank general strength on someone else's distribution, harness, and build. ~80% of benchmark variance is one general factor; your task's specific failure mode (restraint, schema discipline, format) is exactly what they miss. The prior orders the probe — it never replaces it.
"It benches low, scored 0 locally — the model is weak"	A local score far below the public profile means a broken path (chat template, max_tokens vs reasoning, params), not a weak model. Fix the integration, re-probe.

Red flags — stop and run the procedure

• About to delegate without having run kb query
• About to type a model name into a plan with no evidence attached
• About to bulk-read a file of N items into your own context to process them yourself
• About to exclude local models with the word "uncalibrated"
• About to run a full second-model pass "to be safe"
• Finished a delegation and about to move on without kb record