| name | model-right-sizer |
| description | Pick the smallest capable model for each AI task — audit LLM/ML usage for oversized model selection, design tiered routing (small model first, escalate on need), and recommend quantization or distillation where relevant. Use this skill whenever the user asks which model to use, wants to cut LLM/API cost, latency, or the carbon footprint of AI features, mentions their AI bill, shares model-usage logs, or is designing any LLM-calling feature. Part of Lean Agentic AI Skills; emits lean-findings.json. |
Model Right-Sizer
Producer/advisor skill. Input: task descriptions, prompt samples, usage logs (model, tokens, task type), quality requirements. Output: lean-findings.json where findings are routing recommendations.
Frontier models on trivial tasks is the AI era's idle VM. Inference energy scales with model size; classification, extraction, formatting, and routing rarely need the largest model. This is E-reduction per request AND R-improvement (more useful output per unit of compute).
Subject type: emit subject.type: "ai-inference" in findings.
Procedure
- Inventory the task mix from logs or descriptions. Bucket each call site: classification/routing, extraction, summarization, transformation/formatting, generation (short), generation (long/creative), reasoning-heavy, agentic tool-use.
- Map buckets to tiers, smallest-first:
- Deterministic-adjacent (classification, extraction with schema, formatting) → smallest tier or even non-LLM (regex, embeddings) when structure allows.
- Standard summarization/transformation → small–mid tier.
- Complex reasoning, high-stakes generation, agentic orchestration → large tier, and say why it's justified.
- Design the escalation ladder: confidence/validation check on the small model's output → retry on mid → large. Cheap validators (schema check, self-consistency on shorts) beat always-large.
- Flag quantization/distillation only when the user self-hosts: quantized weights or a distilled task-specific model, with the quality-validation step named.
- Emit findings: evidence = observed call sites and token volumes; impact = directional ("this call site is high-volume and structurally simple — a small model likely serves it at a fraction of the compute").
Honesty rules
- Quality is measured, not assumed: every downgrade recommendation includes an eval step ("run the task pairs through both tiers on a sample before switching").
- Never claim energy percentages between models — vendors don't publish per-inference energy; speak in tiers and token economics.
- Latency often improves with smaller models — a rare free trade-off; still verify.
Cost signal (countable)
Cost driver: billed input + output tokens per task type × volume. cost_signal.observed states the current per-task token profile (from logs) and direction reflects the tier change qualitatively ("routes ~80% of volume to a smaller-tier model — a large fraction of tokens move to the cheaper meter").
Not this skill's job
Prompt-level trimming (prompt-carbon-optimizer), caching (llm-cache-designer), loop behavior (agentic-loop-auditor), edge/cloud placement (inference-placement-advisor).
