| name | fine-tune-vs-prompt-advisor |
| description | Decide the cheapest path to an AI capability — prompting, RAG, fine-tuning, or distillation — with the honest amortization math for each. Use this skill whenever the user asks "should we fine-tune", weighs prompt engineering against training a custom model, wants a capability at lower ongoing cost, or is designing a new AI feature and choosing its approach. Part of Lean Agentic AI Skills; emits lean-findings.json. |
Fine-Tune vs Prompt Advisor
Subject type: emit subject.type: "ai-inference" in findings.
Advisor skill. Input: the capability description, quality bar, expected volume, latency needs, data available for training, and current approach if one exists. Output: a recommendation with the trade-off math structure (not invented numbers) + lean-findings.json.
Teams reach for fine-tuning by instinct and prompting by inertia; both instincts are expensive when wrong. The decision is an amortization problem: one-time costs (data curation, training runs, eval builds) against per-request costs (prompt tokens, retrieval, model tier) at the actual expected volume.
Decision ladder (cheapest first — exit at the first rung that meets the bar)
- Better prompting — instructions, few-shot, output constraints on an existing model. Zero one-time infra; per-request cost = the prompt (pairs with prompt-carbon-optimizer to keep it lean). Right when the base model has the capability latent.
- RAG — when the gap is knowledge, not behavior. Adds retrieval cost per request and index maintenance; avoids training entirely and stays current as data changes (pairs with rag-pipeline-optimizer).
- Fine-tuning — when the gap is behavior/format/style at volume: a tuned smaller model can replace a larger prompted one, cutting per-request tokens AND tier. One-time training cost amortizes only past a volume threshold — state the threshold structure explicitly: (training GPU-hours + data effort) vs (per-request savings × requests/month). Below the threshold, fine-tuning is vanity.
- Distillation — fine-tuning's industrial cousin: teacher-generated data trains a much smaller student for one narrow task at very high volume. Highest one-time cost, lowest per-request cost; justified only by sustained scale.
What the recommendation must contain
The chosen rung and why the cheaper rungs fail the bar; the amortization structure with the user's own numbers where supplied (volume, current tokens/request) and named placeholders where not; the maintenance tail honestly stated (fine-tuned models freeze knowledge and need re-training on drift — a recurring cost prompting doesn't have); and the eval gate: no path switch ships without a quality eval on the user's task.
Cost signal (countable)
Cost drivers, per rung: prompt tokens/request (rungs 1-2), training GPU-hours (one-time, rungs 3-4), per-request tokens at the new tier (rungs 3-4). All countable from the user's logs and training configs; the advisor's job is arranging them into the honest comparison, never inventing the missing terms.
Honesty rules
Volume forecasts are the user's claim, not yours — mark the break-even as conditional on them. "Fine-tuning improves quality" is not assumable; only the eval says so. Knowledge-cutoff staleness of tuned models is a named cost, not a footnote.
Not this skill's job
Executing the tuning efficiently (training-efficiency-audit), the prompt itself (prompt-carbon-optimizer), retrieval design (rag-pipeline-optimizer), serving the result (llm-serving-optimizer).
