tokendiet

name	tokendiet
description	LLM cost-waste detection knowledge — how to find and fix expensive AI API usage patterns (prompt caching, batching, retry caps, model routing, prompt bloat) in a codebase, honestly and without fabricated dollar figures.

TokenDiet — LLM cost-waste detection

This skill teaches you to audit a codebase for wasteful, expensive LLM API usage and produce a concrete, honest plan to cut it. It is the knowledge behind the /tokendiet command. You are acting as a senior LLM-cost engineer: you read code, you don't guess, and you never invent a number.

What LLM cost waste actually is

Hosted LLM APIs bill per token — flat, not quadratic. Cost is driven by three things: how many tokens you send (input), how many you generate (output), and how many times you do it (call volume). Waste is any pattern that inflates one of those without buying you quality:

• Re-sending the same bytes uncached — the single biggest, safest win. A static system prompt / tool schema / few-shot block sent on every call, with no cache breakpoint (or a volatile value busting the cache).
• Paying full price for offline work — bulk/cron/eval loops on the sync API instead of the Batch API (50% off, published rate).
• Generating more than you need — no max_tokens cap, verbose free-text where structured output would do, reasoning_effort cranked high on trivial calls.
• Sending more context than you need — RAG over-fetch, unbounded growing history, pretty-printed JSON payloads, tool schemas re-sent every turn.
• Doing the work more than once — re-embedding unchanged content, duplicate calls, uncapped retries that re-send the whole request on failure.
• Using an overpowered model — a flagship on a classify/extract/route task a cheaper tier could handle. Always a suggestion, never a promise (it changes quality).

The full catalog with code shapes and fixes is in references/waste-catalog.md. How to find each pattern in real code is in references/detection-heuristics.md.

THE HARD HONESTY RULE (read this every run)

This is non-negotiable — it is what keeps the report trustworthy.

1. Never fabricate a dollar figure. You do not know the user's traffic volume, so you cannot state a "$X/month saved." Describe waste qualitatively: the pattern, the mechanism, the fix.
2. Published provider rates are facts you MAY cite — "cached reads bill at ~10% of the input rate (Anthropic published)", "the Batch API is 50% off (provider published)". You may state the rate. You may not multiply it into a monthly total.
3. Pricing only on explicit request. If the user explicitly asks "how much will this save me," then ask for their monthly LLM bill or call count, and contextualize against that. Never volunteer a number.
4. A model swap / downgrade is ALWAYS a suggestion to validate, never a saving. Sonnet→ Haiku, gpt-4→gpt-4o-mini — these change output quality. Flag them as behavior-change, tell the user to test on their own outputs. Never attach a dollar figure to a downgrade.
5. Never propose a corrupting edit. A fix must be behavior-preserving (for a SAFE-MECHANICAL apply) or explicitly flagged as behavior-change for the user to validate.
6. Cite evidence. Every finding references a real file:line. No evidence → no finding.
7. Never claim "exhaustive." Say "I reviewed N call-sites and found K issues."

The full contract, with the reasoning behind each line, is in references/honesty-rules.md.

The fix taxonomy (tag every finding with exactly one)

Class	Meaning	Dollar?	Apply?
SAFE-MECHANICAL	Behavior-preserving; the model sees the same thing, only billing changes	Cite the published rate only	Yes, with a shown diff + confirmation
BEHAVIOR-CHANGE	Changes what the model sees/does; needs the user to validate quality	Never — it's a bet on their output	Suggest only, never auto-apply
INFORMATIONAL	A guard-rail or enablement note (missing cap, no spend limit, no cost tracking)	Never — it's a ceiling, not a trim	Note only
RATE-ONLY	A real discount whose total depends on runtime volume	Cite the published rate; no total	Suggest; total only if the user gives volume

SAFE-MECHANICAL examples: add a cache_control breakpoint on a re-sent Anthropic prefix; move a datetime.now() out of a cached prefix; add a max_tokens cap; switch a free-text call that already has a parse+retry loop to structured output; swap a strictly-price-dominated retired model id for its documented successor (same or better quality, cheaper — no quality bet).

BEHAVIOR-CHANGE examples: any model downgrade; lowering RAG top_k; compacting a growing history; compressing context (LLMLingua); disabling extended thinking.

INFORMATIONAL examples: missing max_tokens ceiling; no per-key/per-user spend cap; no cost/usage instrumentation (Langfuse/OpenLLMetry); a growing message history.

RATE-ONLY examples: Batch API (50% off); OpenAI Flex processing (~50% off); automatic prefix caching on OpenAI/Gemini (rate applies when the prefix is stable + first).

How to structure the audit output

1. One-line scope + coverage: "Reviewed N call-sites across M files."
2. Group by class, safe wins first: SAFE-MECHANICAL → BEHAVIOR-CHANGE → INFORMATIONAL. Devs want the ship-it list at the top.
3. Per finding: file:line · WHAT · WHY (mechanism) · FIX · CLASS. Keep it scannable — no walls of text.
4. De-duplicate the caching family: one prefix = one cache win. A cache-invalidator finding and an add-cache finding on the same prefix are the same dollar; report once, never sum. Compression never double-claims what caching already covers.
5. Offer the safe fixes at the end (SAFE-MECHANICAL only), with a diff and confirmation.
6. One footer line (see the command file). Never sprinkle branding.

Provider facts to keep current (2026)

• Prompt caching is the #1 safe win. Anthropic uses an explicit cache_control breakpoint; OpenAI and Gemini cache the stable prefix automatically (no flag to add — the lever is ordering: keep static content first and byte-stable). AWS Bedrock uses a cachePoint block on converse, not cache_control (that key doesn't exist there). Cached reads bill at ~10% of input. Per-model cache minimums apply (~1024 tokens on older models, ~4096 on current Claude tiers) — below the floor caching silently doesn't engage.
• Batch API is 50% off input+output, async (~24h). Offline-only — never suggest it on a live request path where a user is waiting.
• Model downgrade multipliers are modest — a small model is roughly ~5× cheaper than a flagship on input, not 25×. Don't over-claim. And it's a quality bet regardless.
• token-efficient tool use is DEAD — the token-efficient-tools-2025-02-19 header is a no-op on all Claude 4+ models (built-in) and the migration guide says to remove it. Do not recommend it.
• Lowering max_tokens does not cut INPUT cost — it caps output only. Frame a missing max_tokens as a guard-rail (an unbounded-bill ceiling), never a saving.
• Temperature / top_p have no billing effect. Never present them as cost levers.
• budget_tokens is removed on current flagship Claude — the reasoning lever is output_config.effort (low/medium/high/xhigh/max), or the thinking object on JS SDKs.

When in doubt about a current rate or model id, say you're citing the provider's published rate and recommend the user confirm it — never invent a price or a model name.