llm-cost-optimizer

license	Apache-2.0
name	llm-cost-optimizer
description	Track and reduce LLM API costs with per-request token tracking, model routing, budget alerts, and prompt compression. Activate on: reduce AI costs, token tracking, model routing, LLM budget, prompt compression. NOT for: general cloud cost optimization (cost-accrual-tracker), model training costs (ai-engineer).
allowed-tools	Read,Write,Edit,Bash(python:*,pip:*,npm:*,npx:*)
category	AI & Machine Learning
tags	["cost-optimization","token-tracking","model-routing","budget-management","prompt-compression"]
pairs-with	[{"skill":"cost-accrual-tracker","reason":"General cost tracking infrastructure complements LLM-specific optimization"},{"skill":"llm-response-caching-layer","reason":"Caching is the highest-impact cost reduction lever"},{"skill":"model-serving-api-builder","reason":"Self-hosted serving eliminates per-token API costs at scale"}]

LLM Cost Optimizer

Track per-request token usage, implement intelligent model routing, set budget alerts, and compress prompts to reduce LLM API costs by 40-70%.

Decision Points

When to Optimize Which Dimension

High API costs (>$500/month) AND unknown spend breakdown?
├─ YES → Start with token tracking middleware
│   ├─ Instrument all LLM calls for 1 week
│   └─ Generate cost breakdown report
└─ NO → Skip to routing or compression

Cost breakdown shows 80% from 20% of endpoints?
├─ YES → Implement model routing for top cost drivers
│   ├─ Simple tasks (classify/extract) → Haiku/GPT-4o-mini
│   ├─ Medium tasks (summarize/explain) → Sonnet/GPT-4o  
│   └─ Complex tasks (reason/create) → Opus/GPT-4
└─ NO → Focus on prompt compression

System prompts >1000 tokens AND high request volume?
├─ YES → Compress prompts first (highest ROI)
│   ├─ Enable prompt caching (Anthropic/OpenAI)
│   ├─ Reduce few-shot examples to 2-3 best
│   └─ LLMLingua compress system instructions
└─ NO → Set budget alerts and monitoring

Budget overruns happening frequently?
├─ YES → Implement automated throttling
│   ├─ Daily caps with 80% soft limit warnings
│   ├─ Auto-downgrade expensive → cheap models
│   └─ Emergency circuit breaker at 95%
└─ NO → Set up monitoring dashboards

Model Selection Decision Tree

Request Analysis:
├─ Input tokens <200 AND structured output needed?
│   └─ Route to: Haiku ($0.80/M) or GPT-4o-mini ($0.15/M)
├─ Single-step reasoning OR summarization <2000 tokens?
│   └─ Route to: Sonnet ($3/M) or GPT-4o ($2.50/M)
├─ Multi-step reasoning OR creative writing OR high-stakes?
│   └─ Route to: Opus ($15/M) or GPT-4 ($30/M)
└─ Latency <100ms required?
    └─ Route to: Fastest model regardless of cost

Failure Modes

Quality Degradation Spiral

Symptom: Model performance drops after cost optimization Detection: Eval metrics decline >5% after routing/compression changes Diagnosis: Over-aggressive optimization sacrificing capability for cost Fix:

• A/B test compressed vs original prompts on eval suite
• Route complex reasoning back to capable models
• Implement quality gates before cost optimizations go live

Misclassification Cascade

Symptom: Simple tasks routed to expensive models, complex to cheap Detection: Haiku/mini models showing high retry rates or error responses Diagnosis: Complexity classifier is miscalibrated or missing edge cases Fix:

• Log classification confidence scores
• Manual review of misclassified requests
• Retrain classifier with corrected examples
• Add safety net: retry on cheap model failure with expensive model