cost-aware-llm-pipeline

name	cost-aware-llm-pipeline
description	Cost optimization patterns for LLM API usage: model routing by task complexity, budget tracking, retry logic, and prompt caching. USE WHEN LLM API spend is too high, picking models per task, adding budget caps, or wiring prompt caching.
origin	ECC
cluster	ai-agents-meta
version	1.0.0

Cost-Aware LLM Pipeline

Patterns for controlling LLM API costs while maintaining quality. Combines model routing, budget tracking, retry logic, and prompt caching into a composable pipeline.

When to Activate

• Building applications that call LLM APIs (Claude, GPT, etc.)
• Processing batches of items with varying complexity
• Need to stay within a budget for API spend
• Optimizing cost without sacrificing quality on complex tasks

Core Concepts

1. Model Routing by Task Complexity

Automatically select cheaper models for simple tasks, reserving expensive models for complex ones.

MODEL_SONNET = "claude-sonnet-4-6"
MODEL_HAIKU = "claude-haiku-4-5-20251001"

_SONNET_TEXT_THRESHOLD = 10_000  
_SONNET_ITEM_THRESHOLD =      

 () -> :
    
     force_model   :
         force_model
     text_length >= _SONNET_TEXT_THRESHOLD  item_count >= _SONNET_ITEM_THRESHOLD:
         MODEL_SONNET  
     MODEL_HAIKU