| name | groq-cost-tuning |
| description | Optimize Groq costs through model routing, token management, and usage monitoring.
Use when analyzing Groq billing, reducing API costs,
or implementing usage monitoring and budget alerts.
Trigger with phrases like "groq cost", "groq billing",
"reduce groq costs", "groq pricing", "groq expensive", "groq budget".
|
| allowed-tools | Read, Grep |
| version | 1.11.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| tags | ["saas","groq","api","monitoring","cost-optimization"] |
| compatibility | Designed for Claude Code, also compatible with Codex and OpenClaw |
Groq Cost Tuning
Overview
Optimize Groq inference costs through smart model routing, token minimization, and caching. Groq pricing is already extremely competitive, but at high volume the savings from routing classification to 8B vs 70B are 12x per request.
Prerequisites
- A Groq account with an API key exported as the
GROQ_API_KEY environment variable — the groq-sdk client reads it automatically (new Groq()).
- Node.js with the
groq-sdk package installed (npm install groq-sdk).
- Access to the Groq Console to set spending caps and read the usage dashboard.
Groq Pricing (per million tokens)
| Model | Input | Output |
|---|
llama-3.1-8b-instant | ~$0.05 | ~$0.08 |
llama-3.3-70b-versatile | ~$0.59 | ~$0.79 |
llama-3.3-70b-specdec | ~$0.59 | ~$0.99 |
meta-llama/llama-4-scout-17b-16e-instruct | ~$0.11 | ~$0.34 |
whisper-large-v3-turbo | ~$0.04/hr | — |
Check current pricing at groq.com/pricing.
Instructions
Apply these six levers in order. Each compounds on the last — routing alone is
the biggest win (~12x), and caching plus batching halve the remainder. The lean
skeleton below shows the routing core; the full code for every step lives in
references/implementation.md.
- Smart model routing — map each use case to the cheapest model that meets its quality bar (classification/extraction/summarization →
llama-3.1-8b-instant; reasoning/code review/chat → llama-3.3-70b-versatile; vision → llama-4-scout).
- Minimize tokens per request — trim verbose system prompts and cap
max_tokens so a one-word answer never bills for a paragraph.
- Batch to reduce overhead — fold many items into one request; 10-in-1 cuts per-request overhead and RPM pressure ~90%.
- Cache deterministic requests — at , hash identical prompts into a cache for zero-cost, zero-latency repeat hits.
