| name | groq-observability |
| description | Set up observability for Groq integrations: latency histograms, token throughput,
rate limit gauges, cost tracking, and Prometheus alerts.
Use when instrumenting Groq API calls, building a metrics dashboard, or wiring latency/cost/rate-limit alerts.
Trigger with phrases like "groq monitoring", "groq metrics",
"groq observability", "monitor groq", "groq alerts", "groq dashboard".
|
| allowed-tools | Read, Write, Edit |
| version | 1.11.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| tags | ["saas","groq","monitoring","observability","dashboard"] |
| compatibility | Designed for Claude Code |
Groq Observability
Overview
Monitor Groq LPU inference for latency, token throughput, rate limit utilization, and cost. Groq's defining advantage is speed (280-560 tok/s), so latency degradation is the highest-priority signal. The API returns rich timing metadata (queue_time, prompt_time, completion_time) and rate limit headers on every response.
Prerequisites
- A Groq account with an API key exported as the
GROQ_API_KEY environment variable — the groq-sdk client reads it automatically (new Groq()).
- Node.js with
groq-sdk and prom-client installed (npm install groq-sdk prom-client).
- A Prometheus scrape target and (optionally) Grafana for the dashboard panels.
Key Metrics to Track
| Metric | Type | Source | Why |
|---|
| TTFT (time to first token) | Histogram | Client-side timing | Groq's main value prop |
| Tokens/second | Gauge | usage.completion_time | Throughput degradation |
| Total latency | Histogram | Client-side timing | End-to-end performance |
| Rate limit remaining | Gauge | x-ratelimit-remaining-* headers | Prevent 429s |
| Token usage | Counter | usage.total_tokens | Cost attribution |
| Error rate by code | Counter | Error handler | Availability |
| Estimated cost | Counter | Tokens * model price | Budget tracking |
Instructions
Apply these six steps in order. Steps 1-2 are the core instrumentation loop —
wrap the client, then feed a Prometheus instrument set from each call. Steps 3-6
add rate-limit tracking, alerting, structured logs, and dashboards on top. The
lean client skeleton is below; the full code for every step lives in
references/implementation.md.
- — wrap so latency, tokens, queue time, and estimated cost are captured on the same path as the request ().
