groq-observability

name	groq-observability
description	Set up observability for Groq integrations: latency histograms, token throughput, rate limit gauges, cost tracking, and Prometheus alerts. Use when instrumenting Groq API calls, building a metrics dashboard, or wiring latency/cost/rate-limit alerts. Trigger with phrases like "groq monitoring", "groq metrics", "groq observability", "monitor groq", "groq alerts", "groq dashboard".
allowed-tools	Read, Write, Edit
version	1.11.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","groq","monitoring","observability","dashboard"]
compatibility	Designed for Claude Code, also compatible with Codex and OpenClaw

Groq Observability

Overview

Monitor Groq LPU inference for latency, token throughput, rate limit utilization, and cost. Groq's defining advantage is speed (280-560 tok/s), so latency degradation is the highest-priority signal. The API returns rich timing metadata (queue_time, prompt_time, completion_time) and rate limit headers on every response.

Prerequisites

• A Groq account with an API key exported as the GROQ_API_KEY environment variable — the groq-sdk client reads it automatically (new Groq()).
• Node.js with groq-sdk and prom-client installed (npm install groq-sdk prom-client).
• A Prometheus scrape target and (optionally) Grafana for the dashboard panels.

Key Metrics to Track

Metric	Type	Source	Why
TTFT (time to first token)	Histogram	Client-side timing	Groq's main value prop
Tokens/second	Gauge	usage.completion_time	Throughput degradation
Total latency	Histogram	Client-side timing	End-to-end performance
Rate limit remaining	Gauge	x-ratelimit-remaining-* headers	Prevent 429s
Token usage	Counter	usage.total_tokens	Cost attribution
Error rate by code	Counter	Error handler	Availability
Estimated cost	Counter	Tokens * model price	Budget tracking

Instructions

Apply these six steps in order. Steps 1-2 are the core instrumentation loop — wrap the client, then feed a Prometheus instrument set from each call. Steps 3-6 add rate-limit tracking, alerting, structured logs, and dashboards on top. The lean client skeleton is below; the full code for every step lives in references/implementation.md.

1. Instrumented client — wrap groq.chat.completions.create so latency, tokens, queue time, and estimated cost are captured on the same path as the request ().