| name | groq-rate-limits |
| description | Implement Groq rate limit handling with backoff, queuing, and header parsing.
Use when handling rate limit errors, implementing retry logic,
or optimizing API request throughput for Groq.
Trigger with phrases like "groq rate limit", "groq throttling",
"groq 429", "groq retry", "groq backoff".
|
| allowed-tools | Read, Write, Edit |
| version | 1.11.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| tags | ["saas","groq","api"] |
| compatibility | Designed for Claude Code, also compatible with Codex and OpenClaw |
Groq Rate Limits
Overview
Handle Groq rate limits using the retry-after header, exponential backoff, and request queuing. Groq enforces limits at the organization level with both RPM (requests/minute) and TPM (tokens/minute) constraints -- hitting either one triggers a 429.
The workflow builds up in five composable layers: parse the rate-limit headers, wrap calls in retry-with-backoff, gate concurrency through a queue, monitor remaining capacity proactively, and fall back across models when one pool is exhausted. Read SKILL.md for the high-level flow, then drill into the full implementation for every code block and the reference tables + worked examples for header definitions and composed clients.
Prerequisites
- A Groq API key (
GROQ_API_KEY) — get one at console.groq.com.
groq-sdk installed: npm install groq-sdk.- For queuing (Step 3):
p-queue installed: npm install p-queue.
- Node.js 18+ (for native
fetch and the SDK).
- Know your plan's limits — check console.groq.com/settings/limits.
Rate Limits at a Glance
Groq applies RPM, RPD, TPM, and TPD limits simultaneously — you must stay under every one, and either RPM or TPM can trip a 429. Every response (even a success) carries x-ratelimit-* headers describing remaining capacity and reset timing; 429 responses add a retry-after header. Full header and constraint tables: reference.md.
Instructions
Compose these five steps into one client wrapper (queue → monitor → retry). Each step's complete, copy-pasteable code is in implementation.md.
Step 1: Parse Rate Limit Headers
Read the x-ratelimit-* headers off every response into a typed RateLimitInfo so downstream logic can reason about remaining capacity. Groq reports reset times as strings like "1.2s" or "120ms" — normalize them to milliseconds.
Step 2: Exponential Backoff with Retry-After
Wrap each API call in a retry loop. Prefer Groq's retry-after header when present; otherwise back off exponentially with jitter, capped at . Retry only and — other errors are not retryable.
