groq-rate-limits

name	groq-rate-limits
description	Implement Groq rate limit handling with backoff, queuing, and header parsing. Use when handling rate limit errors, implementing retry logic, or optimizing API request throughput for Groq. Trigger with phrases like "groq rate limit", "groq throttling", "groq 429", "groq retry", "groq backoff".
allowed-tools	Read, Write, Edit
version	1.11.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","groq","api"]
compatibility	Designed for Claude Code

Groq Rate Limits

Overview

Handle Groq rate limits using the retry-after header, exponential backoff, and request queuing. Groq enforces limits at the organization level with both RPM (requests/minute) and TPM (tokens/minute) constraints -- hitting either one triggers a 429.

The workflow builds up in five composable layers: parse the rate-limit headers, wrap calls in retry-with-backoff, gate concurrency through a queue, monitor remaining capacity proactively, and fall back across models when one pool is exhausted. Read SKILL.md for the high-level flow, then drill into the full implementation for every code block and the reference tables + worked examples for header definitions and composed clients.

Prerequisites

• A Groq API key (GROQ_API_KEY) — get one at console.groq.com.
• groq-sdk installed: npm install groq-sdk.
• For queuing (Step 3): p-queue installed: npm install p-queue.
• Node.js 18+ (for native fetch and the SDK).
• Know your plan's limits — check console.groq.com/settings/limits.

Rate Limits at a Glance

Groq applies RPM, RPD, TPM, and TPD limits simultaneously — you must stay under every one, and either RPM or TPM can trip a 429. Every response (even a success) carries x-ratelimit-* headers describing remaining capacity and reset timing; 429 responses add a retry-after header. Full header and constraint tables: reference.md.

Instructions

Compose these five steps into one client wrapper (queue → monitor → retry). Each step's complete, copy-pasteable code is in implementation.md.

Step 1: Parse Rate Limit Headers

Read the x-ratelimit-* headers off every response into a typed RateLimitInfo so downstream logic can reason about remaining capacity. Groq reports reset times as strings like "1.2s" or "120ms" — normalize them to milliseconds.

Step 2: Exponential Backoff with Retry-After

Wrap each API call in a retry loop. Prefer Groq's retry-after header when present; otherwise back off exponentially with jitter, capped at maxDelayMs. Retry only 429 and 5xx — other 4xx errors are not retryable.

Step 3: Request Queue with Concurrency Control

Gate all requests through a p-queue sized to your plan's RPM (intervalCap over a 60s interval) so bursts never exceed the limit in the first place.

Step 4: Proactive Rate Limit Monitor

Track remaining requests/tokens from response headers and pause before hitting zero (shouldThrottle() → waitIfNeeded()), instead of reacting to 429s after the fact.

Step 5: Model-Aware Rate Limit Strategy

Different models draw from different limit pools. When the preferred model is throttled, fall back to another model to keep making progress without waiting for a reset.

Output

Applying this skill produces:

• A withRateLimitRetry() wrapper that transparently retries 429/5xx with retry-after-aware backoff.
• A RateLimitMonitor that surfaces live status (getStatus() → "Requests: N remaining | Tokens: M remaining") and throttles proactively.
• A p-queue-backed client that caps throughput to your RPM so 100 fan-out calls complete without tripping the limit.
• Console diagnostics on every backoff/throttle event (e.g. Rate limited (attempt 2/5). Waiting 1.4s...).

The observable end state: sustained request volume that stays under RPM/TPM with zero unhandled 429s.

Error Handling

Scenario	Symptom	Solution
Burst of requests	Many 429s in quick succession	Use queue with p-queue interval limiting (Step 3)
Large prompts burn TPM	429 on tokens, not requests	Reduce max_tokens, compress prompts
Free tier too restrictive	Constant 429s	Upgrade to Developer plan at console.groq.com
Multiple services sharing key	Cascading 429s	Use separate API keys per service
retry-after absent on 429	Retries hammer too fast	Fall back to exponential backoff + jitter (Step 2)

Examples

Start from this minimal 429 handler, then graduate to the composed client (queue + monitor + retry) in reference.md:

try {
  await groq.chat.completions.create({ model, messages });
} catch (err) {
  if (err instanceof Groq.APIError && err.status === 429) {
    const retryAfter = parseInt(err.headers?.["retry-after"] || "0");
    console.log(`Rate limited. retry-after says wait ${retryAfter}s.`);
    // -> feed retryAfter into withRateLimitRetry (Step 2)
  }
}

• Full five-step implementation (every code block, verbatim): implementation.md
• Composed client + header/limit tables + a 100-request fan-out: reference.md

Resources

• Groq Rate Limits Documentation
• Groq Pricing / Plans
• p-queue on npm
• Full implementation: references/implementation.md
• Reference tables & worked examples: references/reference.md

Next Steps

For security configuration, see the groq-security-basics skill in this pack, which covers API key storage, rotation, and request signing to complement the throughput handling above.