cohere-rate-limits

Implement Cohere rate limiting, backoff, and request queuing patterns. Use when handling 429 errors, implementing retry logic, or optimizing API request throughput for Cohere. Trigger with phrases like "cohere rate limit", "cohere throttling", "cohere 429", "cohere retry", "cohere backoff".

Jump to install

Repository: jeremylongshore/claude-code-plugins-plus-skills
Last source activity: July 17, 2026 at 22:18
Detected SKILL.md language: English
Stars: 2,604
Forks: 378

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

Back to search Back to installation

name	cohere-rate-limits
description	Implement Cohere rate limiting, backoff, and request queuing patterns. Use when handling 429 errors, implementing retry logic, or optimizing API request throughput for Cohere. Trigger with phrases like "cohere rate limit", "cohere throttling", "cohere 429", "cohere retry", "cohere backoff".
allowed-tools	Read, Write, Edit
version	1.5.0
license	MIT
author	Jeremy Longshore <jeremy@intentsolutions.io>
tags	["saas","ai","nlp","cohere"]
compatibility	Designed for Claude Code

Cohere Rate Limits

Overview

Handle Cohere rate limits with exponential backoff, request queuing, and proactive throttling. Real rate limits from Cohere's documentation.

Prerequisites

cohere-ai SDK installed
Understanding of async/await patterns

Actual Cohere Rate Limits

Key Type	Endpoint	Rate Limit	Monthly Limit
Trial	Chat	20 calls/min	1,000 total
Trial	Embed	5 calls/min	1,000 total
Trial	Rerank	5 calls/min	1,000 total
Trial	Classify	5 calls/min	1,000 total
Production	All endpoints	1,000 calls/min	Unlimited

Trial keys are free. Production keys require billing at dashboard.cohere.com.

Instructions

Step 1: Exponential Backoff with Jitter

import { CohereError, CohereTimeoutError } from 'cohere-ai';

interface RetryConfig {
  maxRetries: number;
  baseDelayMs: number;
  maxDelayMs: number;
}

const DEFAULT_RETRY: RetryConfig = {
  maxRetries: 5,
  baseDelayMs: 1000,
  maxDelayMs: 60_000,
};

async function withBackoff<T>(
  :  <T>,
  config = 
): <T> {
   ( attempt = ; attempt <= config.; attempt++) {
     {
        ();
    }  (err) {
       (attempt === config.)  err;

      
       shouldRetry = ;
       :  | ;

       (err  ) {
         (err. === ) {
          shouldRetry = ;
          
          
        }   (err. && err. >= ) {
          shouldRetry = ;
        }
      }   (err  ) {
        shouldRetry = ;
      }

       (!shouldRetry)  err;

      
       exponential = config. * .(, attempt);
       jitter = .() * config.;
       delay = .(exponential + jitter, config.);

      .();
        ( (r, retryAfterMs ?? delay));
    }
  }
    ();
}

import PQueue from 'p-queue';

// Match rate limits: trial=20/min for chat, production=1000/min
function createCohereQueue(callsPerMinute: number) {
  return new PQueue({
    concurrency: 5,
    interval: 60_000,
    intervalCap: callsPerMinute,
  });
}

// Trial key queue
const trialChatQueue = createCohereQueue(20);
const trialEmbedQueue = createCohereQueue(5);

// Production key queue
const prodQueue = createCohereQueue(1000);

// Usage
async function queuedChat(params: any) {
  return trialChatQueue.add(() =>
    withBackoff(() => cohere.chat(params))
  );
}

class RateLimitTracker {
  private windows: Map<string, number[]> = new Map();

  constructor(private limitsPerMinute: Record<string, number>) {}

  canProceed(endpoint: string): boolean {
    const limit = this.limitsPerMinute[endpoint] ?? 1000;
    const now = Date.now();
    const window = this.windows.get(endpoint) ?? [];

    // Remove entries older than 1 minute
    const active = window.filter(t => now - t < 60_000);
    this.windows.set(endpoint, active);

    return active.length < limit;
  }

  record(endpoint: string): void {
    const window = this.windows.get(endpoint) ?? [];
    window.push(Date.now());
    this.windows.set(endpoint, window);
  }

  waitTime(endpoint: string): number {
    const limit = this.limitsPerMinute[endpoint] ?? 1000;
    const window = this.windows.get(endpoint) ?? [];
    const now = Date.now();
    const active = window.filter(t => now - t < 60_000);

    if (active.length < limit) return 0;
    return 60_000 - (now - active[0]); // Wait until oldest entry expires
  }
}

// Trial key tracker
const tracker = new RateLimitTracker({
  chat: 20,
  embed: 5,
  rerank: 5,
  classify: 5,
});

// Use before each call
async function trackedEmbed(params: any) {
  const wait = tracker.waitTime('embed');
  if (wait > 0) {
    console.log(`Throttling embed: waiting ${wait}ms`);
    await new Promise(r => setTimeout(r, wait));
  }
  tracker.record('embed');
  return withBackoff(() => cohere.embed(params));
}

// Embed supports up to 96 texts per call — maximize batch size to reduce calls
async function efficientEmbed(
  texts: string[],
  inputType: 'search_document' | 'search_query' = 'search_document'
): Promise<number[][]> {
  const BATCH_SIZE = 96; // Cohere max per request
  const allVectors: number[][] = [];

  for (let i = 0; i < texts.length; i += BATCH_SIZE) {
    const batch = texts.slice(i, i + BATCH_SIZE);

    const response = await trackedEmbed({
      model: 'embed-v4.0',
      texts: batch,
      inputType,
      embeddingTypes: ['float'],
    });

    allVectors.push(...response.embeddings.float);
  }

  return allVectors;
}

// 960 texts = 10 API calls (not 960)
const vectors = await efficientEmbed(largeTextArray);

class TokenBudget {
  private tokensUsed = 0;
  private readonly resetInterval: NodeJS.Timer;

  constructor(
    private maxTokensPerMinute: number,
    private alertCallback?: (used: number) => void
  ) {
    // Reset every minute
    this.resetInterval = setInterval(() => { this.tokensUsed = 0; }, 60_000);
  }

  canAfford(estimatedTokens: number): boolean {
    return this.tokensUsed + estimatedTokens <= this.maxTokensPerMinute;
  }

  record(actualTokens: number): void {
    this.tokensUsed += actualTokens;
    if (this.tokensUsed > this.maxTokensPerMinute * 0.8) {
      this.alertCallback?.(this.tokensUsed);
    }
  }

  dispose(): void {
    clearInterval(this.resetInterval);
  }
}

Scenario	Detection	Action
429 from trial key	`CohereError.statusCode === 429`	Wait 60s, retry
429 from prod key	Same	Backoff, check concurrency
Monthly limit hit (trial)	429 with limit message	Upgrade to production key
Burst of requests	Queue depth > threshold	Add backpressure