brightdata-rate-limits

name	brightdata-rate-limits
description	Implement Bright Data rate limiting, backoff, and idempotency patterns. Use when handling rate limit errors, implementing retry logic, or optimizing API request throughput for Bright Data. Trigger with phrases like "brightdata rate limit", "brightdata throttling", "brightdata 429", "brightdata retry", "brightdata backoff".
allowed-tools	Read, Write, Edit
version	1.6.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","scraping","data","brightdata"]
compatibility	Designed for Claude Code

Bright Data Rate Limits

Overview

Handle Bright Data rate limits and concurrent request limits. Unlike traditional API rate limits, Bright Data limits are per-zone and based on concurrent connections and requests per second. The Web Scraper API trigger endpoint is limited to 20 requests/min and 60 requests/hour.

Prerequisites

• Bright Data zone configured
• Understanding of async/await patterns
• p-queue or similar concurrency library

Instructions

Step 1: Understand Bright Data Rate Limits

Product	Concurrent Limit	Per-Minute	Notes
Residential Proxy	Based on plan	No hard cap	Charged per GB
Web Unlocker	Based on plan	No hard cap	Charged per request
Scraping Browser	Based on plan sessions	No hard cap	Charged per session
SERP API	Based on plan	No hard cap	Charged per search
Web Scraper API (trigger)	N/A	20/min, 60/hr	Async collections
Datasets API	N/A	20/min	Snapshot requests

Step 2: Implement Concurrent Request Limiter

// src/brightdata/limiter.ts
import PQueue from 'p-queue';

// Match concurrency to your Bright Data plan limits
const scrapeQueue = new PQueue({
  concurrency: 10,       // Max concurrent proxy requests
  interval: 1000,        // Per second
  intervalCap: 20,       // Max 20 requests per second
  timeout: 120000,       
  : ,
});

   (): <> {
   scrapeQueue.( () => {
     client = ();
     response =  client.(url);
     response.;
  });
}


scrapeQueue.(,  {
  .();
});