apify-rate-limits

name	apify-rate-limits
description	Handle Apify API rate limits with proper backoff and request queuing. Use when hitting 429 errors, optimizing API request throughput, or implementing rate-aware client wrappers. Trigger with "apify rate limit", "apify throttling", "apify 429", "apify retry", "apify backoff", "too many requests apify".
allowed-tools	Read, Write, Edit
version	1.5.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","scraping","automation","apify"]
compatibility	Designed for Claude Code

Apify Rate Limits

Overview

The Apify API enforces rate limits per resource. The apify-client library auto-retries 429s (up to 8 times with exponential backoff), so most workloads never notice a limit. You reach for this skill when bulk operations, custom API calls, or large fan-outs push past what the built-in retry can absorb — you then batch, queue, stagger, and monitor to stay under the ceiling.

Full runnable code for every step is in implementation.md; combined scenarios are in examples.md.

Apify rate limit rules

Scope	Limit	Notes
Per resource (default)	60 req/sec	Applies to each Actor, dataset, KV store independently
Dataset push	60 req/sec per dataset	Batch items to reduce call count
Actor runs	60 req/sec per Actor	Start runs in sequence or with delays
Platform-wide	Higher limit	Aggregate across all resources

"Per resource" means: calls to dataset A and dataset B each get 60 req/sec independently. Every response carries X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset (epoch seconds) headers.

Prerequisites

• An Apify account with API access and APIFY_TOKEN set in the environment.
• The apify-client package installed (npm install apify-client).
• For custom queuing: p-queue (npm install p-queue); crawlee for sleep and crawler-level concurrency.

Instructions

The workflow is five steps. Each is summarized here with its core lever; the full runnable code for every step is in implementation.md.

1. Understand built-in retries — apify-client already retries 429/500+ with exponential backoff. Tune maxRetries / minDelayBetweenRetriesMillis only when the defaults are wrong for your endpoint:

   import {  }  ;
    client =  ({
     : process..,
     : ,                      
     : ,  
   });