Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Google Search, Google Trends, Reddit, Airbnb, Yelp, and 15+ more platforms. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, review analysis, SEO intelligence, recruitment, or any data extraction task.
Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, LinkedIn, X/Twitter, Google Maps, Google Search, Google Trends, Reddit, Airbnb, Yelp, and 15+ more platforms. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, review analysis, SEO intelligence, recruitment, or any data extraction task.
Universal web scraper
AI-driven data extraction from ~100 Actors across 15+ platforms via the Apify CLI.
Rule: Pass --json and redirect stderr with 2>/dev/null on data-returning commands (actors call, actors start, actors info, actors search, datasets get-items, runs info). JSON output is stable across CLI versions. stderr contains progress messages and version warnings that break JSON parsers if not redirected.
This rule does not apply to status/auth commands (apify info, apify --version, apify login). For those, use 2>&1 so authentication and version errors are visible.
Exception: if --input returns no data, re-run with 2>&1 to confirm whether the cause is a missing schema vs. a network/auth error.
Prerequisites
Apify CLI v1.4.0+ (npm install -g apify-cli)
Authenticated session (see below)
Authentication
If a CLI command fails with an auth error, authenticate using one of these methods:
Identify the target platform and use case. Read references/actor-index.md to find the right Actor.
Prefer apify-tier actors; use community-tier only when no apify actor covers the task.
For input schemas, fetch dynamically: apify actors info "ACTOR_ID" --input --json 2>/dev/null
If the output is empty, re-run without the redirect (2>&1) to surface auth or network errors before proceeding.
If the task involves a multi-step pipeline, also read the matching workflow guide:
From results: items[].username/items[].name (Actor ID), items[].title, items[].stats.totalUsers30Days, items[].currentPricingInfo.pricingModel.
Step 2: Fetch Actor schema and check gotchas
Some Actors don't register an input schema with the platform (their schema lives in code). Try schema sources in this order — fall through on empty/error:
Input schema (human-readable):
apify actors info "ACTOR_ID" --input 2>/dev/null
If output is Error: No input schema found for this Actor, skip to source 2.
Grep the README for an "Input" / "Example input" section to copy the JSON shape.
Last resort — call with minimal known input (e.g. {"startUrls":[{"url":"..."}]} for crawlers) and let the Actor surface validation errors that reveal required fields. See references/gotchas.md for known-good minimal inputs for common Actors.
Also read references/gotchas.md to check for common pitfalls and cost guardrails for the selected Actor.
Step 3: Configure and run
Skip user preferences for simple lookups (e.g., "Nike's follower count"). Go straight to running with quick answer mode.
For larger tasks, confirm output format (quick answer / CSV / JSON) and result count.
Before starting the run, double-check whether the task is short enough for a blocking call or should use the async pattern from Step 0.
