spider-rs

Use when fetching or scraping the contents of a web page or URL, especially when a plain HTTP request returns a block page, a CAPTCHA, a Cloudflare challenge, an empty shell, or a 403/429 — gottem routes the request through a tiered ladder of scraping vendors (and a local browser), lowest-cost-first, escalating until it gets clean content. Also use to compare scraping vendors, force a specific provider, or race/hedge providers for latency.

Browser automation SDK for Spider's cloud browser fleet. Use when building web scrapers, browser automation, AI-powered browsing, or any task requiring headless browser control with stealth, retry, and CAPTCHA solving.

Use Spider Rust CLI for crawling, scraping, and extraction tasks from the terminal. Trigger this skill when a user asks to run or compose `spider` CLI commands, tune crawl/scrape options, export links or HTML, or control runtime browser mode with `--headless` or `--http`.