torch

Reverse-engineer a website's internal APIs, encrypted endpoints, WebSocket streams, and obfuscated JavaScript. Activates when the target data isn't in the HTML, when the site uses encrypted CloudFront/CDN payloads, when real-time streaming data is needed, or when the scrape skill's Phase 0-1 finds API calls that are encrypted, signed, or behind a custom protocol. Escalation ladder from simple network capture to protobuf schema reconstruction.

Scrape a website. Activates when user provides a URL to scrape, asks to extract data from a site, hits 403/blocking errors, or needs anti-bot evasion.

Give AI agents their own email inboxes using the AgentMail API. Use when building email agents, sending/receiving emails programmatically, managing inboxes, handling attachments, organizing with labels, creating drafts for human approval, or setting up real-time notifications via webhooks/websockets. Supports multi-tenant isolation with pods.

Use Camoufox — a Firefox fork with C++-level fingerprint spoofing — for browser scraping when the user's real Chrome profile is not available. Activate on VMs, headless CI servers, remote machines without a GUI, or when torch needs multiple concurrent personas with rotated fingerprints. Detects via TORCH_CAMOUFOX_ENDPOINT env var. Drives from Node via playwright-core connecting to a Camoufox-hosted Playwright server.

Proven scraping playbook for amazon.com search result pages (/s?k=...). CloudFront + CAPTCHA wall blocks bare curl with HTTP 503, but a real Chrome session via the real Chrome debug port walks right through — no stealth, no proxy, no captcha. HTML is server-rendered; cheerio parses 22 results per page with stable `[data-component-type="s-search-result"]` blocks. Activate for any amazon.com /s search URL.

Proven scraping playbook for booking.com searchresults.html pages. CloudFront-fronted JS challenge blocks bare curl (HTTP 202 with a script-only interstitial), but a real Chrome session via the real Chrome debug port walks through on first navigation — no captcha, no proxy, no login. Listings are rendered client-side into [data-testid="property-card"] cards; pagination uses a "Load more results" button after the first 25. Prices only populate when the URL carries checkin/checkout/group_adults params. Activate for any booking.com /searchresults.html scrape.

Proven scraping playbook for costco.com category listings (e.g. /laptops.html, /computers.html). Akamai Bot Manager hard-blocks bare curl with HTTP 403 on the HTML pages, but Costco's internal catalog API at gdx-api.costco.com/catalog/search/api/v1/search is a public JSON POST endpoint with no auth, no cookies, and no anti-bot. Skip the browser entirely. Activate for any costco.com category listing scrape.

Proven scraping playbook for digikey.com /en/products/category/ listings. Cloudflare challenge (cf-mitigated: challenge) blocks bare curl with HTTP 403, but a puppeteer connection to the user's real Chrome via the real Chrome debug port walks straight through with zero stealth, no captcha, no proxy, no auth. Listings are server-rendered into a single `<table>` whose rows contain `a[href*="/en/products/detail/"]`; every parametric column (Core Processor, Speed, RAM, package, etc.) is just a `<th>`/`<td>` pair you can zip. Activate for any digikey.com /en/products/category/ scrape.

Proven scraping playbook for ebay.com search result pages (/sch/i.html?_nkw=...). eBay gates first requests with a "Pardon Our Interruption" splash challenge, but it clears automatically in a real Chrome session via the real Chrome debug port — no 2Captcha, no proxies, no login needed. Activate for any ebay.com /sch/ target.

Proven scraping playbook for etsy.com search result pages (/search?q=...). DataDome protects the HTML edge (bare curl gets HTTP 403 with `x-datadome: protected` and a JS captcha shell), but a puppeteer-core connection to the user's real Chrome via `127.0.0.1:9222` walks through on first navigation — no 2Captcha, no proxy, no login needed. HTML is server-rendered; cheerio parses ~60 `div.listing-link[data-listing-id]` cards per page with stable selectors. Activate for any etsy.com /search target.

Proven scraping playbook for homedepot.com search results (/s/<query>). Akamai Bot Manager hard-blocks raw curl with HTTP 403 (AkamaiGHost), but a puppeteer connection to the user's real Chrome via the real Chrome debug port walks straight through with zero challenges — no stealth plugin, no proxy, no captcha. Products are server-rendered in the HTML as `[data-testid="product-pod"]` blocks (each pod appears twice in the DOM, dedupe by product ID). Activate for any homedepot.com /s/ search scrape.

Proven scraping playbook for ikea.com /cat/ category pages (PLP). Server-rendered HTML with 24 products per page embedded as `[data-testid="plp-product-card"]` blocks, each carrying `data-ref-id`, `data-product-name`, `data-price`, `data-currency` attributes — no JSON blob needed, cheerio parses them straight. Cloudflare fronts the site but does NOT block bare curl for HTML (200 OK, no challenge); however the CDN caches `?page=N` aggressively and can return page 1 content to anonymous curl even for `?page=2`. Fix: drive pagination through a real Chrome via the real Chrome debug port — it bypasses the stale CDN cache and each `?page=N` navigation returns its own SSR batch of 24. Activate for any ikea.com /cat/ URL.

Proven scraping playbook for imdb.com. Next.js SPA behind CloudFront + AWS WAF (x-amzn-waf-action challenge on raw curl). Real Chrome via the real Chrome debug port sails past the challenge on first navigation — no captcha, no proxy. The clean path is __NEXT_DATA__ JSON on chart/list pages. Activate for any imdb.com target.

Proven scraping playbook for producthunt.com homepage. Cloudflare-protected Next.js-ish SPA (no __NEXT_DATA__), but a puppeteer.connect() to the user's real Chrome via the real Chrome debug port walks straight through the CF challenge with zero stealth plugins. All homepage posts are server-rendered into the HTML as `section[data-test^="post-item-"]` blocks, grouped under `[data-test="homepage-section-{today|yesterday|last-week|last-month}"]`. Activate for any producthunt.com homepage scrape.

Proven scraping playbook for stackoverflow.com tag/question listing pages (e.g. /questions/tagged/<tag>). Server-rendered HTML, but Cloudflare flags raw curl/node-fetch with HTTP 403 after the very first request. Connecting to the user's real Chrome via the real Chrome debug port walks straight through with zero challenges — no stealth plugin, no proxy, no captcha. All questions are embedded in `.s-post-summary` blocks. Activate for any stackoverflow.com listing scrape.

Proven scraping playbook for stockx.com listing pages (e.g. /sneakers/most-active, /sneakers/release-date, category leaderboards). Cloudflare hard-blocks bare curl with `cf-mitigated: challenge` + HTTP 403, but a puppeteer connection to the user's real Chrome via the real Chrome debug port walks straight through with zero challenges — no stealth plugin, no proxy, no captcha. Products are server-rendered as `[data-testid="productTile"]` blocks; `__NEXT_DATA__` exists but does NOT contain product data (hydrated client-side). Activate for any stockx.com listing scrape.

Proven scraping playbook for target.com. Skip the browser entirely — replay the internal redsky.target.com plp_search_v2 endpoint directly with fetch. Public API, no cookies, no auth, no anti-bot (HumanSecurity/PerimeterX is only enforced on www.target.com HTML, not on redsky). Activate for any target.com search (/s?searchTerm=) or category (/c/) listing.

Proven scraping playbook for walmart.com. Next.js SPA behind Akamai Bot Manager — curl is 307-redirected to /blocked, but a puppeteer connection to the user's real Chrome (`127.0.0.1:9222`) walks straight through with zero challenges. All product data is embedded in `__NEXT_DATA__`; no API replay needed. Activate for any walmart.com search, browse, or PDP scrape.

Proven scraping playbook for yelp.com search result pages (/search?find_desc=...&find_loc=...). Yelp is behind DataDome — bare curl gets HTTP 403 with `server: DataDome` and a captcha-delivery interstitial. A real Chrome session via the real Chrome debug port walks straight through after the first navigation (no captcha solver, no proxy needed) because the persistent `datadome` cookie is already trusted on the user's profile. HTML is server-rendered; cheerio parses 10 organic results per page. Activate for any yelp.com /search target.

Proven scraping playbook for zillow.com /homes/for_sale/ search pages. CloudFront + PerimeterX (HumanSecurity) gates raw curl with HTTP 403 `x-px-blocked: 1`. A puppeteer.connect() to the user's real Chrome via the real Chrome debug port walks through the challenge — but only if you (a) clear poisoned cookies first and (b) warm up via Google referer. All listings ship in `__NEXT_DATA__`. Activate for any zillow.com search/listing scrape.

Solve CAPTCHAs using the 2Captcha API — reCAPTCHA v2/v3, Cloudflare Turnstile, hCaptcha, image captchas, Geetest, FunCaptcha. Use when a scraper hits a visible CAPTCHA that stealth/headed mode cannot bypass (Layer 5+ in anti-blocking escalation). Requires TWOCAPTCHA_API_KEY env var. Human-backed workers, ~$1/1000 solves, 20-40s latency.

Solve CAPTCHAs using CapMonster Cloud — reCAPTCHA v2/v3, Cloudflare Turnstile (including cf_clearance cookies), GeeTest v3/v4, FunCaptcha, DataDome, AWS WAF, Imperva. AI-based solver, ~$0.60/1K reCAPTCHA (cheaper than 2Captcha), 10-30s latency. Use for cost-sensitive scraping or when 2Captcha fails. Requires CAPMONSTER_API_KEY env var.

How to contribute to torch — the PR workflow, quality bar, skill authoring conventions, and what not to contribute. Activate when the user asks "how do I contribute / upstream / share / PR this", wants to submit a new skill, or needs the contribution guidelines for any kind of change to the repo. Covers site skills, core skills, code changes, and documentation.

Use authenticated residential, datacenter, or mobile proxies with Puppeteer to bypass IP-based rate limiting, geo-blocks, and bot-detection. Covers Bright Data, Oxylabs, Smartproxy, and generic proxy-chain usage. Use when getting 403/429 from the target's IP reputation system, when stealth/CAPTCHA solving alone isn't enough, or when you need geo-targeted exit nodes.

Proven scraping playbook for airbnb.com search result pages (/s/<location>/homes). Server-streamed React app — listing data is in a JSON blob inside `<script id="data-deferred-state-0">` (or `data-injector-instances` on cached responses). No anti-bot, no captcha, no proxy, no browser needed. Plain Node `fetch` + a UA header works. Pagination is a flat `?cursor=<base64>` param taken from `paginationInfo.pageCursors`. Activate for any airbnb.com /s/ search URL.

Proven scraping playbook for apple.com /shop/buy-* configurator pages (Mac, iPhone, iPad, Watch). Server-rendered HTML with all product, pricing and configuration data embedded as JS object literals on `window.PRODUCT_SELECTION_BOOTSTRAP` (and friends). Plain fetch — no anti-bot, no browser, no auth — but the blob is a JS object literal, not strict JSON, so you must brace-balance. Activate for any apple.com /shop/buy-mac, /shop/buy-iphone, /shop/buy-ipad, /shop/buy-watch URL.

Proven scraping playbook for arxiv.org listing pages (e.g. /list/<category>/recent). Plain server-rendered HTML, no anti-bot, no JS needed — just fetch + cheerio. Use `?show=2000` to get every entry on a single page. Activate for any arxiv.org /list/ or /abs/ URL.

Proven scraping playbook for doordash.com. React SPA behind Cloudflare with server-rendered deal carousels on /browse/deals. Cloudflare challenge clears automatically with puppeteer-extra stealth (no captcha, no proxies, no login needed). Activate for any doordash.com target. Covers carousel selectors, price-token parsing, geo-fenced deals, and known virtualization pitfalls.

Proven scraping playbook for github.com. Server-rendered HTML, no anti-bot on public pages (trending, repos, users). Plain fetch + cheerio works — no browser, no headers, no auth. Activate for any github.com scrape that doesn't need the REST/GraphQL API.

Proven scraping playbook for news.ycombinator.com. Server-rendered static HTML (Arc/CL app), no CDN, no anti-bot, no JS required. Plain fetch + cheerio parses the front page in milliseconds. Activate for any news.ycombinator.com target. Covers front-page selectors, pagination, and the official Firebase API alternative.

Proven scraping playbook for huggingface.co model/dataset/space listings. Skip the browser entirely — Hugging Face exposes a public, unauthenticated JSON API at /api/models, /api/datasets, /api/spaces that mirrors every filter and sort on the HTML listing pages. No cookies, no tokens, no anti-bot. Cursor pagination via the `Link: rel="next"` header. Activate for any huggingface.co /models, /datasets, or /spaces listing URL.

Proven scraping playbook for nike.com. Skip the browser entirely — replay the internal api.nike.com product_wall endpoint directly with fetch. Public API, no cookies, no tokens, no anti-bot. Requires the nike-api-caller-id header and count=24 per page. Activate for any nike.com gridwall (/w/ URLs). Covers endpoint template, pagination, response shape, and known API quirks.

Proven scraping playbook for pypi.org. The /search HTML page is behind a FullStory client challenge (_fs-ch-*), but PyPI publishes an official RSS feed for newest packages at /rss/packages.xml with zero anti-bot. Use the RSS feed (or the JSON API) instead of scraping /search. Activate for any pypi.org target.

Proven scraping playbook for reddit.com. Skip the browser entirely — Reddit exposes a public JSON API by appending `.json` to any listing URL (subreddit, user, comments, search). No auth, no cookies, no anti-bot beyond a User-Agent check. Activate for any reddit.com listing or thread URL.

Proven scraping playbook for redfin.com listings. The HTML pages are CloudFront-blocked (403) on bare curl, but Redfin exposes a public undocumented endpoint at /stingray/api/gis-csv that returns the entire region's listings as CSV with no auth, no cookies, no anti-bot. Skip the browser entirely. Activate for any redfin.com /city/, /zipcode/, /county/, or /neighborhood/ listing scrape.

Proven scraping playbook for ubereats.com category pages (/category/<city>/<cuisine>). Fully server-rendered HTML behind Cloudflare, no anti-bot challenge on curl — plain GET returns every store card in the markup. No browser, no stealth, no proxy, no auth. One gotcha — Uber ships a 40KB+ CSP header that overflows Node's built-in fetch (undici HeadersOverflowError), so shell out to curl. Activate for any ubereats.com /category/ scrape.

Proven scraping playbook for en.wikipedia.org (and other language wikis). Server-rendered static HTML, no anti-bot, no JS required. Plain fetch + cheerio works. One important gotcha — since ~2023, section headings are wrapped in `<div class="mw-heading mw-heading2">` so `h2` is no longer a direct child of `.mw-parser-output`. Activate for any wikipedia.org article URL. Prefer the REST/Action API for structured data.