web-scraping

name	web-scraping
description	scrape websites, extract data, crawl pages, and browse the web at scale using kernel cloud browsers with stealth mode and proxies.

web scraping with kernel

kernel cloud browsers let you scrape at scale with stealth mode, residential proxies, and up to 72-hour sessions. no charges for idle time.

stealth mode

stealth mode automatically adds a recaptcha solver and residential proxy to your browsers. use it for any site with bot detection.

const browser = await kernel.browsers.create({ stealth: true });

proxies

proxy quality for bot detection avoidance, best to worst:

• mobile — highest quality, most expensive
• residential — good for most sites (included with stealth mode)
• ISP — dedicated IPs, good balance
• datacenter — cheapest, most likely to be blocked

scraping patterns

extract structured data

await page.goto("https://example.com/products");
await page.waitForSelector(".product-card");

const products = await page.$$eval(".product-card", cards =>
  cards.map( ({
    : card.()?.?.(),
    : card.()?.?.(),
    : card.()?.,
  }))
);

 products;