fetch-firecrawl

name	fetch-firecrawl
description	Use to scrape a URL into clean Markdown when the page is JS-rendered, behind anti-bot, or a PDF — and FIRECRAWL_API_KEY is available. Degrades gracefully to warn when key is missing.
version	1
languages	["en","zh","mixed"]
methods	[{"id":"scrape","impl":"methods/scrape.py","requires":["env:FIRECRAWL_API_KEY"]}]
fallback_chain	["scrape"]
when_to_use	{"query_languages":["en","zh","mixed"],"query_types":["web-fetch","url-scrape","deep-read"],"avoid_for":["keyword-search","batch-crawl"]}
quality_hint	{"typical_yield":"high","chinese_native":false}
layer	leaf
domains	["generic-web"]
scenarios	["url-to-markdown","js-rendered","anti-bot","pdf-scrape"]
model_tier	Fast
auth_required	true
cost	paid

Overview

Firecrawl turns any URL into clean Markdown via a managed browser cluster. Handles JS rendering, anti-bot defenses, and PDFs. Paid per-page; requires FIRECRAWL_API_KEY. When the key is missing, doctor() reports warn.

When to Choose It

• Page is heavily JS-rendered (SPA, React, Next.js)
• Target site blocks fetch-jina or fetch-crawl4ai
• PDF or document conversion needed

How To Use

Call via run_channel("fetch-firecrawl", url) where the query IS the URL to scrape. Returns one Evidence item with body = full Markdown content.

Quality Bar

• Evidence items have non-empty title and url.
• No crash on empty or malformed API response.
• Source channel field matches the channel name.