Skip to main content Startseite Ersteller jeremylongshore tons-of-skills-marketplace firecrawl-advanced-troubleshooting
firecrawl-advanced-troubleshooting Debug hard-to-diagnose Firecrawl issues with systematic isolation and evidence collection.
Use when standard troubleshooting fails, investigating why scrapes return empty content,
crawl jobs hang, or webhooks don't fire.
Trigger with phrases like "firecrawl hard bug", "firecrawl mystery error",
"firecrawl impossible to debug", "firecrawl deep debug", "firecrawl not scraping".
Zur Installation springen Skills Marktplatz Entdecken und erkunden Sie KI-Skills, die von der Community erstellt wurden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Prompt-Details anzeigen Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
npx skills add https://github.com/jeremylongshore/tons-of-skills-marketplace --skill firecrawl-advanced-troubleshootingDer Befehl bleibt in einer Zeile. Scrollen Sie horizontal, um ihn vor dem Kopieren vollständig zu prüfen.
Sie bevorzugen eine lokale Kopie? Laden Sie die Dateien herunter, die SkillsMP derzeit vorliegen.
ZIP herunterladen Herunterladen... Mehr aus diesem Repository langchain-deploy-integration Deploy a LangChain 1.0 / LangGraph 1.0 app to Cloud Run, Vercel, or LangServe correctly — with timeouts sized for chain length, cold-start mitigation, SSE anti-buffering headers, and Secret Manager over .env. Use when prepping a first production deploy, debugging a stream that hangs behind a proxy, or diagnosing p99 latency spikes. Trigger with "langchain deploy", "langchain cloud run", "langchain vercel python", "langchain langserve", or "langchain docker".
langchain-langgraph-agents Build a correct LangGraph 1.0 ReAct agent with create_react_agent — typed tools, error propagation, recursion caps, and stop conditions that actually stop. Use when writing a first tool-calling agent, migrating from AgentExecutor or initialize_agent, or diagnosing an agent that loops on vague prompts. Trigger with "langgraph agent", "create_react_agent", "langgraph tool calling", "AgentExecutor migration", or "agent loop cost".
langchain-langgraph-human-in-loop Build LangGraph 1.0 human-in-the-loop approval flows with interrupt_before /
interrupt_after and Command(resume=...) — JSON-serializable state, clean
resume semantics, and UI wiring for approval decisions. Use when adding an
approval gate before an expensive tool call, wiring a Slack/web UI for agent
approvals, or debugging a graph that crashes on interrupt.
Trigger with "langgraph human in loop", "langgraph interrupt_before",
"langgraph approval flow", "Command resume", "langgraph HITL".
name firecrawl-advanced-troubleshooting description Debug hard-to-diagnose Firecrawl issues with systematic isolation and evidence collection.
Use when standard troubleshooting fails, investigating why scrapes return empty content,
crawl jobs hang, or webhooks don't fire.
Trigger with phrases like "firecrawl hard bug", "firecrawl mystery error",
"firecrawl impossible to debug", "firecrawl deep debug", "firecrawl not scraping".
allowed-tools Read, Grep, Bash(curl:*), Bash(node:*) version 1.11.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","firecrawl","debugging","scaling"] compatibility Designed for Claude Code
Firecrawl Advanced Troubleshooting
Overview
Deep debugging techniques for complex Firecrawl issues: empty scrapes on certain domains, crawl jobs that never complete, inconsistent extraction results, and webhook delivery failures. Uses systematic layer-by-layer isolation.
Instructions
Step 1: Minimal Reproduction
import FirecrawlApp from "@mendable/firecrawl-js" ;
async function minimalRepro (const firecrawl = new FirecrawlApp ({
apiKey : process.env .FIRECRAWL_API_KEY !,
});
console .log ("Test 1: Basic scrape" );
const basic = await firecrawl.scrapeUrl ("https://example.com" , {
formats : ["markdown" ],
});
console .log (` Success: ${basic.success} , Length: ${basic.markdown?.length} ` );
console .log ("Test 2: Target URL" );
const target = await firecrawl.scrapeUrl ("https://YOUR-FAILING-URL.com" , {
formats : ["markdown" ],
});
console .log (` Success: ${target.success} , Length: ${target.markdown?.length} `
console
log
"Test 3: With JS wait"
const
await
scrapeUrl
"https://YOUR-FAILING-URL.com"
formats
"markdown"
waitFor
10000
onlyMainContent
true
console
log
` Success: ${withWait.success} , Length: ${withWait.markdown?.length} `
console
log
"Test 4: With actions"
const
await
scrapeUrl
"https://YOUR-FAILING-URL.com"
formats
"markdown"
"screenshot"
actions
type
"wait"
milliseconds
3000
type
"scroll"
direction
"down"
type
"wait"
milliseconds
2000
console
log
` Success: ${withActions.success} , Length: ${withActions.markdown?.length} `
Step 2: Layer-by-Layer Isolation async function diagnose (url : string const firecrawl = new FirecrawlApp ({ apiKey : process.env .FIRECRAWL_API_KEY ! });
const results : Array <{ test : string ; pass : boolean ; detail : string }> = [];
try {
await firecrawl.scrapeUrl ("https://example.com" , { formats : ["markdown" ] });
results.push ({ test : "API connectivity" , pass : true , detail : "OK" });
} catch (e : any ) {
results.push ({ test : "API connectivity" , pass : false , detail : `${e.statusCode} : ${e.message} ` });
return results;
}
try {
const result = await firecrawl.scrapeUrl (url, { formats : ["markdown" ] });
const hasContent = (result.markdown ?.length || 0 ) > 50 ;
results.push ({
test : "Target scrape" ,
pass : result.success && hasContent,
detail : `Success: ${result.success} , Chars: ${result.markdown?.length} , Status: ${result.metadata?.statusCode} ` ,
});
} catch (e : any ) {
results.push ({ test : "Target scrape" , pass : false , detail : e.message });
}
try {
const result = await firecrawl.scrapeUrl (url, {
formats : ["markdown" , "html" ],
onlyMainContent : true ,
waitFor : 5000 ,
});
const md = result.markdown || "" ;
const isErrorPage = /404|403|access denied|captcha|blocked/i .test (md);
results.push ({
test : "Content quality" ,
pass : md.length > 100 && !isErrorPage,
detail : `Chars: ${md.length} , Error page: ${isErrorPage} , Has headings: ${/^#{1 ,3 }\s/m.test(md)} ` ,
});
} catch (e : any ) {
results.push ({ test : "Content quality" , pass : false , detail : e.message });
}
try {
const map = await firecrawl.mapUrl (url);
results.push ({
test : "Map endpoint" ,
pass : (map.links ?.length || 0 ) > 0 ,
detail : `Found ${map.links?.length} URLs` ,
});
} catch (e : any ) {
results.push ({ test : "Map endpoint" , pass : false , detail : e.message });
}
return results;
}
const results = await diagnose ("https://YOUR-URL.com" );
console .table (results);
Step 3: Debug Empty Scrapes
async function debugEmptyScrape (url : string const firecrawl = new FirecrawlApp ({ apiKey : process.env .FIRECRAWL_API_KEY ! });
const result = await firecrawl.scrapeUrl (url, {
formats : ["markdown" , "html" , "screenshot" ],
waitFor : 10000 ,
});
console .log ("=== Scrape Debug ===" );
console .log (`URL: ${result.metadata?.sourceURL} ` );
console .log (`Status: ${result.metadata?.statusCode} ` );
console .log (`Markdown length: ${result.markdown?.length || 0 } ` );
console .log (`HTML length: ${result.html?.length || 0 } ` );
console .log (`Title: ${result.metadata?.title} ` );
if ((result.html ?.length || 0 ) > 1000 && (result.markdown ?.length || 0 ) < 100 ) {
console .log ("DIAGNOSIS: HTML has content but markdown extraction failed" );
console .log ("FIX: Content may be in iframes or shadow DOM. Try with actions." );
}
if (/captcha|cloudflare|access denied|please verify/i .test (result.html || "" )) {
console .log ("DIAGNOSIS: Bot detection / CAPTCHA detected" );
console .log ("FIX: Site blocks automated scraping. Contact Firecrawl support." );
}
return result;
}
Step 4: Debug Stuck Crawl Jobs async function debugCrawlJob (jobId : string const firecrawl = new FirecrawlApp ({ apiKey : process.env .FIRECRAWL_API_KEY ! });
const status = await firecrawl.checkCrawlStatus (jobId);
console .log ("=== Crawl Job Debug ===" );
console .log (`Status: ${status.status} ` );
console .log (`Completed: ${status.completed} /${status.total} ` );
console .log (`Error: ${status.error || "none" } ` );
if (status.status === "scraping" && status.completed === status.total ) {
console .log ("DIAGNOSIS: All pages scraped but job not marked complete" );
console .log ("FIX: This is a Firecrawl backend issue. Wait or start a new crawl." );
}
if (status.completed === 0 && status.status === "scraping" ) {
console .log ("DIAGNOSIS: Crawl started but no pages scraped" );
console .log ("FIX: Check if start URL returns content. Try scrapeUrl first." );
}
}
Step 5: Timing Analysis async function timeScrape (url : string , iterations = 5 const firecrawl = new FirecrawlApp ({ apiKey : process.env .FIRECRAWL_API_KEY ! });
const times : number [] = [];
for (let i = 0 ; i < iterations; i++) {
const start = Date .now ();
await firecrawl.scrapeUrl (url, { formats : ["markdown" ] });
times.push (Date .now () - start);
}
times.sort ((a, b ) => a - b);
console .log (`p50: ${times[Math .floor(times.length * 0.5 )]} ms` );
console .log (`p95: ${times[Math .floor(times.length * 0.95 )]} ms` );
console .log (`min: ${times[0 ]} ms, max: ${times[times.length - 1 ]} ms` );
}
Error Handling Issue Cause Solution Empty markdown, HTML exists Shadow DOM or iframes Use actions to interact with page Scrape returns CAPTCHA Bot detection Try with mobile: true, contact Firecrawl Crawl stuck at 0 pages Start URL blocked Verify URL loads in browser first Inconsistent results JS rendering timing Increase waitFor, use selector-based wait Webhook never fires URL unreachable Test with curl to your endpoint first
Support Escalation Template Subject: [P1/P2/P3] [Brief description]
URL: [failing URL]
API Key prefix: fc-xxx (first 6 chars)
Timestamp: [ISO 8601]
Expected: [what should happen]
Actual: [what happens]
Diagnostic output: [paste from diagnose() above]
Screenshot: [if available from screenshot format]
Workarounds tried:
1. [what you tried] — result: [outcome]
Resources
Next Steps For load testing, see firecrawl-load-scale.
