web-crawler

Automated web crawling + scraping + recon workflow. Trigger: crawl, crawler, scrape, spider, 鐖櫕, 鎶撳彇, web crawl, site crawl, recon crawl.

Jump to install

Repository: FuDie0915/Super-Instruct-Codex-5.6
Last source activity: August 1, 2026 at 09:54
Detected SKILL.md language: English
Stars: 367
Forks: 69

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/FuDie0915/Super-Instruct-Codex-5.6 --skill web-crawler

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

name	web-crawler
description	Automated web crawling + scraping + recon workflow. Trigger: crawl, crawler, scrape, spider, 鐖櫕, 鎶撳彇, web crawl, site crawl, recon crawl.

Web Crawler Workflow

Auto-chain without confirmation:

Phase 1 鈥?Discover: Spider the target domain. Extract all links, forms, endpoints, JS files. Identify: login pages, admin panels, API endpoints, upload forms.

Phase 2 鈥?Fingerprint: Tech stack: server, framework, CMS, JS libraries, CDN. Headers: security headers missing, cookie flags, CORS config.

Phase 3 鈥?Attack surface: skills/web-pentest/scripts/sqli_scanner.py 鈫?all form/query params skills/web-pentest/scripts/xss_fuzzer.py 鈫?all reflected inputs skills/web-pentest/scripts/auth_bypass.py 鈫?login pages, JWT tokens

Phase 4 鈥?Deep scan: Directory brute-force on discovered paths. API endpoint fuzzing for hidden endpoints. File upload testing on discovered upload forms.

Phase 5 鈥?Data extraction: All discovered data written to disk. Structured JSON output: {urls, forms, endpoints, vulns, tech}

Chat shows: [OK] 鈫?$PATH/crawl-results.json

web-crawler

Source facts

Install options

Review the source files

More from this repository

More from this repository

Web Crawler Workflow

Web Crawler Workflow