Menu
Scans a website deeply, converting HTML pages to markdown, respecting robots.txt, and updating the snapshots log.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | scanner |
| description | Scans a website deeply, converting HTML pages to markdown, respecting robots.txt, and updating the snapshots log. |
Use this skill to scan and analyze all relevant pages under a target website domain to build or refresh a local customer support corpus.
python skills/scanner/scripts/scan.py <URL> [--force]
| Argument | Description |
|---|---|
<URL> | The start/seed URL of the website to analyze (e.g. https://example.com) |
--force | Force scanning and bypass the 24-hour cache check |
robots.txt before parsing. If restricted, aborts scanning. Make sure to perform scanning only on sites that are allowed..agents/workspace/snapshots.json with mapping and timestamp..agents/workspace/pages/index.md listing and explaining all files in a structured, clickable table format so agents can locate matching topics immediately.Guides the agent on how to handle git operations and generate .patch files instead of submitting PRs.
Guides the agent on how to read and search GitHub issues to understand the repository's problems.
Records and appends every user-agent customer support interaction to the memory.md log in the workspace.
Exposes a 100% local, offline PDF batch extraction utility (extract_to_markdown.py) that isolates invoices under invoices/ and translates PDFs into clean Markdown files for LLM-native parsing.
Reconcile loaded expenses against the pre-parsed invoice database, flagging discrepancies like amount mismatches, missing invoices, and merchant mismatches locally.
Verify whether vendors/merchants from expense records are real, registered business entities by querying the free Wikidata open API.