| name | tiny-web-crawler |
| description | Crawl from one or more starting web pages, fetch readable content, search within pages, follow relevant links, and stop when the requested information is found or a bounded limit is reached. |
| metadata | {"author":"Louis Grenard <louis@getleon.ai>","version":"1.0.0"} |
Tiny Web Crawler
Use this skill to inspect web pages by fetching content, searching within it, and following relevant links from the starting page.
Scripts
Use the bundled scripts for the actual web fetching and bounded crawling:
scripts/fetch-page.mjs: fetch one page, extract compact readable text, links, and query snippets.
scripts/crawl-web.mjs: crawl from one or more start URLs, follow relevant links, and stop at limits or strong matches.
Run scripts with node. Leon's shell tool injects runtime shims, so node
uses Leon's managed Node.js binary when available and falls back to PATH only
when the managed binary is missing.
Example:
node scripts/crawl-web.mjs --url "https://example.com" --query "target phrase" --max-pages 8 --max-depth 2
Do not build $LEON_HOME/bin/node/... or /bin/node/... paths manually.
Workflow
- Clarify the target only when the requested information or starting point is ambiguous.
- Start from the owner-provided URL when one is given.
- Use
scripts/crawl-web.mjs to fetch pages, search within content, and follow relevant links.