| name | web-site |
| description | Discover or crawl a known public site; optionally select Tavily or Firecrawl. |
Web Site
Operate on a known site through the configured provider registries. For ambiguous web intent, apply workflows/web-capability-routing.md. Select one explicit mode:
map: discover URLs and structure through registry/web-map-providers.yaml.
crawl: collect many pages through registry/web-crawl-providers.yaml.
Default to map when the exact page is unknown. Use crawl only when multi-page content is requested. Preserve crawl limits, include/exclude paths, depth, and actual provider in output. Do not silently substitute a user-requested provider.
Run
python3 <installed-arkspace-path>/scripts/arkspace.py site map --provider firecrawl https://docs.example.com --output json
python3 <installed-arkspace-path>/scripts/arkspace.py site crawl --provider firecrawl https://docs.example.com/docs --output json
For setup or a failed provider check, use provider-manager.