| name | site-map |
| description | Discover all URLs on a website without downloading content via the Tavily MCP. Use when the user wants to find a specific page on a large site, list all pages, see a site's structure, or says "map this site", "what pages exist on", "find the page about X on this site", "list all URLs", or "find the URL for". Faster than crawling — returns URLs only. Pair with extract-page once the right URL is identified. For bulk content extraction, use site-crawl. |
Site Map
Call tavily_map to discover URLs on a site without downloading the body
of any page. Faster and cheaper than tavily_crawl because it skips
extraction.
When to use this skill
- The user knows the site but not the exact page ("there's a pricing page
on acme.com somewhere — find it").
- You need to enumerate URLs before deciding which ones to extract.
- The user wants a high-level view of how a site is structured.
For bulk content extraction in one call, use site-crawl — that's map +
extract combined.
How to call the tool
Pass these arguments to tavily_map:
| Argument | When to set it |
|---|
url | Required. Root URL to map from. |
max_depth | How many link-hops from the root. Default 1. Bump cautiously — depth 3 on a docs site can return thousands of URLs. |
max_breadth | URLs per level. Default 20. |
limit | Total URL cap. Default 50. Raise for big sites. |
instructions | Natural-language hint of what to look for. E.g. "product pages", "pricing-related URLs", "blog posts about AI". Tavily uses this to bias which links it follows. |
select_paths | Regex include patterns for URLs. |
exclude_paths | Regex exclude patterns. E.g. exclude /tag/, /author/, archives. |
select_domains | Restrict to specific subdomains. |
allow_external | Bool. Follow links to external domains. Default false. |
Patterns
- Find a specific page: pass
instructions describing what you want.
Cheaper than crawling the whole site.
- Survey a docs site: depth 2, instructions
"documentation pages",
exclude /tag/.
- Blog inventory: depth 1 from
/blog/, exclude /author/, /tag/,
/page/.
- Subdomain mapping: leave defaults but set
select_domains to
["docs.example.com"].
Returning results
Default to a categorized list:
Marketing pages: 12
Docs pages: 87
Blog posts: 43
Other: 8
Then surface the 5–10 most relevant URLs based on the user's intent. Offer
to extract any of them with extract-page, or to crawl a section with
site-crawl.
Failure modes
- Too many URLs: tighten with
select_paths / exclude_paths, lower
limit, or pass a more targeted instructions.
- Too few URLs: raise
max_depth, raise limit, drop overly-narrow
filters, or check that url is reachable.
- JS-rendered nav: some sites lazy-load nav links. Map will miss them
— if you suspect this, fall back to a
tavily_search with
include_domains set to the site.
