Map Url Discovery

Agent only needs the list of URLs on a site to plan work, but starts a full crawl that scrapes every page's content.

Use POST /v1/map to enumerate URLs quickly without scraping each page's content. Reserve /v1/crawl for when page bodies are actually needed. Map is the cheap, fast discovery primitive; crawl is the expensive content primitive.

Agent maps example.com but needs docs.example.com and blog.example.com URLs too, which are on subdomains.

Set includeSubdomains=true when subdomain coverage is required; otherwise map stays on the apex/host. Confirm the returned links span the intended subdomains before assuming full coverage.

Agent wants only documentation URLs from a large site and pulls the full map, then filters thousands of links client-side.

Pass the map search parameter (e.g. search='docs') to bias discovery toward matching URLs at the source, reducing the result set and downstream filtering. Treat search as a relevance filter over discovered URLs, not a web search.