Website Scraper API Map Site API
Lists page URLs by default, with sitemap-discovered scripts, styles, fonts, images, media and downloadable files available behind include_assets so they never consume the URL budget.
- 1 credit per call
- Failed calls are free
- Updated
- web.site.map
What you send
The input
object of the request, with every field the endpoint accepts.
| Field | Type | Required | What it does | Example |
|---|---|---|---|---|
| url | string | yes | Public http(s) site URL to inventory. | https://example.com/ |
| cursor | string | no | ||
| discover_sitemaps | boolean | no | Default: true. |
true |
| exclude_patterns | array<string> | no | ||
| fetch_titles | boolean | no | When true, fetch up to 200 returned page URLs to fill each one's title and description. Adds per-URL requests; off by default so the map stays fast. The seed page title is always returned without extra requests. Default: false. |
|
| include_assets | boolean | no | When false (default) the list holds page URLs only, and scripts, styles, fonts, images, media and downloadable files never consume max_urls. When true, every discovered URL is returned. Default: false. |
|
| include_patterns | array<string> | no | ||
| include_subdomains | boolean | no | When true, hosts remain limited to the seed's registrable domain; unrelated hosts are never added. Default: false. |
|
| max_sitemaps | integer | no | From 1 to 500. Default: 50. |
|
| max_urls | integer | no | Maximum URLs returned in this response page. From 1 to 5000. Default: 1000. |
100 |
| respect_robots | boolean | no | Default: true. |
true |
Call it from your code, or ask your agent
Set UPSCRAPE_API_KEY to your key and copy a working request.
curl --request POST \ --url https://data.upscrape.com/execute \ --header "Authorization: Bearer $UPSCRAPE_API_KEY" \ --header "Content-Type: application/json" \ --header "Prefer: wait=30" \ --data '{ "capability": "web.site.map", "input": { "discover_sitemaps": true, "max_urls": 100, "respect_robots": true, "url": "https://example.com/" }}'
import jsonimport osimport requestspayload = json.loads(r'''{ "capability": "web.site.map", "input": { "discover_sitemaps": true, "max_urls": 100, "respect_robots": true, "url": "https://example.com/" }}''')response = requests.post( "https://data.upscrape.com/execute", headers={ "Authorization": f"Bearer {os.environ['UPSCRAPE_API_KEY']}", "Content-Type": "application/json", "Prefer": "wait=30", }, json=payload,)response.raise_for_status()print(json.dumps(response.json(), indent=2))
const response = await fetch("https://data.upscrape.com/execute", { method: "POST", headers: { "Authorization": `Bearer ${process.env.UPSCRAPE_API_KEY}`, "Content-Type": "application/json", "Prefer": "wait=30", }, body: JSON.stringify({ "capability": "web.site.map", "input": { "discover_sitemaps": true, "max_urls": 100, "respect_robots": true, "url": "https://example.com/" }}),});if (!response.ok) { throw new Error(`Upscrape request failed: ${response.status}`);}console.log(JSON.stringify(await response.json(), null, 2));
https://data.upscrape.com/mcp
With Upscrape, get the Website Scraper API map site for max urls "100" and https://example.com/. Tell me what it costs before you run it, then show the results as a table.
The other website tools
Website Scraper API Map Site API FAQ
The short answers. The docs have the long ones.
How much does the Website Scraper API Map Site API cost?
1 credit per successful call, shown before it runs. You buy credits once, and paid credits never expire.
What do I send?
Send url. The input fields below list every option with its type and limits.
Which sites does it work on?
Any public page a visitor can open without logging in. A page that is blocked or gone fails with its reason.
Can my AI agent call it?
Yes, over MCP: the agent finds web.site.map, checks its price and runs it with the same credits as the REST API.
Your first page is free.
Up to 100 free credits to start, no card needed.