Universal Web Scraper scraper API

Universal browser-first page capture and schema-first extraction.

Web & maps 3 capabilities stable 38.2% success rate

Capabilities

Every capability is one POST /execute call that returns structured JSON.

Archive Page


            web.page.archive

Archive one public webpage into durable offline artifacts for saved-page collections. Returns best-effort self-contained HTML and ZIP snapshots with CSS/images/fonts/media/scripts rewritten or packed, plus optional PDF when a Chrome/Chromium backend is configured.

Request

curl -X POST https://data.upscrape.com/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":{"formats":["single_html","zip"],"include_scripts":false,"url":"https://example.com/"},"capability":"web.page.archive"}'

Input

Field	Type	Required	Description
`formats`	array<string>	optional	Archive artifact formats to return. single_html is a self-contained HTML snapshot, zip contains index.html plus local assets, and pdf requires a configured Chrome/Chromium backend. If a requested single_html artifact is too large for the worker-result budget and zip was not requested, the module may return a zip fallback.
`include_scripts`	boolean	optional	Preserve external and inline scripts. Defaults to false because archived arbitrary JavaScript should only be replayed in a sandboxed viewer.
`max_asset_bytes`	integer	optional	Requested maximum bytes to download for a single CSS/image/font/script/media asset. The module may clamp this lower to keep the worker result under platform size limits.
`max_total_asset_bytes`	integer	optional	Requested maximum bytes to download across all archived assets. The module may clamp this lower to keep the worker result under platform size limits.
`url`	string	required	Public http(s) URL to archive.

Output

Field	Type	Required	Description
`artifacts`	array<object>	required	Archive artifacts. Text artifacts use encoding=utf-8 and binary artifacts use encoding=base64.
`assets`	array<object>	required	Downloaded, inlined, skipped, or locally packed resources used by the archive artifacts.
`capture_backend`	string	required
`captured_at`	string	required
`content_type`	string	required
`final_url`	string	required
`page`	object	required
`rendered`	boolean	required
`stats`	object	required
`status_code`	integer	required
`url`	string	required
`warnings`	array<string>	required

Capture Page


            web.page.capture

Capture one public webpage as a browser-rendered, domain-neutral artifact graph: DOM, text, elements, URLs, links, images, media, resources, structured data, forms, tables, and frames.

Request

curl -X POST https://data.upscrape.com/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":{"url":"https://example.com/"},"capability":"web.page.capture"}'

Input

Field	Type	Required	Description
`detail`	string	optional	How much of the artifact graph to return. summary: metadata, counts, and warnings. standard: everything except raw HTML and the per-element dump. full: the complete graph.
`url`	string	required	Public http(s) URL to capture.

Output

Field	Type	Required
`artifacts`	object	required
`capture_backend`	string	required
`content_type`	string	required
`dom`	object	required
`elements`	array<object>	required
`final_url`	string	required
`forms`	array<object>	required
`headers`	object	required
`iframes`	array<object>	required
`images`	array<object>	required
`links`	array<object>	required
`media`	array<object>	required
`page`	object	required
`resources`	array<object>	required
`stats`	object	required
`status_code`	integer	required
`structured_data`	object	required
`tables`	array<object>	required
`text`	object	required
`url`	string	required
`urls`	array<object>	required
`warnings`	array<string>	required

Extract Page


            web.page.extract

Capture one public webpage and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction (metadata, evidence graph, filtered URL enumeration) runs first; internal AI resolves remaining fields per the ai mode (never/auto/always), grounded to captured page content.

Request

curl -X POST https://data.upscrape.com/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":{"ai":"never","fields":{"canonical_url":"canonical url of the page","description":"short page description","title":"page title"},"url":"https://example.com/"},"capability":"web.page.extract"}'

Input

Field	Type	Required	Description
`ai`	string	optional	Extraction mode. never: deterministic only. auto: deterministic first, internal AI only for unresolved fields (degrades gracefully when AI is unavailable). always: deterministic plus AI. Defaults to auto when fields is used, otherwise never.
`ai_enabled`	boolean	optional	Deprecated alias for ai: always. Prefer the ai parameter.
`fields`	object	optional	Shorthand alternative to output_schema: field name mapped to a natural-language description of what to extract. Compiled into a schema internally. Provide exactly one of output_schema or fields.
`instructions`	string	optional	Optional extraction guidance. Do not include secrets.
`output_schema`	object	optional	JSON Schema object describing the desired data shape. Property descriptions double as per-field extraction hints. Constraints such as items.pattern filter deterministic URL enumeration. Provide exactly one of output_schema or fields.
`url`	string	required	Public http(s) URL to extract from.

Output

Field	Type	Required	Description
`artifacts`	object	optional	Optional legacy compatibility field. New extract responses normally omit artifacts; use web.page.capture for full page artifacts.
`data`	object	required
`evidence`	array<object>	required
`extractor_version`	string	required
`fields`	object	required
`schema_valid`	boolean	required
`stats`	object	required
`unresolved_fields`	array<string>	optional	Schema fields no extraction pass could resolve. Agents can retry these with ai: always.
`validation_errors`	array<object>	required
`warnings`	array<string>	required

Run Universal Web Scraper in under a minute

Get an API key