MarkUDown
Dataset API — MarkUDown paginated listing scraper docs
POST
/api/datasetDataset
Automatically paginate a listing page and extract every item as a structured dataset. On page 1 it discovers CSS selectors for the items, uses them for fast extraction on the following pages, and falls back to the LLM when they stop matching.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| url | string | required | — | URL of the first page of the listing. |
| goal | string | required | — | Plain-language description of what to extract from each item. |
| schema | object | optional | null | Field definitions for each item, e.g. {"title": "string", "price": "number"}. |
| max_pages | integer | optional | 10 | Maximum pages to paginate through (1–50). The job is charged this many credits when it starts. |
| timeout | integer | optional | 180 | Per-page timeout in seconds. |
| country | string | optional | null | Two-letter uppercase country code (ISO 3166-1), e.g. "BR": the country the pages are loaded from. Without it, the country is inferred from the URL's domain. |
| city | string | optional | null | Lowercase city name with no spaces or separators, e.g. "saopaulo". Requires country. |
| callback_url | string | optional | null | Webhook URL to POST the result to when the job completes. |
Request
{
"url": "https://books.toscrape.com",
"goal": "Extract all books with title, price and rating",
"schema": {
"title": "string",
"price": "string",
"rating": "string"
},
"max_pages": 50
}Response
{
"success": true,
"id": "job_abc123",
"url": "/api/dataset/job_abc123"
}
// Poll GET /api/dataset/{id} — "data" when completed:
{
"success": true,
"url": "https://books.toscrape.com",
"goal": "Extract all books...",
"total_records": 1000,
"pages_scraped": 50,
"output_format": "json",
"data": [
{ "title": "A Light in the Attic", "price": "£51.77", "rating": "Three" }
],
"processing_time_ms": 124300
}