MarkUDown

Dataset API — MarkUDown paginated listing scraper docs

POST/api/dataset

Dataset

Automatically paginate a listing page and extract every item as a structured dataset. On page 1 it discovers CSS selectors for the items, uses them for fast extraction on the following pages, and falls back to the LLM when they stop matching.

ParameterTypeRequiredDefaultDescription
urlstringrequired—URL of the first page of the listing.
goalstringrequired—Plain-language description of what to extract from each item.
schemaobjectoptionalnullField definitions for each item, e.g. {"title": "string", "price": "number"}.
max_pagesintegeroptional10Maximum pages to paginate through (1–50). The job is charged this many credits when it starts.
timeoutintegeroptional180Per-page timeout in seconds.
countrystringoptionalnullTwo-letter uppercase country code (ISO 3166-1), e.g. "BR": the country the pages are loaded from. Without it, the country is inferred from the URL's domain.
citystringoptionalnullLowercase city name with no spaces or separators, e.g. "saopaulo". Requires country.
callback_urlstringoptionalnullWebhook URL to POST the result to when the job completes.

Request

{
  "url": "https://books.toscrape.com",
  "goal": "Extract all books with title, price and rating",
  "schema": {
    "title": "string",
    "price": "string",
    "rating": "string"
  },
  "max_pages": 50
}

Response

{
  "success": true,
  "id": "job_abc123",
  "url": "/api/dataset/job_abc123"
}

// Poll GET /api/dataset/{id} — "data" when completed:
{
  "success": true,
  "url": "https://books.toscrape.com",
  "goal": "Extract all books...",
  "total_records": 1000,
  "pages_scraped": 50,
  "output_format": "json",
  "data": [
    { "title": "A Light in the Attic", "price": "£51.77", "rating": "Three" }
  ],
  "processing_time_ms": 124300
}