MarkUDown

Batch Scrape API — MarkUDown bulk web scraping API docs

POST/api/batch-scrape

Batch Scrape

Scrape multiple URLs in parallel in a single request. More efficient than issuing separate scrape calls.

ParameterTypeRequiredDefaultDescription
urlarray<string>required—List of URLs to scrape. Each URL costs 1 credit.
main_contentbooleanoptionaltrueExtract only the main content (drops navigation, header and footer).
include_linkbooleanoptionalfalseInclude the links found on the page.
include_htmlbooleanoptionalfalseAlso return the page HTML alongside the Markdown.
exclude_tagsarray<string>optional["header","nav","footer"]CSS selectors or tag names to strip, e.g. ["nav",".ads","footer"].
timeoutintegeroptional60Per-page timeout in seconds.

Request

{
  "url": [
    "https://example.com/page1",
    "https://example.com/page2",
    "https://example.com/page3"
  ],
  "main_content": true
}

Response

// POST returns { "success": true, "id": "abc123", "url": "/api/batch-scrape/abc123" }
// Poll GET /api/batch-scrape/{id} — "data" when completed:
{
  "success": true,
  "total": 3,
  "completed": 3,
  "failed": 0,
  "data": [
    {
      "url": "https://example.com/page1",
      "markdown": "# Page 1\n\nContent...",
      "metadata": { "title": "Page 1", "statusCode": 200 }
    }
  ]
}