MarkUDown

异步任务 — MarkUDown 网页抓取 API 文档

异步任务

长时间运行的操作以异步方式处理。API 立即返回 id;轮询状态端点直到任务完成。

提交异步任务

POST /api/crawl
X-API-KEY: your-api-key

{
  "url": ["https://docs.example.com/"],
  "max_depth": 3,
  "limit": 100
}

// Response
{
  "success": true,
  "id": "abc123",
  "url": "/api/crawl/abc123"
}

轮询结果

GET /api/crawl/abc123
X-API-KEY: your-api-key

// Waiting / running ("progress" appears on some jobs):
{ "success": false, "id": "abc123", "status": "pending" }
{ "success": false, "id": "abc123", "status": "processing", "progress": 50 }

// When complete:
{
  "success": true,
  "id": "abc123",
  "status": "completed",
  "data": {
    "success": true,
    "total": 42,
    "data": [...]
  }
}

// On failure:
{ "success": false, "id": "abc123", "status": "failed", "error": "job failed: ..." }

// Result no longer kept (see below):
{ "success": false, "id": "abc123", "status": "not_found" }
// A job id that is not yours: HTTP 404 { "detail": "Job not found" }
支持的异步端点:
crawlmapextractprompt-extractsearchrssbatch-scrapechange-detectiondeep-researchagentsmart-extractrankdataset

Prompt Extract 任务在 /api/extract/{id} 轮询。已完成的结果只保留有限时间(最多 1–2 小时,负载高时更短)——请及时读取或设置 callback_url;之后状态会变为 not_found。只有提交任务的 API 密钥才能读取该任务。

取消任务

Crawl、Extract、Agent、Smart Extract 和 Dataset 任务可以对同一 URL 发送 DELETE 来取消。

DELETE /api/crawl/abc123
X-API-KEY: your-api-key

{ "success": true, "id": "abc123", "status": "cancelled" }
// or, when the job can no longer be removed:
{ "success": false, "id": "abc123", "status": "not_cancellable" }

Webhook(callback_url)

支持 callback_url 的端点会在任务结束时向该地址 POST 以下 JSON。该 URL 必须是公网可访问的 http(s) 地址。

{
  "event": "completed",          // or "failed"
  "queue": "crawl",
  "jobId": "abc123",
  "data": { ... },               // same as "data" when polling; "error" instead on failure
  "timestamp": "2026-11-16T14:30:00.000Z"
}