PlaybookMarkUDown
MarkUDownPython电商Webhooks中级

实时竞争对手价格监控

两个 MarkUDown 端点,无本地调度器,无需管理浏览器。/api/monitor 创建持久订阅,当内容变化时触发 webhook。/api/extract 通过 LLM 将价格提取为类型化数字——与页面格式无关。

2026年4月30日15 分钟阅读作者:Scrape Technology

场景

价格是电商中最敏感的变量。5% 的差距已经足以将客户推向竞争对手。周五晚上的限时促销可能在你意识到发生了什么之前就已经耗尽整个周末的需求。

问题不是缺乏数据。价格就在那里——公开的,可见的,在竞争对手的网站上。问题是速度。如果没有专门的团队或昂贵的 SaaS 工具,你无法持续监控多个竞争对手的数十种产品。

痛点

经典方法:

  • 每周一早上有人打开电子表格
  • 手动进入每个竞争对手网站
  • 记录价格,比较,更新表格
  • 两个小时的重复工作——其余六天是盲的

竞争对手不等周一。他们在周四下午、周六早上改变价格。在那些时间窗口,你是暴露的。每笔失去的销售都是不可见的——你不知道你没看到什么。

解决方案

两个端点。无需配置浏览器。无本地调度器。

/api/monitor — 持久订阅

一次注册每个竞争对手 URL。MarkUDown 按你配置的间隔检查该页面,当检测到内容变化时向你的服务器发送 webhook。无 cron job。无后台服务器。

/api/extract — 价格作为类型化数字

当监控器检测到变化时,你调用 /api/extract 获取结构化字段。LLM 规范化任何格式——"R$ 699,90"、"699.90"、"6x of 116.65"——并将 current_price 作为 float 返回,可直接比较。

Abrasio 已包含——无需配置

Abrasio——我们的具有 Chromium 和人类指纹的反检测浏览器——是 MarkUDown 的第三个内部层。如果更快的层被竞争对手网站封锁,MarkUDown 会自动升级。你无需配置任何内容。

运作方式

register.py → POST /api/monitor para cada URL (rode uma vez)

↓

MarkUDown → verifica cada URL no intervalo configurado

↓ (quando detecta mudança)

POST /webhook → seu servidor recebe a notificação

↓

webhook.py → POST /api/extract para extrair o preço

↓

Comparação → detecta queda acima do threshold

↓

Slack → alerta com produto, concorrente e preços

教程

你需要一个 MarkUDown API key 和一个有公共 URL 的服务器来接收 webhook。

1

创建账户并获取 API key

前往 MarkUDown 控制台,创建免费账户并复制你的 API key。它以 header X-API-KEY 的形式用于所有调用。

2

安装依赖

terminal
pip install requests python-dotenv fastapi uvicorn

配置 .env:

.env
# .env
MARKUDOWN_API_KEY=sua_chave_aqui
WEBHOOK_URL=https://seu-servidor.com/webhook
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/XXX/YYY/ZZZ
PRICE_DROP_THRESHOLD=0.05
INTERVAL_MINUTES=60

公共 Webhook URL

WEBHOOK_URL 需要从互联网可访问——这是 MarkUDown 发送变化通知的地方。在开发中,使用 ngrok http 8090 来公开你的本地服务器。
3

配置产品

products.json 定义要监控哪些产品以及每个竞争对手的 URL。你可以拥有任意数量的产品和竞争对手。

products.json
[
  {
    "id": "nike-air-max-270",
    "name": "Nike Air Max 270",
    "our_price": 699.90,
    "competitors": [
      {
        "store": "Concorrente A",
        "url": "https://concorrente-a.com.br/nike-air-max-270"
      },
      {
        "store": "Concorrente B",
        "url": "https://concorrente-b.com.br/tenis/nike-air-max-270"
      }
    ]
  },
  {
    "id": "adidas-ultraboost-22",
    "name": "Adidas Ultraboost 22",
    "our_price": 849.90,
    "competitors": [
      {
        "store": "Concorrente A",
        "url": "https://concorrente-a.com.br/adidas-ultraboost-22"
      }
    ]
  }
]
4

定义提取 schema

schema 告诉 /api/extract 要提取哪些字段以及什么类型。LLM 规范化任何价格格式并返回已类型化的值——无需正则表达式,无需手动解析器。

schema
PRICE_SCHEMA = [
    {"name": "product_name",        "type": "String", "active": True},
    {"name": "current_price",       "type": "Number", "active": True},
    {"name": "original_price",      "type": "Number", "active": True},
    {"name": "discount_percentage", "type": "Number", "active": True},
    {"name": "availability",        "type": "String", "active": True},
]
5

注册监控器(运行一次)

register.py 为每个竞争对手 URL 调用 POST /api/monitor,并将 subscription_id 保存到 subscriptions.json。你只需运行一次——MarkUDown 从那时起负责调度。

register.py
"""
register.py — Registra um monitor por URL de concorrente via /api/monitor.
Execute uma vez. O MarkUDown cuida do agendamento — sem scheduler local.
"""
import json
import os
import requests
from pathlib import Path
from dotenv import load_dotenv

load_dotenv()

MARKUDOWN_URL = "https://api.scrapetechnology.com"
API_KEY = os.getenv("MARKUDOWN_API_KEY")
WEBHOOK_URL = os.getenv("WEBHOOK_URL")
INTERVAL_MINUTES = int(os.getenv("INTERVAL_MINUTES", "60"))

PRODUCTS_FILE = Path("products.json")
SUBSCRIPTIONS_FILE = Path("subscriptions.json")


def register_monitors():
    products = json.loads(PRODUCTS_FILE.read_text(encoding="utf-8"))
    subscriptions = {}

    for product in products:
        for competitor in product["competitors"]:
            url = competitor["url"]
            store = competitor["store"]

            print(f"Registrando monitor: {store} — {url}")

            resp = requests.post(
                f"{MARKUDOWN_URL}/api/monitor",
                headers={"X-API-KEY": API_KEY},
                json={
                    "url": url,
                    "callback_url": WEBHOOK_URL,
                    "interval_minutes": INTERVAL_MINUTES,
                    "main_content": True,
                },
                timeout=30,
            )
            resp.raise_for_status()
            data = resp.json()
            sub_id = data["subscription_id"]

            subscriptions[sub_id] = {
                "product_id": product["id"],
                "product_name": product["name"],
                "our_price": product["our_price"],
                "store": store,
                "url": url,
            }
            print(f"  OK — subscription_id: {sub_id}")

    SUBSCRIPTIONS_FILE.write_text(
        json.dumps(subscriptions, ensure_ascii=False, indent=2), encoding="utf-8"
    )
    print(f"\n{len(subscriptions)} monitors registrados em subscriptions.json")


if __name__ == "__main__":
    register_monitors()

无本地调度器

运行 register.py 后,你不需要为监控检查维护任何后台进程。MarkUDown 在配置的间隔验证 URL,当某些内容发生变化时调用你的 webhook。
6

Webhook 服务器

webhook.py 是一个 FastAPI 服务器,接收 MarkUDown 通知。当 payload 中有 changed: true 时,它调用 /api/extract 获取当前价格,与历史记录比较,如果降幅超过配置的阈值则发送 Slack 警报。

webhook.py
"""
webhook.py — Recebe notificações do MarkUDown e alerta quando preço cai.
Execute: uvicorn webhook:app --host 0.0.0.0 --port 8090
"""
import json
import os
import time
from datetime import datetime
from pathlib import Path

import requests
from fastapi import FastAPI, Request
from dotenv import load_dotenv

load_dotenv()

MARKUDOWN_URL = "https://api.scrapetechnology.com"
API_KEY = os.getenv("MARKUDOWN_API_KEY")
SLACK_WEBHOOK_URL = os.getenv("SLACK_WEBHOOK_URL")
PRICE_DROP_THRESHOLD = float(os.getenv("PRICE_DROP_THRESHOLD", "0.05"))

SUBSCRIPTIONS_FILE = Path("subscriptions.json")
PRICES_DB_FILE = Path("prices_db.json")

PRICE_SCHEMA = [
    {"name": "product_name",        "type": "String", "active": True},
    {"name": "current_price",       "type": "Number", "active": True},
    {"name": "original_price",      "type": "Number", "active": True},
    {"name": "discount_percentage", "type": "Number", "active": True},
    {"name": "availability",        "type": "String", "active": True},
]

app = FastAPI()


def load_json(path: Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8")) if path.exists() else {}


def save_json(path: Path, data: dict) -> None:
    path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")


def extract_price(url: str) -> dict | None:
    """Chama /api/extract e aguarda o job completar."""
    resp = requests.post(
        f"{MARKUDOWN_URL}/api/extract",
        headers={"X-API-KEY": API_KEY},
        json={
            "url": url,
            "extract_query": "nome do produto, preco atual, preco original, desconto e disponibilidade",
            "schema": PRICE_SCHEMA,
            "extraction_scope": "single_page",
        },
        timeout=30,
    )
    resp.raise_for_status()
    job_id = resp.json()["job_id"]

    for _ in range(40):
        time.sleep(3)
        result = requests.get(
            f"{MARKUDOWN_URL}/api/extract/{job_id}",
            headers={"X-API-KEY": API_KEY},
            timeout=10,
        ).json()
        if result.get("status") == "completed":
            items = result.get("items", [])
            if items:
                data = items[0].get("extracted_data", [{}])
                return data[0] if data else None
    return None


def send_slack_alert(meta: dict, current_price: float, previous_price: float) -> None:
    if not SLACK_WEBHOOK_URL:
        return
    drop_pct = round((previous_price - current_price) / previous_price * 100, 1)
    our_price = meta["our_price"]
    gap_pct = round((our_price - current_price) / our_price * 100, 1)
    lines = [
        ":rotating_light: *Queda de preco detectada!*",
        f"*Produto:* {meta['product_name']}",
        f"*Concorrente:* {meta['store']}",
        f"*Preco anterior:* R$ {previous_price:.2f}",
        f"*Novo preco:* R$ {current_price:.2f} ({drop_pct}% de queda)",
        f"*Nosso preco:* R$ {our_price:.2f} ({abs(gap_pct)}% {'acima' if gap_pct > 0 else 'abaixo'})",
        f"*Link:* {meta['url']}",
    ]
    requests.post(SLACK_WEBHOOK_URL, json={"text": "\n".join(lines)}, timeout=10)


@app.post("/webhook")
async def handle_monitor_event(request: Request):
    payload = await request.json()

    if not payload.get("changed"):
        return {"ok": True}

    subscription_id = payload.get("subscription_id")
    url = payload.get("url")

    subscriptions = load_json(SUBSCRIPTIONS_FILE)
    meta = subscriptions.get(subscription_id)
    if not meta:
        return {"ok": True}

    print(f"Mudanca detectada: {meta['store']} — {url}")

    extracted = extract_price(url)
    if not extracted:
        return {"ok": True}

    current_price = extracted.get("current_price")
    if not current_price:
        return {"ok": True}

    print(f"  Preco atual: R$ {current_price:.2f}")

    db = load_json(PRICES_DB_FILE)
    key = f"{meta['product_id']}::{meta['store']}"
    previous_price = db.get(key, {}).get("current_price")

    if previous_price and previous_price > 0:
        change_ratio = (previous_price - current_price) / previous_price
        if change_ratio >= PRICE_DROP_THRESHOLD:
            print(f"  ALERTA: R$ {previous_price:.2f} -> R$ {current_price:.2f}")
            send_slack_alert(meta, current_price, previous_price)

    db[key] = {
        **extracted,
        "url": url,
        "store": meta["store"],
        "product_id": meta["product_id"],
        "checked_at": datetime.now().isoformat(),
    }
    save_json(PRICES_DB_FILE, db)
    return {"ok": True}
7

运行

terminal
# 1. Instalar dependências
pip install -r requirements.txt

# 2. Registrar os monitors (rode uma vez)
python register.py

# 3. Subir o servidor de webhook
uvicorn webhook:app --host 0.0.0.0 --port 8090

就绪

MarkUDown 开始按配置的间隔检查 URL。当检测到变化时,调用 /webhook。服务器提取价格并发出警报。

替代方案:使用 /api/extract 轮询

如果你没有带公共 URL 的服务器来接收 webhook,可以直接在轮询模式下使用 /api/extract——每小时检查每个 URL 的本地调度器。效率较低(即使没有变化也会提取),但无需额外基础设施即可工作。

poll_monitor.py
"""
poll_monitor.py — Alternativa sem webhook: polling com /api/extract.
Útil quando você não tem um servidor com URL pública.
Execute: python poll_monitor.py --once  (teste)
         python poll_monitor.py          (roda a cada hora)
"""
import json, os, time, sys, requests
from datetime import datetime
from pathlib import Path
from dotenv import load_dotenv

load_dotenv()

MARKUDOWN_URL = "https://api.scrapetechnology.com"
API_KEY = os.getenv("MARKUDOWN_API_KEY")
SLACK_WEBHOOK_URL = os.getenv("SLACK_WEBHOOK_URL")
PRICE_DROP_THRESHOLD = float(os.getenv("PRICE_DROP_THRESHOLD", "0.05"))

PRODUCTS_FILE = Path("products.json")
PRICES_DB_FILE = Path("prices_db.json")

PRICE_SCHEMA = [
    {"name": "product_name",        "type": "String", "active": True},
    {"name": "current_price",       "type": "Number", "active": True},
    {"name": "original_price",      "type": "Number", "active": True},
    {"name": "discount_percentage", "type": "Number", "active": True},
    {"name": "availability",        "type": "String", "active": True},
]

def load_json(path):
    return json.loads(path.read_text(encoding="utf-8")) if path.exists() else {}

def save_json(path, data):
    path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")

def extract_price(url):
    resp = requests.post(
        f"{MARKUDOWN_URL}/api/extract",
        headers={"X-API-KEY": API_KEY},
        json={
            "url": url,
            "extract_query": "nome do produto, preco atual, preco original, desconto e disponibilidade",
            "schema": PRICE_SCHEMA,
            "extraction_scope": "single_page",
        },
        timeout=30,
    )
    resp.raise_for_status()
    job_id = resp.json()["job_id"]
    for _ in range(40):
        time.sleep(3)
        result = requests.get(
            f"{MARKUDOWN_URL}/api/extract/{job_id}",
            headers={"X-API-KEY": API_KEY},
        ).json()
        if result.get("status") == "completed":
            items = result.get("items", [])
            if items:
                data = items[0].get("extracted_data", [{}])
                return data[0] if data else None
    return None

def run():
    print(f"Monitor: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
    products = load_json(PRODUCTS_FILE)
    db = load_json(PRICES_DB_FILE)
    for product in products:
        print(f"Verificando: {product['name']}")
        for comp in product["competitors"]:
            print(f"  {comp['store']}...", end=" ", flush=True)
            extracted = extract_price(comp["url"])
            if not extracted:
                print("falhou")
                continue
            current = extracted.get("current_price")
            if not current:
                print("sem preco")
                continue
            print(f"R$ {current:.2f}")
            key = f"{product['id']}::{comp['store']}"
            previous = db.get(key, {}).get("current_price")
            if previous and (previous - current) / previous >= PRICE_DROP_THRESHOLD:
                print(f"  ALERTA: {previous:.2f} -> {current:.2f}")
            db[key] = {**extracted, "url": comp["url"], "store": comp["store"],
                       "checked_at": datetime.now().isoformat()}
    save_json(PRICES_DB_FILE, db)
    print("Ciclo concluido.")

if __name__ == "__main__":
    if "--once" in sys.argv:
        run()
    else:
        from apscheduler.schedulers.blocking import BlockingScheduler
        s = BlockingScheduler()
        s.add_job(run, "interval", hours=1, next_run_time=datetime.now())
        print("Monitor ativo. Ctrl+C para parar.")
        try:
            s.start()
        except KeyboardInterrupt:
            pass

Slack 警报

当检测到超过阈值的降价时,会收到这样的消息:

Slack — #价格监控

🚨 Queda de preço detectada!

Produto: Nike Air Max 270

Concorrente: Concorrente A

Preço anterior: R$ 749,90

Novo preço: R$ 599,90 (20,0% de queda)

Nosso preço: R$ 699,90 (14,3% acima)

Link: concorrente-a.com.br/...

下一步

完整历史

将 prices_db.json 替换为 PostgreSQL,以存储完整历史记录并绘制随时间变化的价格变化图表。

实时控制台

Next.js 页面,包含按产品和竞争对手分类的价格比较表,直接从数据库获取数据。

类别监控

使用 /api/map 发现竞争对手类别中的所有产品,并为每个 URL 自动注册监控器。

取消非活跃监控器

调用 DELETE /api/monitor/{subscription_id} 取消已停产产品的订阅,避免在服务器上留下垃圾。

立即开始使用 MarkUDown

创建你的免费账户,配置产品,监控器将在 30 分钟内激活。