How to Build an E-Commerce Price Monitor with Web Scraping

Ecommerce price monitoring is one of the most practical applications of web scraping. Whether you're an online retailer tracking competitors, a consumer hunting for deals, or a marketplace operator ensuring pricing integrity, automated price tracking delivers real business value. This tutorial walks you through building a working price monitor — from a single product check to a full multi-site scheduled system.

Why Price Monitoring Matters

In competitive markets, pricing can change multiple times per day. Amazon updates prices on millions of products continuously. Shopify stores run flash sales. Airline and hotel prices fluctuate by the hour. Manual monitoring is impossible at any meaningful scale.

Automated price monitoring enables:

Competitive repricing — automatically adjust your own prices to match or beat competitors
MAP compliance — detect resellers violating minimum advertised price agreements
Deal alerts — notify customers when a product they want drops below a threshold
Market research — track pricing trends over time for category analysis
Procurement optimization — buy from suppliers at the right time based on price history

What Data to Track

A useful price monitor captures more than just the current price:

Current price — the actual selling price right now
Original/list price — to calculate discount percentage
Stock status — in stock, low stock, out of stock
Variant pricing — different prices for sizes/colors/configurations
Shipping cost — total cost is what matters, not just product price
Seller — for marketplace sites (Amazon, eBay) the featured seller matters
Timestamp — when you checked, for historical trending

Step 1: Check a Single Product Price

const API_KEY = process.env.PAPALILY_API_KEY;

async function checkPrice(productUrl) {
  const res = await fetch('https://api.papalily.com/scrape', {
    method: 'POST',
    headers: {
      'x-api-key': API_KEY,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      url: productUrl,
      prompt: 'Extract the product name, current price, original price if discounted, stock status (in stock / out of stock / low stock), and currency symbol.',
      no_cache: true,  // Always get fresh prices
    }),
  });

  const result = await res.json();

  return {
    ...result.data,
    url: productUrl,
    checked_at: new Date().toISOString(),
  };
}

// Test it
const price = await checkPrice('https://www.amazon.com/dp/B0CX5SH23T');
console.log(price);
// {
//   product_name: 'Sony WH-1000XM6',
//   current_price: '$329.99',
//   original_price: '$399.99',
//   stock_status: 'In Stock',
//   currency: '$',
//   url: 'https://...',
//   checked_at: '2026-03-17T09:00:00.000Z'
// }

Step 2: Monitor Multiple Products (Batch API)

For monitoring multiple products, use Papalily's batch endpoint to check them in parallel. This reduces total time significantly — 10 products checked in parallel takes the same time as checking 1 product.

const PRODUCTS = [
  { name: 'Sony WH-1000XM6',  url: 'https://www.amazon.com/dp/B0CX5SH23T' },
  { name: 'Bose QC Ultra',     url: 'https://www.amazon.com/dp/B0CCZ7XN8S' },
  { name: 'Apple AirPods Max', url: 'https://www.amazon.com/dp/B08PZHYVJS' },
];

async function batchPriceCheck(products) {
  const res = await fetch('https://api.papalily.com/batch', {
    method: 'POST',
    headers: {
      'x-api-key': process.env.PAPALILY_API_KEY,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      requests: products.map(p => ({
        url: p.url,
        prompt: 'Extract product name, current price, original price, and stock status.',
        no_cache: true,
      })),
    }),
  });

  const { results } = await res.json();
  const timestamp = new Date().toISOString();

  return results.map((r, i) => ({
    ...products[i],
    ...r.data,
    success: r.success,
    checked_at: timestamp,
  }));
}

const results = await batchPriceCheck(PRODUCTS);
console.log(JSON.stringify(results, null, 2));

Step 3: Store Results and Detect Price Drops

const fs = require('fs');
const DB_FILE = 'price-history.json';

function loadHistory() {
  if (!fs.existsSync(DB_FILE)) return {};
  return JSON.parse(fs.readFileSync(DB_FILE, 'utf8'));
}

function saveHistory(history) {
  fs.writeFileSync(DB_FILE, JSON.stringify(history, null, 2));
}

function parsePrice(priceStr) {
  return parseFloat(String(priceStr).replace(/[^0-9.]/g, ''));
}

async function runPriceMonitor() {
  const history = loadHistory();
  const current = await batchPriceCheck(PRODUCTS);  // from above
  const alerts = [];

  for (const item of current) {
    const key = item.url;
    const newPrice = parsePrice(item.current_price);

    if (!history[key]) {
      history[key] = { prices: [] };
    }

    const lastPrice = history[key].prices.at(-1)?.price;

    // Detect price drop
    if (lastPrice && newPrice < lastPrice) {
      const drop = (((lastPrice - newPrice) / lastPrice) * 100).toFixed(1);
      alerts.push({
        name: item.name,
        oldPrice: lastPrice,
        newPrice,
        dropPct: drop,
        url: item.url,
      });
    }

    // Store new price
    history[key].prices.push({ price: newPrice, at: item.checked_at });
    // Keep last 90 days of data
    if (history[key].prices.length > 90) history[key].prices.shift();
  }

  saveHistory(history);
  return { current, alerts };
}

const { current, alerts } = await runPriceMonitor();
if (alerts.length > 0) {
  console.log('📉 Price drops detected!');
  alerts.forEach(a =>
    console.log(`${a.name}: $${a.oldPrice} → $${a.newPrice} (-${a.dropPct}%)`)
  );
}

Step 4: Schedule It with Cron

Run your monitor on a schedule. On Linux/Mac, add a cron job. On Windows, use Task Scheduler. For simplicity, here's a Node.js version using node-cron:

const cron = require('node-cron');

// Run every 6 hours
cron.schedule('0 */6 * * *', async () => {
  console.log(`[${new Date().toISOString()}] Running price check...`);
  const { current, alerts } = await runPriceMonitor();

  if (alerts.length > 0) {
    await sendSlackAlert(alerts);  // See below
  }

  console.log(`Checked ${current.length} products. ${alerts.length} price drops found.`);
});

console.log('Price monitor started. Checking every 6 hours.');

Step 5: Slack Alerts for Price Drops

async function sendSlackAlert(alerts) {
  const blocks = alerts.map(a => ({
    type: 'section',
    text: {
      type: 'mrkdwn',
      text: `*📉 Price Drop: ${a.name}*\n$${a.oldPrice} → *$${a.newPrice}* (-${a.dropPct}%)\n<${a.url}|View product>`,
    },
  }));

  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ blocks }),
  });
}

Scaling Up

Once your basic monitor is working, here's how to scale:

More products: Papalily's batch API handles up to 20 URLs per call, all running in parallel. For hundreds of products, split into batches of 20 with a small delay between batches.
Database instead of JSON file: Switch from a flat JSON file to SQLite (easy, local) or PostgreSQL (for multi-server or multi-user scenarios). Use the same price schema — just write to a price_history table instead of a file.
Multiple competitors: Track the same product across multiple stores by including multiple URLs for the same item (Amazon, Walmart, Best Buy, direct brand site).
Rate your own price: Compare scraped competitor prices against your own pricing API to trigger automatic repricing rules.

Build Your Price Monitor Today

Papalily's AI extraction works on any e-commerce site — Amazon, Shopify, WooCommerce, custom-built stores. Get your free API key and have your first price check running in minutes.

Get Free API Key on RapidAPI →

Full docs at papalily.com/docs