How to Scrape Supplier Websites for WooCommerce When They Don’t Have an API

You finally negotiated a great deal with a new supplier, but when you ask for their product feed, they reply: “We don’t have an API, just copy the products from our website.”

Manually copying and pasting thousands of titles, descriptions, and SKUs is out of the question. Generic scraping software (like browser extensions) often breaks, requires your computer to stay on, and struggles to import complex variable products. To automate this reliably, you need a server-side PHP scraper.

1. Why Generic Scraping Tools Fail

Desktop scraping software and generic WordPress import plugins try to solve every problem at once. They rely on visual DOM selection, which breaks the moment your supplier redesigns a button or adds a promotional banner.

Furthermore, running a desktop scraper means you have to manually export a CSV, clean it, and run a separate WooCommerce import tool every time prices change. It is a fragile, multi-step process.

2. Building a Server-Side PHP Scraper

A custom PHP scraper runs directly on your server, acting as an invisible bridge between the supplier’s website and your WooCommerce database. It uses cURL to fetch the raw HTML of the supplier’s catalog, and DOMDocument (or a library like Guzzle/Symfony DomCrawler) to extract the exact data points.

$html = file_get_contents('https://supplier.com/product/123');
$dom = new DOMDocument();
@$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$price = $xpath->query('//span[@class="product-price"]')->item(0)->nodeValue;

Because this code is hardcoded to the supplier’s specific HTML structure, it is lightning-fast and bypasses the overhead of visual interfaces.

3. Bypassing IP Bans and Rate Limits

If your script tries to download 5,000 products in one minute, the supplier’s firewall (like Cloudflare) will instantly ban your server’s IP address.

The Fix: A professional scraping script must respect server limits. We implement sleep() delays between requests, rotate user-agent headers to mimic real browsers, and utilize proxy networks if the supplier has aggressive anti-bot protections.

4. Automating the Sync

Once the script successfully extracts the data, it bypasses CSV files entirely. It hooks directly into the WooCommerce database via PHP, executing wp_insert_post() and update_post_meta() to create the product instantly. Hook this script to a daily server cron job, and your inventory syncs automatically while you sleep.