Guides · landing_page_crawl_error
Product page could not be crawled
What Google's message means
Google reached the server but could not crawl the page — most often robots.txt or a firewall denying Googlebot, or a page that needs JavaScript that never finished rendering.
Same as an unavailable page: the product is disapproved until Google can read it, and earns nothing meanwhile.
The fix in Shopify
- Online Store → Themes → Edit code →
robots.txt.liquid(only if it exists): delete any Disallow line covering/products/. Save. - Online Store → Preferences → Password protection: untick ‘Restrict access to visitors with the password’ — a password page blocks Google from every product.
- Settings → Markets: the country Google shows the product in must be in an active market, or Shopify redirects Google's crawler away from the product page.
- Paste the product URL in Search Console → URL Inspection → Test live URL: it names the exact blocker (robots.txt, 403, timeout).
- Google re-checks the product at its next crawl (usually within a day); the next RevenueCue scan confirms it's clear.
The fix in WooCommerce
- Yoast SEO → Tools → File editor (or Rank Math → General Settings → Edit robots.txt): delete any Disallow line covering
/product/or/wp-content/uploads/. Save. - Settings → Reading: untick ‘Discourage search engines from indexing this site’.
- If a security plugin or Cloudflare sits in front of the store, allow Google's crawlers: Wordfence → Firewall → Rate Limiting → ‘How should we treat Google's crawlers’ = Verified Google crawlers have unlimited access; Cloudflare → Security → Bots → Bot Fight Mode off.
- Paste the product URL in Search Console → URL Inspection → Test live URL: it names the exact blocker (robots.txt, 403, timeout).
- Google re-checks the product at its next crawl (usually within a day); the next RevenueCue scan confirms it's clear.
What it costs while it's down
Every product Google flags this way is out of Google Shopping — no ads, no free listing — until it's fixed. Its listed price is what isn't being shown; a free scan names each product and adds up the listed value at stake.
See which of your products are out — free
Enter your store's domain. The scan reads your public catalogue the way Googlebot does, names every product behind each leak and shows the listed value affected.
No card, no signup. Built for Shopify and WooCommerce.