Shopify robots.txt
Shopify generates robots.txt automatically and, for years, wouldn't let you touch it. Since the introduction of robots.txt.liquid, you can customize it — carefully. Here's what Shopify blocks by default and what you should and shouldn't change.
What Shopify blocks by default
Shopify ships a sensible default robots.txt. It disallows checkout, cart, account, and internal search/filter URLs to protect crawl budget and avoid duplicate content, and it declares your sitemap. For most stores, the default is genuinely good — don't change it without a reason.
- Disallows /checkout, /cart, /account, /orders.
- Disallows internal search (/search) and many filtered collection URLs.
- Declares the Shopify-generated sitemap.xml.
Editing with robots.txt.liquid
To customize it, add a template named robots.txt.liquid under your theme (Online Store → Themes → Edit code → Templates → Add a new template → robots.txt). You can append rules or filter the defaults with Liquid. Edits apply to your primary domain's /robots.txt.
{% comment %} Append a rule to Shopify's defaults {% endcomment %}
{{ content_for_header }}
User-agent: AhrefsBot
Disallow: /Edit conservatively
Blocking AI crawlers on Shopify
If you want to keep product copy and content out of AI training, append AI crawler groups in robots.txt.liquid. This is the most common legitimate reason to customize a Shopify robots.txt.
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Google-Extended
Disallow: /Common Shopify mistakes
Unblocking checkout and cart
These pages are noise for crawlers and create duplicate content. Keep Shopify's disallows.
Blocking collection or product URLs you want ranked
Over-eager filtering can hide pages you actually want indexed. Test before shipping.
Editing robots.txt on a non-primary domain
robots.txt applies per host. Make sure you're editing the canonical domain.
Assuming you can't change it at all
Outdated advice — robots.txt.liquid has allowed customization for a while.
Can you edit robots.txt on Shopify?
Yes. Add a robots.txt.liquid template to your theme to customize the generated robots.txt. You can append rules or filter Shopify's defaults with Liquid.
What does Shopify block in robots.txt by default?
Shopify disallows checkout, cart, account, and internal search/filter URLs to save crawl budget and avoid duplicate content, and it declares your sitemap. The defaults are well-tuned for most stores.
How do I block AI crawlers on Shopify?
Add a robots.txt.liquid template and append groups for GPTBot, ClaudeBot, CCBot, and Google-Extended with Disallow: /. This keeps your store content out of AI training without affecting search engines.
Should I remove Shopify's default disallow rules?
Usually not. They prevent crawlers wasting budget on checkout, cart, and filtered URLs. Append your own rules instead of removing Shopify's, unless you have a specific, tested reason.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.