Google-Extended in robots.txt
Google-Extended is not a crawler — it's an opt-out token. It controls whether Google may use your content to train Gemini and ground AI features. Blocking it has zero effect on your Google Search ranking, because Googlebot is separate.
What Google-Extended is
Google-Extended is a robots.txt user-agent token Google introduced so publishers can opt out of having their content used for AI training and grounding (Gemini, Vertex AI) while remaining fully crawlable for Google Search. There is no separate bot called Google-Extended fetching your pages — Googlebot still does the crawling. The token simply governs how the already-crawled content may be used.
This is the safest AI opt-out
Google-Extended vs Googlebot
| Token | Controls | Affects Search ranking? |
|---|---|---|
Googlebot | Crawling for Google Search | Yes — don't block it |
Google-Extended | Use of content for Gemini / AI training | No — safe to block |
Never block Googlebot to stop AI
How to block Google-Extended
Opt out of Google AI training while staying in Search:
User-agent: Google-Extended
Disallow: /
User-agent: Googlebot
Allow: /To allow it (the default), use an empty Disallow:
User-agent: Google-Extended
Disallow:Common mistakes
Blocking Googlebot instead
The classic error: Disallow: / under Googlebot deindexes your site. Use Google-Extended for the AI opt-out.
Expecting a crawl change
You won't see a new bot in your logs — Google-Extended is a policy token, not a separate crawler.
Assuming it covers other engines
Google-Extended only governs Google. Bing, OpenAI, and Anthropic need their own rules.
What does Google-Extended do?
It lets you opt out of Google using your content to train its AI models (Gemini) and ground AI features. It does not affect crawling for Google Search, which is handled by Googlebot.
Does blocking Google-Extended hurt my Google ranking?
No. Google-Extended is completely separate from Googlebot. Blocking it stops AI training use only and has no effect on how you rank or whether you're indexed in Google Search.
How do I block Google-Extended?
Add User-agent: Google-Extended followed by Disallow: / to robots.txt. Keep Googlebot allowed so you remain in Google Search.
Is Google-Extended a crawler?
No. It's an opt-out token, not a bot. Googlebot still crawls your pages; Google-Extended only controls whether that content may be used for AI training and grounding.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.