Block GPTBot in robots.txt
GPTBot is OpenAI's web crawler. It gathers content to train OpenAI's models. If you don't want your pages used for training, two lines of robots.txt stop it — without touching your Google ranking.
What GPTBot does
GPTBot crawls publicly accessible pages and feeds them into the training data for OpenAI's foundation models. It is a training crawler only — it is not the bot that fetches live pages for ChatGPT Search (that's OAI-SearchBot) and it is not a search engine. Blocking GPTBot keeps your content out of future model training but does not remove you from any search results.
| Property | Value |
|---|---|
| User-agent | GPTBot |
| Operator | OpenAI |
| Purpose | Model training data collection |
| Honors robots.txt | Yes (per OpenAI docs) |
| Affects Google ranking | No |
Why site owners care
- Content licensing — publishers who sell or license content often block training crawlers by default.
- Competitive risk — you may not want your proprietary writing absorbed into a model.
- Bandwidth — large crawls cost server resources on big sites.
- It's free and reversible — a robots.txt rule costs nothing and can be removed any time.
How to block GPTBot
Add a group targeting the GPTBot user-agent with Disallow: / to block your whole site:
User-agent: GPTBot
Disallow: /To block only part of your site, list specific paths instead:
User-agent: GPTBot
Disallow: /premium/
Disallow: /members/To explicitly allow GPTBot (the default if you say nothing), use an empty Disallow:
User-agent: GPTBot
Disallow:GPTBot vs OAI-SearchBot
OpenAI runs two distinct crawlers. GPTBot is for training; OAI-SearchBot fetches pages to cite in ChatGPT Search and can send you referral traffic. Blocking one does not block the other, so decide them separately.
# Opt out of training, stay eligible for ChatGPT Search
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow:Common mistakes
Blocking Googlebot by accident
GPTBot is not Googlebot. Make sure your Disallow: / sits under User-agent: GPTBot, not User-agent: *.
Assuming it deindexes content
Blocking GPTBot stops training, not indexing. It has no effect on whether you appear in any search engine.
Typos in the token
It must be exactly GPTBot. GPT-Bot, ChatGPT, or OpenAI won't match.
Forgetting OAI-SearchBot
If your goal is to stay out of all OpenAI products, you also need to decide on OAI-SearchBot.
How do I block GPTBot?
Add User-agent: GPTBot followed by Disallow: / to your robots.txt. That blocks GPTBot from your entire site. OpenAI documents that GPTBot honors robots.txt.
Does blocking GPTBot hurt my SEO?
No. GPTBot is OpenAI's training crawler and is completely separate from Googlebot and Bingbot. Blocking it has zero effect on your search rankings or indexing.
What is the difference between GPTBot and OAI-SearchBot?
GPTBot collects training data for OpenAI's models. OAI-SearchBot fetches pages to show and cite in ChatGPT Search. They are separate user-agents, so you can block training while still allowing ChatGPT Search.
Does GPTBot obey robots.txt?
Yes. OpenAI states that GPTBot respects robots.txt directives. To confirm your rule is working on a live site, run it through the Analyzer.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.