OAI-SearchBot in robots.txt
OAI-SearchBot is OpenAI's search crawler. It fetches pages to show and cite them in ChatGPT Search — it does not collect training data. Allowing it can send you referral traffic; blocking it keeps you out of ChatGPT's web answers. Decide it separately from GPTBot.
What OAI-SearchBot does
When ChatGPT searches the web, OAI-SearchBot is the crawler that discovers and fetches the pages ChatGPT links and cites. Pages it can access are eligible to appear — with attribution and a link — in ChatGPT Search results. OpenAI documents that it is not used to train models: training collection is GPTBot's job, under a separate user-agent you control independently.
| Property | Value |
|---|---|
| User-agent | OAI-SearchBot |
| Operator | OpenAI |
| Purpose | Fetch and cite pages for ChatGPT Search |
| Trains AI models | No (that's GPTBot) |
| Honors robots.txt | Yes (per OpenAI docs) |
| Affects Google ranking | No |
Source: OpenAI's crawler documentation, which lists every OpenAI user-agent and what it is used for.
Should you allow or block it?
This is a traffic trade-off, not a content-protection question. Unlike training crawlers, OAI-SearchBot's output links back to you.
- Allow it if you want citations and referral clicks from ChatGPT Search — most publishers and businesses do.
- Block it if you don't want your pages quoted in AI answers at all, even with attribution.
- Blocking GPTBot does NOT block OAI-SearchBot — they are separate user-agents and separate decisions.
Example robots.txt rules
User-agent: OAI-SearchBot
Disallow:User-agent: OAI-SearchBot
Disallow: /# Opt out of training, stay visible in ChatGPT Search
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow:Verify your rule works
After editing robots.txt, confirm the rule does what you intended: run your domain through the Analyzer to see OAI-SearchBot's resolved access, or paste the file into the URL Tester and check a specific path against the OAI-SearchBot user-agent.
Robots.txt AnalyzerAnalyze a siteCommon mistakes
Assuming GPTBot rules cover it
A Disallow under User-agent: GPTBot has no effect on OAI-SearchBot. Each OpenAI crawler needs its own group.
Blocking it to “protect content from AI”
OAI-SearchBot doesn't train models. Blocking it costs you ChatGPT Search visibility without changing what trains on your content.
Typos in the token
It must be exactly OAI-SearchBot. OpenAI-SearchBot or SearchBot won't match.
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's search crawler. It fetches web pages so ChatGPT Search can show and cite them, with links back to the source. It is separate from GPTBot and is not used for model training.
Does OAI-SearchBot train ChatGPT?
No. OpenAI documents that OAI-SearchBot is used only to surface and link pages in ChatGPT Search. Training data collection happens under the GPTBot user-agent, which you control separately in robots.txt.
Should I block OAI-SearchBot?
Only if you don't want your pages cited in ChatGPT's web answers. Allowing it can bring referral traffic, and it has no effect on what trains AI models — most sites allow it even when they block GPTBot.
Does OAI-SearchBot respect robots.txt?
Yes. OpenAI states that all of its published crawlers, including OAI-SearchBot, honor robots.txt rules targeting their user-agent token.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.