PerplexityBot in robots.txt
PerplexityBot fetches pages for Perplexity's AI answer engine. Unlike pure training crawlers, blocking it is a genuine trade-off: you stay out of AI answers, but you also forgo the referral traffic those citations can send.
What PerplexityBot does
Perplexity is an AI answer engine that responds to questions with synthesized answers and source citations. PerplexityBot crawls pages so they can be surfaced and linked in those answers. Because Perplexity links its sources, appearing there can drive real visitors — which makes the allow/block decision different from a training-only crawler.
| Property | Value |
|---|---|
| User-agent | PerplexityBot |
| Operator | Perplexity |
| Purpose | AI answer engine — fetch & cite pages |
| Sends referral traffic | Yes, via citations |
| Honors robots.txt | Partial — see caveat below |
The allow-vs-block trade-off
- Allow it if discovery matters more than control — citations in Perplexity can send qualified readers to your pages.
- Block it if you don't want your content summarized in AI answers, or if you license content commercially.
- Unlike GPTBot or ClaudeBot, this is not a no-cost block — weigh the lost referrals.
How to allow or block PerplexityBot
Block PerplexityBot across the whole site:
User-agent: PerplexityBot
Disallow: /Allow it explicitly (the default) with an empty Disallow:
User-agent: PerplexityBot
Disallow:Compliance caveat
Reported robots.txt evasion
robots.txt is a voluntary standard. For crawlers with a questionable record, treat a Disallow rule as a request and confirm it with the Analyzer, then escalate to firewall or WAF rules if the crawler ignores it.
Common mistakes
Treating it like a training crawler
Blocking PerplexityBot costs you referral traffic, unlike blocking GPTBot. Decide deliberately.
Relying on robots.txt alone
Given reported non-compliance, add server-side enforcement if the content is sensitive.
Not verifying
Always confirm the block took effect — run your domain through the Analyzer.
Should I block PerplexityBot?
It depends. Blocking it keeps your content out of Perplexity's AI answers but also gives up the referral traffic its citations can send. If you want discovery, allow it; if you want control or license content, block it.
Does PerplexityBot respect robots.txt?
Only partially. Perplexity states it honors robots.txt, but researchers have reported it fetching pages without doing so in some cases. For sensitive content, pair robots.txt with server-side blocking and verify with the Analyzer.
Does blocking PerplexityBot affect SEO?
No. PerplexityBot is unrelated to Googlebot or Bingbot, so blocking it does not change your search rankings. It only affects whether you appear in Perplexity's answers.
How do I block PerplexityBot in robots.txt?
Add User-agent: PerplexityBot followed by Disallow: /. Because compliance has been inconsistent, confirm the block on a live site and consider firewall rules as a backup.
Robots.txt Analyzer
Fetch and audit any site's live robots.txt in one report.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.