Applebot-Extended in robots.txt
Applebot-Extended is Apple's crawler. Here's exactly what it does, whether it respects robots.txt, and the rules to control it. To apply a policy in one click, use the AI Crawler Manager.
What Applebot-Extended is
Applebot-Extended is a robots.txt control token, not a separate crawler. It lets you opt out of Apple using content (gathered by the regular Applebot) to train Apple Intelligence and other generative models — without affecting Siri or Spotlight.
| Property | Value |
|---|---|
| User-agent | Applebot-Extended |
| Operator | Apple |
| Category | AI search crawlers |
| Honors robots.txt | Yes |
| Affects search ranking | No |
Official documentation: Apple crawler docs.
What Applebot-Extended does
- Acts as an opt-out signal: blocking Applebot-Extended tells Apple not to use your content for AI model training.
- Does not fetch pages itself — it modifies how content already crawled by Applebot may be used.
Why site owners care
- Blocking it keeps your content out of Apple's generative-AI training while preserving Siri Suggestions and Spotlight features.
- It is the Apple equivalent of Google-Extended — an AI-use control separate from the search crawler.
How to allow or block Applebot-Extended
Add a group targeting the Applebot-Extended user-agent. Disallow: / blocks it from your whole site; an empty Disallow: allows it.
User-agent: Applebot-Extended
Disallow: /User-agent: Applebot-Extended
Disallow:No effect on search ranking
How to verify Applebot-Extended
Applebot-Extended does not crawl, so there's nothing to verify in logs — it only changes how Applebot's data is used. Verify the standard Applebot via reverse DNS to an applebot.apple.com host.
Does it honor robots.txt?
Recommendation
Recommended: Depends
What is the difference between Applebot and Applebot-Extended?
Applebot is the crawler that fetches pages for Siri and Spotlight. Applebot-Extended is a separate robots.txt token that controls whether Apple may use that already-crawled content for AI model training. Blocking Applebot-Extended opts you out of training without affecting Siri or Spotlight.
Does blocking Applebot-Extended hurt Siri or Spotlight?
No. Applebot-Extended only governs AI training use. Siri Suggestions, Spotlight, and Safari results rely on the standard Applebot, which is unaffected.
AI Crawler Manager
Allow or block GPTBot, ClaudeBot, PerplexityBot and more in one place.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.