Applebot-Extended
Applebot-Extended doesn't crawl websites: it's a switch in robots.txt that decides whether Apple can use the content already collected by Applebot to train its AI models, the ones behind Apple Intelligence. Blocking it doesn't affect your presence in Siri or Spotlight, but it does leave your content out of Apple's generative AI knowledge.
- User-agent
Applebot-Extended- Does it respect robots.txt?
- Yes
- Official documentation
- https://support.apple.com/en-us/119829
How to allow it in your robots.txt
User-agent: Applebot-Extended
Allow: /How to block it (not recommended)
User-agent: Applebot-Extended
Disallow: /Frequently asked questions
Should I block Applebot-Extended?
For most businesses, no. Allowing it lets Apple Intelligence learn from your content, which can translate into mentions when its AI features answer about your industry. Blocking it improves nothing in return.
Does blocking Applebot-Extended remove me from Siri or Spotlight?
No. Apple confirms that blocking Applebot-Extended doesn't stop your content from still appearing in its search products; it only excludes it from generative AI training.
Will I see Applebot-Extended in my logs?
No, because according to Apple this agent doesn't crawl web pages. The crawling is done by Applebot; Applebot-Extended only determines the usage rights of the data already collected.