Applebot - Not Obeying robots.txt


"Applebot is the web crawler for Apple. Products like Siri and Spotlight Suggestions use Applebot. It respects customary robots.txt rules and robots meta tags, and it originates in the net block."



Applebot/0.1 does not fully obey robots.txt as it interprets allow entries for Googlebot as implied permission for Applebot, illustrated by the following example:

User-agent: Googlebot

User-agent: *
Disallow: /


This scenario presents potential privacy implications for site owners who may want their content made available to Google but not Apple.


September 19, 2019 - Provided the details to Apple via
September 19 , 2019 - Apple sent an auto acknowledgment
October 1, 2019 - Apple responded stating that they are investigating
October 29, 2019 - Asked for a status update
November 4, 2019 - Apple responded stating that they do not see any security implications
July 7, 2020 - Published an advisory to document the issue


If you do not want Apple to index your site, explicitly restrict Applebot in robots.txt using the following example:

User-agent: Applebot
Disallow: /


Contact Information