Amazonbot
Amazonbot is Amazon's crawler. It collects web content to improve its products and services, for example so the Alexa assistant gives more accurate answers, and according to its documentation can also be used to train Amazon's AI models. If your customers ask Alexa about services like yours, this bot is the gateway.
- User-agent
AmazonbotMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) Chrome/119.0.6045.214 Safari/537.36- Does it respect robots.txt?
- Yes
- Official documentation
- https://developer.amazon.com/amazonbot
How to allow it in your robots.txt
User-agent: Amazonbot
Allow: /How to block it (not recommended)
User-agent: Amazonbot
Disallow: /Frequently asked questions
Should I block Amazonbot?
It's not advisable for a business. Alexa is in millions of homes and Amazonbot feeds its answers. Blocking it reduces your chances of showing up when someone asks by voice about products or services in your industry.
Does Amazonbot respect robots.txt?
Yes, it honors standard user-agent and allow/disallow directives, as well as tags like noindex. That said, according to its documentation it doesn't support the crawl-delay directive.
How do I know if Amazonbot visits my site?
Search for "Amazonbot" in your server logs. Amazon also publishes its official IP ranges so you can verify the visits are genuine and not an impostor.