Amazonbot: who runs it and how to allow or block it
Amazonbot crawls the web to improve Amazon's products and services. Amazon states the content it collects may be used to train Amazon AI models.
Amazonbot at a glance
- Operator
- Amazon
- Purpose
- Training It collects content for AI model training.
- Kind
- Crawler. Visits your pages under its own user agent and reads the robots.txt group that names Amazonbot.
- What it feeds
- Amazon products, services and AI models
- What Amazon says about robots.txt
- Amazon states its crawlers follow robots.txt user-agent and allow/disallow rules, may use a copy cached within the last 30 days, and ignore crawl-delay.
- User agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36- Verified
- against Amazon's official docs (opens in a new tab).
Allow or block Amazonbot.
Paste one group into the robots.txt at the root of your domain. A group that names Amazonbot replaces your * group for Amazonbot, so copy any * Disallow lines you still want applied into it.
User-agent: Amazonbot
Allow: /
User-agent: Amazonbot
Disallow: /
# AI crawler access, generated by Suede AI Agent Studio
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: Amzn-SearchBot
Allow: /
User-agent: Amzn-User
Allow: /
User-agent: Meta-WebIndexer
Allow: /
User-agent: Meta-ExternalFetcher
Allow: /
User-agent: Amazonbot
Disallow: /
Opens OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Amzn-SearchBot, Amzn-User, Meta-WebIndexer and Meta-ExternalFetcher, the crawlers behind live answers in ChatGPT, Claude, Perplexity, Alexa and Meta AI, while Amazonbot stays out.
How Amazonbot reads your robots.txt
Major crawlers apply the robots.txt standard (RFC 9309) the same way. Three rules decide every path.
- A named group beats *. When a group says
User-agent: Amazonbot, Amazonbot follows that group and ignores the * group entirely. With no named group, the * group applies; with neither, every path is allowed. - The longest matching rule wins. Inside the group, the Allow or Disallow line with the longest matching path decides, so
Allow: /blog/pressoverridesDisallow: /blogfor/blog/press.*matches any run of characters and$marks the end of the path. The example below keeps your articles and audio out of model training while press pages stay readable. - Allow wins a tie. When an Allow and a Disallow match with the same length, the path is allowed.
User-agent: *
Disallow: /
User-agent: Amazonbot
Disallow: /blog
Allow: /blog/press
Disallow: /*.mp3$
| Path | Result | Deciding rule |
|---|---|---|
/ | Allowed | No rule in the Amazonbot group matches, so it is allowed |
/blog/how-we-price | Blocked | line 5: Disallow: /blog |
/blog/press | Allowed | line 6: Allow: /blog/press |
/demo-track.mp3 | Blocked | line 7: Disallow: /*.mp3$ |
Line 2 still blocks every crawler without a group of its own. Amazonbot reads only lines 4 to 7, which is why the homepage stays open to it.
See which line decides Amazonbot on your site.
Amazonbot, answered.
What is Amazonbot?
Amazonbot crawls the web to improve Amazon's products and services. Amazon states the content it collects may be used to train Amazon AI models. Amazonbot is run by Amazon and collects content for AI model training.
How do I allow Amazonbot in robots.txt?
Add a group that names it: a line reading "User-agent: Amazonbot" followed by "Allow: /". A group that names Amazonbot replaces your * group for Amazonbot, so repeat any Disallow lines from your * group that should still apply to it. To block it instead, use "Disallow: /" in that group.
Does blocking Amazonbot remove me from AI answers?
Blocking Amazonbot governs whether your content is used for Amazon products, services and AI models. Live answers come from answer bots such as Amzn-SearchBot, which follows its own robots.txt group, so you can block Amazonbot and keep Amzn-SearchBot open. Amazon states its crawlers follow robots.txt user-agent and allow/disallow rules, may use a copy cached within the last 30 days, and ignore crawl-delay.
Crawlers to set alongside it.
Every crawler, its operator and its docs are in the full AI crawler list.
Last updated .
Keep Amazonbot where you want it. Every Monday.
Run the free check for the full crawler table and ranked fixes, unlock the $9.99 fix kit for paste-ready files, then let the AI Visibility Watch template read your robots.txt weekly and flag any answer bot that becomes blocked.