Claude-SearchBot
Claude-SearchBot is Anthropic's search crawler. It indexes content so Claude's search features return relevant, up-to-date results. If you block this bot, Anthropic won't index your site for those searches, and your business will lose chances to appear as a source in Claude's answers.
- User-agent
Claude-SearchBot- Does it respect robots.txt?
- Yes
How to allow it in your robots.txt
User-agent: Claude-SearchBot
Allow: /How to block it (not recommended)
User-agent: Claude-SearchBot
Disallow: /Frequently asked questions
Should I block Claude-SearchBot?
No, unless you have a very specific reason. Blocking it prevents Claude from indexing your site for its search results, which reduces your chances of being cited as a source.
How is it different from ClaudeBot?
ClaudeBot collects data to train the models; Claude-SearchBot indexes content for Claude's real-time searches. They're separate robots.txt controls: you can allow one and block the other.
How do I know if Claude-SearchBot visits my site?
Search for "Claude-SearchBot" in your server logs or your CDN. If it shows up, your content is being indexed for Claude search.