Claude-User
Claude-User is the bot Anthropic uses when a real Claude user asks a question that needs current information from a specific website. It doesn't crawl on its own: it only goes where the conversation leads. If it visits your page, your content is likely to end up cited with a link in the answer that user sees.
- User-agent
Claude-User- Does it respect robots.txt?
- Yes
How to allow it in your robots.txt
User-agent: Claude-User
Allow: /How to block it (not recommended)
User-agent: Claude-User
Disallow: /Frequently asked questions
Should I block Claude-User?
It's not advisable. If you block it, Claude won't be able to read your site when a user asks about your products or industry, and it will answer with information from other sources, probably your competitors.
Does Claude-User affect my visibility in Claude?
Yes, directly. When Claude checks your site through Claude-User, it usually includes your content and a link to your page in the answer. Blocking it removes those citations.
How do I know if Claude-User visits my site?
Search for "Claude-User" in your server logs. Its visits indicate that real Claude users are asking questions that lead the assistant to your site.