GrokBot
This is the crawler associated with Grok, the AI from xAI (Elon Musk's company). Specialized directories like CrawlerCheck document that it identifies itself with the user-agent "GrokBot" and that you can control it with standard robots.txt rules. Important caveat: xAI doesn't publish its own official documentation, and an independent analysis with controlled tests (Stackfox, 2026) found that part of Grok's live traffic arrives with generic browser user-agents from datacenter proxies, in which case rules targeting "GrokBot" don't cover it.
- User-agent
GrokBot- Does it respect robots.txt?
- Partially
- Official documentation
- https://crawlercheck.com/directory/ai-bots/grokbot
How to allow it in your robots.txt
User-agent: GrokBot
Allow: /How to block it (not recommended)
User-agent: GrokBot
Disallow: /Frequently asked questions
Should I block GrokBot?
For a business, it's not advisable. Grok responds to millions of X users, and being cited there is visibility. If you still want to block it, standard robots.txt rules targeting "GrokBot" work for the traffic that identifies itself as such.
Does GrokBot respect robots.txt?
Partially. Bot directories document that traffic identified as "GrokBot" responds to standard robots.txt rules. But xAI publishes no official documentation, and Stackfox's independent analysis (2026) found Grok requests with regular browser user-agents, which those rules don't reach.
How do I know if Grok checks my site?
Search for "GrokBot" in your server logs. Keep in mind not all of its traffic may show up: according to Stackfox's analysis, part of Grok's requests arrive with Chrome or Safari user-agents from datacenter IPs, so logs may underestimate its visits.