Are you blocking AI without knowing it?
Check in seconds which AI crawlers can read your site and which ones you're blocking in your robots.txt.
Takes 5 seconds · No sign-up · We don't store your site
How it works
Enter your site
Type your web address. No sign-up, no email required.
We read your robots.txt
We fetch your site's robots.txt and compare it against the AI crawlers we track.
See what you block
We show which AIs can read your site, which ones you block, and what each case means for your business.
Frequently asked questions
What is an AI bot or crawler?
It's a program from companies like OpenAI, Google or Anthropic that crawls the web to feed their AIs. If it can read your site, that AI is more likely to know and recommend your business.
Where are these bots blocked?
In a file called robots.txt, at the root of your site (yourdomain.com/robots.txt). A wrong rule there can block AIs without you knowing.
Should I unblock all of them?
For most businesses that want customers, yes: the more AIs can read your site, the more chances to appear in their answers. The exception is very aggressive bots that overload your server.
Do you store my site or email?
No. This tool is single-use: we read your robots.txt on the spot and show you the result. No email, nothing stored.
How do I monitor this continuously?
Surfeo tracks your AI visibility and alerts you to changes. This tool is a snapshot; Surfeo is the ongoing monitoring.
Do you actually show up in ChatGPT, Gemini and Perplexity?
Unblocking the bots is step one. Surfeo measures whether AIs mention you and tells you how to improve.
Try Surfeo free