Baiduspider
Baiduspider is the official web crawler from Baidu, the dominant search engine in China with billions of users. It visits publicly accessible pages and adds them to Baidu's search index so they appear when someone searches for related topics. If your business has potential customers in Chinese-speaking markets or sells internationally, allowing Baiduspider access could open a visibility channel that most Western SMBs tend to overlook.
- User-agent
BaiduspiderMozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)- Does it respect robots.txt?
- Yes
- Official documentation
- https://help.baidu.com/question?prod_id=99&class=0&id=3001
How to allow it in your robots.txt
User-agent: Baiduspider
Allow: /How to block it (not recommended)
User-agent: Baiduspider
Disallow: /Frequently asked questions
Should I block Baiduspider?
It depends on your target market. If your business only operates locally or in English-speaking markets, blocking it has no real impact. If you sell internationally or want visibility within the Baidu ecosystem, letting it through makes sense. Baiduspider respects your robots.txt file, so you have full control over which sections of your site it can access.
Does Baiduspider affect my visibility in AI tools like ChatGPT?
Not directly. Baiduspider feeds Baidu's search index, not Western generative AI models like ChatGPT, Gemini, or Claude. Its effect on visibility within those AI assistants is negligible. Its real value lies in positioning within the Baidu ecosystem, which is primarily relevant for the Chinese market.
How do I know if Baiduspider is visiting my site?
Look for "Baiduspider" in your server logs or in your hosting or CDN dashboard, for example in Cloudflare. If you see visits with that name in the user-agent field, the crawler is active on your site. To verify they are legitimate Baidu visits, check that the reverse DNS hostname of the IP address ends in .baidu.com or .baidu.jp.