AI Bot Checker
We check your robots.txt and send live requests with each AI crawler's user-agent — catching server and firewall blocks a robots.txt check can't see.
Search crawlers vs training crawlers
AI companies run two very different kinds of bots. Answer-engine crawlers (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User) retrieve pages when someone asks a question — they are how your content gets found and cited in AI answers. Training crawlers (GPTBot, ClaudeBot, Google-Extended and friends) collect data to train future models.
Blocking training bots is a reasonable content-protection choice with zero effect on your AI visibility. Blocking answer-engine bots removes you from AI search results entirely — and blocks can hide in your firewall or CDN, not just robots.txt, which is why this tool sends live requests too.
Want the Full Picture?
Our AI scanner checks 100+ signals beyond just bot access — including llms.txt, Schema markup, E-E-A-T, and more.