Analyze a site
What is this tool?
The AI Crawlability Checker analyzes your robots.txt file to determine if major AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.) are allowed to access your site.
Why is it important?
- AI crawlers index your content for responses generated by ChatGPT, Claude, Perplexity...
- Blocking these crawlers reduces your chances of being recommended by these AIs
- A good AI crawlability score is essential for GEO (Generative Engine Optimization)