Can AI answer engines actually reach your site?
This check tells you whether ChatGPT, Perplexity, Claude and Google AI Overviews can crawl a page — and whether anything is silently stopping them. It reads your robots.txt crawler by crawler, checks that the content exists without JavaScript, and looks for snippet directives that block AI use entirely.
Free, no signup. We fetch your robots.txt and the page itself — nothing is stored.
Why does crawler access matter more than content?
Crawler accessibility is the strongest-evidenced factor in generative search. In Zyppy’s meta-analysis of 54 studies it scores 9.5 out of 10 — ahead of search rank itself. The logic is blunt: a page an engine cannot fetch is a page it cannot cite, regardless of how well it is written. Almost no GEO tool checks this.
Which crawlers actually produce citations?
AI crawlers do three different jobs, and confusing them is the most expensive mistake in this field. Training crawlers collect model training data — blocking them costs you nothing in visibility. Retrieval crawlers build the index an answer engine searches. Live-fetch crawlers pull a page the moment a real user’s question needs it. Block one of the last two and you disappear from that engine’s answers.
| Crawler | Job | Blocking it costs you |
|---|---|---|
| OAI-SearchBot | ChatGPT index | ChatGPT citations |
| ChatGPT-User | ChatGPT live fetch | ChatGPT citations |
| GPTBot | OpenAI training | Nothing |
| PerplexityBot | Perplexity index | Perplexity citations |
| Claude-SearchBot | Claude index | Claude citations |
| Googlebot | Search + AI Overviews | Search and both AI surfaces |
| Google-Extended | Gemini training | Nothing |
Why check before 15 September 2026?
Cloudflare begins default-blocking mixed-use crawlers on ad-bearing pages from 15 September 2026, for free accounts, new customers and new sites. Many sites will lose AI visibility without anyone choosing it. The scan above takes about two seconds and tells you where you stand today.
Frequently asked questions
Does blocking GPTBot stop ChatGPT citing me?
No. GPTBot collects training data only. The crawler behind ChatGPT citations is OAI-SearchBot, and live user questions are fetched by ChatGPT-User. You can block GPTBot and still be cited — they are independently controllable.
Is Google-Extended safe to block?
Yes. Google-Extended governs Gemini model training only. Google Search, your rankings, and AI Overviews are entirely unaffected by it. It is the safest block on the list, and it is the one people most often avoid out of caution.
I rank well in Bing. Will ChatGPT cite me?
Not necessarily. OAI-SearchBot is not Bingbot — OpenAI operates its own crawler and its own index. Bing rankings feed Copilot, not ChatGPT.
Do AI crawlers run JavaScript?
No major AI crawler executes JavaScript. Vercel and MERJ analysed over 500 million GPTBot fetches and found zero evidence of JS execution. Content that only appears after hydration does not rank poorly in AI search — it does not exist there.
What is changing on 15 September 2026?
Cloudflare begins blocking mixed-use crawlers by default on ad-bearing pages for free accounts, new customers, and new sites from existing customers. Sites can lose AI visibility without anyone making a decision, which is why checking now is worth the two minutes.
Does llms.txt help?
There is no evidence that it does. SE Ranking tested roughly 300,000 domains and found no meaningful relationship between llms.txt and AI citation frequency — their model actually got more accurate when the variable was removed. No AI provider has confirmed reading it.
Access is step one. Being worth citing is step two.
ActiveGeo writes content the way answer engines read it — passage by passage — and scores every draft against 15 checks you can audit.
See how it worksLast updated 1 August 2026 · activegeo.io/geo-check