IndexDoctor.io
AI visibility

Google-Extended blocked: what it does and does not affect

Google-Extended is the cleanest example of "training vs. search" separation. You can block it without losing Google Search ranking.

Run AI Crawler CheckerOpen AI Crawler Checker
What this usually means

Your robots.txt disallows Google-Extended, either explicitly or via User-agent: *. Google honors the Google-Extended token for decisions about using your content in AI products, but it does not treat Google-Extended as a condition for classic Google Search ranking.

Why it matters

Google-Extended is separate from Googlebot. Blocking Google-Extended removes your pages from training data for Google's AI products such as Gemini and AI Overviews. It does not affect whether Googlebot crawls your pages, nor whether those pages rank in traditional Google Search. Many teams misread Google-Extended as "blocking Google," which is not what happens.

Common causes
  • A broad "block AI" robots.txt template added Google-Extended alongside GPTBot and ClaudeBot.
  • Legal or editorial policy requested an opt-out from AI use without understanding the user-agent distinction.
  • A catch-all User-agent: * Disallow: / also covers Google-Extended as a side effect.
  • Someone assumed Google-Extended was a typo for Googlebot and "fixed" it, breaking the intended policy.
How to diagnose it
  1. Open AI Crawler Checker.
  2. Look at the Google-Extended row in the matrix and at the Googlebot row right below it.
  3. Confirm Googlebot is allowed and Google-Extended matches your intent.
  4. Check that the Disallow group for Google-Extended uses that exact token, not a typo or wildcard.
How to fix it
  1. 1

    Keep Googlebot allowed

    Ensure User-agent: Googlebot is either implicitly or explicitly allowed. Never disallow Googlebot, that really does remove you from Google Search.

  2. 2

    Use an explicit Google-Extended group

    Add User-agent: Google-Extended with Allow: / or Disallow: / depending on your stance on AI use. Relying on the catch-all for such an important decision is fragile.

  3. 3

    Keep the policy documented

    Add a brief comment in robots.txt or your deploy config explaining why Google-Extended is (dis)allowed. This prevents the decision from being quietly reverted during refactors.

  4. 4

    Verify with the checker

    Re-run AI Crawler Checker. Confirm Googlebot is allowed and Google-Extended resolves the way you intended.

FAQ
Is Google-Extended the same as Googlebot?

No. Googlebot is Google's search crawler. Google-Extended is a separate token that controls whether Google can use your content for AI products like Gemini and AI Overviews. They are independent in robots.txt.

Does blocking Google-Extended hurt rankings?

Blocking Google-Extended does not affect classic Google Search ranking. Google has stated that it is a policy signal about AI use, not about search crawling.

How do I allow Googlebot but block Google-Extended?

Add a User-agent: Googlebot group with Allow: / (or no Disallow) and a separate User-agent: Google-Extended group with Disallow: /. Put both above any catch-all rule.

Related fixes

Ready to diagnose your URL?

AI Crawler Checker runs the exact checks discussed above.

Run AI Crawler Checker