Google-Extended is not a crawler, and that one fact settles most of the debate.
People treat Google-Extended like a bot creeping around their site. It is a robots.txt token that fetches nothing. Googlebot already crawls your pages for Search; Google-Extended only decides whether that crawled content can be used to train future Gemini models. Blocking it opts you out of AI training and grounding, and leaves your Google Search visibility exactly where it was. The real question is not whether it is safe to block. It is whether you want to be in the training set at all.
What is Google-Extended?
Google-Extended is a standalone robots.txt product token that controls whether content Google crawls from your site may be used to train future Gemini models and to ground answers in Gemini and Vertex AI. Google introduced it in September 2023 as a publisher control. Per Google's own documentation, it does not crawl pages itself; Googlebot does the fetching, and the token only governs downstream use.
This is the single most-mislabeled thing in the whole topic. Directories and blog posts call it a crawler or a bot. Google calls it a product token. Getting that right changes how you reason about blocking it.
Google crawler family: where Google-Extended fits
Google runs several crawlers and tokens, and only one of them is about Search. Googlebot fetches your pages for Google Search. Google-Extended is a usage token for AI training. GoogleOther runs internal research crawls. Google-CloudVertexBot crawls on a site owner request when building a Vertex AI agent. The Gemini and agent fetchers act only when a user asks for a specific page.
| Token | What it does | Fetches pages? |
|---|---|---|
| Googlebot | Crawls for Google Search + AI Overviews | Yes |
| Google-Extended | Controls Gemini/Vertex AI training use | No, it is a token |
| GoogleOther | Internal research and dev crawls | Yes |
| Google-CloudVertexBot | Crawls on your request for Vertex agents | Yes |
| Google-Agent, GeminiNotebook | User-triggered page fetches | Yes, per user |
One catch worth knowing: the user-triggered fetchers, like the Gemini Notebook and agent fetchers, generally ignore robots.txt, because a person explicitly asked for that page. So no robots.txt rule, Google-Extended included, stops a user from pulling your page into Gemini by hand.
Does blocking Google-Extended hurt your Search or AI Overviews visibility?
No, on both counts, and this is the part most guides get wrong. Google states that Google-Extended does not affect your inclusion in Google Search and is not a ranking signal. AI Overviews and AI Mode are part of Search and run on Googlebot, so Google-Extended does not touch them either. Blocking it removes you from Gemini training, not from anything you can see in Search.
Should you allow or block Google-Extended?
Block Google-Extended if you would rather your content not train Gemini or ground its answers. Allow it, which is the default, if you are fine contributing to that, or you actively want your brand represented in how Gemini learns about your category. Because blocking costs you nothing in Search, the decision is purely about training, not about traffic or visibility.
Blocking Google-Extended is one of the few AI controls with no downside to your search traffic. You give up a spot in Gemini training data, and keep everything you can actually see.The whole decision, in one line
How do you block, allow, and verify Google-Extended?
To block it, add a Google-Extended rule to the robots.txt at your domain root. To allow it, do nothing, since content is eligible by default. Then verify by loading your robots.txt in a browser and confirming the rule is there. The config is short, and because Googlebot is left alone, your Search presence is untouched.
# Opt out of Gemini / Vertex AI training
User-agent: Google-Extended
Disallow: /
# Googlebot is untouched, so Search and
# AI Overviews visibility stay exactly the same.
How Google-Extended compares to GPTBot and ClaudeBot
All three let you opt out of a vendor AI training, but they are not the same kind of thing. GPTBot and ClaudeBot are real crawlers that fetch your pages; blocking them stops the fetch. Google-Extended fetches nothing, because Googlebot already has your content, so the token just flags whether that content may be used. The shared lesson across all three: opting out of training does not remove you from that engine's answers.
Frequently asked questions
What is Google-Extended?
Does blocking Google-Extended affect my Google rankings?
Will blocking Google-Extended remove me from AI Overviews or AI Mode?
What is the difference between Googlebot and Google-Extended?
How do I block Google-Extended in robots.txt?
Is Google-Extended the same as GPTBot?
Set the rule, keep your search traffic
Do this today: decide whether you want to be in Gemini training data. If not, add the Google-Extended disallow to your robots.txt and verify it. Googlebot stays allowed, so your Google Search and AI Overviews visibility does not move.
Then measure the part that pays the bills: whether AI actually cites you. Pull that baseline with MentionsAPI, keep your crawler rules clean across every engine, and watch whether your Gemini and AI Overviews mentions hold up.