> Site index: https://usehardal.com/llms.txt
> Every content route also serves markdown at <url>.md

# How to Measure Googlebot

Googlebot is Google Search's crawler and still feeds AI Overviews. Measure it in Search Console and logs. Do not confuse it with Google-Extended or ChatGPT Ads.

Source: https://usehardal.com/blog/how-to-measure-googlebot-traffic
Published: 2026-08-14
Updated: 2026-08-14
Author: Ecem Bircan
Category: Marketing

---

**Googlebot** is still the crawler that matters for Google Search. Google's AI Overviews and AI Mode also depend on that search index. It is a different agent from [Google-Extended](https://usehardal.com/blog/how-to-measure-google-extended-traffic), which is the training token, and it has nothing to do with Google Ads.

Block Googlebot to "stop AI" and you leave Search. Blocking Google-Extended is the switch for Gemini training. Hardal's [isvisible.ai](https://usehardal.com/blog/why-we-built-isvisible-ai) groups Googlebot with Bingbot as search crawlers that power AI-assisted results.

## How to measure it (the boring, correct stack)

1. **Search Console** for coverage, crawl stats, and rendering. This remains the source of truth for Google's view of your site.
2. **Origin logs** for `Googlebot` in the user-agent, verified against Google's published IP and reverse-DNS guidance when you care about spoofing.
3. **Hardal AI Visibility** for the named agent under platform Google, so Googlebot is not mixed with GPTBot in one "bot" pie chart ([docs](https://usehardal.com/ai-visibility)).

Googlebot does execute more of the web than GPTBot, but you still should not use a marketing pixel as your Googlebot counter. Crawl stats are an ops and SEO dataset.

## What to report to a marketing budget owner

- Crawl vs index vs clicks (Search Console), not "Googlebot ROAS"
- Whether AI-shaped Google features can see the same URLs as classic Search (they share the crawl)
- Side-by-side with [Bingbot](https://usehardal.com/blog/how-to-measure-bingbot-traffic) if Copilot / Bing is in the story
- Explicit note: ChatGPT Ads measurement is a different pipeline ([hub](https://usehardal.com/blog/how-to-measure-chatgpt-ads))

## robots.txt

Allow Googlebot unless you intend to disappear from Google:

```
User-agent: Googlebot
Allow: /
```

Training opt-out stays on Google-Extended. Do not copy a "block all AI bots" snippet that includes Googlebot.

## How Hardal measures it

AI Visibility categorizes search-engine agents separately from LLM trainers and fetchers. Use it as the named-bot layer next to Search Console, not instead of it. Human Google Ads conversions still go through your existing Google Ads / GA4 destinations on the first-party server path.