GPTBot crawls public pages that may be used to train OpenAI's generative models. Its job is training data. Ad clicks, search citations, and live fetches each arrive under a different user agent.
OpenAI documents this in Overview of OpenAI Crawlers. Disallowing GPTBot in robots.txt is the signal that your content should not be used for that training. Allowing OAI-SearchBot at the same time is a separate choice. OpenAI says the two robots.txt tags are independent.
If you "measure GPTBot" the way you measure Meta, you will invent a channel that does not exist. The useful measurement is: how often it hits you, which URLs it prefers, and whether that matches the pages you actually want a model to learn.
How GPTBot shows up
Match on GPTBot in the user-agent, not a pinned version. OpenAI's example string looks like compatible; GPTBot/1.4; +https://openai.com/gptbot. Version numbers change. Published IPs are at openai.com/gptbot.json. User-agents can be spoofed; IP ranges are the check when the volume matters.
GA4 will not show this, because GPTBot does not execute your JavaScript tag. That holds for every other trainer too (AI Visibility Playbook).
What to report
- Hit count and unique URLs, weekly or monthly, next to Search Console, not inside Ads Manager
- Top paths: product, pricing, docs, and old posts that still get crawled
- robots.txt intent vs observed hits (a Disallow that still shows volume is a config or spoof problem)
A thousand GPTBot hits on /pricing means the model pipeline is interested in how you sell. It does not mean a thousand ChatGPT users landed. For live fetches see ChatGPT-User. For ChatGPT search indexing see OAI-SearchBot. For paid clicks see How to measure ChatGPT Ads.
robots.txt
User-agent: GPTBot
Disallow: /
blocks future training crawls. It does not untrain models that already ingested you, and it does not change Google rankings. Leave Googlebot alone.
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
is the common split: no training corpus, still eligible for ChatGPT search citations.
How Hardal measures it
Hardal AI Visibility names GPTBot and attributes it to OpenAI on the same first-party, server-side pipeline as Hardal Analytics. No extra pixel. isvisible.ai is the one-shot robots.txt / live UA check. The playbook has the log queries if you still pull CDN files yourself.
Decisions after the report: What marketers should do about AI crawler traffic.