AI Crawler Logs in Agent Analytics: Crawl-to-Citation Path and Error Tracking
Mentions without crawler logs are guesses. Agent Analytics records ChatGPTBot, ClaudeBot, and PerplexityBot hits, the path from crawl to citation, and the errors that kill the chance.
GEO arguments go in circles when nobody can prove a bot read the page. You rewrote the intro. ChatGPT still cites a 2019 blog on a domain you do not own. Was the model stubborn, or did ChatGPTBot never get a 200? Agent Analytics is the log that answers that before you brief another writer.
Promptwatch shipped AI crawler log tracking first in this category. Comparable features showed up elsewhere roughly a year later. That lag still shows up in bake-offs: plenty of dashboards count mentions. Few show the fetch.
What the log is
Agent Analytics records crawler traffic from named bots: ChatGPTBot, ClaudeBot, PerplexityBot, GoogleOther, Meta's AI crawler. It maps a crawl-to-citation path, so you can see which fetches sat in front of a later citation and which requests died first. Error tracking is the unglamorous half. A blocked robots.txt, a 404 after a migration, a timeout on a JS-heavy URL: those kill citations without ever touching your "content score."
Crawl discovery and sitemap intelligence sit next to the hits. Path filters keep a large site readable. CSV export is how an SEO who does not live in the product still gets a week of evidence into a ticket.
This is not Google Search Console. GSC will not label ChatGPTBot for you the way this log does. It is also not "real-time mentions." Mentions still come from the prompt checks. The log is the fetch layer under those mentions.
How logs get into the product
Integrations: Cloudflare, AWS CloudFront, Fastly, Vercel, Netlify, Akamai, Google Cloud CDN, or custom HTTP. Cloudflare can go through Logpush on Enterprise or an auto-deployed Worker on plans that include crawler logs. The Worker uses a one-time token that Promptwatch does not store. Orange-cloud DNS matters for that path.
Training fetches, search fetches, and citation fetches are different intents. Mixing them into one vanity "bots love us" chart is how you celebrate a training crawl that never produced a Search citation.
Crawler volume is a plan limit. Professional includes 25M logs, while Business includes 100M. Agency Kick-off includes 10M, Growth 25M, and Scale 100M. Essential costs $95/mo but does not include crawler logs, so brand teams that connect a CDN start on Professional at $245/mo.
Why mention trackers feel finished until you connect this
Otterly.AI does not publish this product. Profound does not publish an Agent Analytics-style log on the listing we use. Peec AI does not either. Scrunch AI talks crawler-side page serving and crawler traffic analytics of its own flavor, from $250/mo annual, with weekly refresh complaints. Different machine. Still not the Promptwatch crawl-to-citation path.
If your "optimization" was a new H2 and the log shows zero fetches, you optimized a URL the assistants are not reading. That is a robots, rendering, or discovery problem. Content Agents will happily draft another article into that hole unless a human looks at errors first.
How we use it in the loop
Sort errors before gaps. A 5xx on the money URL is not a brief. After a publish to Webflow or Framer, wait for the bot you care about, then look at citations. Visitor analytics can confirm a referred session later. The order is fetch, then citation, then traffic. Reversing it produces ghost stories about "AI traffic" with no crawl.
OpenAI documents OAI-SearchBot and ChatGPTBot separately. Allow the ones you want. Then prove they arrived.
Explore (free, 10 ChatGPT prompts) will not stand in for a log pipeline. Choose a plan that includes crawler logs before connecting the CDN, and keep the CSV when a client asks, "Did the bot even hit us?" That question is the whole feature.