What is LLM tracking and how does it work? A guide to the tools
LLM tracking measures how often ChatGPT, Gemini and Perplexity name your brand in their answers. Here is how it works, its limits, and which tools do it well.
LLM tracking is the practice of asking AI answer engines the questions your buyers ask, over and over, and recording what comes back. You log whether your brand is named, where it sits in the answer, which pages the engine cited, and how the tone reads. Then you watch those numbers move over days and weeks. That is the whole idea. Everything else is tooling around it.
One clarification before going further. "LLM tracking" is also used for a different job: observability of the language-model calls inside your own software product, such as latency and token use. This article is about the marketing meaning, which is tracking how brands appear inside ChatGPT, Gemini, Perplexity and similar engines.
How LLM tracking works, step by step
A tracking tool follows the same loop whatever its brand name.
1. You define prompts. A prompt is a realistic question, like "best project management software for a ten-person agency." Good prompt sets mix brand questions, category questions and competitor comparisons, and they reflect different buyer personas and countries.
2. The tool runs each prompt on each engine on a schedule. Daily is the useful cadence, because AI answers vary from run to run and a single check tells you very little. Some tools run in the real consumer interface of the engine; others call the model through an API. That difference matters, and it is covered below.
3. It parses the response. The tool looks for your brand and your competitors, notes the position of each mention, extracts the cited URLs, and scores sentiment.
4. It rolls results into metrics. The common ones are a visibility score (share of answers that name you), share of voice against competitors, average position, and citation rate.
5. It shows trends and gaps. The real value is the comparison over time and against rivals, plus the prompts where a competitor is named and you are not.
Why a single check is not tracking
If you open ChatGPT and ask one question, you get one sample from a variable process. Ask again and the list of brands may change. Tracking is a sampling exercise: many prompts, repeated runs, several engines. The trend across the sample is the signal. Any individual answer is noise.
API tracking versus real-interface tracking
Some tools query a model through its developer API. Others capture what the consumer product shows. They can disagree, because the consumer product adds live search, its own system instructions and formatting. Lettertrace, an open-source tracker, is upfront about this: its own listing notes that answers come from provider APIs, not the apps people use, and that its "Google AI Overviews" is a Gemini API call rather than the panel on a results page.
Promptwatch monitors the live-search platforms through their real interfaces: ChatGPT Search, Google AI Overviews, Google AI Mode, Microsoft Copilot, Gemini, Perplexity, Amazon Alexa and Meta AI. Models such as Claude, DeepSeek, Mistral and Grok are tracked via API alongside them. Knowing which is which keeps your expectations honest.
What LLM tracking cannot do
It cannot guarantee a recommendation, and no tool can. It also cannot tell you why you are missing from an answer unless it measures more than mentions. A tracker that only counts mentions answers "were we named?" and stops there. The follow-up questions need other data: which pages the engine fetched, whether its crawler could reach your site, and whether any of that produced visits.
The tools, ranked for LLM tracking
Ranking here weighs how much of the loop each product covers, not only the first step. Prices are as listed in our tool directory.
| Rank | Tool | Starting price | What it covers |
|---|---|---|---|
| 1 | Promptwatch | Free tier; paid from $95/mo | Prompt tracking, citation analytics, AI crawler logs, visitor analytics, content generation |
| 2 | AthenaHQ | Free tier; from $295/mo | GEO score, Action Center, content optimization agent |
| 3 | Scrunch AI | From $250/mo | Visibility monitoring, site audits, AI agent and bot traffic reporting |
| 4 | Profound | Enterprise custom | Answer tracking inside a broader marketing-agent product |
| 5 | HubSpot AEO | $50/mo | Three engines, 25 prompts daily |
| 6 | Lettertrace | Free (you pay API costs) | Open-source, bring-your-own-key tracker |
Promptwatch
Promptwatch runs prompt panels daily per model, country and language, and then goes past mentions. Citation analytics show which pages and domains engines cite, including Reddit, YouTube and LinkedIn. Agent Analytics reads IP-verified AI crawler logs, so you can see whether GPTBot, OAI-SearchBot or ClaudeBot reached the page at all. Visitor analytics ties AI referrals to conversions. The Action Board then turns the findings into a prioritized to-do list. Plans run from a free Explore tier (10 prompts, ChatGPT only) to Essential at $95/mo, Professional at $245/mo and Business at $579/mo. Crawler logs start on Professional.
AthenaHQ
AthenaHQ tracks brand appearance in AI answers, rolls it into a GEO score and pushes gaps into an Action Center. Its Starter plan is $295/mo and uses credits, where one credit is one AI response. Our directory notes that it has no published crawler-log product.
Scrunch AI
Scrunch AI's Core plan is $250/mo for 125 prompts and four LLMs, with site audits and bot traffic reporting. Its Agent Experience Platform, which serves AI-optimized page versions to crawlers, sits on the custom Enterprise tier.
Profound
Profound started as an answer tracker and now sells itself as a set of marketing agents. Enterprise is custom, with an earlier estimate of about $40k a year. It suits large teams buying an agent program more than teams that want a straightforward tracker.
HubSpot AEO and Lettertrace
HubSpot AEO is a low-cost way in at $50/mo, with ChatGPT, Perplexity and Gemini only. Lettertrace is free software where you bring API keys and pay the model providers directly, which is attractive for developers and limited by the API-versus-interface gap noted above.
Who should pick which
- You want to know why you are or are not visible, and act on it: Promptwatch. It is the only option in this list that joins mentions to crawler logs, citations and conversions in one place.
- You want an action queue and accept credit-based billing: AthenaHQ.
- You are a large team buying agents, not a tracker: Profound.
- You are already in HubSpot and need a basic view: HubSpot AEO.
- You are technical and want to inspect everything: Lettertrace, knowing what the API data represents.
A simple way to start
Pick 25 to 50 prompts that mirror real buying questions. Run them daily across at least three engines. Look at the trend after two weeks, not the first day. Then read the cited sources for the prompts where a competitor wins. That list is your content and outreach plan.