An AI mention is either a measurement or a memory

Ask ChatGPT the same question twice and you can get two different answers. Not two phrasings of one answer. Two different lists of companies, with different reasoning behind them.

The usual explanation is that models are non-deterministic, which is true and beside the point. The larger difference is that one of those answers was produced after the model went and read the web, and the other was produced from memory. Most AI visibility tools count both as a mention.

What is a grounded answer?

A grounded answer is one the model produced after retrieving live pages during the request. A recalled answer is one it produced from training data alone, with no retrieval at all.

Both look identical on the screen. Both name companies, both sound current, and neither says which it is. The difference is in the response body rather than the prose, and each engine exposes it differently:

• OpenAI returns a web_search_call item in the response output when the model chose to search. No item, no retrieval.
• Gemini returns groundingMetadata on the candidate, containing the chunks it actually pulled. An empty or absent field means the answer came from the model.
• Perplexity retrieves on every request as a matter of product design, so the question does not arise.

None of this is hidden. It is simply thrown away by anything that reads only the answer text, which is most of what gets built on top of these APIs.

Why the difference changes what you can conclude

Say you run a scan and ChatGPT names you among the top three options in your category. What have you learned?

If the answer was grounded, you have learned something about your website as it exists now. The model issued a search, something of yours came back, and it was good enough to use. That chain has four links, and every one of them is a thing you control: your site was crawlable, it was indexed by whatever backs the search, it matched the query, and the passage that came back was usable.

If the answer was recalled, you have learned something about a training corpus assembled months ago. That is not nothing. It is durable, it is hard to displace, and it is the reason incumbents keep appearing in answers long after their products stop being the best ones. But it is evidence about the past, and no change you make to your site this quarter will move it.

Treating those two as one number produces a metric that goes up when you buy an old domain and does not move when you fix your robots.txt.

The failure this hides

Here is the case that makes the distinction worth instrumenting rather than merely knowing about.

A company blocks OAI-SearchBot in robots.txt, usually by accident, usually as part of a broader rule aimed at scrapers. Their AI visibility does not drop. It stays flat for months, because the model keeps naming them from memory and every scan records a mention.

What has actually happened is that they have stopped being able to earn new mentions. The recalled ones decay slowly as models are retrained, and by the time the number moves, the cause is a year old and unrecoverable within the reporting period. If you split the metric, the grounded half goes to zero the week the rule ships, which is when it is still a five-minute fix.

This is why which crawlers you allow is a more urgent question than it looks, and why we wrote about which AI crawlers actually matter before writing about anything else.

What we tried that did not work

The obvious approach is to infer grounding from the answer. If the response cites URLs, it searched. This fails in both directions.

Models cite URLs from memory, confidently and sometimes incorrectly, so the presence of a link proves nothing. And a grounded answer often cites nothing at all, because the model read three pages, synthesised them, and saw no reason to attribute. We measured this before deciding to read the provenance fields instead, and the inference was wrong often enough that a metric built on it would have been noise.

The second thing that did not work was forcing the issue. You cannot require an engine to search. You can phrase a prompt so that searching is the obviously correct move, which raises the rate, but the model still decides per request. Any measurement here has to accept that the same question can ground today and not tomorrow, which means single runs are close to meaningless and the useful unit is a rate across repeated runs.

Known limitations

Grounding tells you retrieval happened. It does not tell you the model retrieved your page, only that it retrieved something. Gemini returns the chunks, so there you can check. OpenAI reports the call without an exhaustive list of what it read, so a grounded mention is strong evidence and not proof.

None of these fields are contractual. They are implementation details of products under active development, and any of them could change shape without notice. Anything built on them, including our own pipeline, has to treat a missing field as unknown rather than as false.

And the split says nothing about whether being mentioned is worth anything commercially. That is a separate question, and we have not seen anyone answer it convincingly yet.

What to do with this

Record provenance per answer, not per scan. Report the two numbers separately and never average them. Watch the grounded rate as the leading indicator, because it responds to your work within days, and treat the recalled rate as a slow-moving asset you influence over quarters rather than sprints.

If you are choosing a tool for this, the question to ask is not how many engines it queries. It is whether it can tell you, for a specific answer on a specific day, whether the model looked. If it cannot, the number it gives you is two measurements added together.