Meta description: One number cannot tell you how you show up in AI search. It averages two systems that move independently, personalises per user, and the tools selling you that number are counting on you not knowing the difference.
The question lands on my desk almost every week now, phrased with the same reasonable confidence every time. “How is my AI visibility?” Clients ask it the way they used to ask where they ranked for a keyword, expecting a number back, ideally one that went up since last quarter.
I cannot give them that number. Not because we are not looking, but because the honest version of the answer is more complicated than a single figure can carry, and I would rather say something true than something tidy. That gap, between what people want to be measured and what can actually be measured, is the whole story of AI visibility right now.
Let me be fair to the fan-out first
The standard move today is a query fan-out. You take the questions a buyer might ask, expand each into a spread of natural-language variations, fire them at an AI engine, and count how often your brand comes back. It is a sensible instinct and I do not want to sneer at it.
It is cheap. It is repeatable. It turns a vague anxiety into something you can actually look at, and it is miles better than the alternative of guessing or assuming you are invisible because a founder typed one prompt into ChatGPT and did not see themselves. As a starting point, running a fan-out is genuinely the right first step. Anyone doing it is already ahead of the crowd still arguing about whether any of this matters.
So the fan-out is not the problem. Treating the fan-out as the finish line is the problem.
One engine is not “AI”
The first crack shows up the moment you remember there is no single “AI” to be visible in. There is a room full of them, and they do not agree.
Run the same buyer question across five engines yourself and you will usually see the split. ChatGPT puts a brand in the top three and cites its pricing page. Perplexity does not mention it and builds the whole answer around a competitor. Gemini describes it with a tagline the company dropped eighteen months ago. Google’s AI Overview hedges and names nobody. Same brand, same question, same afternoon. Report only the engine that flatters it and you would say “you are winning.” Report only the one that ignores it and you would say “you are invisible.” Both true. Neither useful.
A fan-out run against a single engine, on a single day, and then reported as “your AI visibility” is not measuring your visibility. It is measuring one engine’s mood on a Tuesday. Do it properly and you are running the same deliberate spread across multiple LLMs and across AI Overviews, in the same wording, repeatedly, because the answers move over time as the models update underneath you. That is more work than a dashboard wants to admit. It is also the only version that tells you anything real.
The crack that actually matters: two systems, not one
Here is the part most scoring tools walk straight past, and it is the part Duane Forrester has been making the case for more clearly than anyone. Your brand does not live in one memory inside these engines. It lives in two, and they fail in completely different ways.
One is parametric memory, the knowledge baked into the model during training and then frozen until the next training run. The other is retrieval, the content the engine pulls in live at the moment someone asks. When an answer about you is wrong, it is wrong because of one of these two, and the fix for one does nothing for the other. A single score cannot tell you which you are looking at, which means it cannot tell you what to do next. It just tells you that you feel bad.
It is really important to know the difference between these two memories:
| Parametric problem | Retrieval problem | |
|---|---|---|
| The tell | Confident answer, no citations | Live citations, but not yours |
| The cause | Stale or wrong version of you was learned during training | Your page was not selected, or a competitor’s was |
| What will not fix it | Publishing a correction today (the model already trained) | Waiting for the next training run |
| What actually helps | Consistent, corroborated, crawlable content now, so the right version is learned next time | Structure for clean extraction, answer the sub-questions, strengthen third-party corroboration |
Same wrong answer on the screen. Two entirely different causes, two entirely different jobs. Collapse them into one green number and you will confidently spend a quarter fixing the layer that was never broken.
And then there’s who is asking
There is a third axis I have skated over, and it earns a line. Even within one engine, the answer is personalised to the person asking. An LLM carries saved memory, custom instructions and chat history for each signed-in user, held on the account rather than the device, so the same category question from two different buyers can return two different shortlists. That is not the parametric-versus-retrieval split, it is a layer sitting on top of it: who is asking, and what the engine already remembers about them. It is the same point I made in Your LLM Doesn’t Know You, That’s Why Its Answers Are Average, from the other direction. You are not one number to one engine. You are a different number to every signed-in user it thinks it knows.
Even the big players are only just starting
I want to be fair here too, because it would be easy to read this as “the tools are rubbish.” They are not. They are early.
Watch what the platforms themselves are shipping, because it tells you where this is going. Google added Generative AI performance reports in Search Console so you can see how you surface in AI Overviews and AI Mode, which is worth reading alongside this piece of my valued colleague Gregory Pinas on how Search Console has a habit of finally showing you data it was already sitting on. Microsoft added an AI performance view in Bing Webmaster Tools. Google is even rolling out an AI share-of-voice report inside Merchant Center, benchmarking your products on AI surfaces against similar brands.
This is good. It is also, by their own framing, a start. Each of these looks at that platform’s own surface, which is exactly the single-engine limitation from a few paragraphs ago, now with an official logo on it. When the companies with the most data are shipping v1, you should be sceptical of any third-party tool quietly implying it has already solved the whole thing with one figure.
This is not a solved problem, and I would be lying if I told you otherwise. It is not solved by the platforms, it is not solved by the tools, and it is not solved by us…..yet. We are still working out the best way to show the correct data, and I suspect the answer is not one number but several: different engines, different moments in time, tracked so you can watch the picture move instead of freezing it into a single figure that either flatters you or panics you. When a client asks how their AI visibility is, that is the honest shape of the answer, which engine, checked more than once, with or without a citation, and which of the two memory systems the problem sits in. Collapse all of that into one score and you throw away the only parts that tell you what to do. It sells worse in a pitch than a confident number would. It is also the truth, and the truth is worth more than a tidy gauge.
What I actually tell the client
So I give them the unsatisfying, useful answer instead of the satisfying, useless one.
Do not treat “AI visibility” as a number. Treat it as a question you re-ask across several engines and both memory systems, in that order, before you spend a single euro on fixing anything. Check more than one engine. Read the citations, because the presence or absence of them is the tell that tells you which system you are dealing with. Sort your problems by cause before you sort them by size. And keep half an eye on the platforms, because Google and Microsoft will keep expanding these reports, and the ground will keep moving under whatever you measured last month.
Do not take today’s snapshot as gospel, on your own brand or on software that is changing this fast. The number you were handed last week is already a little bit fiction.
The vendors are not wrong that AI visibility matters. They are wrong that it fits on a gauge. Anyone who can hand you one clean score for something this messy is not measuring your visibility. They are measuring how badly you wanted a number.
