Search Engine Journal tests AI visibility metrics against real traffic
A 0.62% traffic figure and four disagreeing datasets show why most AI visibility dashboards won't survive a finance committee's questions.
Key takeaways
- ChatGPT sends roughly 0.62% of Google's organic referral traffic, per Search Engine Journal's cited dataset.
- Four AI-visibility datasets measuring exposure, citation frequency, recall, and consideration often disagree with each other.
- Most commercial AI-visibility dashboards collapse these distinct layers into one misleading score.
- Referral traffic is the wrong yardstick for AI answers, since exposure can occur with zero clickthrough.
- Defensible budget cases need at least two independent measurement signals, not one dashboard score.
ChatGPT sends the equivalent of 0.62% of Google's organic traffic, according to the dataset Search Engine Journal cites in its review of four separate measurement sources. That figure alone should puncture a thousand vendor slide decks. Yet the article's real argument is not that AI visibility is trivial. It is that most of the dashboards built to measure it are.
Search Engine Journal's Greg Jarboe lines up four datasets, each measuring a different layer of the problem: referral traffic, citation frequency, brand mention recall, and share of model "consideration" in generated answers. The datasets disagree with each other, sometimes wildly. A brand can rank first in citation frequency and nowhere in consideration share. That is not noise. It is four different instruments measuring four different things, badged under one misleading label: "AI visibility."
The dashboard problem is a definitions problem
Most AI visibility tools on the market today collapse these layers into a single score, typically derived from prompting a model a few hundred times and counting mentions. Search Engine Journal's point, worth sitting with, is that this number rarely correlates with anything a CFO would recognise as business impact. Exposure (how often a brand appears in a generated answer) is not the same as recall (whether users remember it) or consideration (whether it shapes a purchase decision). Conflating them produces a tidy chart and a false sense of precision.
This matters more for the sectors that cannot afford vanity metrics. A bank building a marketing case for AI-answer optimisation needs to show the finance committee that citation share converts into something measurable, pipeline, inquiries, brand lift. An institution like CGAP or UNDRR, operating on donor and member-state budgets, needs an even higher bar: visibility spend has to map to a policy outcome, not a dashboard glow. If the underlying metric is exposure-only and the real driver of behaviour is consideration, the budget is being defended with the wrong number. Finance committees are good at spotting that gap, eventually.
The 0.62% figure deserves scrutiny in both directions. Treated alone, it is an invitation to deprioritise AI search entirely: why build a measurement practice around a channel sending under one percent of Google's referral volume? That reading is too quick. Referral traffic is the wrong yardstick for a channel whose entire value proposition is answering the question before the click happens. A model that cites a bank's methodology paper in a generated answer about inflation hedging has delivered brand exposure with zero referral trace. Judging AI visibility by clickthrough is like judging a billboard by how many people scan its QR code. Most won't. The exposure still happened.
What actually travels between the four layers
Search Engine Journal's useful move is treating exposure, recall, and consideration as a funnel rather than synonyms. Exposure is necessary but not sufficient; a brand mentioned in a ChatGPT answer about "best pension providers" has exposure. Whether that translates into recall when the user later searches, or consideration when they shortlist a provider, is a separate measurement question requiring different instruments: brand-tracking surveys, not just prompt audits.