AI search agents score worse when they search, not clarify
A new benchmark shows agents that search rather than clarify ambiguous queries perform worse than those that guess, with accuracy gaps of up to 40 points.
Key takeaways
- AI search agents that search repeatedly on ambiguous queries score 51.9% accuracy, worse than guessing.
- The best-performing model on DiscoBench reaches only 43% overall accuracy across multi-step research tasks.
- Removing ambiguity from queries raises accuracy by up to 40 percentage points, revealing a design gap, not a data gap.
- Brands with broad, discursive content face higher distortion risk when agents run retrieval loops on unclear queries.
- Precisely scoped content with explicit qualifications is harder to misrepresent in ambiguous retrieval scenarios.
Searching is not the problem. Knowing when to stop and ask is.
That is the finding that DiscoBench, a new benchmark for AI search agents, delivers with unusual precision. Reported by The Decoder, the study tests how models handle ambiguous queries across multi-step research tasks. The headline result: agents that respond to an unclear question by searching repeatedly score 51.9% accuracy. Agents that simply guess score higher. When researchers strip ambiguity from the queries entirely, accuracy rises by up to 40 percentage points.
The correct response to an ambiguous query is a clarifying question. Every model tested got this wrong, at least some of the time. The best performer reached only 43% overall accuracy. That gap between a clean-query world and a messy-query world is the actual measure of how far AI search agents are from production reliability.
The mechanism that breaks retrieval
The failure is architectural, not incidental. Current search agents are optimised for retrieval: given a query, find information. Ambiguity resolution is a different cognitive task, one that requires the agent to model what the user does not yet know they need to specify. Retrieval pipelines were not built for that. They were built to return results.
So when a query is unclear, the agent does what it was designed to do: it searches. It searches again. It searches differently. Each iteration is confident and wrong, because the underlying question was never settled. The benchmark makes this concrete. Searching harder than guessing is worse than guessing. That should unsettle anyone who has assumed that agentic AI search is simply better search.
The practical implication for brands is immediate. If a user asks an AI search agent something like "what is [company]'s position on sustainable infrastructure financing," and the question is underspecified, the agent does not return a blank or prompt for more context. It constructs an answer from whatever it retrieves, with the compounding errors that repeated ambiguous queries produce. The brand gets cited, or not cited, on the basis of retrieval luck rather than content quality.
What this means for sectors where precision is not optional
For multilateral institutions and policy bodies, this failure mode is particularly costly. UNDRR or CGAP, for example, publish technical guidance documents where a single misattributed position can have reputational or operational consequences. An AI agent misreading an ambiguous query about disaster risk finance and retrieving adjacent but incorrect material will not flag the error. It will present the result as a finding.
Financial services face the same exposure. A compliance professional querying an AI search agent about regulatory requirements in a particular jurisdiction, with a question that conflates two distinct frameworks, will not be prompted to clarify. The agent searches, compounds the ambiguity, and returns something. The 40-point accuracy gap between clean and ambiguous queries is the magnitude of that risk.