Why LLMs fail to credit data sources, and what it costs brands

The conventional account of LLM citation treats the problem as one of accuracy: does the model point to a real document that supports its claim? That framing, published across dozens of studies, solves the wrong problem. A paper released on arXiv reframes the challenge entirely, and the implications for organisations whose authority rests on proprietary data are considerably more serious than the document-citation debate suggests.

The distinction the paper draws is precise. Scholarly citation does three things: it verifies a claim, it traces provenance, and it assigns credit. Document-citation research in the LLM field addresses the first. It ignores the second and third almost entirely, and it addresses only text. Data, meaning structured datasets, knowledge graphs, statistical series, and curated corpora, presents a categorically different problem. A model trained on a World Bank poverty dataset does not cite that dataset when it produces a poverty statistic. It produces the number as if the number were its own.

Three failure modes, one structural gap

The arXiv paper identifies three distinct research directions where data citation currently breaks down, and each maps to a different moment in the model's lifecycle.

The first is training data attribution. When a model absorbs a dataset during pre-training, influence estimation techniques can, in principle, identify which training examples shaped a given output. Turning those estimates into citable references is unsolved. The granularity problem alone is substantial: a model may have ingested a dataset at the row level, the file level, or the corpus level, and those are not the same citation target.

The second is inference-time citation. When a retrieval-augmented model pulls from a database or knowledge graph to answer a query, it must identify not just the source system but the specific subset queried, the version, and the fixity of that subset at retrieval time. A citation to "the World Health Organisation dataset" is not a citation; it is a gesture in the direction of one.

The third is knowledge graph citation. Facts extracted from structured graphs present their own problem of what, precisely, constitutes a citable unit. A triple in a knowledge graph is not a document. There is no standard for what a reference to it should contain or how it should be rendered.

All three problems have one cause in common. The citation infrastructure of the LLM field was built by people thinking about documents, because documents are what traditional information retrieval handles. Data is not a document, and the infrastructure does not fit.

What this costs institutions that produce data

For multilateral organisations, national statistics agencies, and policy research institutions, this gap is not academic. The UN system, the World Bank group, and bodies like CGAP produce some of the most-cited datasets in development economics. When an LLM produces outputs drawing on that data, no credit flows back. No provenance chain exists. The institution's role as the originating authority is invisible to the model's output, and therefore invisible to any user relying on that output.

The consequences compound. First, if a model produces an incorrect figure derived from a dataset, and no citation chain exists, error correction is nearly impossible. The institution that produced the underlying data cannot be identified as the place to check. Second, the reputational logic of AI visibility breaks down. Organisations that have invested in producing authoritative data, and that currently benefit from citation in academic and policy literature, receive none of that benefit when their data is absorbed into a model's parameters or retrieved at inference time without attribution.

For major industrial groups and financial institutions that commission proprietary data, the problem has a third dimension. Competitive intelligence derived from their own research can be reproduced by a model without attribution, making it impossible to distinguish their authoritative output from a model's reconstruction of it.

The document-citation advances of the past two years, which have made models marginally better at pointing to text sources, have created a false sense of progress. Citation recall for documents has improved; citation of data has not started. Brands and institutions that assume improving LLM citation behaviour will eventually protect their data-based authority are working from a model of the problem that the research has now made obsolete.

The institutions with the most to lose are precisely those that compete not on narrative but on evidence: the statistical agencies, the multilateral research arms, the policy think-tanks. Their authority in AI-mediated information flows depends on a citation infrastructure that does not yet exist. Building it is a collective action problem with no obvious convener, which means the gap is likely to persist long after document-level citation becomes routine.