How to verify statistics, figures and claims from AI

A person reviewing printed reports at a desk with a laptop open beside them
TL;DR

Generative AI tools produce statistics by predicting plausible text, not by retrieving verified data from official sources. Every AI-generated figure should be treated as unverified until checked against a primary source. The practical approach is to classify the risk level of each claim, ask the AI for its source as a starting point, and confirm the figure directly with the original publisher before it appears in any external document or client communication.

Key takeaways

- Generative AI predicts plausible-sounding output, including statistics, rather than retrieving verified facts. Every figure it produces should be treated as unverified until independently checked against a named primary source. - The ICO can fine UK businesses up to £17.5 million or 4% of global annual turnover for serious UK GDPR breaches, which can include mishandling client personal data in public AI tools. - The UK Government AI Playbook recommends aligning verification controls to the risk level of the output, rather than applying the same process to every number an AI produces. - For UK business statistics, the ONS, ICO, CMA, FCA, and NCSC are the primary sources. Knowing where to look is the core practical skill required. - A two-step sign-off where the author documents sources and a reviewer confirms them handles the bulk of AI figure verification without significant overhead.

A finance manager at a UK professional services firm included a statistic in a client proposal recently. The figure came from a ChatGPT conversation: roughly 70% of UK SMEs already using AI. It sounded credible, it fitted the narrative, and the proposal went out. When the client asked for the original source, there was none. ONS data from 2023 shows 15.1% of UK businesses were using any AI technology at that time. The figure in the proposal was off by a factor of four.

This is how AI verification failures tend to happen. The tool produces the figure with the same confidence whether it is right or wrong. There is no built-in indicator to prompt a check. By the time the error surfaces, it has already reached someone who matters.

What does verifying an AI statistic actually involve?

Every statistic, percentage, or figure an AI tool gives you is a plausible prediction, not a database lookup. The model has no live connection to official sources. It reproduces patterns from its training data and presents them with the same confidence whether they are accurate or fabricated. Verification means checking the claim against an independently published primary source before you use it in any external document or communication.

OpenAI’s own guidance acknowledges that models may generate incorrect or misleading information and should not be relied on as a sole source of truth. Google’s AI Overviews service produced false claims publicly in May 2024, including advice that required public corrections. These were documented failures in mainstream production systems, not early prototypes.

The UK Government’s 2024 AI Playbook describes generative models as systems that “may produce outputs which are factually inaccurate or fabricated” and calls for human oversight before AI-assisted outputs are used in consequential decisions. That principle applies to any services firm using AI to draft client materials, reports, or public communications.

Why does using an unverified AI figure put your business at risk?

When an AI-generated figure appears in client work, a regulated document, or a public marketing claim, you carry the liability for its accuracy. UK data protection law, sector regulation, and basic contract law create no exceptions for “the AI said so.” The practical question is what happens to your business when an AI number turns out to be wrong, and whether you can show you took reasonable steps to check it first.

The ICO can fine UK businesses up to £17.5 million or 4% of global annual turnover for serious UK GDPR breaches. One live risk is staff pasting client personal data into a public AI tool without a data-processing agreement in place. Financial records, client contact details, and other personal data go into AI chat interfaces regularly in services firms.

The FCA holds regulated firms responsible for the accuracy of their outputs, even when those outputs draw on third-party AI tools. A wrong regulatory threshold in client advice does not become the AI vendor’s liability.

In 2023, a US law firm submitted court filings containing fabricated case citations generated by ChatGPT. The judge sanctioned the lawyers. The tool offered no warning that the citations it produced were invented. The CMA identified “false or misleading information” in AI outputs as a consumer and business risk in its 2023 foundation models report.

Where do AI-generated figures most often go wrong?

AI tools are most likely to produce unreliable figures when the claim depends on recent data the model was not trained on, when it involves arithmetic inferred from a table, and when the model is generating a plausible-sounding estimate for a topic where verified public data is thin. AI adoption percentages, market size claims, and regulatory deadlines are the three categories where errors appear most frequently.

ONS data from 2023 shows 15.1% of UK businesses were using at least one type of AI technology at that point, with higher uptake in information and communications sectors. When an AI tool states that 70% of UK businesses already use AI, that figure conflicts sharply with official data. The gap is wide enough to mislead a board paper or a client recommendation.

AI tools have also been observed making basic arithmetic mistakes. Testing of models including Google’s Bard in early 2023 found miscalculations and mis-stated figures from table data. Any percentage or growth rate an AI tool produces is worth recalculating in a spreadsheet before it appears in external work.

Out-of-date figures create a separate problem. The EU AI Act was formally adopted in 2024, with compliance obligations phasing in over several years. An AI tool describing it as a draft or proposal is now incorrect, and using that framing in client work signals you have not verified the source.

When do you need to verify, and when can you move fast?

The level of checking required depends on the consequences if the figure is wrong. A rough estimate clearly labelled as illustrative can be used for internal planning without a full source check. A figure going into a client deliverable, a regulated communication, or a public marketing claim needs a proper primary source confirmed first. Calibrating verification effort to risk is more practical than applying the same process to every number an AI produces.

The UK Government AI Playbook frames this as aligning controls to the potential impact of errors. For a small services firm, a simple three-level classification works well.

Low-risk uses include internal planning and brainstorming where numbers are labelled as estimates. Medium-risk covers external marketing content and non-regulated client advice. High-risk covers client deliverables, financial projections, legal or compliance statements, and anything going to a regulator.

For medium and high-risk uses, the process is consistent. Ask the AI for the original source, organisation, and year. Treat that response as a lead to investigate rather than as confirmation of accuracy. Find the original source directly and confirm the figure is there, correctly stated, and current enough to be relevant. For high-risk items, keep a brief note of the check: the source URL, who confirmed it, and when. A comment in a shared document is sufficient.

What habits and sources make ongoing verification manageable?

A short list of trusted UK sources and a brief team sign-off step will handle the bulk of AI number-checking without adding significant time to any task. UK official data is freely accessible online. Knowing where to look is the practical skill, and it takes an afternoon to build the reference list rather than months of training to acquire the expertise.

For UK data and regulatory matters, the primary sources are the ICO for data protection and AI guidance, the NCSC for security aspects of AI use, the FCA for regulated sectors, the CMA for competition and consumer concerns from foundation model outputs, and ONS for business and economic statistics. All publish freely accessible material online. A short reference page with links to these sites takes an hour to put together for your team.

The governance step that makes the most practical difference is a two-step sign-off for external content containing AI-generated figures. The person who drafted the content documents where each key figure came from. The reviewer confirms sources are credible and arithmetic is correct. A comment in a shared document is enough. The discipline is in doing it consistently rather than occasionally.

The NCSC advises incorporating AI-related issues into standard incident-management processes. For a small team, that means logging any AI factual errors found in drafts and reviewing the log quarterly to spot patterns. If one tool consistently generates wrong figures for a certain type of task, that is a reasonable basis for restricting its use in that area.

If you want to get this right before an error reaches a client document, a conversation about your AI verification process is a practical starting point. Book a conversation to talk through what makes sense for your team.

Sources

- OpenAI (n.d.). Safety Best Practices for Developers. Acknowledges that models may generate incorrect or misleading information and should not be relied on as a sole source of truth. https://platform.openai.com/docs/guides/safety-best-practices - UK Government (2024). Artificial Intelligence Playbook for the UK Government. Describes generative AI as capable of producing factually inaccurate or fabricated outputs and calls for human oversight and independent verification before consequential use. https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government/artificial-intelligence-playbook-for-the-uk-government-html - Competition and Markets Authority (2023). AI Foundation Models: Initial report. Identifies false or misleading information in AI outputs as a consumer and business risk requiring regulatory attention. https://www.gov.uk/government/publications/ai-foundation-models-initial-report - ICO (n.d.). UK GDPR: Accountability and governance, enforcement and penalties. Sets out fines of up to £17.5 million or 4% of global annual turnover for serious breaches, relevant to personal data mishandled in public AI tools. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/guide-to-uk-gdpr/accountability-and-governance/enforcement-and-penalties/ - ONS (2023). Artificial intelligence adoption in the UK. Reports 15.1% of UK businesses using at least one AI technology in 2023, providing a primary-source benchmark for AI adoption claims. https://www.ons.gov.uk/businessindustryandtrade/business/businessservices/articles/artificialintelligenceadoptionintheuk/2023-10-04 - NCSC (2023). The security of artificial intelligence: a system of systems. Advises incorporating AI systems into standard risk and incident-management processes rather than treating them as reliable black boxes. https://www.ncsc.gov.uk/whitepaper/security-of-artificial-intelligence - FCA (2016, updated). FG16/5: Guidance for firms outsourcing to the cloud and other third-party IT services. Holds regulated firms responsible for the accuracy of their outputs even when those outputs use third-party AI tools. https://www.fca.org.uk/publication/finalised-guidance/fg16-5.pdf - BBC News (2023). ChatGPT: US lawyers sanctioned after using AI for fake court citations. Documents a case where fabricated AI-generated citations reached a court filing, with professional sanctions following. https://www.bbc.co.uk/news/world-us-canada-66030857 - BBC News (2024). Google AI Overview criticised for giving dangerous advice. Reports public failures in a mainstream production AI system, illustrating that factual errors occur in widely used tools. https://www.bbc.co.uk/news/technology-69090217 - Scottish AI Playbook (n.d.). How to write an AI policy. Provides practical SME-oriented guidance on creating verification controls and restricting AI use by risk level. https://www.scottishaiplaybook.com/how-to-write-an-ai-policy

Frequently asked questions

Why can't I just trust the source an AI gives me?

AI tools can generate plausible-sounding citations, including publication names and URLs, that do not exist or do not support the claim made. OpenAI's guidance acknowledges this explicitly. Any citation an AI provides should be treated as a starting point for your own search, not as confirmation that the figure is accurate.

Which types of AI statistics are most likely to be wrong?

AI adoption rates, market size claims, and regulatory deadlines are the most common failure points. The underlying data changes quickly and models may not have been trained on the most recent figures. ONS data from 2023 shows 15.1% of UK businesses used any AI technology at that point, which conflicts sharply with the much higher figures AI tools often cite.

Do I need to verify AI figures for internal use too?

For internal brainstorming and planning material clearly labelled as estimates, you can move quickly. For any figure that will inform a decision, appear in a document shared with others, or support a public claim, a primary source check is required. The liability is proportionate to use. A number in a compliance document carries far more exposure than one in a private planning note.

This post is general information and education only, not legal, regulatory, financial, or other professional advice. Regulations evolve, fee benchmarks shift, and every situation is different, so please take qualified professional advice before acting on anything you read here. See the Terms of Use for the full position.

Ready to talk it through?

Book a free 30 minute conversation. No pitch, no pressure, just a useful chat about where AI fits in your business.

Book a conversation

Related reading

If any of this sounds familiar, let's talk.

The next step is a conversation. No pitch, no pressure. Just an honest discussion about where you are and whether I can help.

Book a conversation