On this page
To verify an AI statistic, demand the source before accepting the number, locate the original document, check its methodology and publication date, read the full sentence around the statistic to catch missing context, and cross-check against other credible sources. If you cannot reach the source or verify the number appears in it, exclude the statistic from your work.
- AI systems hallucinate statistics at measurable rates, from 0.7% to 30% depending on the model and domain, and as high as 69-88% on legal queries.
- Never accept a statistic from an AI without asking where it came from; AI confidence is not evidence of accuracy.
- A statistic can be accurate but obsolete or based on flawed methodology, so evaluate the source's authority, publication date, and study design before using it.
- AI commonly mangles statistics by rounding incorrectly, confusing units, citing outliers as averages, or stripping away qualifications like 'up to' or 'in one study'.
- If you cannot locate the original source, verify the number appears in it, or confirm it through independent sources, exclude the statistic from your report.
When an AI tool gives you a statistic, you face a choice: use it or verify it. The cost of using it wrong is high. A misquoted figure can undermine a report, mislead a decision, or damage your credibility. The cost of verifying it is time. This guide teaches you the workflow to catch the most common ways AI statistics fail before they reach your work.
AI hallucinations are instances where an AI system produces outputs that sound plausible but are factually wrong, irrelevant or entirely fabricated. Studies report hallucination rates ranging from 0.7% to nearly 30% depending on model and domain. On legal queries specifically, rates climb as high as 69-88%. In one documented case, ChatGPT fabricated six court cases with fake docket numbers and invented legal reasoning, resulting in a $5,000 sanction against the lawyers. The risk is real. The good news is that most AI errors follow predictable patterns, and you can learn to spot them.
What you'll be able to do
After working through this guide, you will be able to evaluate whether an AI-provided statistic is safe to cite in your own work. You will know how to ask the right questions about a source, how to recognize when an AI has given you a source that does not exist, and how to spot the most common ways a real statistic gets mangled in the retelling. You will also know when to exclude a statistic because you cannot verify it, and how to flag that uncertainty if you must mention it anyway.
Demand the source before you demand the number
The first rule is simple: never accept a statistic from an AI without asking where it came from. When an AI tells you a number, your immediate response should be to ask for the source. Not "Is this true?" but "Where did this come from?"
AI's confidence is not evidence. An AI can state a false statistic with the same tone and certainty as a true one. The system has no internal alarm that goes off when it is guessing. It has no way to distinguish between a real study it learned about during training and a plausible-sounding study it invented. Your job is to supply that alarm by demanding proof.
When you ask for a source, you are asking for one of these: a peer-reviewed journal article, a government report or dataset, a company's own filing or press release, a book or textbook, a news article from a credible outlet, or a specialized database. You are not asking for "the internet" or "research" or "studies show". You are asking for something specific enough that another person could find it and verify the number themselves.
Locate the original source document
Once you have a source name, your job is to find the actual document and confirm the statistic appears in it. This is where many AI errors reveal themselves. The AI may have cited a source that does not exist, misattributed a statistic to the wrong source, or paraphrased a number so badly that the original is unrecognizable.
Search for the source by author, date, and title
Start with the most specific information the AI gave you. If it cited a journal article, search by author name and publication year. If it cited a company report, search by company name and year. If it cited a government dataset, search by agency and dataset name. Use quotation marks around exact phrases to force the search engine to find that exact string. Search for the title of the document first, then the author, then the year. If you cannot find it after searching three different ways, the source likely does not exist. Stop there and mark the statistic as unverifiable.
Verify the source actually contains the statistic
Finding a document with the right title and author is not enough. You must open it and search for the specific number the AI cited. Use your browser's find function (Ctrl+F or Cmd+F) to search for the statistic. Search for the number itself, then search for key words around it. If the number does not appear in the document, the AI misattributed it. If a different number appears in the same sentence, the AI rounded, converted units, or conflated two different statistics. Do not assume the AI's version is correct. Write down what the document actually says.
Check the source's methodology and date
A statistic can be accurate and still be wrong for your purposes. It may be based on outdated data, a flawed study, or a narrow population that does not apply to your question. Before you cite a number, evaluate the source that produced it.
Confirm the publication date and relevance window
Check when the source was published. A statistic from a study conducted in 2010 may not reflect current conditions. A survey of consumer behavior in one country may not apply to another. Ask yourself: Is this statistic current enough for my use case? If I am writing about trends in 2024, is a 2015 study still relevant, or has the landscape changed? If the source is behind a paywall or too old to be useful, exclude it or note the limitation explicitly in your work.
Evaluate the source's authority and potential bias
Not all sources are equally credible. Peer-reviewed research has been evaluated by field experts before publication to ensure quality, validity, and originality. A company's own earnings report is authoritative on its own revenue but may be selective in what it highlights. An advocacy group's report may be accurate but shaped by the group's mission. Government data is often reliable but can be collected with different standards across agencies or time periods. Ask: Who produced this source? Do they have a reason to bias the number in one direction? Has the methodology been scrutinized by others? If the source is a peer-reviewed study, look for the study design. If it is a company report, check whether the number has been independently verified. If it is advocacy material, cross-check the statistic against neutral sources.
Verify the number hasn't been misquoted or decontextualized
Even when the AI cited a real source and found the right number, the statistic can be mangled by stripping away crucial context. This is where AI most commonly fails. The system may round a number incorrectly, confuse units, cite an outlier as the average, or drop a qualification that changes the meaning entirely.
Read the sentence around the statistic in the original source
Do not read the number in isolation. Read the full sentence in the original source. Look for qualifications: "In one study" is not the same as "across all studies". "Among men aged 65 and older" is not the same as "in the general population". If the AI dropped a qualifier, the statistic is decontextualized. If the original says "up to" and the AI said "is", the meaning has changed. Write down the full sentence from the source, not the AI's paraphrase.
Check for unit conversions or aggregations
Watch for unit confusion. If the source reports a value in millions and the AI states it in billions, the AI may have converted correctly or may have made an error. Check the math. If the source reports a percentage and the AI reports a percentage point, these are different things. When comparing percentages, percentage points should be used to describe the difference between them, not percent change. For example, a diagnosis rate that increased from 1% in 1981 to 5% in 2021 rose 4 percentage points, not 4 percent; the rate actually jumped 400 percent or five-fold. If the source reports nominal values and the AI compares them across decades, check whether the AI adjusted for inflation. Nominal values must be adjusted for inflation to arrive at real or inflation-adjusted indicators that show what is actually happening to sales levels relative to any time period. Comparing nominal values across different years without adjustment is highly misleading.
Cross-check against other credible sources
The strongest verification is independent confirmation. If you can find the same statistic reported by at least one other credible source, you have caught both AI errors and source errors. Cross-referencing is the practice of checking multiple sources to confirm the validity of a statement and helps ensure accuracy, identify bias, and strengthen credibility. Search for the statistic in a different database, a different news outlet, or a different research organization. If the number appears consistently, you have more confidence. If other sources report a different number, investigate why. The difference may reveal that the AI cited an outlier, a subgroup, or an outdated version of the data.
Recognize when you cannot verify a statistic
Sometimes you will reach a dead end. The source does not exist. The source is behind a paywall you cannot access. The source is so obscure that no search engine can find it. The number appears in the source but is qualified in a way that makes it unsuitable for your use. In these cases, you have three options.
First, exclude the statistic. If you cannot verify it, you should not cite it. Your credibility depends on being able to defend every number you use. If you cannot defend this one, leave it out.
Second, find a different statistic that makes the same point. Often there are multiple ways to express an idea. If one statistic is unverifiable, another may not be.
Third, if the statistic is essential and you cannot verify it, flag the uncertainty explicitly in your work. Write: "According to [source], [statistic], though this figure could not be independently verified." This tells your reader that you are aware of the limitation. It is not ideal, but it is honest.
Common failure modes and how to spot them
AI statistics fail in predictable ways. Learning to recognize these patterns will help you catch errors before they reach your work.
Hallucinated sources are the most dramatic failure. The AI invents a study, a report, or a dataset that sounds plausible but does not exist. You will catch this when you search for the source and find nothing. The AI may cite "a 2019 study by researchers at Stanford" or "data from the World Health Organization" that, when you search, does not exist in the form described. If you cannot find the source after searching by author, date, and title, assume it is hallucinated.
Conflated studies happen when the AI mixes up two different sources. It may cite the right author but the wrong year, or the right title but attribute it to the wrong journal. It may combine findings from two studies into one. You will catch this when you find the source the AI cited and the number is not there, but you find a similar number in a different study. The AI confused the two.
Outdated data presented as current is common when the AI was trained on older information. It may cite a statistic from 2015 as though it reflects today's conditions. You will catch this by checking the publication date. If the source is more than a few years old and your question is about current conditions, the statistic may be stale.
Numbers that sound too precise are a red flag. If an AI gives you a statistic like "47.3%" or "$2.847 billion", search for that exact number in the source. Precise numbers are less likely to be hallucinated, but they are also less likely to be rounded the way the AI reports them. If the source says "approximately 47%" and the AI says "47.3%", the AI invented the decimal places.
Missing context is the most subtle failure. The AI may cite a real statistic from a real source but drop a qualifier that changes the meaning. "In a small early-stage trial" becomes invisible. "Among patients who completed the full treatment" becomes "in patients". You will catch this by reading the full sentence in the original source and comparing it to what the AI said.
Do this automatically
Manually verifying each AI-provided statistic requires searching for source documents, locating the original data, cross-referencing the numbers, and evaluating methodology, a process that multiplies in time cost across dozens of statistics in a single project. Software can automate the source-finding, cross-referencing, and methodology-checking steps, reducing verification from hours to minutes per statistic. IMBeacon.ai runs this kind of analysis end to end and cites every figure back to its original source, across as many subjects as you need, with Mimir attaching a citation to every figure. It will not make the decision for you. It gathers and cites the evidence; the judgment stays yours, and deliberately so. Every plan includes it, including the free one.
Start your first report and run it on something you already know well.
This article is general information and research commentary. It is not professional advice, and it does not account for your particular circumstances.
