What an AI "hallucination" means and how to check each kind of failure

"Hallucination" is a convenient word, but it can hide important differences. It does not describe an AI having a human experience, nor does it prove any intent to deceive: in practice, it names an output that looks useful or safe but is not sufficiently supported by the facts. Lumping it all together prevents action, because each kind of failure is checked differently. The capability you take away from here is not distrusting every generated text, but identifying which kind of failure you are facing and applying the check that fits it.

It is worth starting with the word itself, because part of the problem lives there. "Hallucination" borrows a term from human perception and, in doing so, suggests the model "sees" something that is not there — a deceptive mental image. But a language model does not perceive: it chains probable words. Calling it a hallucination invites two opposite and equally dangerous errors: humanizing it, as if it had an experience to correct, or excusing it, as if the failure were a mysterious accident rather than measurable behavior. The four classes that follow replace the metaphor with something actionable: not "what happened to the model," but "what I would have to check."

Factual error

A factual error asserts something checkable that is false: a date, a figure, a name, or a property. The right check is to go to a primary source or a reliable record and verify the exact datum. The TruthfulQA benchmark measures precisely this kind of truthfulness, and its design teaches something uncomfortable: its 817 questions, across 38 categories such as health, law, finance, and politics, are deliberately built to provoke false answers that many humans would also give out of a widespread misconception.

The result worth keeping is twofold. First, the size of the gap: in that study, the best model was truthful on 58% of the questions, versus 94% for humans. Second, and more revealing, the direction: the largest models were generally the least truthful, unlike almost any other language task, where growing improves the result. The authors read it as a deeper warning: scaling the model alone promises less truthfulness than fine-tuning it with objectives other than imitating web text. For the reader, the consequence is direct: a newer or larger model is not more reliable on a specific fact — that fact still has to be looked up.

Fabricated citation or source

Here the problem is not only the answer: the model attributes a link, author, study, or page that does not exist or does not contain what it claims. The rule is simple: open the source. Check the title, author, date, and the relevant passage. A citation that looks academic is not evidence until the document sustains the specific sentence.

The GopherCite work illustrates both the help and its limit. It is a 280-billion-parameter model trained to accompany each answer with verbatim quotes drawn from real documents — from a search engine or from a user-provided document — so that the claim can be checked. In human evaluation, its answers were rated high-quality 80% of the time on one set of search questions and 67% on another of open questions; and when allowed to abstain on the most uncertain third, it rises to 90% and 80%. But the authors themselves mark the limit that matters: not all claims supported by a quote are true. A real source can exist and still fail to support the decisive sentence. That is why you check the relation, not just the existence — that the document says what the answer attributes to it.

Over-inference

Sometimes all the cited data are real, but the conclusion goes further than they support. "This company presented a pilot" does not prove "the technology has already transformed its operation"; "a study observed a relationship" does not prove causation, nor does "it beat a human on a test" amount to "it works for your case." It is the hardest failure to see, because each link is true and only the jump between them fails. The check is not of data but of logic: ask the answer to separate explicitly what is written in the source, what is the model's conclusion, and what remains unknown. When those three layers blend into a terse paragraph, the confidence of the tone replaces the strength of the argument.

Acknowledged uncertainty

The most reliable answer is sometimes "I cannot verify this." Acknowledging a limit is not a failure if it avoids inventing — but the evaluation systems themselves push the other way. A paper from OpenAI and Georgia Tech, signed by Adam Tauman Kalai, Ofir Nachum, and Santosh Vempala in September 2025, argues that many benchmarks reward guessing and penalize abstaining: under a score that counts only correct answers, guessing is the dominant strategy, because a model that answers almost everything outscores a cautious one that says "I don't know" when it does not know, even if the second makes many fewer errors. The authors propose the opposite: directly penalize incorrect answers and reward well-expressed uncertainty. The example they give is instructive: on the SimpleQA test, a model that answers almost every question — and gets more than three out of four wrong — earns a better raw score than one that abstains when unsure and makes far fewer errors. The scoring rewards the reckless and punishes the prudent. The authors also frame the phenomenon at its root: a hallucination is neither magic nor malice, but a binary-classification error; if a system cannot always tell a true statement from a false one, those errors emerge from the sheer statistical pressure of training. OpenAI's plain-language version sums up the idea. For the user, the practical reading is clear: a confident answer without evidence deserves more caution than an honest uncertainty.

The routine, proportional to the risk

The four classes call for different checks — a primary source for the datum, opening the link for the citation, separating facts from inferences for the logical jump, and valuing abstention as a virtue rather than a defect — and the discipline is to match the effort to the risk. For a minor datum, verify the source and move on. For an important decision, ask for documents, method, date, and human review before acting. And there is a useful asymmetry that orders the effort: the first two failures — the false datum and the fabricated citation — are the cheapest to catch, because they are resolved by opening a source; the last two — the logical jump and false confidence — are the most expensive, because every piece is correct and only the glue between them fails. That is why the trained reader spends distrust where it pays: not in doubting every date, but in watching the moment a well-written answer slides from what the source says to what you would like to believe. This routine survives any specific model, because it does not depend on which system generated the text, but on what would have to be true for the answer to be true. Knowing what question to ask when an answer sounds too neat is a capability that will keep working when today's model is history.