What an AI “Hallucination” Means and How to Check Each Kind of Failure
Not every false answer fails in the same way. Separating factual error, invented citation, overreach and acknowledged uncertainty helps verify AI without assigning intent.
“Hallucination” is a convenient word, but it can hide important differences. It does not mean an AI has a human experience, and it does not prove intent to deceive. In practical use, it names an output that looks useful or confident but is not sufficiently supported by facts. To decide what to do, classify the failure.
Factual error
A factual error states something checkable that is false: a date, number, name or property. The right check is to go to a primary source or reliable record and compare the exact claim. TruthfulQA studies false answers to questions designed to invite misconceptions. It measures one kind of truthfulness; it does not establish model intent.
Invented citation or source
Here the problem is not only the answer: the model attributes a link, author, study or page that does not exist or does not say what it claims. Open the source. Check title, author, date and the relevant passage. An academic-looking citation is not evidence until the document supports the specific sentence.
GopherCite explains why verified citations are only part of the solution: real evidence can still fail to support the decisive claim. Check the relation, not just existence.
Overreaching inference
Sometimes cited facts are real but the conclusion goes further. “A company announced a pilot” does not show “the technology transformed operations.” “A study observed a relationship” does not prove causation. Ask an answer to separate facts, inferences and what remains unknown.
Acknowledged uncertainty
The most reliable response is sometimes “I cannot verify that.” Acknowledging a limit is not failure when it prevents invention. OpenAI’s research argues that many evaluation systems reward guessing and penalise non-answers. A confident answer without evidence deserves more caution than clear uncertainty.
Use proportional checks: for a minor fact, verify the source; for an important decision, request documents, method, date and human review. The goal is not to distrust every generated sentence. It is to know what question to ask when an answer sounds too complete.
Primary sources: TruthfulQA, verified quotations and OpenAI.
This article was produced with artificial intelligence under human editorial oversight.
