A support chatbot tells a customer that a discount can be claimed after the trip, while the company's own policy page says it cannot. A coding assistant suggests installing a library that was never published. A lawyer files a brief that quotes court decisions no court ever wrote. Each answer was fluent and specific, without a trace of doubt, and each was wrong. The AI field has a name for this failure: hallucination.
This post explains what AI hallucination is, why language models produce it, the main types, documented cases, how to spot one, how to reduce it with grounding and verification, and why it cannot be removed entirely.
What is AI hallucination?
AI hallucination is content generated by an AI model that is presented as fact but is wrong, made up or unsupported by its input. The word is borrowed from psychology as a metaphor: the model does not see or hear anything. It produces text, and some of that text does not match reality or the document it was asked about.
The US standards institute NIST prefers another word. Its Generative AI Profile (NIST AI 600-1) calls the risk confabulation, "the production of confidently stated but erroneous or false content", colloquially known as hallucination or fabrication. The definition also covers output that drifts from the prompt or contradicts the model's earlier statements in the same conversation.
Two things are not hallucinations in the strict sense: repeating an error that was already in the document you supplied, which is a source problem, and invention in a story or an image, where inventing is the task. Hallucination is a problem when the reader expects facts.
Why do AI models hallucinate?
AI models hallucinate because a language model is trained to predict plausible text, not to check it: it picks the next word piece (token) that best fits the patterns it learned, then the next. Our guide to how large language models work walks through that loop. Nothing in it looks up a fact, so a plausible wrong sentence and a plausible right one come out of the same process.
Researchers and standards bodies point to the same causes:
- Prediction rewards plausibility. NIST calls confabulation a natural result of how generative models are designed: they approximate the statistical patterns of their training data, and text that sounds right scores well whether or not it is true.
- Rare or wrong facts in the data. A fact that appears many times in the training data is usually reproduced well; a fact seen once or twice is not, and mistakes in the data are repeated fluently. In Why Language Models Hallucinate, researchers at OpenAI and Georgia Tech argue that for arbitrary facts with no pattern behind them, such as a person's birthday, some errors are statistically unavoidable for a base model.
- Training and tests reward guessing. The same paper notes that most benchmarks grade answers as simply right or wrong, so "I don't know" earns nothing while a guess sometimes earns a point. A model tuned for such tests learns to guess.
- Training data has an end date. A model knows nothing after its knowledge cutoff and can answer about a later event, price or rule from older patterns.
- Vague prompts and false premises. Asking why a product won an award it never won invites an answer built on that premise. Preference tuning can also make a model too eager to agree with the user, which researchers call sycophancy.
- Long or noisy context. In the study "Lost in the Middle" (Liu and colleagues, 2023), models answered best when the relevant passage sat at the start or end of a long input and worse when it sat in the middle.
- Decoding settings. Most tools sample from the predicted probabilities instead of always taking the top word; a higher randomness setting, usually called temperature, makes unlikely words and claims more frequent.
What types of AI hallucination are there?
AI hallucinations fall into two broad groups: answers that conflict with real-world facts and answers that do not follow the instructions or the text the model was given. A 2023 survey of hallucination in large language models by Huang and colleagues calls them factuality and faithfulness hallucinations. Earlier research on text generation used the terms intrinsic (the output contradicts the source) and extrinsic (the output adds something the source cannot confirm).
In practice the types look like this (the examples are invented):
| Type | What it looks like | Example |
|---|---|---|
| Factual error | A wrong date, figure or name | "Brand A was founded in 2009"; its site says 2012 |
| Invented source | A citation, quote, link or case that does not exist | A study with real-sounding authors and a dead link |
| Invented package | Code that imports a library nobody published | An install command for an unknown name |
| Contradicts the source | A summary that reverses the document | "Returns within 30 days"; the policy says 14 |
| Adds to the source | Details the document never mentions | A CEO quote that is not in the article |
| Self-contradiction | Conflicts with an earlier answer | Two different totals in one chat |
Invented sources are convincing because a citation is a pattern (authors, title, journal, year, DOI) that a model can reproduce with no paper behind it. Code works the same way: a 2024 study of code-generating models in Python and JavaScript, accepted at the 2025 USENIX Security Symposium, found models recommending packages that do not exist and repeating many of the same invented names, so an attacker can publish a malicious package under such a name and wait.
What are real examples of AI hallucination?
The best-documented cases come from courts and tribunals, whose decisions record what was said and what it cost.
Invented case law in a US court. In Mata v. Avianca, a federal court in New York sanctioned two lawyers and their firm on 22 June 2023 for submitting non-existent judicial opinions with fake quotes and citations created by ChatGPT. When one lawyer asked the chatbot whether a case was real, it said yes and that the case could be found in legal databases. The court wrote that using a reliable AI tool for help is not improper in itself, but lawyers must check what they file. It imposed a penalty of USD 5,000 and ordered letters to the client and to each judge falsely named as the author of a fake opinion.
A chatbot's wrong policy answer. In Moffatt v. Air Canada (2024 BCCRT 149), an airline's website chatbot told a passenger that a reduced bereavement fare could be requested within 90 days of the ticket being issued, while the airline's own policy page said it did not apply after travel. On 14 February 2024, British Columbia's Civil Resolution Tribunal rejected the argument that the airline was not liable for its chatbot, held it responsible for all information on its website and ordered it to pay CAD 812.02. The decision does not say what technology the chatbot used; its lesson is that an automated answer is still the company's answer.
Confident and different every time. In the OpenAI and Georgia Tech paper, the authors asked three popular chat models for the title of one author's doctoral dissertation. Each gave a different title and year, and none was correct.
How can you spot an AI hallucination?
You spot a hallucination by checking the specific claims against a source you can open, because the tone of the answer tells you nothing. These signs deserve a second look:
- Precise details without a source. Exact figures, dates, quotes or case numbers with no link or reference.
- Sources you cannot open. A link that returns an error, a DOI that leads nowhere, a quote you cannot find in the cited text.
- Detail on an obscure topic. The rarer the subject, the less the model learned about it and the more it fills in.
- Answers that change. Ask again in a new chat or rephrase. If the facts change, at least one version is wrong.
- Self-confirmation. Asking the same model "Is this real?" proves nothing, as the Mata case showed.
- A summary that knows too much. Any detail you cannot find in the summarised document.
How do you reduce AI hallucinations?
You reduce hallucinations by giving the model the facts it needs, letting it decline when it does not know and checking its claims before use. Teams combine several steps:
- Ground the answer in sources. Retrieval-augmented generation (RAG) fetches relevant passages from a document index when a question arrives and has the model answer from them, citing each one. Preparing web pages for such an index is covered in Web Scraping for RAG.
- Restrict the model to the supplied text. Tell it to answer only from the documents provided, to say when the answer is not there and to quote the sentence each claim rests on.
- Make "I don't know" acceptable. State that declining is better than guessing. The OpenAI and Georgia Tech authors propose the same for benchmarks: a wrong answer should cost more than no answer.
- Ask for sources, then open them. Requested citations can be invented too.
- Lower randomness for factual tasks. A low temperature makes answers more consistent but adds no knowledge; in the package study, changing decoding settings did not reduce invented names.
- Use tools for facts and figures. Calculations, dates and current information should come from a calculator, code, a database or a search, not from the model's memory.
- Keep the context focused. Send the relevant passages, not a whole archive, and put the key material near the start or end of the prompt.
- Review high-stakes output. Legal, medical and financial content, and anything sent to customers, needs a person who checks each claim; NIST advises closer monitoring wherever AI output feeds consequential decisions.
- Measure it. Keep 30 to 50 questions with known answers from your own documents and count the wrong answers after every change.
What does grounding mean, and why do the sources matter?
A grounded answer is one whose claims can be traced to specific source passages supplied at answer time, so every statement can be checked against the text it came from. Grounding does not make a model truthful by itself; it moves the risk to retrieval. If retrieval returns the wrong passage, an outdated page or a page written for another market, the model answers confidently from it.
Retrieval quality is half of the work. Most retrievers find passages by meaning rather than exact words, as explained in What Is Semantic Search?, and each passage should carry its URL and fetch time so answers can cite it.
Location is the part teams often miss. Prices, stock, delivery terms and local rules differ by country and city, and many sites choose the version they show from the visitor's IP address. A pipeline that fetches every page from one data center sees one version and answers users elsewhere about a market they are not in. Collecting public pages as a local visitor sees them, for example through a Residential Proxy with country and city targeting, keeps the source and the user's market aligned. Collect only public pages, respect robots.txt and the site's terms, keep request rates low and use an official API where one exists.
Teams that would rather not run the collection themselves can talk to our sales team about managed page extraction through our Web Scraping API.
Can AI hallucinations be eliminated?
No. Hallucinations can be made rarer and easier to catch, but a general-purpose language model answering open questions will still produce them. NIST treats confabulation as a consequence of how generative models work, and the OpenAI and Georgia Tech paper argues that well-trained base models should still make certain errors; a system avoids them only by declining to answer what it does not know.
RAG does not remove the problem either. The survey cited above devotes a section to its limits: the retriever can miss the right passage, and the model can still misread the context. The realistic goal is fewer errors, easy to detect and caught before they cause harm.
Where do you meet AI hallucinations?
You meet them wherever a model writes text that someone treats as fact: chat assistants, AI-written summaries above search results, coding assistants, customer service bots and tools that summarise contracts, papers or meeting transcripts.
The risk grows when a model acts as well as writes. An AI agent that books, buys, edits files or sends messages can turn an invented fact into an action.
Common mistakes
- Treating fluent text as correct. Confidence and grammar are not evidence.
- Asking the model to verify itself. Use an independent source.
- Trusting a citation without opening it. Fake references look exactly like real ones.
- Assuming RAG means no hallucinations. Retrieval reduces them and adds its own failure points.
- Expecting a low temperature to fix facts. It reduces variation, not ignorance.
- Launching a customer-facing bot without safeguards. It needs a "don't know" path, logs and a hand-off to a person.
- Installing a package because an assistant named it. Check the official registry first.
Decision guide
| Need | What to do |
|---|---|
| A quick general question | Verify any fact you reuse in one reliable source |
| Answers about your own documents | RAG with citations, answers only from the text |
| A customer-facing chatbot | Current policy pages, a decline path, logs, human hand-off |
| Legal, medical or financial content | Expert review of every claim and citation |
| Code from an assistant | Check each package in its registry, run the tests |
| Prices, stock or local rules | Current pages from the market concerned |
| Tracking progress | A fixed test set with known answers |
Frequently asked questions
Why does AI make things up?
AI makes things up because a language model generates the most plausible continuation of a prompt, and plausible is not the same as true. When its training data held little on a topic, it still produces an answer shaped like the right one. Benchmarks that give no credit for "I don't know" also teach models that guessing pays.
Can AI hallucinations be fixed completely?
No. Grounding, permission to decline, tools and human review make hallucinations rarer and easier to catch, but no current method removes them from a general-purpose model. Plan for errors and decide where a person must check the output.
Is an AI hallucination the same as lying?
No. Lying requires an intention to deceive, and a model has none; it produces likely text without knowing whether it is true. The effect on a reader can be the same, which is why whoever publishes an AI answer stays responsible for it, as the airline chatbot decision showed.
How do I check an AI answer?
Check the claims that matter against a source you can open yourself: the official page, the original document, the package registry or the court record. Open every link, search for quotes word for word and ask again in a new chat to see whether the facts hold. Never let the same model be the only judge of its own answer.
Does RAG stop hallucinations?
RAG reduces hallucinations but does not stop them. It gives the model current, relevant text and lets readers check the cited passages, but the retriever can return a wrong or outdated page and the model can still misread what it was given. Citations and spot checks remain necessary.
Why do AI tools invent sources?
AI tools invent sources because a reference is a pattern the model has seen countless times: authors, a title, a journal, a year and a link. It can produce that pattern with no real document behind it and cannot look the reference up by itself. Tools that retrieve and cite real documents reduce the problem; every citation is still worth opening.
Summary
AI hallucination is confident, fluent output that is false, invented or unsupported by the input. It follows from how language models work: they predict plausible text, learn rare facts poorly, stop learning at a cutoff date and are often trained in ways that reward guessing. The documented cases show that whoever uses the answer pays for the error. Grounding in current sources, permission to say "I don't know", tools for facts and human review make errors rarer and easier to catch. Where answers depend on prices, stock or rules that differ by place, the sources have to come from the user's market.




