A fact, citation, case number, or statistic supplied by an LLM turns out not to exist — after it was pasted into a report, an email, or a customer reply. The output looked perfect: real-sounding authors, a plausible journal, a correctly formatted circular or case number, a confident tone. Someone downstream tried to look it up and found nothing. Now the sender's credibility, not the model's, is on the line.
Any workflow where model output travels without a verification step: research summaries, compliance and policy answers, competitor claims, "find me a source for this" prompts. Highest risk on obscure, specific asks — niche papers, exact rule numbers, small-company details, anything after the model's training cutoff. Small local models fabricate more than frontier ones, but every model does it. Linked module: 3.1 (How an LLM works at inference).
This is a hallucination, and it is structural, not a bug. The model generates the most plausible next token, and the PATTERN of a citation — authors, title, journal, year — is easy to produce whether or not the specific paper exists; no truth check runs anywhere in the loop. Knowledge inside the model is smeared statistics, not a fact table: common facts were reinforced millions of times and come out reliably, while rare details get interpolated — filled in with what is statistically likely. Confidence is a learned style, so a fabricated answer reads exactly like a correct one. The trap has a cruel shape: models are most reliable on well-known facts, where you least need them, and least reliable on the obscure ones you cannot eyeball. Confirm a suspected fabrication fast: search for the exact citation, and ask the same question in a fresh chat — an answer that changes between independent runs is a strong fabrication signal.
Contain first: correct the record with whoever received the fake fact, fast and plainly — credibility survives a quick correction far better than a discovered cover-up. Then verify every other load-bearing claim in the same document; fabrications travel in groups. Re-answer the original question properly: ground the model by pasting the real source document and asking about the pasted text, or use a tool with live search and click through to the sources. Log the incident: the prompt, the fabricated output, and the verification that caught it.
Adopt the rule "citations, numbers, quotes, and URLs are unverified until checked" for every model output that leaves your screen. Match scrutiny to stakes: brainstorming needs none, compliance answers need all of it. Prefer grounded workflows — retrieval over recall — for factual work, and add "if you are not sure, say so" to prompts; it helps, though it never fully solves it. Run the two-minute fabrication test from module 3.1 on any new model or tool before trusting it: ask for obscure citations and check them. Never use fluency or confidence as evidence of truth.
Q: "Your team's assistant cited a policy circular that does not exist, and it reached a client. What happened, and how do you stop the repeat?" Strong answer: define hallucination as plausible-text generation without a truth check, explain why obscure specifics are the danger zone, and lay out the fix in two layers — immediate correction and verification of siblings, then a grounded pipeline (retrieval or live search plus a human check for high-stakes facts). The differentiator: saying "the model is not lying — prediction has no concept of truth", because it shows you can design around the real mechanism instead of blaming the tool.