Hallucinations
Hallucinations are generated statements that are unsupported, fabricated, or misleading relative to the task evidence. The governance problem is not that a model “sounds creative”; it is that users may treat fluent unsupported text as fact. The control surface overlaps with factual correctness, prompt injection, and hallucination mitigation.
Where the failure entered
A hallucination review should classify each unsupported output by where the failure entered:
| Failure point | Example | Likely fix |
|---|---|---|
| Retrieval miss | Correct document absent from context | Improve grounding and retrieval tests |
| Context misuse | Evidence present but ignored | Prompt and decoding regression tests |
| Unsupported synthesis | Model combines facts into an unstated conclusion | Claim-level citation checks |
| Knowledge boundary failure | Model answers when source is silent | Abstention policy |
| Attack-induced claim | Retrieved text instructs model to lie | Adversarial evaluation |
This is why “use RAG” is not a complete hallucination control. Retrieval can reduce uncertainty, but the answer still needs claim-level support and an abstention path.
Worked abstention comparison
A four-question review compares a baseline answer policy with an abstention policy:
| question type | baseline supported? | abstention policy supported? |
|---|---|---|
| refund window | yes | yes |
| dental surgery coverage | no | yes, abstains |
| fax cancellation | no | yes, abstains |
| admin MFA | yes | yes |
The baseline has 2 unsupported claims out of 4 answers, so its unsupported-claim rate is 0.50. The abstention policy has 0 unsupported claims out of 4 reviewed outputs because it replaces two guesses with “I cannot determine…” responses. That improves factuality but may reduce apparent helpfulness, so the metric should be reported beside coverage and user escalation rates in the error taxonomy.
Caveats
Unsupported claim rate depends on the evidence standard. A legal assistant, support bot, and brainstorming tool should not share one threshold. Also watch for citation laundering: a model can attach a real citation to a claim the source does not support. Store prompts, retrieved passages, output, and reviewer labels so auditability can reproduce the failure.
References
- NIST AI 600-1: Generative AI Profile
- Ji et al., Survey of Hallucination in Natural Language Generation
- OWASP LLM09:2025 Misinformation
Nav