Glossary
AI hallucination
What an AI hallucination is, and how an AI job is set up to catch one.
Updated
AI hallucination: An AI hallucination is a reply in which a language model states something false as if it were true, such as a price the business never set or a source that does not exist. It happens because the model writes the likeliest-sounding words, and nothing in that step checks them against a fact. NIST's Generative AI Profile of July 2024 calls it confabulation, known colloquially as hallucination, and lists it among the risks of generative AI. The usual guard is to hold the model to the business's own sources and send any question they do not answer to a person.
Where it shows up
People reach for the word when an AI tool states a detail wrongly with full confidence, such as a chat assistant quoting a policy a business never had. The risk is highest when a model answers from memory alone, with none of the business's own material in front of it.
Where an AI job we build answers questions, it is set to answer from the sources its written plan names and to say so when they do not cover a question. The team assistant goes further and shows which record or playbook line each answer came from.
Related terms
- Large language modelWhat a large language model is, and what it needs from a business before it can answer that business's customers.
- Human handoffWhat a human handoff is, and what should happen when a caller or a chat visitor asks an AI for a person.
- Answer engineWhat an answer engine is and how it changes the way customers find a small business online.
- Every term in the glossary
Start with a written plan
Tell us about the business and what you need. We reply with questions, then a written plan that lists the work and its price, and nothing starts until you have read it.