—Glossary
AI hallucination
An AI hallucination is an output from an AI model that sounds fluent and confident but is false, unsupported by its source material, or invented outright.
AI hallucination is the name for a large language model producing something that is not true: a wrong price, a policy that does not exist, a made-up citation. The term is informal. The model is not seeing things; it is generating text that fits the pattern of a good answer without checking it against facts.
Why it happens
A language model predicts likely text. It has no built-in sense of whether a statement is true. Hallucinations are more likely when the answer is not in the information the model was given, when a question asks for specifics such as numbers, names or dates, or when the instructions push the model to always give an answer.
Why it matters in a business
A customer-facing assistant that hallucinates can quote the wrong fee, promise a service you do not offer or give incorrect opening hours. Customers reasonably treat what your assistant says as what your business says.
How to reduce it
Hallucination cannot be removed completely, so good systems are designed around it:
- Ground answers in your own content with retrieval-augmented generation.
- Fetch live facts, such as availability and order status, from the real system instead of letting the model generate them.
- Limit the assistant's scope, and instruct it to say it does not know, then hand off, when the answer is not in its sources.
- Test with real customer questions before launch, and review transcripts after.
How it relates to Worcia
Assistants in Concierge, Worcia's AI receptionist, are built to answer from approved sources and to hand off when they cannot.