Definition
Hallucination (AI)
An AI hallucination is output from a language model that is fluent and confident but factually wrong, fabricated, or unsupported by the model's inputs — invented citations, fake quotes, or made-up details. It arises because models predict plausible text rather than retrieve verified facts, making hallucination an inherent risk that must be mitigated, not a bug that's simply fixed.
Key takeaways
- An AI hallucination is fluent, confident output that is factually wrong, fabricated, or unsupported by the model's inputs.
- It arises because language models predict plausible text rather than retrieve verified facts — fluency is not truth.
- Common forms include invented citations, fake quotes, confident false facts, and contradicting the provided context.
- Mitigation is layered (retrieval grounding, citations, lower randomness, human review), not an absolute fix, because the cause is architectural.
Large language models generate text by predicting the most plausible next token given everything before it. They optimize for fluency, not truth, and have no built-in mechanism to know whether a statement is factual. When the training data is thin or the prompt invites speculation, the model confidently fills the gap with text that reads right but isn't — a hallucination.
Hallucinations take several forms. A model may invent a citation or legal case that never existed, attribute a fabricated quote to a real person, state a false fact with total confidence, or contradict information present in its own context. The danger is that fluency masks the error: the output carries the same authoritative tone whether it's correct or invented.
Mitigation is layered, not absolute. Retrieval-augmented generation grounds answers in retrieved source documents so the model summarizes rather than invents. Asking for citations, constraining the model to provided context, lowering randomness, and adding a verification or human-review step all reduce the rate. But because the cause is architectural, the responsible posture is to design systems that assume hallucination is possible and check accordingly.
Planoda treats AI output as proposals, never authoritative actions: agents operate under propose/approve governance with their suggestions grounded in your workspace data and recorded in an audit trail for human confirmation.
Related terms
- Large Language Model (LLM)A large language model is an AI system trained on vast amounts of text to predict and generate language, enabling it to answer questions, summarize, write, and reason over natural language. LLMs power modern AI assistants and agents. They are probabilistic next-token predictors, which makes them remarkably capable but also prone to confident errors.
- Retrieval-Augmented Generation (RAG)Retrieval-augmented generation (RAG) grounds a language model's answers in your own data by retrieving the most relevant documents at query time and feeding them into the model's context before it responds. Instead of relying solely on what the model memorized in training, RAG lets it answer from current, authoritative, private sources — sharply reducing hallucination.
- Prompt InjectionPrompt injection is an attack on AI systems where malicious instructions hidden in user input or retrieved content trick a language model into ignoring its original directives — leaking data, performing unauthorized actions, or producing harmful output. Because models can't reliably separate trusted instructions from untrusted data in their context, it is a fundamental security risk for any LLM application.
- AI AgentAn AI agent is a software system that uses a large language model to pursue a goal across multiple steps — reading context, choosing tools, and taking actions — rather than answering a single prompt. In a work platform, agents triage issues, draft updates, and execute multi-step tasks as autonomous teammates, bounded by the permissions and approvals their operators set.
- Propose / Approve (AI Governance)Propose/approve is a governance pattern for autonomous software: instead of executing a consequential action directly, an AI agent emits it as a proposal that a human or policy must approve before it runs. It keeps fast, read-only work autonomous while gating destructive or irreversible operations — the practical way to give agents real power without surrendering control.