A hallucination is when the model confidently produces something that does not exist: an API method, a package, a CLI flag, a test run that never happened. It is not lying on purpose. A language model generates the most plausible next text, and when it does not know something, the most plausible text is a confident guess.
What it looks like in code
A method that does not exist. It looks logical, it type-checks in your head, and it is not there:
import { promises as fs } from "node:fs";
// Hallucinated: there is no fs.promises.exists in Node.
if (await fs.exists(path)) { /* ... */ }
// What exists: access() throws if the path is missing.
async function exists(path: string): Promise<boolean> {
try {
await fs.access(path);
return true;
} catch {
return false;
}
}An API from an older version. The model’s knowledge has a cut-off date; your package.json does not care. Typical cases:
getServerSidePropsin a Next.js App Router page. In theappdirectory, data fetching moved to async Server Components withfetch();getServerSidePropsandgetStaticPropsbelong to thepagesdirectory.- Zod 3 style in a Zod 4 project:
z.string().email()instead ofz.email(),messageinstead oferror,.merge()instead of.extend(). Many of the old forms still run in Zod 4 but are deprecated, so the code works today and breaks later.
A package that does not exist. The model suggests npm install with a plausible but invented name. That is also a security hole: attackers register the names that models tend to invent and publish malware under them. This is called slopsquatting. A USENIX Security 2025 study found that about a fifth of the packages suggested by the tested models did not exist, and many of the invented names came back again and again, which makes them predictable targets.
Invented flags and config options. A CLI flag or a config key that sounds right for the tool but was never part of it.
“The tests pass” when nobody ran them.
Why it happens
- Answering from memory instead of reading. The model writes the call from what it has seen in training rather than opening the code or the docs.
- A crowded or compacted context. When the context window is full of other things, or details were lost in compaction, the model fills the gaps with guesses.
- Stale knowledge. The model’s knowledge has a date; your dependencies have their own versions.
What to do about it
Two things work: make the model look things up instead of guessing, and give it a check that catches guesses before you see them. Both are in Verifying agent output. The short version: an agent that runs npx tsc --noEmit and the tests after every change catches most invented methods and fields on its own.
Related
- Verifying agent output: how to reduce hallucinations and catch the rest.
- Context window: why long or compacted sessions hallucinate more.
- Agentic loop: why an agent with a check can fix its own mistakes.