|
 |
When Humans Hallucinate: What Tech Writers Can Learn from AI's Mistakes
Before we judge AI for being confidently wrong, it may be worth pointing out how often we are incorrect, too
|
| |||||||||||||||
Christopher Noessel, an interaction designer, author, and AI experience architect whose work focuses on how people and intelligent systems work together, recently shared a Facebook post responding to a billboard criticizing generative AI. The billboard message was straightforward: you would not hire an employee who was wrong 10 percent of the time, so why would you trust AI?
The criticism is fair. Large language models are known to invent facts, confuse details, and produce answers that sound convincing even when they are wrong. What interested me more was the assumption underneath the slogan. The comparison only works if people are reliable.
Tech writers spend enough time interviewing subject matter experts to know that isn’t always true. Ask several people how a product feature works and you may hear several explanations. One engineer describes the original design. Another explains how the system behaves today. Support remembers the workaround customers use. Product management recalls a decision that was never documented.
Nobody intends to mislead anyone, yet the facts often arrive tangled together. Much of the work of technical writers involves sorting through those competing accounts.
 |
Human Hallucinations Are More Common Than We Admit
Chris’s post assembled several examples that are uncomfortable precisely because they involve ordinary people rather than obvious bad actors.
A YouGov survey found that 2 percent of Americans said they firmly believe the Earth is flat. Another 7 percent said they were unsure about the Earth’s shape. Among younger adults, the uncertainty was even higher.
A separate Economist/YouGov poll found that 12 percent of Americans agreed that the Moon landing was staged.
Related reading: Conspiracy vs. Science: A Survey of U.S. Public Beliefs
The so-called Mandela Effect offers another example. In a 2022 YouGov survey, 40 percent of Americans said they had shared a memory with others that many people remembered differently or believed to be false. When asked about specific examples, 61 percent remembered the children’s books as the “Berenstein Bears,” even though the books have always been published as the “Berenstain Bears.”
The same survey found that 28 percent of respondents considered parallel universes or alternate dimensions a plausible explanation for these shared memories.
 |
These examples are easy to dismiss because they seem unusual, but memory errors appear in situations with much higher stakes. Eyewitness testimony remains one of the most persuasive forms of evidence presented in court. Yet the Innocence Project reports that eyewitness misidentification played a role in roughly 69 percent of convictions later overturned through DNA evidence.
One of the best-known cases involved Jennifer Thompson, who deliberately studied her attacker’s face so she could identify him later. She selected Ronald Cotton from a lineup, testified against him in court, and remained convinced she had identified the correct person. Cotton spent more than a decade in prison before DNA evidence established that another man had committed the crime.
Why Organizations Write Things Down
Organizations create documentation for many reasons. New products need instructions, APIs need references, and customers need help completing tasks. Over time, documentation also becomes a record of decisions, product behavior, terminology, and procedures that people may remember differently or forget altogether.
Most documentation teams eventually discover procedures that nobody follows, product behavior that differs from the manual, and explanations that change depending on who is asked. Generative AI did not create these problems.
When an AI system produces an incorrect answer, the mistake appears immediately. Human errors often move more slowly. They become outdated procedures, undocumented assumptions, conflicting explanations, and organizational folklore that survives long after the product changes.
Every tech writer has heard someone begin a sentence with, “We’ve always done it this way,” only to discover that nobody can explain why.
 |