|
 |
AI systems now answer customer questions, explain features, and guide users of an increasing variety of products and services through tasks they need to complete.
In many organizations, they do this instead of sending people to the documentation.
Sometimes they do it without ever showing the customer the documentation at all.
 |
This means our technical documentation has quietly stopped being reading material and started a new career as knowledge infrastructure. It is now the raw material from which AI systems assemble confident, well-phrased answers — whether those answers are spot-on or otherwise wildly incorrect.
Recent research (👈🏽 PDF) from EMNLP 2025 Conference on Empirical Methods in Natural Language Processing explains why so many of these systems sound smart while being wrong, and why fixing the problem has less to do with “better AI” and more to do with how documentation is structured.
Let’s talk about what the research found (and why tech writers are holding the keys to AI success, whether they asked for them or not).
 |
First, Let’s Be Very Clear About What This Is Not
This is not about documentation of AI products.
This is about AI systems that:
Answer questions using your documentation
Generate explanations based on your docs
Power chatbots, copilots, and AI search (both standalone and embedded in apps and products)
Cheerfully speak on behalf of your product at 2 a.m.
In these systems, documentation is no longer a helpful companion. It is the authority. And if it is vague, inconsistent, or poorly structured, the AI will not hesitate to improvise — politely, fluently, and with great confidence.
The Real Problem (Spoiler: It’s Not the AI)
Most AI systems that use documentation rely on something called Retrieval-Augmented Generation, or RAG. The idea is simple and sounds reasonable enough:
Break docs into chunks
Turn chunks into vectors
Retrieve “similar” text
Ask the AI to answer
The problem is that this treats documentation like a bag of words someone spilled on the floor.
See also: The Problem with Traditional RAG Models
According to the research, this approach reliably produces:
Hallucinated steps
Mixed product versions
Feature explanations that never existed
Confident contradictions
Answers that cannot explain where they came from
This is not because the AI is careless. It is because similar text is not the same thing as relevant knowledge.
 |
What the Research Proposes Instead
The paper introduces Ontology-Grounded Retrieval-Augmented Generation, or OG-RAG.
OG-RAG does something radical: it assumes your documentation actually means something.