Can You Trust an AI-Written Report?
Here is a hypothetical: an AI-generated report summarises a month of customer feedback. It is clear and well organised, and one figure looks far better than all the rest. But can you trust an AI-written report?
You have to decide by lunchtime whether to act on it. Nothing in the report raises a question, and nothing in it looks missing.
A report that reads well is not evidence that it is right. That is the one idea in this post. AI makes confident, fluent writing cost almost nothing, so a leader needs a few simple habits before acting on what it produces.
Good interpretation habits are about stance, not procedure. None of them needs special tools or technical skill. There are five:
- Know what would change your mind before you read.
- Look at some of the raw material yourself.
- Be most suspicious of the number that surprises you.
- Ask what it left out.
- Check anything important against a second source.
Why a good read proves so little
A confident report used to be a fair sign that somebody had worked the problem out. Writing it well took effort, and the effort acted as a rough check. Nobody designed it that way.
It was simply hard to write something clear about a subject you did not understand. That check has gone. Fluent, tidy writing now costs almost nothing, and it costs the same whether the facts underneath are sound or not.
I wrote about this in October, in Is Anybody Actually Reading This?. I argued there that polished output no longer proves that anyone understood anything.
Reports have always been able to be wrong. What has changed is the cost of looking right. Clear headings, a tidy layout and a well-organised summary used to take a skilled writer an afternoon, and now they arrive in seconds.
A report that looks right cannot be held to account. A tool cannot answer for a wrong conclusion or carry the consequences of a bad decision, so accountability has to sit with a person. The natural person is the named reader: the one who has agreed to act on the report.
If a leader hands that accountability to the tool, the report is not the real problem. The real problem is in leadership.
Three more things make a good read a poor guide:
- Expertise is no longer implied. When a person wrote the report, you could rely on their experience and expertise behind the conclusions, and you knew who to ask. AI has undermined that, because a report can now sound just as expert whether or not anyone with experience stands behind it.
- Checking costs more than producing. Producing a plausible report is nearly free, but checking one still takes a person’s time and attention. That cost lands on the reader.
- The reader is over-confident. A good report is direct and says plainly what it found, whoever wrote it, so the tone is not the problem. The trouble is the over-confidence of the reader, who finds a clear report easy to believe and so asks fewer questions.
In my experience, busy leaders often treat a clear summary as finished work, and I understand why. It is easy to read, the tool that wrote it is usually helpful, and the decision is waiting.
This is the reader’s side of something I looked at from the writer’s side in Name the Reader. There I argued that every report should have a named author who stands behind it, and I still think so. But the report that reaches you may come from someone who only skimmed it, or straight from a tool.
Either way, you are the last person to look at it before the decision is made. By the time a wrong figure is noticed, any of these may have happened:
- Money has been spent.
- People have been moved.
- A promise has gone to a customer.
None of this is an argument against using AI. It is useful for reports, and I use it myself. The point is narrower: how a report reads and whether it is right are two separate questions, and only the first is easy to see.
Know what would change your mind before you read
Before you open the report, be sure of what you expect it to say. Then make sure you know what result would make you change the plan.
This matters because a confident report is easy to agree with. Once you have read a clear answer, almost any result starts to feel like support for something you already half believed. If you know in advance what would change your mind, the report is able to disagree with you.
Without it, the report sets the frame. You end up judging the plan against whatever the report happens to say, instead of judging the report against what you needed to know.
I made a related point in What Is Your Team Missing?. There I wrote about deciding beforehand what counts as a real change, so that you do not hunt for a story in every wobble. It applies to a single report in just the same way.
It takes a minute, and it changes how you read everything that follows.
Look at some of the raw material yourself
A summary is a view of the evidence. It is not the evidence. The evidence sits underneath: the customer comments, the rows of data, the messages the summary was built from.
So look at some of it yourself. You do not need to read it all, and you should not try. A handful of the real comments, or a few rows of the actual data, is usually enough.
Pick them yourself instead of taking the ones the summary quotes, because those were chosen to illustrate its point. Then notice whether the real examples sound like the summary. If they do not, you have learned something important in a couple of minutes.
A few examples also show you things a summary cannot. You see:
- How messy the source is.
- What was ambiguous.
- Which judgements the summary made for you without saying so.
A figure is abstract, while a few real examples make it concrete.
In Name the Reader I said that a reader can go deeper if they doubt a report. Here I would go a little further. Look at some of it even when you do not doubt it, because a confident, well-written summary is exactly the kind you are least likely to doubt.
Be most suspicious of the number that surprises you
Most reports have one number that stands out. It is far better than the rest, or far worse, or it contradicts what you thought you knew.
That is the number to be most suspicious of. An unusually good or bad figure is more often a fault in how it was counted than a real change.
Twyman’s Law, named after the audience researcher Tony Twyman, says that any figure that looks interesting or different is usually wrong. Ron Kohavi, a researcher who works on online experiments, set the law out in a short note.
Calling this a “law” might be overstating it. It is really a rule of thumb, so the word “usually” matters. There are many ways to get a figure wrong:
- A duplicate.
- A missing week.
- A definition that changed.
- Two categories merged into one.
Real changes of that size are rare. When one appears, a counting fault is the more likely explanation.
I made a related point in What Is Your Team Missing?: one spike earns a look, not a reaction. This is the look. Before you ask what caused the change, ask how the number was counted:
- What counts as one?
- What was removed?
- Did the definition change?
These are plain questions, and asking them is not an accusation. A careful author hopes you will ask, and a good report can answer in a sentence.
Here is a small example of my own. A dashboard I own showed a healthy-looking traffic figure. When I checked how it was counted, most of it appeared not to be people.
Nothing about the figure suggested a problem until I looked. That is the trouble with a surprising number: it rarely announces that it is suspect.
The good number is the harder case. A bad number tends to start a conversation. A good one tends to be waved through, because nobody wants to be the person who questions good news.
There is a leadership point here too. A leader who does not want to hear bad news tends to teach people to bring only good news, and the best figures then reach the decision unchecked. That is how a counting fault ends up driving a decision.
Ask what it left out
Every report covers some things and ignores others, and a summary rarely says which. Fluent writing hides this well, because nothing in a tidy report looks missing.
So ask:
- What was excluded?
- What period and which sources does it cover?
- What could it not see?
These questions matter more with AI, not less. A model works from what it was given. It cannot tell you about a source nobody connected, a customer who never wrote in, or a month that was left out.
For instance:
- A summary of feedback that covers only the people who wrote in says nothing about those who quietly left.
- A report on a quiet month says little about the busy one before it.
Both will still read as complete.
In What Is Your Team Missing? I described how a fluent summary can feel like coverage while describing only what was fed into it. This is the same gap, seen from the reader’s chair.
You can put the questions to the tool as well as to a person. Treat the answer as a lead to follow up, not a finding, because the tool will answer just as fluently as it wrote the report.
Check anything important against a second source
For anything that matters, check it against a second source gathered a different way.
The phrase “a different way” is the point. A second copy of the same data, run through the same tool, will repeat the same mistake and agree with itself happily. A different source can disagree.
It might be:
- A separate count made by another method.
- Figures from another system.
- A short conversation with someone who deals with customers every day.
Two reports built from the same underlying data are not two sources. If the data has a fault, both will inherit it and both will look equally sure.
When the two sources agree, your confidence has been earned. When they do not, you have found out something before it cost you anything. Either result is useful.
The check does not have to be large. One independent look at the claim the decision rests on is worth more than many looks at things that do not matter.
Back to the report on your desk
Return to the hypothetical:
- Before you read it, you knew what you expected, and what would make you change the plan.
- You then looked at a few of the real comments instead of only the summary.
- The standout figure is the first thing you ask about, and you ask how it was counted.
- You ask what the report left out, and you find that it covers only customers who filled in a survey.
- Before acting, you check it against what the support team is hearing.
None of this means the report was wrong. It may well be right. It means that if you act, you are acting on evidence and not on how the report read.
Suppose the figure survives all of this. Then you have a better reason to act than the one you started with, and you can say what it is. It also took less time than the argument you would have had afterwards, if it had turned out to be a counting fault.
I have skipped this myself
I was working on a quote with an AI assistant. A quote is a report of a kind, built in this instance from several spreadsheets that the assistant pulled together in a series of passes. At some point I found that numbers I had seen in an earlier version had vanished.
The cause was a set of edits to the spreadsheets, which the assistant had not spotted between one pass and the next. That undermined the whole quote. I spent another day rebuilding the sheets and the reports that finally became it.
I probably had not read what the assistant wrote carefully enough the first time. Looking at the spreadsheets underneath, and at what had changed between passes, would have shown me the gap far sooner.
Not every report deserves all five
The care should match what hangs on the decision. A report that changes nothing you will do can be read lightly. I made the same point about where to look in What Is Your Team Missing?, under watching in proportion to consequence.
This is not distrust of the people who write reports, or of the tools that help them. It is the care you would give any single source of evidence.
What I would resist is the opposite habit, which is giving every report the same quick nod because it reads the same way. They all read well now, and that is exactly why the reading cannot be what decides.
One question for this week
None of these habits is clever. They are what a careful person does with any single source of evidence. What is new is that a report no longer shows, by the way it reads, whether it has earned your trust.
Which report did you last act on without looking at anything underneath it?
A team rarely notices where it has started to accept AI-written reports without questions, because the reports look fine and the habit grows slowly. Someone who has not lived with your team can ask which reports you act on and what sits underneath them, without the familiarity that makes every answer sound reasonable.
A Second Pair of Eyes
If it would help to have someone look at this with you, I am happy to talk.
I help teams and leaders work out how to operate now, through process and communication consulting and through mentoring. The first step is a free 15 to 30 minute introductory conversation about the problem you are seeing, where I give you my initial thoughts and proposals. There is no obligation to go any further.
If it makes sense to continue, there are two usual shapes. One is a fixed-price audit of one or two weeks, depending on team size, which looks at process and communication: which feedback loops are broken, which artefacts have no readers, and which practices are doing real work and which are habit. The other is a monthly retainer for ongoing mentoring and advisory work. Even a lightweight second opinion tends to be valuable.
If any of this sounds like your team, get in touch.
About the Author
Tim Huegdon is the founder of Wyrd Technology, a consultancy that helps engineering teams and leaders work out how to operate in the AI era, through process and communication consulting and mentoring. With over 25 years of experience building and maintaining software and software engineering teams.