AI Wrote the Documentation. Nobody Checked It.

Automation is the solution to each and every problem, announces every software developer ever.

The automation of builds, tests, deployments. That little notification that announces to everyone that a build has failed which then no one acts upon until it is their build (rendering the automation pointless, naturally).

Yes, because automation might at its best mask incompetence. But it will not automate it away.

The Documentation Trap

Reading things is a problem for many in the software development game. There, I’ve said it.

Perhaps it’s neural divergence, perhaps it’s the idea that software developers are the best lazy people you can imagine.

All I know is that I’d rather spend an hour at the keyboard typing than 5 minutes reading a document on confluence which only has a passing resemblance to English.

Which has always meant a problem, and the problem is doubled when managers think that the solution to all future problems is to document a past problem. Or worse to automate the documentation.

You can’t automate something which won’t really help anyone anyway, because nobody validates whether they are true.

Documentation Has Become a Checkbox

Documentation is often treated as something that must exist rather than something that must be useful.

When we have an incident at work the answer is always more observability. More monitors which wake developers in the middle of the night.

It makes everyone happy as something has been done. Yet no-one maintains the documentation, no-one monitors .

Previously, a developer had to misunderstand the system and write inaccurate documentation manually. Now a machine can misunderstand the system and produce twice as much inaccurate documentation before lunch.

Progress.

Generated Does Not Mean Verified

AI-generated documentation can be useful, and provides us with a great first draft. The issue comes when people don’t think of it as a first draft, and think it can replace the thoughtful developer.

Because no matter the complexity of a model, it cannot accept responsibility. The AI will not be paged when the production system fails.

It will not sit in the incident review while someone asks why the authentication guide confidently recommended an option that disables authentication.

It will not explain why the onboarding document refers to three services that were deleted last year.

That privilege belongs to the development team.

Using AI does not remove the need for though review. It increases it.

We Already Know How This Goes

Get someone competent to review that code please. We all know the reason, the code that is being pushed by AI isn’t always fit for purpose.

The Process

The person generating the documentation should remain responsible for it.

That means checking the claims, running the commands, testing the examples and confirming that the words describe the system as it actually exists.

A reviewer should then look at the documentation from the reader’s perspective.

We all have our own duty, and we should all produce the best result we can in order to deliver shareholder value.

Wait, that’s probably a justification for not producing documentation. Just because AI tools make it easy to generate a page for every class, function and configuration option. That does not mean we should.

The process we should be following is thinking about things, right? Just because we are not doesn’t mean we should not.

Not an Alibi

The useful way to apply AI is straightforward, but at no point should we believe it.

When you approach things, consider who should be responsible.

Protip: It’s not the AI.

About The Author

Professional Software Developer “The Secret Developer” can be found on Twitter @TheSDeveloper.

The Secret Developer wonders if when they are replaced by AI the world will be better.