// free checklist
The 90-Minute Test Suite Audit
Six things to look for in an AI-written test suite, and what each one costs you.
Your team merges more code than it did last year. The test suite grew with it, most of it written by an agent, and it looks fine. Coverage is up. The dashboard is green.
None of your instruments can tell you whether that is true, because every one of them reads green either way. This is the audit I run instead. Six checks, about 90 minutes, by hand, on your own repo.
There is no tool to install and no account to create. Nothing leaves your machine.
Ninety minutes is what it takes. I'm not going to call it fifteen to get your email address. If the number is the thing stopping you, this probably isn't your problem yet.
// what you run
Two lines each. The names and the questions are public. The method, the thresholds and what each finding costs are what the checklist adds.
The mutation spot check
Break three functions your business would hate to lose, one small change at a time, and watch whether the suite notices. It's the closest thing to a fault-detection measurement you can run in an afternoon.
Is the thing under test mocked?
In your ten most recent test files, count how often the unit being tested is itself replaced by a stub. These are the tests that pass forever, including after the behaviour breaks.
Tests that agree with the code instead of checking it
Assertions that restate the implementation instead of pinning the intended behaviour. They raise coverage and survive mutation at the same time, which is what makes them hard to see.
Assertion-free and assertion-crowded tests
Two greps. How many tests assert nothing at all, and how many assert so much that a failure tells whoever is on call nothing useful.
Drift and flakiness
Run the suite three times unchanged, then look for validation patterns nobody on your team would have written. Both defeat the tools you already own, for different reasons.
Coverage against the paths that matter
Name the five things your product must never get wrong, then find the test that would fail if each one broke. This is the only check in the set stated in the business's own terms.
// the checklist
The how-to-check steps, the thresholds, what each finding costs, and the scoring sheet. A nine-page PDF, built to be printed and worked through.
No email needed. It’s just a download.
// before you ask
How do I audit our test suite without installing anything?
By hand, in about 90 minutes, using the test command you already run. The six checks in this audit are deliberately tool-free: three of them are reading, two are greps, and one is editing three functions on a branch and reverting them. Nothing gets uploaded and nothing is sent anywhere.
Does high test coverage mean our AI-written tests are good?
Coverage measures lines executed, not behaviour verified. A test that mocks the unit under test and asserts that the mock was called will raise coverage and catch nothing. That is why the first check in this list breaks code on purpose and watches whether the suite notices, rather than reading the coverage number.
What is actually different about AI-written tests?
They're flakier, even though they assert more. But assertions aren't checking. A test can assert six things and still only describe what the code already does. Change the behaviour and it keeps passing. You can't see that by reading the tests. You see it by breaking something and watching which tests stay green when they shouldn't. That's check 1, and it's first for that reason.
Want the occasional email?
New articles, more free resources, and what I’m working on. Usually once or twice a month, often less.
What happens with your address
It goes to me, Thomas Langhorst, and to Brevo, who send the emails. It is used for the occasional emails about testing and code quality you just asked for, and for nothing else. Whether you open those emails and what you click in them is measured, which is what the second sentence of the box is about. You can unsubscribe at any time with one click, which is as easy as subscribing was. Your signup and your confirmation click are logged, and that record is kept for three years after you unsubscribe so the consent can still be proven. Nothing is shared with anyone else and nothing is sold. Full detail in the privacy policy.
You can object to advertising at any time, and the emails stop.