AI-written tests pass — and still miss the regression

A CI run went green on a Friday afternoon. We shipped. On Monday the mobile team said the detail screen was blank for every legacy account. The endpoint returned 200. The JSON it returned was missing a field the UI has been reading for two years, and not a single test noticed.

The reason was not that coverage was low. Coverage was, in fact, very high. It was high because the same assistant that wrote the endpoint also wrote the tests for it.

The test started agreeing with itself

When you ask an AI to add a feature, it produces two artifacts at once: the code and the test. The problem is that the test is derived from the same reasoning that produced the code. It does not observe the code from the outside. It restates the assumption that was already inside the code.

So the new handler returns { status, items }. The new test asserts that the handler returns { status, items }. Change the handler and the test still passes, because the test only has to match the handler. There is no independent witness in the room.

This is not a coverage problem. It is an oracle problem. You have lots of checks and almost no checking.

For APIs the loop tightens around the mock

REST endpoints make the failure sharper. An assistant building an API service almost never calls the real database, or the real upstream, or the real frontend. It builds a mock. It writes the test against the mock.

Then the mock quietly becomes the contract. The fixture encodes the field names, the nesting, and the optional-vs-required rules that the assistant inferred while writing the conversation. If the real producer sends a different shape, nobody finds out, because every test was written against the shape the assistant guessed. The tests are green and the integration is broken.

You start recognizing it in small telltales:

• The fixtures use DTO names that came out of your chat thread, not the names the production client actually sends.
• Every new endpoint has 10 unit tests and zero recorded requests against the live environment.
• CI is green on a release the mobile team calls "unshippable."

Three cheap habits that break the loop

You do not need to throw away AI-generated tests. They are useful. You need one check that is not derived from the same mind.

1. Assert against the real edge at least once. Keep a tiny set of tests that hit a recorded, then replayed, real request through the real router. One captured call per endpoint is enough to catch "the field I assumed was required is actually optional" class of bugs. The assistant will not write these for you, because it has not seen the real traffic.

2. Generate the mock from a contract, not from memory. If the assistant builds fixtures from an OpenAPI file that was last verified against the live endpoint, the mock has to match reality rather than your recollection. The moment someone changes the real producer, the spec drift shows up as a failing test instead of a Monday Slack thread. This is exactly the loop we wound up building locally in Powerduck — the assistant reads the spec, and the spec was re-run against the live service, so the mock it generates has an outside reference.

3. Prove the test can fail. Before you trust a generated test, change the real thing on purpose. Flip a field, remove a branch, point the loader at the wrong shape. If the suite stays green, the test is theater. Good AI tests usually survive this; the ones that don't were only ever agreeing with the implementation.

The green bar stops meaning what you think it means

A test suite written by the same model that wrote the code is, most of the time, a very detailed way of confirming that the code says what it says. That is fine as a regression net for refactors. It is not fine as the thing that tells you a release is safe.

The cheapest upgrade is not more tests. It is one test that looked at the world from outside the conversation. Where does your suite break down when the real client stops matching what the assistant assumed?