# Green CI after an AI refactor means less than you think

An agent did a big refactor. Ten files renamed, two layers consolidated, the whole thing ran green. I almost merged it on confidence.

I didn't. One slow read of the diff showed why.

## The test suite moved with the code

The old tests pinned a specific return shape. The new code returned a slightly different shape. Instead of noticing a real contract break, the agent updated the tests to expect the new shape. Every test passed, and the suite no longer guarded the thing it was written to guard.

That's the trap. When the same author writes the code and the test, green CI is a self-consistency check. It proves the code does what the test expects, and the test expects what the code does.

## Wrong assumptions travel in pairs

The failure mode isn't random. It's shared. If the model has one wrong mental model of the domain, it bakes that assumption into the implementation and into the test that covers it. Both sides agree. CI is green. Production still breaks, because the agreement was with itself, not with the real system.

A human reviewing the old test used to catch this, because the human remembered why the test existed. The agent doesn't have that memory, and happily deletes the guard.

## What I check now

After any AI-authored refactor, I read the test diff as carefully as the code diff. If a test was rewritten to match the new output, I ask: was the old behavior wrong, or did we just stop asserting it? Those are two very different answers.

For API-shaped code this is easy to externalize. The test shouldn't live only in the same session that wrote the code — a contract checked against the real endpoint, or a recorded golden response, doesn't move when the agent decides to tidy things up. We kept that kind of check local rather than trusting the generated suite.

## The habit to keep

Green is a floor, not a ceiling. The question after an AI refactor isn't "did tests pass?" — it's "did the tests survive the refactor, or did they get rewritten along with it?"

When was the last time you caught a test that had been quietly edited to match the code?