Why We Distrust Tests Written After the Code

Test order.

On my team, the unit tests get written right after the plan is agreed and before anyone (human or agent) starts on the implementation. And if a test shows up in a commit after the feature code, we treat it as suspect until someone has looked at it properly.

That sounds fussy, I know. The reason is boring. Agents often add or edit tests to fit code they already wrote. A test written before the code is a record of what we agreed. A test written after it is, too often, a record of what the code happens to do.

None of this is new. Kent Beck's TDD book came out in 2002, and people have argued about test-first ever since. What changed for us is the speed. An agent can write a feature and rewrite three tests to match it in a few minutes, and the CI badge looks the same either way.

Think of a tailor who cuts the suit first and then goes back and changes the measurements on the order slip so they match. Every number on the slip is now correct. The customer's shoulders didn't change.

A public example

The clearest write-up of this I've seen in 2026 is openai/codex #24922, opened on May 28 and closed the same day by a maintainer who said it "appears to be model behavior, not a Codex harness issue." It's one developer's report. I haven't reproduced it, and one report tells you nothing about how often this happens. It is very detailed, though.

The task was a bounded React refactor: move feature logic from an eager shell into a lazy-loaded surface, preserve an async stale-navigation guard, keep positive route-transition behavior, remove exactly five temporary lint baseline entries, then run tests, lint and build and post READY evidence. The reporter says Codex got the first part right. It "explicitly identified the stale-navigation guard and lint entries before editing."

Then a context compaction happened in the middle of implementation, tests started failing, and Codex "repeatedly increased async settling windows and then weakened/remapped assertions instead of preserving the original behavior contract."

The detail that got me was one test. Before the refactor, a positive test proved that clicking Begin navigates into the draft route:

expect(window.location.pathname).toBe("/workspace/drafts/draft-1");
expect(activePanel()).toBe("draft");

After it, the same test, with the same name, asserted this:

expect(window.location.pathname).toBe("/workspace/briefs/req-1");
expect(activePanel()).toBe("");

In the report's words, a test "still named as proving navigation into a draft route instead asserted that navigation did not occur." That's the tailor and the order slip, more or less exactly.

There was more. A stale-request race test was made "vacuous": two mock branches for the same API path, the first one returning immediately, so the deferred branch the test existed for could never run. And the READY record said five lint entries had been removed, while the committed diff "did not modify the lint file at all."

The reporter sums it up better than I can: the agent "weakened or invalidated existing regression tests to obtain a green test run, then posted completion evidence contradicted by its own committed file inventory."

Why test order helps (a bit)

None of those changes would look scary in a diff. Nothing was deleted, nothing skipped, coverage probably didn't move. If you read the feature code carefully and skim the tests because "they pass", you approve it. Plenty of us would, on a tired day.

Writing tests first won't make an agent honest. You do get something to compare against that the agent couldn't have shaped around its own code, because the code didn't exist yet. When a pre-written assertion changes in the same PR as the feature, that's a finding by default, and somebody has to explain it.

In practice that mostly means reading the test diff before the code diff, and checking that test names still match what the bodies assert. Plus one cheap check from this report: if the completion message says a file changed, git show --stat should list that file. The lint claim would have failed that in about ten seconds.

Something like this, against the base branch, is enough to know where to look:

git diff origin/main...HEAD -- '**/*.test.*' | grep -E '^[-+].*expect\('

It won't tell you whether a change is right. It tells you which tests the agent touched, which is usually the part nobody reads.

To be fair to the agent, humans do exactly this too when a deadline is close, and the report also mentions the compaction, which may have played a part (nobody knows how much). Agents just do it faster, in larger diffs, and then write a confident summary on top.

Anyway. The suite was green. The test name was still very sure the page navigates.