Everything the spec does not say, the model says for you

I set out to raise coverage in one package. What I found was a live runtime panic in code that had been shipping green for a week, three distinct ways a passing test can be a lie, and a fix loop that had spent a week throwing away half of every default this project ever wrote. Ten reds today. Nine were upstream of the model. The tenth was an honest model limit — and it took two go/ast gates to close, because six escalating versions of a prompt rule could not.

The money shot: coverage was not a vanity metric

The service package sat at 48.6% and 54.2%. I pulled the uncovered blocks out of the coverprofile and wrote the tests the spec had implied but never named. One of them crashed:

panic: runtime error: slice bounds out of range [-1:]

paginate had no offset < 0 clamp. Any HTTP request carrying ?offset=-1 slices items[-1:] and takes the process down. The spec itself said "never panic."

The bug is not the sharp part. The sharp part is that the model wrote that guard correctly in one run and dropped it in another. Nothing noticed, because nothing ever asked. Same branch, same purpose, three backends: guard present in one, absent in two, tested in none. All three shipped green.

An untested branch is not merely unverified. It is a branch where the model's output is a coin flip — and the coin has been flipping in production.

Three ways a passing test can be a lie

All three came from the spec, not the model.

1. An assertion that cannot fail

The spec said: "List with limit=1 → at most 1 item." The model faithfully wrote the check. A paginate that returns nothing at all satisfies it. I proved it by mutation — broke paginate to return an empty slice, and:

if len(got) > limit { Errorf(...) }   // "at most 1" — SILENT. Passes.
if len(got) != limit { Errorf(...) }  // "exactly 1" — catches it instantly.

2. An assertion that cannot run

In a test that expects an error, the model opened with the file's dominant rhythm — and then wrote the assertion the spec asked for underneath it:

_, err := svc.List(ctx, 1, 0, "")
if err != nil {
        t.Fatalf("List: %v", err)      // fires on the very error we asked for
}
if !errors.Is(err, errBoom) {          // DEAD CODE. Never runs.
        t.Fatalf("want errBoom, got %v", err)
}

It did not fail to understand the ask. It added the habit on top of it. In the function directly above, it got the same thing right — a rhythm, not a rule.

3. An assertion that was never asked for

The spec said TestHealthz: GET /healthz -> 200. The model wrote the status check, then invented a second one: it decoded the body into struct{ Status string } and asserted Status == "ok". The handler writes the bare JSON string "ok". The struct decode silently yields the zero value, and the test fails against a handler that is working correctly.

And then the part that should worry anyone running a repair loop: left alive, that invented assertion started rewriting the product. The machinery's next move was widening fix targets to package impl — it was about to bend the handler to satisfy a test nobody had specified.

The controlled evidence

In the same run, in the same file family: the two tests where I had written "and NOTHING ELSE" came out clean. The one test where I had not, the model filled the silence — and failed.

"One test, one outcome" is not enough. You must also say and nothing else. Otherwise the model fills the silence — with a habit, or with an invention.

Mechanism #12: the fixer never saw the defaults

Chasing why a bug I had already written a default for kept coming back, I found it. test_rule — every test-authoring rule this project has built over a week: HTTP hygiene, isolate-state, one-test-one-outcome, struct literals, the shadowed t — was used in _generate_prompt and nowhere else. It was not in _fix_prompt.

The generator got the rules. The fixer did not. And a fix round rewrites the whole file — so every fix round rewrote a test file blind to every default the project had ever built, and regressed to the exact prior each one exists to prevent. Half the investment, thrown away one round at a time.

It compounds with mechanism #11, found the same day: the fix prompt embeds task.spec.purpose verbatim. So when the purpose is what produced the wrong code, the loop hands that same purpose back as the instruction for repairing it. Five rounds of "fixing" are five re-readings of the same bad sentence. In the log it reads as a model that is struggling. It is a spec that is winning.

Both repair layers are helpless against a spec bug. The gates cannot even hear a parse error — a file that does not parse never reaches the type checker, and the go/ast gates need the tree that does not exist. And the model is handed the bug back, every round, in the prompt meant to fix it.

A nudge is not a gate

The drained-body bug — a request built once and served twice, so the second call sends an empty body and a "POST twice → 409" case reports want 409, got 400 against a handler that is perfectly correct — survived six escalating versions of its prompt default. The rule names the variable. It forbids the reuse. It explains the drain. It predicts the exact status code. It quotes the wrong code verbatim as "exactly the bug". It anchors the example to the duplicate test by name.

I checked that the rule reaches the prompt. I checked that the retrieval corpus does not teach it. The model writes it anyway — because the duplicate case is the one test in which both requests are a POST of the same body to the same URL, and that identity is exactly what makes reusing req look correct.

That is a nudge-resistant defect, and Report #14 already named the answer. Two gates followed — both go/ast, both structural rather than error-driven, and that is the interesting part. The failures surface as want 409, got 400 and List: boom: messages the spec wrote, not the compiler. There is no sentence for a gate to listen for. So each keys instead on an unconditional contradiction in the source:

• a body-bearing request reaching ServeHTTP twice with no reassignment between is always a drained body;
• a test cannot both demand that err be nil and assert what err wraps.

Whichever way you read those, the code is wrong. That is what makes the repair safe without the compiler's help. And the repair has to be careful in a way the first draft was not: rebuilding the request naively drops req.Header.Set("Authorization", …), so the "fixed" test fails 401 instead of 400 — one silent breakage traded for another. It replays the mutations.

The criterion for building a gate was wrong

The model wrote w.count.Inc() on a sync/atomic.Int64, which has no such method. The compiler named the type, the method, the file and the line — every round. The model rewrote it five times, in a file where it used w.count.Load() correctly two lines below. The project went red on one line, and the entire fix budget was spent on it.

By frequency (n ≥ 2 across projects), that gate was not worth building.

A gate's value is not how often the error occurs. It is whether the model can fix it once the compiler has named it. An error re-committed after being told is deterministic-layer work by definition — even at n=1.

Numbers

The deterministic layer, measured on the archive of red artifacts — projects it drives to green with no model involved:

a week ago       0
this morning     5
tonight         10        // 0 stuck. Suite 290 green.

Coverage, after the fronts closed. Green was never traded for it — every number below is from a run that passed build, vet and test-race:

service   workapi      54.2%  →  100.0%
service   taskapipro   48.6%  →   94.3%
api       taskapi      64.6%  →   93.0%
Recover   usersapi     66.7%  →  100.0%     // 5 specs declared it. 0 tested it.
Recover   taskflow     66.7%  →  100.0%
paginate  taskflow     80.0%  →  100.0%     // the panic, closed

10 / 10 specs green.

The mandatory regression sweep that gated the global default change did exactly what it exists for: five specs I had not touched, five green. Three came back red — and all three were pre-existing spec gaps the sweep made visible for the first time. Not one was a regression.

Write the answer to your own spec first — and its limit

Four times today my spec was wrong, and each time I learned it only after a 40-minute GPU run. So: copy the artifact, write the test file yourself, run go test -cover. Thirty seconds, no GPU. If you cannot write it, the model cannot. If it compiles and moves the number you predicted, the ask is proven.

It paid immediately — my own reference implementation showed me, twice, that my "fixed" spec was still incomplete. But it has a limit, and the dead-assertion bug is exactly it:

Writing the answer yourself proves the ask is SATISFIABLE. It does not prove the ask is UNAMBIGUOUS. My reference implementation passed precisely because I did not happen to have the model's habit.

Yesterday's lesson was that a default is only as good as the name the model moves over. Today it completed itself twice more: an example is only as good as the scope it pins, and only as good as the case it is anchored to. The model read every rule, understood it, and applied its shape to the wrong place — because a shape that fits somewhere else will land somewhere else. None of that is a model limit. What is a model limit gets a gate, and the gate is not a louder sentence: it is a contradiction in the source that no reading can rescue.

Postscript — the ruler lied eight times

Every finding above survived a second measurement. Eight did not:

• a defect looked rare because it was missing from logs that had never printed error output at all — until I added it thirty minutes earlier. A signal absent from records that never recorded it is not rare; it is newly visible.
• a loose regex turned valid method values into five phantom bugs, and nearly turned n=1 into n=5 — I was one keystroke from publishing "the corpus teaches the bug."
• a spec check flagged <= 1 — inside the sentence I had written warning against <= 1.
• go build … | head && echo PROVEN printed PROVEN while the build was failing, because && reads head's exit code. The shell's own version of an oracle that cannot fail — in the script verifying a gate written to catch oracles that cannot fail.

A number is not true because it looks plausible. It is true when it could not have been anything else.