Renaming a single letter — the gate that needed a parser

After the sweep and the breadth run, one failure was left standing in the whole suite, and it was a single shadowed identifier. The model writes for _, t := range tasks; the loop variable steals the *testing.T; every t.Fatalf in the body stops compiling. Report #14 argued that a prompt nudge lowers a mistake rate but cannot be relied on to eliminate a class. This bug proved it in the field: an explicit never name a local variable t in the spec did not hold across repeated rolls. So this report does the thing the thesis says to do — builds the gate — and finds that this particular gate cannot be written with a regex at all.

The bug, exactly as the model writes it

Here is the real generated file, unedited. It is almost right, which is what makes it interesting:

func TestStoreListSorted(t *testing.T) {
	s := NewStore()
	tasks := []Task{{Title: "b"}, {Title: "a"}}
	for _, t := range tasks {                    // t now shadows *testing.T
		if err := s.Create(t); err != nil {  // this t IS the Task — correct
			t.Fatalf("Create: %v", err)  // this t was meant to be the tester
		}
	}
	...
}

The compiler is precise about it: ./store_test.go:33:6: t.Fatalf undefined (type Task has no field or method Fatalf). Note what the author actually meant, because the fix depends on it entirely: inside that loop the bare t is the Task and is exactly right, while t.Fatalf is a reach for the tester that the shadow silently stole. Both readings of the same letter live three lines apart.

By the taxonomy from Report #14 this is a gate's job, not a prompt's: the compiler names the defect the same way every time, with a file, a line, a column, the variable, the method and the type. What it is not is regex-able.

Why the obvious fix is not safe

The tempting one-liner is to rewrite t to tk inside the loop body. A single letter is the worst possible thing to substitute textually: it occurs as a struct field in x.t, as a map key, as a label, inside strings and comments, and — worst — inside a nested loop that re-shadows it again, where the same token means a third thing. And the rename must be selective: it has to rewrite s.Create(t) but deliberately leave t.Fatalf alone, because leaving it alone is the entire point. A text-level tool cannot tell those two ts apart. A scope-aware one can.

So this gate is the first one in the project that shells out to a real parser. The rewrite lives in tools/shadowfix.go and runs on go/ast: it finds functions that actually take a t *testing.T, finds the declarations that shadow it, computes the scope each shadow is visible over, and renames the declaration plus every non-tester use inside that scope to a fresh name that collides with nothing in the function. t.<testing method> is left untouched, so it binds to the un-shadowed tester again. The Go toolchain was already a hard dependency of the build loop — every candidate is compiled, vetted and raced — so leaning on Go's own parser costs nothing new.

What it refuses to do

The gate is written to be safe by construction, which mostly means knowing when to do nothing. It bails out — leaving the file exactly as the model wrote it — on every construct whose meaning it cannot settle without a full type-checker:

• An inner re-shadow. Two nested bindings of t mean a use in the inner body could belong to either. Refuse.
• A declaration in an if/for/switch init clause. if t := tasks[0]; t.ID == 0 {…} scopes t to that statement only — but the innermost enclosing block is wider than that, so renaming across it would rewrite the real tester further down. This one was a genuine soundness hole in the first draft, caught while re-reading the scope resolution, and it is now an explicit refusal.
• A domain type that declares a testing-shaped method. If Task itself has an Error() — and a Go type very often does — then t.Error(...) inside the shadow is genuinely ambiguous. Refuse rather than guess.
• A key or a label named t. A struct field key and a map key are indistinguishable in the AST without types. Refuse.
• A shadow that is never used as a tester. for _, t := range tasks { total += t.ID } compiles perfectly well and is none of our business.

There is also a structural safety property worth stating plainly, because it bounds the blast radius of the whole idea: this gate can only ever touch a file the compiler has already rejected with this specific error. A green file never produces the error, so the gate never fires on one. It cannot regress working code; the worst it can do is fail to help.

Verification

Seventeen unit tests, and the ones that matter are the refusals — every bullet above has a test that asserts the gate does nothing. The rewrite itself is checked on the real, unedited artifact that produced the bug: before, go vet stops at t.Fatalf undefined; after, that error is gone and vet has moved on to an unrelated pre-existing defect elsewhere in the file. The output is gofmt-clean, because it is printed by go/format rather than patched as text.

Then we ran the spec, three times, to watch the gate fire in a real build. It never fired. What happened instead is the actual subject of this report.

The gate could not see the bug

All three rolls failed, and none of them failed on the shadow. They failed on undefined: NewStore — and the shadow bug was sitting right there in the artifact, three lines further down a file the compiler never got to. A gate cannot repair an error the compiler declines to print. go build does not compile test files at all, and go vet stops at the first error; while anything ahead of it was broken, the shadow was invisible.

So the failing thing was store.go. The spec told it, emphatically, to be a single concrete struct with a constructor named exactly NewStore. Every roll wrote a Store interface with a StoreImpl built by NewStoreImpl instead — so nothing named NewStore existed, and every file the spec had pinned to that name failed to compile.

Pulling that thread found three separate defects, none of them in the model:

• The fix loop was beating on the wrong file. The compiler reports undefined: NewStore at the USE site — the test — so the loop spent all five rounds regenerating the test, and never once regenerated the file that owed the constructor. There was already root-cause routing for exactly this, but only for undefined: pkg.Sym, where the package name says who owes it. A single-package project gets a bare undefined: NewStore and no hint, so the routing never ran. It now routes to the non-test file whose spec purpose promises the symbol.
• The root-cause routing believed the shadow error. t.Fatalf undefined (type Task has no field or method Fatalf) reads exactly like a genuinely missing method, and the routing took it at face value: it added task.go to the fix targets and invited the model to give Task a Fatalf method. It now recognises the shadow and stays out of the way.
• A regex had been quietly eating a letter. The undefined-symbol pattern matched the name as (\w+)(?!\s*\.), and \s* crosses newlines: with undefined: NewStore followed by a line starting ./store_test.go, the engine backtracks onto that leading dot and the match shrinks to NewStor — a symbol that does not exist. It had been silently truncating symbols in multi-error output all along. A \b forbids the shrink.

With the routing fixed, store.go was regenerated five times with undefined: NewStore right there in the prompt. The model wrote NewStoreImpl every single time. The prior does not move — so it got a gate too: when exactly one zero-argument constructor in the package builds the same thing under another name, alias the promised name onto it (func NewStore() *StoreImpl { return NewStoreImpl() }), which compiles by construction. On the real artifact the gate chain now carries the project from undefined: NewStore to build-clean and vet-clean with no model involved at all.

But the spec was the one at fault

Two more rolls failed a new way: a Store interface and a Store struct, in the same file — Store redeclared in this block. And that is the tell. The model wants a Store interface in every single generation. The spec was demanding that the concrete type take the name the model reserves for the interface, so its two natural declarations collided head-on. The collision was manufactured by the ask.

So stop fighting it — the workapi lever again. The spec now asks for exactly what the model already wants to write, with the names pinned so that nothing can collide: a Store interface, a MemStore struct implementing it, and func NewStore() Store { return &MemStore{...} }. The redeclaration is now structurally impossible, because the two types no longer share a name. tasks-api greens 2/2, first try, race-clean. The last stochastic spec in the suite is closed.

Credit where it is due, though: the gates did not fire in those green rolls. The green came from the spec realignment. The gates are proven separately — unit tests, and a real artifact driven from broken to vet-clean by the gate chain alone — but they are not what made this spec green, and it would be easy and wrong to imply otherwise.

The probe, and an honest inconclusive

A gate that never fires proves nothing, so — the subtractive method from Report #14 — we deleted the anti-shadowing instruction from the spec entirely and ran it three times. The shadow bug came back in all three, which at least settles that the nudge was doing real work. But the probe cannot answer the question it was asked: all three runs failed on other stochastic test-authoring bugs (a test reaching for the concrete constructor; a test wiring the router without the API in between), and those failures masked the shadow from the compiler all over again. Given the shadow error directly, the gate fires and repairs the probe's own file correctly. "Green without the nudge" was not demonstrated. Recorded as inconclusive rather than dressed up.

And a sweep, which found a spec arguing with itself

A regex used by one of the oldest gates had changed, so the whole suite got re-run. Seven of eight specs green; taskapi red — and none of the new gates had fired in that run at all (build and vet were clean, only a test failed), so it was not a regression but something the sweep simply exposed.

Its spec said "each case builds a FRESH router+store" and then asked, in the same breath, for a duplicate id → 409 case and a GET existing → 200 case. Neither can pass against a store nobody has written to. The model obeyed literally — one shared table loop, a fresh empty store per subtest — and got 201 where it wanted 409, and 404 where it wanted 200. The instruction defeated itself, and the model was right. Split into focused functions that each POST their own precondition before asserting on it, it greens 2/2, race-clean — the fifth spec closed by that same lever.

The headline is not the gate. It is that building the gate uncovered three pieces of machinery mis-handling the very error it was written for — a fix loop routing to the wrong file, a root-cause widener that believed a shadow was a missing method, and a regex that had been truncating symbol names in every multi-error build — and then two specs that were manufacturing their own failures: one demanding a name the model reserves for something else, one asking for a duplicate on an empty store. Not one of those five was a model limitation. That keeps being the finding. The moat is still the system: a fixed 7B, an agentic loop, a verified corpus, a growing set of gates, and specs written like an engineer. The Builder's test suite stands at 227.

Postscript — three more, and the last one is the worst

The thread kept pulling. Three findings after this report was first published, each one the same shape as the ones above.

A gate that existed and never fired. The regression sweep put taskapipro back on the middleware wall — the one documented since Report #7 — and there is already a gate for it. It did nothing. The reason: that gate repairs a definition written in the wrong shape, and this generation's definition was right. The call site was wrong. The model declared func Logging(next http.Handler) http.Handler, which already is the Middleware type, and then invoked it — Chain(mux, Logging(logger)) — where it should have handed it over: Chain(mux, Logging). A function whose signature is exactly the wanted named func type is assignable to it by construction, and that is the condition the new gate proves before touching anything. On the real artifact the build goes from failing to build-clean and vet-clean with no model involved.

A gate can't fix what the compiler never printed. This was the report's own central finding, and it turned out to have a direct mechanical fix. Each Go stage shows a different slice of the truth: go build skips _test.go files entirely, go vet typechecks them but bails at the first type error in a package, and go test compiles the test binary, where the compiler reports up to ten errors. The loop was fixing against the narrow surface. Widening it, measured on two real artifacts by how much the gates can repair from each surface in a single round:

In both, the second file is the one carrying the shadowed tester — previously unreachable, now repaired in the same round as the error that was hiding it.

And the worst one: the Builder's own default was teaching the model the failing pattern. Six specs in a row — usersapi, kvservice, workapi, tasks-api, taskapi, taskapipro — died the same death: a duplicate → 409 case that got 201, and a GET existing → 200 case that got 404. I had been patching them one spec at a time, which should have been the tell. The cause was upstream of all of them.

The Builder's ISOLATE-STATE prompt default told the model, correctly, that every test case must construct its OWN fresh instance of the system under test. That is half a rule. A fresh instance is empty — so a "duplicate" case has nothing to duplicate and an "existing record" case has nothing to fetch, and the case fails no matter how correct the handler is. The model obeyed the default literally, and the default walked it into the bug. Six times. The rule is now whole: isolate state, then seed it — any case with a precondition creates that precondition itself, and for stateful CRUD, separate focused functions beat a shared table loop, which pushes you toward exactly this mistake. One fix, upstream, free to every future spec.

That is now the count for this session: five gates, three machinery defects, and five specs — including the Builder's own default — that were manufacturing their own failures. Thirteen things that looked like a model ceiling. Not one of them was.