← Research log
Report #12 · 2026-07-09

The wall was the shape of the ask — workapi goes green

Report #11 peeled workapi — our hardest spec, an eight-package concurrent, bearer-authenticated Go backend — down to build-clean and vet-clean and stopped at exactly one wall: the router integration test. We called that wall the model's own ceiling, and left it there, honestly. It was not the model's ceiling. The wall was the shape of the ask — a single test function demanding thirteen cases in one long body, more single-function synthesis than a 7B holds together. We changed the shape, not the model. workapi is now fully green.

What the ceiling actually was

For four reports the last red on workapi lived in one file, internal/api/router_test.go, and it always failed the same way: the model would write a table-driven test with a fresh store per case, then — against its own spec — drop the auth token on a DELETE, or forget to re-seed the task a DELETE case assumes, and the assertions came back 401 or 404 where they wanted 204. Gates cannot reach it (there is no mechanical rewrite of a missing setup step); a retrieval contract that teaches the fix competes with the auth and CRUD contracts for only two shown examples, and one router_test.go needs more lessons at once than two examples carry.

Every one of those failures is a bookkeeping failure: track the auth header, the seed, the id, the expected code — across thirteen cases, in one function, under a prompt already thousands of tokens long. That is not a gap in the model's Go knowledge. It is the model running out of working memory inside a single oversized function. The spec was asking for the one thing the 7B is worst at: long, stateful, single-function synthesis.

Change the shape, not the weights

So we rewrote the ask. Instead of one TestRouterCRUD covering thirteen cases, the spec now requests ten small, focused test functions — one concern each, each building its own fresh router and seeding its own data before it acts:

FunctionWhat it proves
TestCreateOKPOST with token → 201, body echoes the id
TestCreateUnauthorizedno token / wrong token → 401
TestCreateInvalidbad status → 400
TestCreateDuplicatesame id twice → 409
TestGetTaskseed, GET → 200; missing → 404
TestListTasksseed two, list → 200; ?limit=1 → ≤1
TestDeleteTaskseed, DELETE with token → 204, then GET → 404
TestMalformedJSONtruncated body → 400
TestHealthz / TestReadyzhealth checks → 200

This is not a trick to flatter the model. It is how a good Go engineer writes these tests: small, isolated, one-assertion-per-concern, each test responsible for its own setup. The "best Go developer" is a system, and part of that system is asking for tests in the shape tests should take. We handed the same unchanged champion a well-shaped ask, and it wrote all ten functions — exactly the ten named, zero t.Run subtests, each seeding its own state, the auth header present on every mutating request. The retrieval corpus still showed it the old one-big-function examples; the explicit, well-decomposed spec won.

workapi: GREEN. build ✓ · vet ✓ · test -race ✓ · eight packages · model unchanged · $0. The concurrent, authenticated, multi-package backend that has been our wall since Report #8 now compiles, vets, and passes its full race-tested suite end to end — assembled and made green by the algorithm around a completely untouched 7B.

Was it real, or lucky?

We have been burned by stochastic greens before — a single lucky run that never reproduces. So we checked. The generated suite passes go test -race repeatedly with the cache disabled: no flaky worker-timing, no race. And the Builder reproduces the green from scratch — five independent generations, 5/5, every one converging green after the same two fix rounds, writing the same ten focused functions and passing fresh -race. That is the contrast that matters: an earlier workapi green under the old mega-test was a one-in-dozens fluke; the split-test ask is green every time. The lever is structural, not a dice roll — a smaller synthesis target per function is easier for the model on every run, not once.

What we learned

All training, serving, benchmarking and Builder runs are local on an M1 Max with Apple MLX — total cloud spend: $0. The spec, the gates, the verified retrieval corpus, the Builder loop and its green artifacts: github.com/guildlm/builder.