The wall was the shape of the ask — workapi goes green
Report #11 peeled
workapi — our hardest spec, an eight-package
concurrent, bearer-authenticated Go backend — down to build-clean and
vet-clean and stopped at exactly one wall: the router integration
test. We called that wall the model's own ceiling, and left it there,
honestly. It was not the model's ceiling. The wall was the shape of the
ask — a single test function demanding thirteen cases in one long body,
more single-function synthesis than a 7B holds together. We changed the shape,
not the model. workapi is now fully green.
What the ceiling actually was
For four reports the last red on workapi lived in one file,
internal/api/router_test.go, and it always failed the same way:
the model would write a table-driven test with a fresh store per case, then —
against its own spec — drop the auth token on a DELETE, or forget
to re-seed the task a DELETE case assumes, and the assertions came
back 401 or 404 where they wanted 204. Gates cannot reach it (there is no
mechanical rewrite of a missing setup step); a retrieval contract that teaches
the fix competes with the auth and CRUD contracts for only two shown examples,
and one router_test.go needs more lessons at once than two examples
carry.
Every one of those failures is a bookkeeping failure: track the auth header, the seed, the id, the expected code — across thirteen cases, in one function, under a prompt already thousands of tokens long. That is not a gap in the model's Go knowledge. It is the model running out of working memory inside a single oversized function. The spec was asking for the one thing the 7B is worst at: long, stateful, single-function synthesis.
Change the shape, not the weights
So we rewrote the ask. Instead of one TestRouterCRUD covering
thirteen cases, the spec now requests ten small, focused test
functions — one concern each, each building its own fresh router and
seeding its own data before it acts:
| Function | What it proves |
|---|---|
TestCreateOK | POST with token → 201, body echoes the id |
TestCreateUnauthorized | no token / wrong token → 401 |
TestCreateInvalid | bad status → 400 |
TestCreateDuplicate | same id twice → 409 |
TestGetTask | seed, GET → 200; missing → 404 |
TestListTasks | seed two, list → 200; ?limit=1 → ≤1 |
TestDeleteTask | seed, DELETE with token → 204, then GET → 404 |
TestMalformedJSON | truncated body → 400 |
TestHealthz / TestReadyz | health checks → 200 |
This is not a trick to flatter the model. It is how a good Go engineer writes
these tests: small, isolated, one-assertion-per-concern, each test responsible
for its own setup. The "best Go developer" is a system, and part of
that system is asking for tests in the shape tests should take. We handed the
same unchanged champion a well-shaped ask, and it wrote all ten functions —
exactly the ten named, zero t.Run subtests,
each seeding its own state, the auth header present on every mutating request.
The retrieval corpus still showed it the old one-big-function examples; the
explicit, well-decomposed spec won.
workapi: GREEN. build ✓ · vet ✓ · test -race ✓ · eight packages · model unchanged · $0. The concurrent, authenticated, multi-package backend that has been our wall since Report #8 now compiles, vets, and passes its full race-tested suite end to end — assembled and made green by the algorithm around a completely untouched 7B.
Was it real, or lucky?
We have been burned by stochastic greens before — a single lucky run that never
reproduces. So we checked. The generated suite passes go test -race
repeatedly with the cache disabled: no flaky worker-timing, no race. And the
Builder reproduces the green from scratch — five independent
generations, 5/5, every one converging green after the same two fix
rounds, writing the same ten focused functions and passing fresh
-race. That is the contrast that matters: an earlier
workapi green under the old mega-test was a one-in-dozens fluke;
the split-test ask is green every time. The lever is structural, not a dice
roll — a smaller synthesis target per function is easier for the model on every
run, not once.
What we learned
- A "model ceiling" can be an artifact of the ask. The wall we had named the 7B's own limit for four reports was, in large part, the cost of demanding thirteen cases in one function. Decompose the function and the limit moves. Name ceilings carefully — some are the harness's, not the model's.
- The prompt's structure is a free lever. No gate, no contract, no weight touched — only the shape of what we asked for. It is the cheapest lever we have found and it cleared the hardest wall in the suite.
- Good engineering and model-friendliness point the same way. Small focused tests are better Go and easier for a small model. When the well-shaped ask and the human-preferred one coincide, take it.
- Explicit decomposition beats a competing example. Retrieval still offered the old mega-test shape; a spec that names ten functions and their concerns overrode it. Be specific about structure when structure is the thing that's hard.