The algorithm composes — peeling an eight-package backend to the ceiling
Report #10 shut a door: making
the model bigger — a longer training budget on more of its own verified
output — overfit and regressed. The champion stayed the champion. So we did the
only honest thing left and pushed the other lever, the one nine reports
keep vindicating: the algorithm around the model. We aimed it at the hardest
target we have — workapi, an eight-package
concurrent, bearer-authenticated Go backend — and, with the 7B completely
unchanged, peeled its failures off one at a time. Each recurring failure
became a durable, verified asset. The implementation went green: it compiles and
go vets clean across all eight packages. What remains is a single
documented wall we have named for months.
Six levers, one model, zero training
The method is the whole thesis in miniature: never touch the weights;
turn every recurring compiler or test failure into a reusable rule. Two
kinds of rule. A deterministic gate — Python that reads the compiler's
own error and rewrites the exact defect, the same way every time. A verified
retrieval contract — a known-good, compiler-checked Go example the model is
shown when it writes a similar file. Over this campaign, six of them, each earning
its place by killing a specific, repeatable failure on workapi:
| Lever | Kind | The wall it removes |
|---|---|---|
_fix_middleware_arity | gate | middleware defined (next, cfg) but used as a Chain value |
| requalify type positions | gate | sibling-package types used bare in tests ([]Event, not models.Event) |
| paginated List | contract | len(svc.List(...)) on a two-value (items, error) return |
| auth CRUD router | contract | the bearer token dropped on the DELETE request → 401 |
| worker drain | contract | Processed() read before Stop() drains → 0 |
| scenario seed | contract | a fresh-store subtest not re-creating the task it deletes → 404 |
The peel
Run the spec; read the first wall; convert it; run again. workapi
began stuck behind a build error — the middleware-arity confusion, a
knowledge-free scaffolding defect that retrieval alone had never fixed because
the model copies the correct call shape but writes the wrong definition. The gate
rewrites the definition in one edit and the eight-package implementation
compiled clean. Then the walls moved inward, exactly as the
thesis predicts: build errors gave way to vet errors gave way to
single failing test assertions — a foreign type used bare, a two-valued
List misused, a DELETE that forgot its token, a worker
counter read too early. Each fell to its lever. The run that carried all of them
reached the deepest point any workapi attempt has: go build
clean, go vet clean across all eight packages, with only the
router's integration test still red.
Build ✓, vet ✓, eight packages, model unchanged. A concurrent, authenticated, multi-package Go backend — the kind of thing the 7B cannot write in one shot and never could — assembled and made to compile and vet clean by the algorithm around it, not by any change to the model. That is Report #10's negative turned into its positive: the lever that works is the one we can actually move for free.
The ceiling we did not pretend to clear
What is left is the wall we have reported since Report #8:
the 7B's router integration-test logic. The model writes a
table-driven test with a fresh store per case, then — against its own spec — fails
to re-seed the task a DELETE case assumes, so the empty store answers
404 where the test wants 204. We wrote the contract that teaches the seed; on the
single hardest file it competes with the auth and CRUD contracts for only two
shown examples, and the lessons a correct router_test.go needs at
once exceed what two examples can carry. That is an honest limit of retrieval at
this spec's complexity, not a bug to paper over. We did not hand-write
the green; the implementation-clean result is the result.
Three ways a corpus poisons itself — and the discipline that caught them
Adding examples to a retrieval corpus is not free, and this campaign paid the tuition three times — each caught by re-checking, not by luck, and each fixed:
- Ranking contamination. A paginated-
Listexample whose service defined only aTasksurfaced for the models file and diluted its "define every type" signal — the model then dropped theEventtype. Fix: make the contract test-only, so it ranks last for implementation files. Verified against the retriever, not assumed. - Name collision. A test helper named
storebled into generated code that imports a package namedstore— "store already declared." Fix: rename in-example helpers to something no project package will ever be (taskStore). - Helper leakage. A shared
do(t, h, method, path, token)test helper got copied and mis-called with the wrong arity. Fix: contracts carry no copyable helper — inline every request, so there is nothing to copy wrong.
The rule that falls out: a retrieval contract must be test-only, use identifiers
no project package can collide with, contain no shared helper, and be checked —
with the real retriever — to surface for its target and nothing else. And
the safety net held: re-running the champion spec taskapi against the
grown corpus converged green with deterministic fixes only. Six
new assets, zero regressions.
What we learned
- The algorithm composes. Gates and contracts stack: each removes one wall and reveals the next, and together they carried an eight-package concurrent+auth backend to build-clean and vet-clean with the model untouched. Capability came from the scaffolding, exactly as the thesis says.
- Scaffolding beats knowledge on scaffolding defects. The middleware-arity wall was never a Go-knowledge gap — it was a mechanical mistake retrieval couldn't reach and a gate fixes in one edit. Match the lever to the defect.
- The remaining wall is genuinely the model's. Router integration-test logic is semantic synthesis under a long prompt — the 7B's documented ceiling, and the point where free levers run thin.
- Corpus growth needs an integrity discipline. Three distinct self-poisonings, all caught by checking rather than trusting. A corpus that silently rots teaches broken Go; the guard is verification, every time.