The algorithm scales: a six-package production backend from a 7B
Report #6 ended with 3–4 file specs and a thesis: with a small, fixed model, the loop and its data are the lever. This report pushes that thesis up the only axis that matters for the mission — project size. Same 7B specialist the whole way (go-dev-mixed-v2, at base parity on the unit benchmark, never retrained), same Builder loop, and a ladder of increasingly real backends. The question: where does it break, and is the break in the algorithm or the model?
The ladder
| Spec | Files / packages | Shape | Autonomous |
|---|---|---|---|
usersapi | 7 / 1 | REST users API: repository, middleware chain, JSON handlers, graceful shutdown | 3/3 |
taskflow | 11 / 1 | Layered: errors→models→store→middleware→pagination→handlers→router→main | 3/3 |
taskapi | 13 / 5 | Real multi-package: cmd/server + internal/{config,models,store,api}, interface DI, slog, Go 1.22 routing | 3/3 |
taskapipro | 16 / 6 | + service layer, pagination, /healthz + /readyz, config validation, context-aware store/service | 2/3 → 3/3† |
Score is score_backend as always: go build +
go vet + go test, real toolchain, 0–3.
†The asterisk on taskapipro is the honest part; it gets its own
section below.
What made multi-package work: the Builder learned what a package is
The jump from 11 files in one package to 13 files in five is where naive
file-at-a-time generation dies: cross-package references need imports and
qualifiers, same-named symbols in different packages are legal, and only
exported identifiers cross the boundary. The Builder became package-aware —
package = directory; same-directory files are shown in full while other
packages appear as their exported API only (signatures, not bodies),
exactly how a human reads a codebase. Redeclaration-stripping and sibling
context were scoped to the directory so a legitimate New() in two
packages isn't "repaired" away.
The result surprised us: on taskapi's first run the whole
five-package architecture — every import path, every qualified reference,
every package clause, the config→models→store→api→cmd dependency graph —
compiled clean on the first try. The hard part of
multi-package wasn't hard once the prompts carried the right context.
Every failure class became a deterministic gate
Ten runs up the ladder surfaced failure modes one at a time — and each became a permanent, model-free repair or prompt rule. The pattern that keeps repeating: a 7B's mistakes are not random noise, they're enumerable mechanical classes, and every class you close deterministically is closed forever, for free:
- Truncation: the serving default silently cut the biggest file mid-statement (fixed max_tokens; strip orphan code fences).
- Redeclaration collapse: a file re-declares a sibling's
symbols and the fix loop bounces forever (deterministic
strip_redeclarations). - Cross-package misqualification:
service.ErrExistswhen the sentinel lives instore; bare unqualified calls; and the killer — goimports cannot add local-module imports, so the Builder adds them itself (_requalify_undefined+_ensure_import). - The oscillation: the deterministic pass adds the import, the same round's model fix regenerates the file and drops it again — forever. Fixed twice: requalified files are excluded from that round's model targets, and their imports are pinned — re-added after every later model edit.
- Assignment arity:
2 variables but f returns 1 value— blank_identifiers are dropped or padded; named variables are never touched. - Root-cause blindness: a failing assertion always pointed the fix loop at the test file, so a genuinely-wrong implementation could never be repaired. Now a package that keeps failing at runtime pulls its implementation files into the fix targets too.
- Plus prompt rules with teeth: Go 1.22 ServeMux method-routing, no
interface-by-pointer, full method implementations (no stubs), never shadow
*testing.T, fresh fixtures per subtest.
The honest gap: taskapipro stopped 20 human lines short
Three full autonomous runs of the six-package spec all landed at exactly 2/3 —
all six packages build and vet clean every time, and the
failures were the same two test-semantics bugs each run, down to the
line. That reproducibility is the finding. The model wrote correct
env-with-default helpers for five of six config fields and missed the same one
every time; its table test correctly isolates state per subtest (the fix-time
diagnosis landed) but keeps expected values that assume the old shared-store
accumulation. The fix loop, even pointed at the right files with the right
diagnosis, could not make those two semantic expected-value repairs —
and a maintain pass asked to make both fixes dug a fresh hole in
the service layer and was correctly rolled back by the non-regressing gate.
So we finished it by hand and counted: ~20 lines of human edits in
1,225 (one missing default helper; three corrected expected values
plus the coverage the table shape couldn't express), documented line-by-line
in the artifact's HUMAN_EDITS.md. go build,
go vet, go test -race: green across all six
packages, including the model's own router, service and config tests.
What we learned
- The algorithm was the scaling lever, full stop. The model never changed. 7→11→13→16 files and 1→6 packages came entirely from package-aware context, verified retrieval, and deterministic gates.
- Mechanical failure is a solved genre. Compile-level correctness across six packages is now routine for a 7B — every mechanical class we met is gated. What's left standing is different in kind.
- The residual is semantic, and it's the model's half. Deriving a consistent set of expected values — "what should this return, given everything else I wrote" — resisted every prompt and every loop we have. That's not a gate; that's knowledge and reasoning. It's exactly the half we're now attacking from the other side: domain-adaptive pretraining of the base on ~45M tokens of real, high-star Go — running on a free cloud GPU as this report is written.