← Research log
Report #14 · 2026-07-10

A nudge is not a gate — promoting the authoring rules

Report #13 closed the whole suite by hardening each spec by hand — a good engineer's clarifications, written into YAML. But a lesson written into one spec's YAML only helps that spec; the next one starts naive and re-earns the same wall. So this report tries to pay down that debt: move the recurring authoring rules out of individual specs and into the Builder's permanent defaults, then prove they carry their weight by deleting the hardening from a spec and watching it stay green. Three rules moved cleanly. The deletion probe failed — and the failure is the most useful thing in the report, because it draws a sharp line between what a prompt nudge can carry and what only a deterministic gate can.

The spec-authoring debt

Across the sweep, the same handful of clarifications kept reappearing, spec after spec. Two of them were already promoted to defaults in earlier reports (never shadow t; isolate per-case state). Three were still being copy-pasted by hand:

Rule (copy-pasted per spec)Specs that repeated itThe failure it prevents
write struct literals with field names, never positionalkvservice, taskapiproa value silently lands in the wrong field — and go vet only flags this for an imported struct, never a same-package test table
interface/impl parity — same method set on both sidestaskflow, taskapiprothe impl has a method the interface never declared (or vice-versa), so a call through the interface is undefined
register the method value, never call the handlertasks-api, ratelimitmux.HandleFunc(pat, h.Create(w, r)) or an invented extra arg instead of h.Create

Copy-paste is debt: it means the system hadn't learned the lesson — I had. So each moved into _generate_prompt as a conditional default, firing only on the file kind it applies to (the 7B degrades on long prompts, so a rule that fires everywhere is a tax): field-named literals on _test.go files, interface/impl parity on implementation files, method-value registration on routing files. Three new unit tests assert each rule appears in the right prompt and stays out of the wrong one. The Builder's test suite went 168 → 171 passing; the verified retrieval corpus still builds 36/36. Nothing in the model changed; this is all prompt scaffolding.

The probe: delete the hardening

A default is only worth its prompt length if it lets a spec drop the hand-hardening. So the test is subtractive. tasks-api's two original walls (Report #13) were exactly (1) the model dropped the Update method, and (2) the router called handlers instead of registering them — both now defaults. I wrote tasks-api-min.yaml: the same spec with the all-methods paragraph and the method-value paragraph stripped out, leaning entirely on the promoted defaults.

It did not go green — twice. The first attempt also tripped an orthogonal string-vs-int id bug I had over-stripped (the fact that this spec's IDs are integers isn't one of the promoted idioms, so restoring it was fair); with that one spec-specific line put back, the second attempt still exhausted all six fix rounds on the same wall. The residual was blunt:

handlers.go: a.store.Update undefined (type Store has no field or method Update)

And here is the clarifying part. The generated store.go looked internally consistent:

type Store interface {
    Create(t Task) error
    Get(id int) (Task, error)
    List() []Task
    Delete(id int) error          // no Update — dropped from the interface…
}
// …and no Update on the impl either, so this compiles:
var _ Store = (*MemStore)(nil)

The model dropped Update from the interface and the implementation together. The parity assertion passes — both sides agree — yet the method the handlers call doesn't exist anywhere. This is not a parity failure, which my new default is built to prevent. It is a completeness failure: a required method missing from both sides.

Why completeness can't be a blanket default

There is a structural reason a prompt nudge can't fix this. The Builder writes files in order — store.go (3/8) is generated before handlers.go (4/8). When the model writes the Store interface, the caller that will need Update does not exist yet. A generic default cannot say "include the methods your callers will need," because at generation time there are no callers to point at. The only thing that knows Update is required is the spec — which is exactly why the original hardening ("handlers call store.Update, so a missing Update breaks the build") was load-bearing, and why it doesn't generalize.

The fix loop tried: five rounds, each correctly widening to "type Store (missing method Update)" and asking the model to repair it. It never converged — the model kept regenerating a Store still missing the method, or fixed one file and broke another. A probabilistic repair of a synthesis gap is still probabilistic.

The honest line: nudge vs. gate

This report is a negative result on its headline claim — you cannot delete tasks-api's hardening — and it is worth more than a green checkmark, because it partitions the work cleanly:

Three authoring rules are now permanent, conditional, regression-safe Builder defaults — kvservice still greens first-generation and the six-package taskapipro still converges green and race-clean, so the additions cost nothing — and the deletion probe drew the boundary the defaults can't cross. The moat is still the system — a fixed 7B, an agentic loop, a verified corpus, a growing set of gates, and specs written like an engineer writes them. This report tightened one screw and, more usefully, marked exactly where the next one goes.

Postscript — the gate, built and bounded

The pointer got picked up: build the gate. It exists now as _fix_interface_missing_method, and building it sharpened the class one more notch. The gate handles the safe half — the taskflow escape, where the model writes a method on the concrete implementation but forgets to declare it on the interface, so a call through the interface is undefined. When a same-package type already implements every method the interface currently lists and has the missing one, the gate lifts that method's exact signature — copied verbatim from real code, never inferred — into the interface. It is safe by construction: the implementer provably satisfies the augmented interface, and it bails on every ambiguity (the type isn't an interface, the method is already declared, no same-package type implements interface-plus-method, or candidate implementers disagree on the signature).

What it pointedly does not do is the case that motivated the whole report — tasks-api-min, where the method is gone from both sides. There is no signature to lift, and synthesizing a method body deterministically would be guessing, so the gate no-ops and leaves that to the model. The boundary from the body of the report holds, one notch finer: the mismatch half of completeness (the impl has it, the interface doesn't) is mechanical and now deterministic; the pure half (which methods must exist at all) is spec knowledge and stays with the model. It is verified the way a deterministic gate should be — not by hoping a stochastic run trips it, but by exercising the exact path: fourteen unit tests over the fire path, the multi-method case, and every guard, plus a real module where go build and go vet fail before the gate and pass after, gofmt-clean. And it did fire in the wild: a live taskflow run shipped a Store interface missing four methods its own in-memory implementation defined, and the gate lifted all four — Create, Get, List, Delete — back onto the interface, verbatim. That particular run did not converge — it foundered on two residuals the gate has no business touching (a *Store pointer-to-interface receiver and a missing struct field, the sort of chaos taskflow's twelve files still throw; a later generation of the same spec greened in six minutes on a single deterministic fix, which is just to say the spec is stochastic) — the honest shape of it being that a correct gate closes its class and no more.

Though "and no more" lasted about an hour. One of those two residuals — the *Store pointer-to-interface receiver — is itself a mechanical, compiler-named class: a pointer to an interface has no methods and is essentially always a bug, so *Interface → Interface became a second deterministic gate the same session (_fix_pointer_to_interface). Verified the same way — seven unit tests including the load-bearing guard that a pointer to a struct is valid and left alone, plus the real taskflow artifact, where it clears all three pointer errors and leaves exactly the one genuine model bug (a reference to a struct field that was never declared) for the model. That is the shape of the whole project in miniature: run it, read the residual, and if the compiler names the defect the same way every time, it becomes a gate; if it doesn't, it stays with the model or the spec. The Builder's test suite stands at 192.

All training, serving, benchmarking and Builder runs are local on an M1 Max with Apple MLX — total cloud spend: $0. The specs, the promoted defaults, the gates, the verified retrieval corpus and the Builder loop: github.com/guildlm/builder.