One run is not a green

The last report ended on a table: ten specs, ten greens. It was true and it was not yet earned. Every one of those greens was a single roll of a stochastic model, and this project's own bar for calling something green is two. So today's job was to go back and pay for a number I had already written down. The suite paid — and the shape of how it paid turned out to be worth more than the number.

The number I had not earned

Yesterday I changed seven specs, four gates, and the repair prompt. Then I ran each spec once, got ten greens, and wrote them down. Every instinct in this project says that is not enough. A 7B model is stochastic; a spec that passes once may be a spec that got lucky once. And the week had been one long lesson in things that look like they work: a harness that deleted its own evidence, a default that taught the wrong pattern, a best-of-N that never chose, a ruler that lied ten times in a single day.

A green that has been measured once is a claim. A green that has been measured twice is a result.

So: all ten specs, from scratch, again. Nothing else touched.

10 / 10, and the pattern underneath

                run 1     run 2
taskapi         88.8%  →  88.8%     // untouched
kvservice       88.0%  →  88.0%     // untouched
logstats        82.0%  →  82.0%     // untouched
usersapi        80.8%  →  80.8%     // untouched
ratelimit       75.0%  →  75.0%     // untouched
shortener       72.7%  →  72.4%     // untouched

taskapipro      73.1%  →  84.3%     // +11.2  spec touched
taskflow        66.2%  →  79.7%     // +13.5  spec touched
tasks-api       56.5%  →  68.3%     // +11.8  spec touched
workapi         81.1%  →  81.6%     //  +0.5  spec touched

10 / 10 green.  build + vet + test -race, every one.

Read the two halves against each other, because that is where the result actually lives. Every spec I touched gained. Every spec I did not touch came back flat to the decimal. Not approximately flat — 88.8 to 88.8, 88.0 to 88.0, 82.0 to 82.0.

One measurement, two proofs

That twin pattern is doing two jobs at once, and I did not design it to:

• The gains are real. If run-to-run noise were large enough to produce a double-digit swing, the six untouched specs would have wandered too. They did not move at all. So the +11.2, +13.5 and +11.8 are not the dice — they are the spec edits.
• Nothing regressed. Six specs I never opened, re-generated from scratch against four new gates and a rewritten repair prompt, landed on their exact previous numbers. That is the regression sweep, for free, from the same run.

And there is a quieter thing in the touched half. Yesterday those four spec fixes were proven against a reference implementation — I hand-wrote the code the spec was asking for and measured that. They had never once been run through the actual model. A reference implementation proves a spec is satisfiable. It does not prove the model will satisfy it. Today was the first time anything checked the second claim, and it held: 89.1% promised on taskapipro's api front, 84.3% delivered on the whole project.

I inferred an actor from an outcome

workapi failed its first compile round with two errors, and both were the kind the compiler names out loud:

w.count.Inc undefined (type "sync/atomic".Int64 has no field or method Inc)

cannot use auth.TokenAuth(...)(http.HandlerFunc(h.Create)) (value of interface
  type http.Handler) as func(http.ResponseWriter, *http.Request) value in
  argument to mux.HandleFunc

A hallucinated method on atomic.Int64, and a handler wrapped in middleware being passed to mux.HandleFunc when it is no longer a HandlerFunc. By round 2 both were gone, and I wrote — in the first draft of this very report — that the model had repaired them itself, and that the honest outcome was therefore two gates I did not need to build.

That was wrong, and the log had already said so. Two lines below the errors:

[guildlm-build]   deterministic fix in internal/api/router.go
[guildlm-build]   deterministic fix in internal/worker/worker.go

Those are the two files that carried the two errors. The repairs came from _fix_atomic_inc and _fix_handle_vs_handlefunc — gates written days ago, for precisely these two classes, firing exactly as designed. The model did not fix them. The deterministic layer did, and it did it silently, which is what a working gate looks like.

I saw the errors are gone and concluded the model removed them. The disappearance of a defect does not tell you who removed it, and the machine was naming the actor on the very next line. In a report whose whole argument is measure it, do not believe it, I asserted something I had not measured. That is the finding, and it is not a comfortable one:

An outcome does not name its cause. "It went green" and "it got fixed" are not attributions — and the moment you skip reading who did it, you will credit whoever you already believed in.

The gate the criterion actually found

The real gate candidate came from somewhere else entirely — a brand-new held-out spec (a double-entry ledger, five packages, a domain the corpus has never seen, written once against the checklist and never tuned). The compiler said:

internal/service/service.go:18:17: unknown field s in struct literal of type Ledger

And the code says why:

type Ledger struct {
    store store.Store            // the field is named store
}

func NewLedger(s store.Store) *Ledger {
    return &Ledger{s: s}         // the model keyed it by the PARAMETER name
}

The model contradicted itself inside a single file, and then could not repair it: the same error, on the same line, in fix round 2 and again in fix round 3, with the compiler naming it precisely each time. That is the criterion, and this is what it looks like when it is genuinely met:

A defect earns a gate not when it happens often, but when the compiler names it and the model still cannot repair it. A defect the model fixes on being told is not a gate — it is the loop working. And a defect a gate fixes is not evidence about the model at all.

What the deterministic layer did do

Which is the other half of the correction: the gates were not idle today, they were doing most of the work, and I had been mis-crediting it. taskflow came back with *Store — a pointer to an interface, the compiler naming it precisely: type *Store is pointer to interface, not interface. The gate written for that class five days ago fired, propagated across both files that carried the bad signature, and the project converged in a single round. tasks-api — historically the most stochastic spec in the suite, the one marked nudge-resistant — went green with converged after fix round 1 (deterministic): the model was never asked to repair anything. The gates did it alone.

green by the gates alone, no model call:  11     // start of the week: 0
stuck:                                     1     // the honest floor

The one stuck artifact is the known one: taskflow's missing POST /projects. An unregistered route is legal Go. It builds, it vets, and the compiler has nothing to say about it — there is no sentence for a gate to listen for. Only a test notices. That is not a gate regression and it is not a backlog item; it is the boundary, and it belongs to the spec.

Result

The suite is green twice over: ten specs, two independent runs each, build and vet and race-detector, nothing traded for it. The single measurement, honestly repeated, confirmed the four gains and ruled out a regression in the other six at the same time. But the sharpest thing today did not come from the sweep — it came from getting the sweep wrong. I watched two defects vanish and credited the model, when the log named two gates on the very next line. A green I had measured once was a claim I had not checked; a cause I had inferred from an outcome was a claim I had not checked either. The first kind of carelessness costs you a false number. The second costs you a false theory — and it was hiding inside a report about not believing things you have not measured.