Repository path: workshop/experiments/E-20260730c-revision-close/critic/dispositions.md · rendered 2026-09-09

Pre-run critic dispositions — E-20260730c

moonshotai/kimi-k3 (P4), one call, stop, in 5,398 / out 4,771, provider Together, $0.0876726, 122 s. Verdict NEEDS-REDESIGN — two BLOCKING, four MANDATORY, one ADVISORY. All seven accepted; none declined. Note (rr), twentieth consecutive session.

This is the first NEEDS-REDESIGN this project has taken since S021, and it was taken rather than argued down. The two BLOCKING findings are both about the same thing: the design's two central self-descriptions — "the attribution gate can fail" and "the rebuild is not tuned" — were assertions, and one of them was false.

Finding 1 [BLOCKING] — F5's tolerance was larger than the effect it gates. ACCEPTED IN FULL.

"F5 can pass in all six cells while a uniform cross-day shift of up to 10 points fully accounts for F3 flipping." This is arithmetic and it is right. E(surface) must move from 12.87 to ≥ 20 — a 7.13-point gap — and F5 tolerated 10 points of drift. The gate could not fail in the one circumstance it existed to catch.

A1. F5's per-cell criterion is tightened to mean |Δ| ≤ 3 points and Pearson r ≥ 0.90, and — this is the part that matters — attribution is made a comparison rather than a threshold:

Any movement in E(surface) that does not exceed the observed cross-day mean |Δ| on the 216 byte-identical items is reported as UNATTRIBUTABLE, regardless of F5's pass/fail verdict.

Finding 2 [BLOCKING] — "not tuned" was unsupported and unverifiable. ACCEPTED; the stronger branch of the fix taken.

"The same designer who saw S057's F3 failure scores … hand-built the replacements under a criterion that was itself derived from that failure." True, and the critic is right that no amendment could repair it after the fact.

A2. The lead's eight hand-built surface controls are WITHDRAWN UNSCORED. They are kept in git history and in this file's first version, and they were never sent to a reader. In their place, five controls built blind by qwen/qwen3.7-max — config/models.md's probed-but-not-selected first reserve, which is not a reader (P1/P3/P5), not the declared reader reserve (P2) and not the critic (P4). A materials-construction role is not a judging role, so charter §5 is not engaged; the seat is declared here rather than chosen later.

What the builder was told: the edit must substitute different English words for the original words while leaving the proposition identical; it must introduce a genuine new English word (not a contraction, not a misspelling, not a reordering, not a punctuation change); and the corpus's span statistics (draft-side median 2 / mean 2.72, revision-side median 2 / mean 3.20). Twelve carrier sentences, of which it chose five. What it was not told: that a gate exists, that its output would be scored, what the axes are, any threshold, or any S057 number. The lead selected nothing — build_blind.py registered "the first five well-formed items, in the order returned" before the call, and that is what was used.

The residual confound, declared rather than dissolved: the prompt was written by a lead that had seen S057's scores. The builder's ignorance is of the scores and thresholds, not of the lead's framing of the problem.

Finding 3 [MANDATORY] — the registered "8 of 8 lexical" claim was false of the materials. ACCEPTED, and the criterion is replaced by a measured one.

The critic caught that C009 was a pure word-order rearrangement with no lexical substitution at all, in a control set the design described as "8 of 8 lexical". A3. The "match on mechanical kind" criterion is struck and replaced with a criterion that was measured before the rebuild:

Does the revision span introduce a word type absent from the draft span? 184 of the 205 real edits do — 89.8%. Of S057's eight surface controls, four do; and all four of those are two contractions (I'd, wasn't) and two misspellings (acquiescense, rheumatismn), i.e. the same word in another form. Not one of S057's eight surface controls substitutes a different English word for a word — which is the operation that makes up nine tenths of the corpus.

That is the diagnosis note (bdu) was reaching for, and it is checkable. The blind builder was given this requirement and no score.

Finding 4 [MANDATORY] — kind and length were still confounded. ACCEPTED, and the blind build answers it better than an amendment could.

The lead's withdrawn controls sat at draft/revision mean spans of 3.750 / 3.750 against the corpus's 2.717 / 3.195 — the critic is right that they did not match. A4. The blind builder was given the corpus's span statistics as a constraint. Realised, from build_items.py: median 2.0 / 2.0, mean 2.800 / 3.200 against the corpus's 2.0 / 2.0, 2.717 / 3.195. Span is therefore matched to within 0.083 and 0.005 of a word, blind. Had it not matched, span would have been reported as an unresolved covariate.

Finding 5 [MANDATORY] — F5 conflates cross-day drift with batch-context effects. ACCEPTED IN FULL.

A5. F5's registered consequence now names both confounds — cross-day drift or within-batch context effects from the substituted items — and neither is claimed over the other. Per-item Δ distributions are reported, not only cell means. On the scatter request: the substituted slots are 56, 96, 108, 158, 174 of 221, which S057's seed-7919 shuffle had already scattered; the figure is printed by build_items.py rather than asserted. The redesign also reduces the change, from 8 substituted items to 5, so 216 items are byte-identical rather than 213.

Finding 6 [MANDATORY] — Q3 and F5 were jointly inconsistent. ACCEPTED IN FULL.

"If F5 fails but F3 passes, the design will report a P4 whose cross-session meaning it has just declared unclaimable." A6. Registered explicitly:

If F5 fails, P4 is reported as a within-session descriptive statistic only; S057's 22.48 stays withheld permanently; and item (a) is recorded as unanswerable with the instruments this project has — which makes ARM-revision's own retired ending live on that ground.

Finding 7 [ADVISORY] — no negative replication. ACCEPTED, and satisfied without changing the batch.

A7. Three of S057's surface controls are kept byte-identical as band N — the non-corpus-kind negative band (C013 and C014 punctuation-only, C016 a pure word-order rearrangement). F3's surface mean is computed on band L (n = 5) only, and a new gate is registered:

F7 — the kind contrast. mean E(band L) − mean E(band N) ≥ 10. If band N scores as high as band L, the substitution hypothesis is refuted and any F3 pass licenses only the weaker claim "any surface perturbation now scores ≥ 20".

Band N also joins the reproduction anchor, which is why 216 items are byte-identical and not 213.

What the redesign cost, stated plainly. The design lost its own control set unscored; its central mechanical criterion was replaced with a different one because the first was false of its own materials; F3's surface mean is now over five items instead of eight; and its attribution claim was converted from a threshold that could not fail into a comparison that can. Four of the seven findings improved the design by taking something away from it.