Repository path: workshop/experiments/E-20260730-grain-clause/critic/dispositions.md · rendered 2026-09-09

Pre-run critic dispositions — E-20260730-grain-clause

Critic: P4 moonshotai/kimi-k3, provider Moonshot AI, one call, stop, in 28,259 / out 3,629, 113s, $0.139212. Chosen because P1, P3 and P5 are the readers and a reader may not critique the instrument it is about to be. Raw: runs/critic.raw, prompt runs/critic.prompt.md, verdict runs/critic.output.md.

Verdict: NEEDS-AMENDMENT. Eleven findings — one BLOCKING, seven MANDATORY, three ADVISORY. All eleven accepted. Every amendment below was applied to design.md before any reader call was dispatched. Note (rr) — an independent pre-run critic pass — fires for the nineteenth consecutive session, and this is its second-largest return: it struck one failure criterion as unfireable, one prediction as unfailable, and rebuilt the arm's own decisive measurement.

The three that changed what the session can conclude

Finding 4 — BLOCKING, and it is the one that matters most. Arm A's warrant measurement was specified as Jaccard between rules' test-3 firing sets. All three rules share test 3's first limb verbatim (established borrowing), so every borrowing site is in all three sets and every pairwise Jaccard is inflated toward 1 regardless of what the second limb does. F3's Jaccard ≥ 0.80 clause was therefore close to unfailable. Accepted. The measurement is replaced by the critic's own proposal — the set of sites at which a variant flips C15's test-3-versus-not-test-3 decision:

D(rule) = { site : the rule fired test 3 there and C15 did not, or C15 fired test 3 there and the rule did not }, per reader.

That quantity is zero when a rewrite preserves the clause's extension and grows as it departs from it, it is computed entirely from reader outputs, and it is not inflated by the shared limb. Q4 and F3 are rewritten on it.

Finding 2 — MANDATORY. Q1 rested on one byte-identical repeat, which yields one agreement number and can neither establish nor refute stability. Accepted, at half the critic's price: the C15 condition is run twice in-session for each primary reader (2 extra calls, not 6 — P5 needs no repeat because it was not a reader at S056 and its numbers are new data either way). That gives three observations of the identical request: S056's, and two this session. F1 now gates on both.

Finding 3 — MANDATORY. F1 gated raw agreement while Q2 is a κ claim, and κ at n = 23 over nine labels is far more volatile than raw agreement. Accepted. F1 gains a κ clause with a registered band, and Q2 is assessed only if it passes.

The rest

Finding 1 — MANDATORY. NORULE is structurally unreachable: test 4's furniture branch returns a handling for anything, so F4 and the NORULE half of Q8 could only ever have fired on reader non-compliance, which F5 already covers. Accepted — F4 re-scoped to UNSURE, Q8 re-scoped to UNSURE. Recorded as the second time in three sessions that a critic found a registered criterion that could not fire.

Finding 5 — MANDATORY. Arm A's primary statistic did not say how two readers' sets combine. Accepted, and taken stricter than the critic's own fix (which was to average): every arm-A prediction is assessed per reader and counts as held only if it holds on both. Averaging would let one reader carry a prediction the other refutes.

Finding 6 — MANDATORY, and it costs the session its most quotable sentence. Arm J changes the pair, the passage, the site count, the label distribution and the instruction wording at once, so no pair-attributive claim is available from it — yet §2 said "arm J is where it shows" and Q6/Q7 planned to draw exactly that claim. Accepted. Q6 and Q7 are demoted to descriptive, and the design now pre-registers that no pair-attributive claim will be made. What survives is arm J's internally valid content, which is the part worth having: C15 against C15′ on the same prompt shell with only the rule differing, i.e. an independent replication of the treatment effect on a second site list. §2's framing sentence is rewritten to say that instead.

Finding 7 — MANDATORY, and it is a defect in the control the design had not seen. C15″ asks the reader to count the letters of "a romanisation of the source item", and romanisation schemes differ (хата → khata / hata; 煙管 → kiseru), so two readers can disagree under C15″ for a reason that has nothing to do with mechanisation versus aptness. Accepted as confound 9, with a pre-commitment: a C15″ κ below C15′'s — the F3 pattern — carries a stated transliteration caveat and does not on its own license F3.

Finding 8 — MANDATORY. F5's fall-through to the reserve would silently turn "P1~P3" into "P1~P2" and destroy the retest. Accepted: any reserve substitution in arm R voids retest status for that reader's conditions, which are then reported as new data.

Finding 9 — ADVISORY. Q2's ≥ 0.15 threshold is smaller than the +0.175 drift S049 measured on a byte-identical repeat. Accepted: the design now states that Q2 is assessed only conditional on the strengthened F1, and that the threshold's justification is the retest variance measured this session rather than an assertion.

Finding 10 — ADVISORY, and the honest one. Confound 4 — the lead wrote the control this session, after reading the nine disagreement sites, while the treatment's wording was fixed by an earlier session — is declared and not mitigated, and it points at the outcome the session expects. Accepted: a registered sentence now says that Q3 holding is weak evidence for the mechanisation reading precisely because the control was tunable, and F2's closure must carry it.

Finding 11 — ADVISORY. The residual exact in the EQUIVALENT label's definition was to be flagged only on a null Q2; an attenuated positive is equally consistent with it. Accepted: the caveat is extended to every Q2 outcome.

Nothing was declined

The critic offered no finding this design refused, which is unusual here — S056, S057, S058 and S060 each declined at least one sub-clause in writing. The one place the critic's fix was narrowed rather than rejected is finding 2 (two extra calls rather than six), and the narrowing is stated with its reason above.

Budget consequence

Two extra arm-R calls at max_tokens 9,000: +≈$0.21 worst case, taking the design's worst case from ≈$1.90 to ≈$2.11 across 22 calls including the critic. Actual critic spend $0.139212 is 31% of its declared $0.45 chain worst case, inside the 15–34% band.