Repository path: workshop/experiments/E-20260728i-coverage-replication/critic/dispositions.md · rendered 2026-09-09

Pre-run critic dispositions — E-20260728i

Critic: P2 google/gemini-3.6-flash, provider Google, one call, temperature 0.2, reasoning: {"effort":"low"}, in 7,153 / out 1,432, finish_reason: stop, 10s, $0.0214695 — 30% of the $0.072 worst case built from max_tokens (note (abc)). No fall-through needed; reserve qwen/qwen3.7-max unused. P2 was chosen because P1 and P3 are the readers this design is about to create and cannot critique the instrument they are about to be; P4 and P5 are off the call list (note (b)).

Verdict: NEEDS-REDESIGN. Nine findings. Seven accepted, one accepted-as-limitation, one declined in writing. Two of the accepted findings withdraw a claim this design made, one of them the design's own reading of its central prediction. Raw request and response: runs/critic__gemini-3.6-flash.json; prompt: runs/critic.prompt.md.

Note (rr) fires for the ninth consecutive session: the most valuable thing the call bought was the refutation of a sentence the design had written that morning.

(The critic numbered ten items under nine headings; the table follows its own numbering, and "nine findings" counts its headings.)

The two that withdraw a claim

A2, and it is the finding of the session. The design argued: the lead had read the candidate list before writing Log B, that priming would make the lead notice inventory-shaped decisions, so coverage would be biased upward, so a null — no rise — would be safe to read. The critic's answer is that priming does not only change which decisions get noticed as inventory-shaped; it changes which decisions get written down at all. A primed lead may record extra fine-grained decisions no candidate reaches, and every such entry enlarges the denominator and lowers the proportion. The two effects push opposite ways and neither is measured. A null is therefore as uninterpretable as a rise, and the sentence claiming otherwise is withdrawn.

The circumstantial evidence for the critic's mechanism is in the counts the design already had: Log A has 21 decisions in 542 source words; Log B has 42 in 1,506 source characters. The design cannot tell whether that is a longer passage, a different language, a revision pass, or an attention effect, and it did not ask.

A4 needed an answer, not an amendment. The charge is that the design cannot come out against closure. It is partly true and partly a misreading of the ordering: ARM-framework's completion criterion was declared at S035, sixteen sessions before this design, and explicitly admits "the arm closes retired with a written statement of what evidence a release is short of" as one of two honest endings; S046 then predicted, in writing, that closure was the likely one. This design is downstream of that, not a device to reach it. What was genuinely missing is a stated outcome that would defeat closure, and Amendment 4 supplies one.

The one declined, and why

B4 — order. The fix is obvious (give the second reader the logs in the opposite order) and it is refused, because it would confound order with reader, and reader-vs-reader agreement is the only measurement that survives A2. Counterbalancing would buy a controlled order effect at the cost of the finding the run now exists for. Order stays fixed, Log A first for both, and the confound is declared on the result page as bearing on the Log A / Log B comparison — which A2 has already reduced to description.

The one accepted as a limitation without a change

B2 — rule 4. Changing it would require the readers to judge pair-applicability, a different and harder task that the design deliberately assigns to D-20260724-04 and the inventory's §3 instead. The critic is right that ignoring the pair line inflates Log A's coverage relative to a pair-aware reading, and therefore shrinks the Log A / Log B gap. Under Amendment 2 that gap is no longer carrying an inference, so the bias is recorded and not corrected.

Amendments, all made before dispatch

1. P4 is evaluated on three Log B subsets, not one: all 42, the draft-only subset B1–B27, and the revision-only subset B28–B42. None is a clean counterpart to Log A's 21 and the result page must say so.
2. §7's "only the null is informative" is withdrawn. What survives the withdrawal, stated now: (a) the reliability measurement — lead-vs-reader and reader-vs-reader agreement on Log A, which uses a published mapping and a log written before the inventory existed, and does not depend on Log B at all; (b) the DECIDES count, which is a near-absolute rather than a proportion. Log B's coverage proportions are reported as description of one log and are not a replication of RS-20260726e's figure. The word "replication" is withdrawn from the design's own title claim.
3. Failure criterion 2 is restated on chance-corrected agreement. Cohen's κ on the binary, with the run's coverage figures unreportable-as-estimates at κ ≤ 0.20; raw agreement and the majority-class baseline are reported alongside so the criterion can be checked.
4. New P6, the closure-defeating outcome, registered before dispatch: if either reader returns DECIDES ≥ 2 on Log B through a candidate other than C12, the inventory contains prescriptive-about-translating content that the lead's own mapping missed, and ARM-framework must be extended rather than closed. Reachable, checkable, and against the session's declared intention.
5. Instruction rule 2 gains one clause before dispatch: a candidate that lists options is INFORMS even where only one listed option fits, and DECIDES requires the candidate itself to do the ranking.
6. §4's blinding claim is corrected: there is no pair blinding. The readers can identify both pairs from the source strings and may treat them differently for reasons the run cannot see.
7. P3 is restated as a check on reader discipline (do the readers respect rule 2?), not as a test of the inventory.
8. P5 is withdrawn. The property — "about a single word or a single figure" — has no operational definition, no assigned classifier and no independent check, and the run has no budget for a third reader to supply one. RS-20260726e §3's grain finding is therefore not re-tested here. Withdrawn rather than weakened.
9. analyse.py and verify.py must join on an explicit D<n> ↔ B<n> map with an assertion on both key sets, and must report three-way agreement alongside the binary.
10. Failure criterion 3 rises to $0.50 per reader call, and a finish_reason: length whose body nonetheless contains all 63 parseable lines is accepted, not voided — the criterion is about completeness, not about the flag.