Repository path: workshop/experiments/E-20260729c-neutral-summary/critic/dispositions.md · rendered 2026-09-09

Pre-run critic pass — findings and dispositions

Model: moonshotai/kimi-k3 (P4), provider read off the response. Verdict: NEEDS-AMENDMENT, five findings. All five accepted. One call, $0.067524, first attempt, finish_reason: stop. Raw body: kimi-k3.json, .raw.

P4 was chosen because it is a subject in nothing here: P1 and P2 are the two voices being re-run, P3 is the stage-1 summariser, P5 is the declared reserve. Its one prior failure in this project was at max_tokens 6,000; note (bdl) says the remedy that worked was raising the cap, so it ran at 16,000 and returned on the first attempt.

The amendments below were made before any other call was dispatched. The design file carries them as §Amendments (v2).

F1 — A1's failure attribution is not licensed by one call. ACCEPTED, by rewording and by adding a second summary.

"a single P2 vote call is one sample. … If 2b returns non-C, the design cannot distinguish (i) the non-lead summary changed the outcome, (ii) P2's run-to-run variance changed the outcome, or (iii) the declared protocol departure … Three candidate causes, one observation. … The power statement is honest about N-vs-L and then quietly overclaims for A1; that is the worst place to overclaim."

This is right and it is the finding that mattered. The design cited S053's result about unmeasurable judgment variance and then wrote "a one-cell question and one cell answers it" four lines later. Two changes:

1. Failure criterion 1 is rewritten to the critic's option (b): a non-C verdict is reported as failure to reproduce in a single instance under a changed protocol, cause unidentifiable between summary authorship, vote variance and protocol departure. It still opens a decision page; it no longer asserts a cause.
2. A second, independently authored non-lead summary is added (N2, model P5), with its own vote call. This does not measure vote variance, and is not claimed to; it bounds summary-authorship variance, which is the one of the three causes this experiment is actually about. If P5 returns no body (note (b)), the arm runs with one summary and F1's rewording carries the result alone — which is why the rewording, not the extra call, is the primary repair.

F2 — the "neutral summary" is asserted, not established. ACCEPTED in full.

"The whole-page repair genuinely removes excerpt selection, but the summarizer still chooses what to include, emphasize, and omit … this design delivers an unattributed-tilt summary and calls it neutral because the author isn't the lead."

Three changes:

1. The word "neutral" is withdrawn from every claim this experiment makes about its own artifact. What is delivered is a non-lead summary. That is exactly the obligation's own wording — "a summary the lead did not write" — and the ratification record's looser phrase "a neutral summary" is a stronger thing that this design does not deliver.
2. A frozen load-bearing-sentence checklist (materials/loadbearing-checklist.json, committed before stage 1 runs) fixes which sentences count, so the lead is not choosing after the fact. Coverage of each is checked mechanically against every summary produced.
3. A mechanical tilt gate is run on each summary and reported whatever it says: counts of directive and verdict-shaped language, and whether the summary reaches a conclusion about condition (ii) it was not asked for.

F3 — B1/B2 are trivially satisfiable and the device classification is circular. ACCEPTED in full.

"An author who knows B1 can satisfy it by simply never writing a gloss — B1 then measures the translator's compliance with B1 … the same agent grading its own homework."

The device classification moves to a non-lead model, blind to the census's predictions and to the design: P1 openai/gpt-5.6-terra, shown the frozen census, the frozen English and the device vocabulary, and nothing else. The lead's own classification is kept and reported beside it as a labelled second reading, internal-judgment-only, with disagreements printed rather than resolved.

What the amendment does not repair, and the design now says so. B1 remains partly a compliance measure: an author who has registered "zero stage-directions" will not write one. The independent classifier fixes the classification, not the authorship. The sentence Part B licenses is therefore narrower than the design first wrote it: these devices exist and an independent reader finds them in this text, not a translator who did not know the prediction would have found them.

F4 — A3 and A5 have no declared failure consequences. ACCEPTED in full, as proposed.

• A3 failure (C returned but scoped to all six senses) → same consequence as A1 failure. The narrowing is the binding part of the 2026-07-25 outcome; a C that does not narrow is not that outcome.
• A5 failure (no voice notices the citation pattern) → condition (iii) is still discharged, with an explicit note that whole-page evidence did not surface the scope problem, which weakens the rationale for the re-derivation repair itself and must be reported in those words.

F5 — "Part B costs nothing" is false as stated, and the contamination exception is unnamed. ACCEPTED.

Reworded to "Part B incurs no API cost". The exceptions are now named in the design and on the translation artifact:

• Andreyev, «Баргамот и Гараська» was considered as the translation limb and rejected because triaging it meant printing the tail of the stored Lowe comparator, which primed the lead at the exact site that made the story attractive. Recorded as method note (bdn).
• For the tale actually chosen, one fragment of Seltzer's English — the final clause, "five kopeks. Now, Muzhik, rejoice." — was printed by the slice-boundary check that extracted the comparator from the Gutenberg anthology. Nothing else of Seltzer has been read. Declared on the artifact and in the log against site S31, which is that clause.

What the critic got right that the design had already half-said, and one thing it declined to attack

The critic explicitly cleared (c) — A1/A2/A4 are falsifiable, and failure criterion 3 correctly makes a null a success — and cleared the analogical distance in (e): "an existence counterexample is a valid refutation of 'only' … the analogical distance is acceptable and arguably strengthens the test." Both are recorded because a critic pass that finds only faults is not being read, it is being deferred to.

Note (rr) fires for the thirteenth consecutive session: $0.067524 bought the withdrawal of a sentence the design had written about itself one hour earlier — "a one-cell question and one cell answers it" — before any number existed to protect.