Repository path: workshop/experiments/E-20260804-displaced-marking-fr/runs/critic-pass1-response.md · rendered 2026-09-09

critic-pass1-response.md

VERDICT: NEEDS-AMENDMENT

FINDING 1 [BLOCKING]: The eight relation statements were written by the same author who wrote the FORCED renderings, after the FORCED renderings existed, and several describe exactly what the FORCED rendering was constructed to add. WHY: The design's own table shows the fit: S2's relation ("a household whose failure to answer is an act of theirs, and at the same time nobody identifiable") is satisfied word-for-word by FORCED's "they did not answer me… whoever they were" and by nothing else; S1's "entertained and resisted in one breath" maps onto FORCED's inserted "and I would rather it were not"; S8's "the arrival of the will is itself the event" maps onto "The need to know took me." The grading question "does an English reader get this relation from it" then measures whether the grader can detect the clause the author embedded to match the relation the author wrote — a closed loop. P1 at ≥5/8 can be met with zero evidence that R1 recovers anything, because both sides of the comparison (the rendering and the yardstick) came from one motivated hand. The design acknowledges unblinded authorship of renderings (§7) but nowhere acknowledges that the grading standard itself is also unblinded and post-hoc relative to FORCED. FIX: Have the relation statements re-authored from the French span and gloss alone, by a party (or model seat) that has never seen any of the four renderings, and freeze them before dispatch; discard the current statements. If no independent author is available, at minimum commit to relations written strictly from source+gloss with the renderings' text provably excluded from the prompt, and record this in the design.

FINDING 2 [BLOCKING]: F1, the sole instrument-validity check, is constructed so that it cannot fail, because POSITIVE states the relation verbatim and DECOY is authored under the instruction "attempts no marking." WHY: P4 requires POSITIVE to be graded YES at 8/8 — but POSITIVE is an explicit metalinguistic gloss of the relation, so a seat would have to fail reading comprehension to grade it NO; this tests the seat, not the instrument's discrimination. Symmetrically, P3 requires DECOY YES at ≤1 site, but DECOY was written by the motivated author with the explicit brief of not conveying the relation, so its pass is manufactured, not measured. An instrument that graded everything YES would trip F1 on DECOY, but an instrument that grades YES exactly when the author wanted YES and NO exactly when the author wanted NO — the actual failure mode given Finding 1 — passes F1 cleanly. The control controls for random graders, not for the aligned-author confound the design itself flags as its weakest joint. FIX: Add at least two DECOYs built by a different rule: renderings written by someone other than the lead (or by a model under a neutral "translate this span well" brief) that incidentally carry a marking, interleaved as unlabelled controls; F1 should also fail if these are graded NO at high rate. Without a control the author did not tune, F1 must be described in the result as a seat-function check only, not an instrument check.

FINDING 3 [BLOCKING]: P5 is a registered prediction that is true by construction, and it inflates the apparent predictive success of the framework. WHY: S5's relation, as shown to graders, contains the clause "automatically rather than by any choice the writer made at this point: the language leaves him no other option." No rendering of the sentence can convey that metalinguistic fact except POSITIVE, which states it outright — FORCED's "She seemed to be alive" conveys femaleness but cannot convey automaticity. So FORCED at S5 is guaranteed NO, P5 is guaranteed to "hold," and the run will record a confirmed framework prediction that was never at risk. Meanwhile S5 still counts in P1's denominator, where its guaranteed failure is absorbed by the ≥5-of-8 margin the lead has already narrated ("P1 lands at exactly 5 of 8 and survives"). A prediction that cannot fail is not a prediction, and reporting it as one converts design into evidence. FIX: Either remove P5 from the scored predictions and treat S5 as a demonstration site outside P1's denominator (making P1 ≥4 of 7 with failure at ≤2 of 7), or rewrite S5's relation to what the marking actually conveys to a source-blind reader (the watch is a she), in which case P5 as registered is withdrawn rather than confirmed.

FINDING 4 [NON-BLOCKING]: The within-device site picks, though stated with reasons "before any rendering was written," are unverifiable as pre-hoc and one pick (S2, "the hardest") is doing argumentative work the run will later claim as conservatism. WHY: The census, the classification, the picks, and the design all originate in one session from one author; the assertion that picks preceded renderings rests on the author's say-so and commit order, not on any independent check. S2's framing as "the conservative pick against R1" pre-loads a narrative: if S2 passes, R1 survived the hard case; if it fails, the design has already written the excuse ("the interesting sentence is not about R1 but about English"). That sentence in §5 is a pre-registered deflection — it converts a P1-relevant failure into a claim about the English language, which the run has no instrument to support. FIX: Keep the picks (they are defensible on their stated reasons), but strike the §5 sentence about S2 from the interpretive apparatus, or re-register it as what it is: a post-result hypothesis requiring a new experiment, not an interpretation this run licenses.

END OF FINDINGS

Recovered verbatim from the first invocation's console output; the stored .raw body for this dispatch was overwritten by the second invocation. See runs/critic_pass1_kimi-k3.meta.json.