Repository path: wiki/findings/results/RS-20260730c-revision-close.md · rendered 2026-09-09

Result — the control was rebuilt blind and the gate failed anyway, for a reason that retires the diagnosis: this instrument does not distinguish a word substitution from a comma

The wire, in one sentence. A frozen 221-item reader batch was re-sent one day later with five of its eight surface controls replaced by controls built blind by a model that had seen no score, three of the old ones kept byte-identical as a negative band, and 216 of the 221 items unchanged at unchanged slots; and a Polish passage was translated in session, draft and revision frozen separately, to supply the first prospective instance of the drift statistic the same arm owed.

Discipline and the freeze chain. Polish source + manifest → T-niewola-tatarska-R06-v1 and its draft log at 14c8989 → T-niewola-tatarska-R04-v1 and its pass-separated revision log at 73ba25c → contamination figures appended at 1bf2831 → design + item batch frozen at 913f5f7 → independent pre-run critic: NEEDS-REDESIGN, 2 BLOCKING / 4 MANDATORY / 1 ADVISORY, all seven accepted → redesign, blind rebuild, amendments at 65aeba4 → six reader calls, all accepted first time.

Standing. provisional: true, internal-judgment-only. The readers are panel models and the panel is NOT CALIBRATED. Nothing here is a quality judgment; the axes are named for change, not repair; the lead never judges its own translation (charter §5).

1. The headline, and it is a double negative that settles something

ARM-revision step 2(a) existed to make P4 reportable: S057 computed mean E − mean M = 22.48 over 205 mechanical edits and withheld it, because gate F3's positive control failed on its two E-side criteria. RS-20260729e §2 diagnosed the failure as "the surface controls were built too weak" — method note (bdu).

Three things happened, in this order, and each one is worse for the diagnosis than the last.

1. The rebuild was done blind and F3 still fails, on the same two criteria.
2. The new gate F7 shows why, and it is not weakness. A pure comma-swap and a pure word-order rearrangement score as high on the E axis as a genuine word-for-word substitution does.
3. And the attribution gate F5 fails, so even a pass could not have been credited to the rebuild.

P4 is therefore withheld for a second session, and it is now withheld permanently in its S057 form — per registered amendment A6. And the number itself makes the point: the same 205 edits that gave 22.48 yesterday give 14.31 today. Had F3 passed at S057, the project would have published "E exceeds M by ≥ 20, confirmed at 22.48"; the identical measurement one day later fails that same registered prediction.

2. F3 and F7: the gate fails, and the reason retires note (bdu)'s diagnosis

Consensus over three readers. Band L = the five blind-built substitutions; band N = three of S057's own controls, kept byte-identical (two punctuation-only, one pure word-order).

F7 is the finding. The eight control items, with their scores:

The single highest E score in the whole control set belongs to band N — a rearrangement of the identical eight words, changing not one lexeme. It outscores every blind-built lexical substitution, including two that swap a word for a different word. The E axis is not weakly sensitive to kind; it is not sensitive to kind at all at this magnitude. What it appears to track is how much of the sentence's shape moved, which a word-order change moves maximally and a one-word synonym barely moves.

So note (bdu)'s diagnosis of S057 — "a control weaker than the material cannot calibrate it" — is narrowed to a case where it does not apply. The controls were not too weak in kind; and building them blind, at the corpus's own span statistics, to the corpus's own dominant operation, raised E(band L) by only 5.46 points and left the gate failing. The gate that S057 read as a defect in its controls is better read as a limit of the axis. Note (bdu) is not withdrawn — it is a true general statement — but it is recorded as not the explanation here, and note (bdu)'s last fired should not be read as a confirmed diagnosis on this instrument.

And the diagnosis it replaced was measured before the rebuild, so it too can be reported as a fact rather than as a story. verify.py asserts both figures: 184 of the 205 real edits (89.8%) introduce a word type absent from the draft span, and of S057's eight surface controls exactly four do — two contractions and two misspellings. Not one of S057's eight surface controls substituted a different English word for a word. True, checkable, and — as F7 now shows — not the reason the gate failed.

3. F5, the attribution gate: FAILS in four of five genuine cells

216 items byte-identical at byte-identical slots, same seats, same parameters, one day apart.

One of the six cells is not a cross-day comparison at all, and the verifier found that, not the analysis. At S057 the P5-M cell fell through to the declared reserve: deepseek/deepseek-v4-pro returned finish_reason: length with an empty body at StreamLake, and the answer of record came from google/gemini-3.6-flash (P2). So "P5 at S057" on the M axis is a different model. That cell is excluded from F5's verdict and reported as a between-model datum. This is method note (bdt) — a verifier that reads <tag>.raw reads the rejected body wherever a declared reserve fired — firing on real data for the first time, and it changed a number.

F5 fails: 1 of 5 genuine cells passes. And the movement is not a common shift: on M two readers went up (P3 +5.83) while on E two went down (P3 −5.14, P5 −7.15). Per-reader, per-axis, in opposite directions.

The attribution comparison (amendment A1) fails independently of the verdict. E(band L) moved +5.46 (12.87 → 18.33) against an E-axis cross-day drift ceiling of 8.83. Registered consequence: unattributable. As frozen, F5's tolerance was ≤10 points — larger than the 7.13-point gap it was gating — and the critic's first BLOCKING finding is the reason this comparison exists at all. Without it the session would have reported "the rebuild moved E by 5.5 points" as progress.

What F5's failure is confounded with, and the design claims neither over the other (A5): cross-day drift, or within-batch context effects from the five substituted items. The five are 2.3% of the batch, at slots 56, 96, 108, 158 and 174; P3-M's mean rose 40% and P5-E's fell 26%. A context effect of that size from 2.3% of the items would be a strong anchoring mechanism. It is not ruled out and is not claimed to be.

A second confound inside the "genuine" cells: P5-E kept its slug and changed provider, GMICloud → SiliconFlow. The S022 routing caution says a slug is not a served model. So of six cells, one is a model change, one is a provider change, and four are as clean as this project can make them — and all four of those fail.

4. Two gates that reproduced, and one whose verdict flipped for a reason that is not drift

• F1 — between-reader reliability: PASSES, and falls. Over the 216 identical items: α_M 0.8208 → 0.7209, α_E 0.8527 → 0.7902. Both clear 0.60, so F1 passes; both dropped, by 0.100 and 0.063, on byte-identical material. RS-20260729e §6 licensed "α = 0.82 / 0.86 on two graded axes over 221 items" — that sentence is now one draw of a quantity whose second draw is 0.72 / 0.79. The graded-axis finding of S054/S057 is not overturned (0.72 and 0.79 are still far above the categorical schemes' 0.51 and 0.066); its precision is.
• F6 — has the material moved? NO, and this is the reassuring row. The corpus's own median real-edit E: 27.33 → 24.00, a shift of −3.33, inside the registered ±10. The 205 real edits are the most stable thing in the run.
• F2 — the verdict flips, and the flip is the model swap, not the day. At S057 the 10%-of-real-edits-above-50-on-M clause fired (0.020 / 0.054 / 0.000); today it does not (0.000 / 0.063 / 0.117). But the third figure is P2's at S057 and P5's today. P1 went 0.020 → 0.000 and P3 0.054 → 0.063; the clause is carried across the threshold by the seat that changed model. Reported this way rather than as a reproduction datum, because it is not one.

5. Item (b), the drift question: a null, and the null has a mechanism

Registered corpus n = 6, fixed by rule before any threshold (§3 of the design), with the six exclusions named. Comparator window: the whole Gutenberg text, boilerplate stripped, identical for both arms of a pair — so absolute runs are upper bounds and only the within-pair difference is used.

Q4 predicted a null and gets one: sign test on the single non-tied pair, p = 1.00. The registered power statement stands as written — at n = 6 only 6-of-6 would have reached p ≤ 0.05 — so this is a null with the count printed, not evidence of absence.

But the tie rate is the finding, and it is exceptionless. In the five tied pairs the longest run is the identical string before and after revision, not merely the same length. In the sixth it is the same string minus its final token (…out of his pocket raising his → …out of his pocket raising). In 6 of 6 pairs, across five source languages, the revision pass did not rewrite the span carrying the longest run shared with an independent published translation. verify.py asserts this as a claim and mutation-tests it.

That is why the statistic cannot move, and it is a better answer than "no drift". If the spans where a translator's English most coincides with an independent translator's are also the spans the reviser leaves alone, then a draft-versus-revision drift measurement is close to degenerate by construction. Two readings, and this run does not separate them: those spans are forced (little room to move, so nothing to revise), or they are fluent (nothing catches the reviser's eye). The first is ARM-forced-defence's hypothesis and this is independent evidence consistent with it.

And the S050 anecdote this item was absorbed from is reconciled, not contradicted. S050 reported bargamot's run moving 7 → 8 and called it "an anecdote, not a measurement". That was Unit A only; S050 never measured Unit B's draft. Over the whole passage the draft's run is 10 — in Unit B — and the revision shortened it to 9. Within one passage, the same revision moved one unit one token toward the comparator and another unit one token away, and the anecdote recorded only the first. Both figures are correct; the denominators differ, which is precisely what amendment A4 withdrew the original P5 for lacking. (Checked against a narrow story-region window as well: 10 → 9 there too, so the wide window inflated nothing on this pair.)

6. Item (c): the axes cannot reach a goodness sense, and Q5 predicted that

Q5 registered the impossibility in advance and it holds. Both axes measure change; every sense in wiki/goodness-senses.md is evaluative. Projecting a change measure onto an evaluative sense requires a quality judgment, and Tier D has not passed. No per-sense number is reported, and producing one would have refuted Q5.

What is available instead is descriptive, and it is offered as description:

These are not reportable as estimates, for the reason §3 gives: the instrument that produced them moved by up to 8.83 points per item across one day, and the between-language spread here is 9.43 points. The spread is the size of the noise. Stated so that a later session does not quote the ordering.

IF Tier D ever passes, the mapping the axes would bear on is: M → accuracy (and only against a source the readers were not shown, so not even then without redesign); E → some combination of naturalness and style-correspondence, which RS-20260729-drift-window-verify and RS-20260729b-graded-drift both suggest are not one axis. Recorded as a pointer, not a result.

7. The translation limb

T-niewola-tatarska-R04-v1 — Sienkiewicz, «Niewola tatarska» I (opening), 985 Polish words into 1,329 English, the project's first Polish source and its first macaronic one; a 30-point draft log frozen at 14c8989 and a 21-item pass-separated revision log frozen at 73ba25c before any measurement of it existed. It is the fourth pair in the corpus with a pass-separated revision log, and the only one written to be a prospective instance of item (b).

Contamination suspected, measured, and it is a finding: two maximal shared runs with Curtin 1898, of 14 and 13 tokens, against a comparator the translator never opened. The 14-token run is function-word dominated (three content words). The 13-token run is not — "Jesus Christ, who lookest into my heart, seest that I would have done" carries two archaic second-person inflections. The artifact records a third mechanism as a conjecture: a run can be produced by two translators independently adopting the same period-register convention, which is neither memorisation nor forcing by the source, and which this session does not test.

The standing selection gate was deliberately not run as a gate, with a reason rather than an excuse — new method note (bel): a selection gate may not be run on the outcome variable of the study it gates. Selecting this locus on a low run would have conditioned item (b)'s sample on item (b)'s dependent variable.

8. What ARM-revision closes as, and why it is retired

retired, at 2 of 2, inside its declared budget, on the ending the arm wrote for itself at birth: "if the reader instrument cannot separate a meaning change from a surface change (registered gate F3), the primary is unanswerable with the instruments this project has, and the arm closes saying so."

The letter needs one correction and the substance does not. The axes do separate meaning from surface — F3's criterion 1 passes at 79.33. What they do not do is discriminate among kinds of surface change (F7 at 1.11), and the instrument does not hold still across sessions (F5, 1 of 5). Either alone makes item (a) unanswerable; together they make it unanswerable for a reason the arm could not have anticipated, which is why the word is retired and not resolved.

What the arm delivered and what stands. Step 1's primary — 42 of 47 mechanical differences accounted for by the translator's log, two of four logs accounting for all of theirs — needs no reader and is untouched by everything above. RS-20260729e §3 stands. What does not stand is RS-20260729e §6 item 1's α figures as estimates, and P4 in any form.

Calling this resolved because the session learned a lot would have been the easier and the wrong choice — the same call S061 made one session earlier, and the second time it is easier to make because the first is on the record.

9. Spend

35.1% is just outside the 15–34% band every output-dominated run in this ledger has landed in, and the reason is the amendment: the builder call was not in the frozen estimate. Excluding it, the run is at 31.2% and inside the band. Recorded rather than smoothed: an amendment that adds a call invalidates the pre-flight fraction, and the honest fix is to state both numbers.

All six reader calls accepted first time. No reserve fired. Note (b) did not fire — the first reader pass on this batch where it did not, and S057's firing is what §3 had to correct for. Reader key-usage delta 0.232554393 against a per-request sum of 0.2400870038: a 0.0075 shortfall, note (bco)'s settling lag, asserted as a bound.

Free and never ledgered: the Polish translation and both its logs, all seven contamination measurements, the item rebuild, item (b) entire, every analysis and the verifier.

10. Verification

analysis/verify.py, importing nothing from analyse.py: 167 checks, 0 failures. It re-reads the stored .raw HTTP bytes, resolves each cell to the body that was actually accepted (which is how the S057 model swap was found), re-derives all 221 scores per cell with a differently written scanner, recomputes Krippendorff's α through the sum-of-squares identity rather than the double loop and Pearson r through the raw-moment form, re-asserts the 216-item byte-identity and the span statistics, re-derives the 89.8% / 4-of-8 diagnosis, and asserts each of the four substantive claims about item (b) rather than describing it. Three mutation tests: a single corrupted score must move F5's mean |Δ| and F1's α, and a corrupted run string must break the identical-span claim. All three break as required.