Repository path: workshop/experiments/E-20260813b-affect-yardstick/design.md · rendered 2026-09-09

E-20260813b — is "closest to what the original does" a judgment about the original, or a

preference for the prose that reads like the description?

ARM-affect-unprompted step 2. The arm's step 2 is written in the arm page as "write the verdict into wiki/goodness-senses.md §affect", with the instruction "scope it from the result, not from here." Scoped from the result, it cannot be done as a writing step, and this design says exactly why.

Everything here is internal-judgment-only and provisional. affect is untested, Tier D is NOT PASSED (config/models.md), and no jury verdict in this project carries evidential weight.

1. Why the writing step is not a writing step

RS-20260812f §5 reported that the two halves of affect — the experience the English produces in its reader (H1) and how comparable that experience is to the one the source produces in its own reader (H2) — separate without anyone pointing the translator at either, and that the separation runs the opposite way to the standard story: the foreignizing arm won H1, the plain arm won H2 at 14 of 14 segments, and every one of 17 discordant cells moved toward the plain arm.

Its own limit 5 says the sharpest thing that can be said against that:

H2 supplies a yardstick and H1 does not, so H2's seats are answering a question with a document in it. A preference for the plainer arm may be a preference for the arm that matches a plainly-written English description.

If that reading is right, the run did not measure a second half of affect at all. It measured whether a passage of English resembles a passage of English — and then the separation between the halves, which is what the ratified rule's reversion condition demands, is an artifact of one question having a document attached and the other not. The discharge and the reversal both hang on it, so writing either into wiki/goodness-senses.md before testing it would put a figure into the senses page that a single obvious control could destroy.

Subject rule (wiki/tracks.md, continue-prompt.md §4.5), stated in one sentence. What this unit teaches about translating literature: whether a translator can be told which of two renderings comes closer to what the original does to its own reader — the oldest evaluative claim in translation and the one every "equivalent effect" argument rests on — or whether that judgment collapses, whenever the judge cannot read the source, into a comparison between the translation and whatever prose the description happens to be written in. The exception the subject rule names also applies on its own terms: a named deliverable — wiki/goodness-senses.md §affect, the arm's declared completion criterion — is blocked by the limit, and the arm says so.

2. Materials

Ion Luca Caragiale, «Căldură mare» (1899), the sketch whole: a man calls at a house in Strada Pacienței on a 33° afternoon, spends four pages failing to leave a message with a servant who answers every question exactly and helpfully, discovers he wants Strada Sapienței, and then asks four more people for Strada Pacienței, which is where he is standing. 120 paragraphs, 916 Romanian words, public domain (Caragiale 1852–1912), read whole. Copy-text: Romanian Wikisource, fetched 2026-08-13, workshop/translations/caldura-mare/source.txt.

The project's first Romanian source.

Two arms, both lead, both $0, both frozen at 783c87b before this design existed:

R07 is not built and is not in this run. RS-20260812f §3's gate found, on two independent seats, that the fluency programme taken whole is the sense's first half under another name, and excluded it from the primary. Building it again in order to exclude it again would be paying for a settled result.

Contamination: none, measured — tools/dependence_check.py, R06 against the whole of Lucy Byng's Caragiale (Roumanian Stories, 1921, Project Gutenberg #38991, 8,748 words): longest common run 5 tokens, 0 shared 7-grams, verdict clean. No English rendering of this sketch exists to compare against; the cross-text figure is reported for what it is on both translation pages.

Segments: 15, cut at paragraph boundaries chosen on the source's structure by code.py before any prompt existed, identical for the source and both arms, with reconstruction asserted. Segment lengths run 31–193 English words.

3. Conditions

All three judging conditions put the same two arms to the same three seats as a forced binary choice, in separate stateless calls, with per-cell label assignment from a fixed seed.

The three H2 conditions differ in exactly one thing: the prose style of the document. The label order for a given (segment, seat) is drawn from the same seed key in all three, so the conditions are paired cell by cell and nothing but the document moves.

The yardsticks

• YP (plain). A source-side seat that judges nothing in this run reads the Romanian segment and writes two fields kept strictly apart: EFFECT — what the passage does to a reader of the Romanian, with every reference to a property of the language forbidden — and CAUSE, where linguistic properties may be named freely. Only EFFECT is ever shown to a judge (RS-20260812f's pre-run critic, BLOCKING 1: a yardstick that names the mechanism turns H2 into feature-matching).
• YM (marked / estranged). The same seat, in a fresh stateless call, is shown only the EFFECT text — not the Romanian, not the arms, not CAUSE — and rewrites it into markedly non-standard English: dislocated and front-loaded syntax, archaism, unidiomatic literalness, fragments left unrepaired. No claim added, removed or altered.
• YO (ornate). The same seat, the same input, rewritten into elaborate high-literary English — long balanced periods, latinate diction, rhetorical figure — that stays completely fluent and idiomatic. No claim added, removed or altered. This document is marked relative to plain English but not marked in R08's direction, and it is in the design because the pre-run critic required it (§10).

Holding the writer constant and varying only the style instruction is the point. P2 never judges in this run, so no seat is judging prose it wrote (charter §5).

4. Predictions, registered before any call

• P1 — replication, with direction. Under H1 the segment majority prefers R08 more often than R06; under H2P it prefers R06 more often than R08; and the segments where the H1 and H2P majorities disagree are one-sided toward R06 under H2P. Primary statistic: exact two-sided binomial at p = 0.5 over discordant segments — the same statistic RS-20260812f §5 used, so the two runs are directly comparable.
• P2 — the confound, as a tracking prediction. The confound limit 5 alleges is that an H2 judge picks the arm whose English resembles the document's English. That hypothesis makes a sharp, symmetric prediction: the H2 choice moves with the style match, whatever the document's style. It is tested in two parts, and both are registered as predictions that can fail.
• P2a — invariance. The H2 segment majority is the same arm under all three documents at ≥ 11 of 15 segments.
• P2b — non-tracking. Take every (segment, document-pair) case in which the GS style match differs between the two documents. In those cases the H2 majority does not differ with it: the proportion that co-move is < 0.5. The confound predicts ≈ 1.0. Exact two-sided binomial over the co-moving count.
• P3 — the manipulation check. GS must actually move the style match: the GS two-seat-agreed match differs between at least one document pair at ≥ 8 of 15 segments, and the plain document is matched to R06 at ≥ 11 of 15.

What each outcome licenses, written now so it cannot be chosen later:

What a P2b failure would and would not establish. It would establish that H2 is sensitive to the document's prose style. It would not establish that the H2P result of RS-20260812f was style-driven rather than comparability-driven, because under a plain document the style match and the comparability answer point the same way and this design cannot separate them. That asymmetry is stated here, before the numbers, and travels with any citation.

5. Gates, run before the primary is read

• GA — arm parity. One seat per segment: do the two arms state the same facts? Wording, register, vocabulary, grammar and word order are explicitly not differences of fact. Note (bmh) applies: a flag is read before it is counted.
• GY — yardstick content parity. One seat per segment per rewrite (YP vs YM, YP vs YO), shown unlabelled: do these two say the same things about the passage, ignoring style entirely?
• GS — the style manipulation check. Two seats × 3 styles × 15 segments, shown one yardstick and both arms: which of the two English versions is written in a style more like this document's? The prompt mentions neither the source, nor effect, nor comparability.

6. Failure criteria, registered

• F1 — GA returns a genuine fact difference at more than 3 of 15 segments → the arms are not parallel and the primary is void.
• F2 — GY returns a content difference for a rewrite at a segment → that (segment, document) drops from the P2 pool. More than 5 of 15 dropping for a document voids that document's arm of P2, and if both rewrites drop that far P2 is void and the run reports the manipulation as unbuildable.
• F3 — P3 fails → P2 is void and is reported as void, not as a null. A style manipulation that did not happen cannot test a style confound. Registered because it is the way this run could most easily fool itself.
• F4 — a yardstick body that is empty, that leaks the CAUSE field, or that contains a Romanian-diacritic character or a source word → that segment's yardstick is void and the segment drops from every H2 condition that would have shown it.
• F5 — more than 10% of the bodies at any judging stage come back empty or finish_reason: length after the note (bmb) remedy → that stage is void.
• F6 — a seat refuses or returns an unparseable choice at a cell → the cell is excluded and counted; a stage losing more than 10 of its 45 cells is void.

7. Seats, caps, and the money

Roles from config/models.md; slugs are logged as provenance by the runner.

Note (bmb) is applied in the FIRST dispatch, not after it fires. Both reasoning-capable seats have their effort pinned and their caps sized: P5 at 4,000 on the judging stages and 2,000 on GS; P3 at 900, which RS-20260812f measured as sufficient on this slug. P4 moonshotai/kimi-k3 is not used: the 4,000-token cap the note prescribes prices its calls out of any ceiling this run could declare, which NEXT.md carries as a standing fact.

360 calls. Pre-flight worst case is computed by run.py --dry-run from the exact prompts and the caps, per note (abc), and recorded in §7.1 below before dispatch. P5's billed rate is priced at the worst plausible provider, not the list rate (config/models.md, the 2026-07-25 routing caution).

7.1 Pre-flight, from --dry-run before dispatch

Worst case built from the caps the requests actually permit, per note (abc), not from an assumed output length.

Ceiling history, recorded rather than adjusted afterwards. $2.00 while the design was being written; $2.20 when the first pre-flight came back at $2.041076 on 255 calls; $3.00 when the pre-run critic's BLOCKING finding was accepted and the third document added, taking the pre-flight to $2.807070 on 360 calls. Today's ledger stands at $1.319019290 of $5.00, so a $3.00 ceiling leaves $0.68 of the day's cap for any later session — the cost of taking the critic's finding seriously, and it is declared here rather than discovered later. The alternative considered and rejected was to cut P5's cap below the 4,000 that note (bmb)'s remedy prescribes.

Lead translation of both arms is $0 and is not ledgered.

8. Verification

verify.py recomputes every reported number from the stored bodies, imports nothing from analyse.py, and asserts the design's own invariants, among them:

1. No H1 prompt contains a yardstick.
2. No H2P or H2M prompt contains the cause field of any yardstick.
3. No judge prompt contains a Cyrillic or Romanian-diacritic character, and none contains the arm labels R06 or R08.
4. For every (segment, seat), the H2P, H2M and H2O prompts differ only in the yardstick block — the arms, their order and the question wording are byte-identical.
5. Each arm's text sits behind the label the seed says it should.
6. GS prompts mention neither the source nor effect nor comparability.
7. Every reported count is recomputed from runs/*.json, and the binomial probabilities are recomputed by exhaustive enumeration rather than by calling a library.

Mutation tests: the verifier is run against deliberately corrupted copies of the record and must catch each corruption.

tools/metric_a.py's clopper_pearson was repaired at S172 (RS-20260813a §5, note (bmy)); this run's proportions are at n = 15 and n = 45, where the repaired path is the one exercised.

9. What this run cannot establish

Written before the numbers exist.

1. Model seats, not readers. Tier D is NOT PASSED. A finding here is a finding about how three seats behave, not about human readers of English or of Romanian.
2. One more work, one more pair, one hand. R06 and R08 are the same translator in one session in that order, so R08 is downstream of R06.
3. A P2 that holds defeats the style reading of limit 5 and nothing else. A yardstick is still a document, and having a document at all — as against H1's having none — remains a difference between the two halves that this design does not remove. It cannot: a comparability question with no description of the original in it is not a comparability question.
4. GS measures what seats call stylistic similarity, which is itself a model judgment and is not anchored.
5. 15 segments. A one-sided 11-of-15 is P = 0.118 by the two-sided sign test; the run is powered to detect a strong effect and not a moderate one, and the registered bars are set where they are for that reason.
6. The YM document is deliberately the strongest form of the manipulation, and the pre-run critic is right that its instruction names features R08's rule set also has. That is why P2 is a tracking prediction across three documents rather than a survival test under one, and why YO is in the run. A citation of P2 carries §4's asymmetry paragraph.

10. Amendment, before dispatch: the pre-run critic's BLOCKING finding, accepted

The independent pre-run critic (qwen/qwen3.7-max, non-panel, runs/critic.txt) returned NEEDS-REDESIGN with one BLOCKING finding, and it is right.

Its finding: YM's instruction — dislocated word order, archaism, unidiomatic literalness, unrepaired fragments, "foreign-sounding, deliberately not fluent" — is a description of what R08's rule set produces. So H2M would cue R08 by stylistic kinship, P2 as originally written would be near-impossible to pass, and the design's result→option map turned that engineered failure into the conclusion "limit 5 is vindicated, withdraw §5's direction". In its words: it "proves only the trivial fact that if you give judges a yardstick that sounds like R08, they pick R08."

Accepted in full, and the design was amended in three places before any study call was dispatched:

1. A third document, YO — the critic's requested control: marked relative to plain English, fluent, and not marked in R08's direction. 105 calls and about $0.9 of worst case.
2. P2 rewritten as a tracking prediction (P2a invariance, P2b non-tracking) rather than as R06-survival under one document. The confound hypothesis predicts co-movement between the GS style match and the H2 choice whatever the document's style; that prediction is symmetric, is not satisfied by construction, and can fail in either direction.
3. The result→option map no longer converts a P2 failure into a withdrawal of §5. §4 now states, before the numbers, exactly what a failure would and would not establish.

The critic's own remedy — replace the estranged style with an ornate one — was not taken, and the reason is on the record: an ornate document is still fluent English, so GS would plausibly match it to R06 as well, the manipulation would not move, F3 would fire, and the run would have no power against the confound at all. Keeping the strong manipulation and adding the neutral one is what makes both the sensitivity and the tracking question answerable.

The amended design was put back to the same critic for a second pass before dispatch (runs/critic2.txt): PROCEED-WITH-AMENDMENT, one MINOR finding — that F4 and verification invariant 4 still named only H2P and H2M and had not been extended to H2O. Both were already repaired in the working copy when the second pass was dispatched, and the committed text carries the repair; the finding is recorded as discharged on arrival rather than quietly dropped.

Critic cost: $0.050053 + $0.057559 = $0.107612, both inside the declared ceiling.