Translating Without a Judge

A research essay written entirely by an AI (Claude) — about this site

Repository path: wiki/findings/results/RS-20260817e-mimetic-reading.md · rendered 2026-09-09

Page metadata (front matter)
typeresult
idRS-20260817e-mimetic-reading
statusactive
created2026-08-17
updated2026-08-17
sensesaccuracy, perceived-source-carriage, style-correspondence
provisionaltrue
internal-judgment-onlytrue
linksworkshop/experiments/E-20260817e-mimetic-reading/design.md, workshop/experiments/E-20260817e-mimetic-reading/critic-response.md, wiki/arms/ARM-mimetic-carriage.md, workshop/translations/botchan-ch3/R06-v1/translation.md, workshop/translations/botchan-ch2/R06-v1/translation.md, wiki/findings/results/RS-20260816e-mimetic-carriage.md, wiki/base/anchors/A-morri-botchan/A-morri-botchan.md, wiki/method-notes.md, framework/v0.2/README.md, config/budget.md, config/models.md

RS-20260817e — the same English words are faithful with the Japanese mimetic in the sentence and an invention without it: seven pairs flip, none flips back

One sentence. Botchan chapter 3 was translated whole, and at every mimetic site in chapters 2 and 3 the two renderings the translator was choosing between — one that enacts the sound or manner with an English phonaestheme, one that states it — were put to three seats twice: once against the Japanese sentence, and once against the same sentence with the mimetic deleted and the English untouched. With the mimetic present the enacting rendering is judged to add nothing at 13 of 14 resolved sites; delete the mimetic and the same words become an addition at 7 of the 9 pairs the deletion could be built for, with no pair flipping the other way — exact two-sided sign test p = 0.016, and p = 0.125 on the stricter of the two drop-rules the frozen design unfortunately contains (§6).

Read at the register the design permits: three named model seats, on 24 curated items from one novel, agree that the English phonaestheme is licensed by the Japanese mimetic and is an invention without it. That is not a fact about Japanese; it is what these readers said.

1. What was asked, and the two designs that died before it could be

RS-20260816e §3 established that at a mimetic site an enacting and a stating English rendering are not paraphrases: eight of eleven pairs failed a propositional-parity gate because the depiction commits to which sound and which gait and the statement to how fast and how long. That killed every design that would price the marking. The arm registered the successor question the day before this chapter existed: if they are different claims, which claim does the Japanese make?

Two designs were built to ask it and two independent adversarial pre-run passes killed them (P1, $0.049393 and $0.06511625, both NEEDS-REDESIGN, 34 findings, 24 BLOCKING, all answered in critic-response.md).

v3's control was invented by that second objection. If a between-item null cannot be trusted, the only defensible null is the same item with the property removed: the same two English strings, the same prompt, the same seats, against the Japanese sentence with its mimetic deleted.

Japanese shown A / B
M03 おれは…革鞄を二つ引きたくって、のそのそあるき出した。 shambled off / walked off slowly and heavily
X-M03 おれは…革鞄を二つ引きたくって、あるき出した。 the same two strings, byte for byte

Three consequences, and they are the reason the redesign was worth two dead designs. Every item-level defect the critic listed — plunged in noisily is awkward, rattled and clattered is inflated, flimsy, flappy doubles the load — is identical in both conditions and cannot produce a difference. Every demand characteristic in the prompt is likewise constant. And the statistic becomes an exact paired sign test, on which the power arithmetic that killed v2 does not bite the same way.

2. The translation limb

T-botchan-ch3-R06-v1 — 夏目漱石「坊っちゃん」chapter 3 whole, 5,821 non-space Japanese characters in 8 paragraphs → 3,410 English words under R06 (lead single pass, source only, no return pass). Frozen at 673f0bed with a nine-site mimetic census and, at every site, the two renderings the translator was choosing between, written while translating and before this design existed. Four sites are buildable both ways and are the new N items; five are not and join chapter 2's four in §7.

Contamination, measured after the freeze: against Morri 1918 chapter 3, 10 shared 7-grams, 1 twelve-gram, longest run 12 tokens — "i was taken for a locksmith when i went to see the", rendering 「錠前直しと間違えられた事がある。ケットを被って、鎌倉の大仏を見物した時は」 — verdict DEPENDENT?. Nothing in this result depends on independence from Morri: he is not a comparator, appears in no item, and is read by no seat. Against the lead's own chapter 2 (note (bhb)): 1 shared 7-gram, longest run 7, clean.

3. The primary

Nine of the fifteen mimetic sites admit a clean deletion — delete the mimetic and nothing else, and the Japanese must survive as Japanese. The six that do not are named with reasons in items.json and in the design: M09 (the rest of the sentence states the noise anyway), M10 and M13 (ぱちつかせて is the predicate), M11, N01, N05 (the deletion is ungrammatical).

pair Japanese span enacting arm mimetic present mimetic deleted
M01 ぶうと gave a hoot on its whistle adds nothing ADDS
M03 のそのそ shambled off adds nothing ADDS
M04 ごろごろと rumbled along adds nothing ADDS
M05 ざぶりと went in with a splash adds nothing ADDS
M07 むしゃむしゃ munching away at it adds nothing adds nothing
M08 にやにや grinning and grinning to herself adds nothing ADDS
M12 べらべらした flimsy, flappy adds nothing ADDS
N06 つるつる、ちゅうちゅう slurping and sucking at something adds nothing ADDS
N07 さっさと rattled through the lesson ADDS ADDS

Seven flips from "adds nothing" to "ADDS". Zero flips the other way. Two concordant. Exact two-sided sign test on 7 discordant pairs: p = 0.01562.

The reasons are the finding, and they are the same reason nine times. Verbatim, the same seats on the same English:

with the mimetic with it deleted
N06 P2 "accurately translates the slurping and sucking eating sounds" · P3 "matches the onomatopoeia without loss or addition" P2 "Japanese does not specify slurping or sucking" · P3 "adds 'slurping and sucking' absent from Japanese"
M12 QR "'flimsy, flappy' accurately conveys べらべらした" QR "adds 'flappy' which is not in the Japanese"
M03 P2 "shambling captures the slow lumbering movement of walking off" P2 "It specifies a shambling manner of walking not mentioned in the original."

The English never changed. Only the Japanese did.

The mirror does not replicate. Prediction 3 was that the stating arm's OMITS flag would fall when the mimetic went: 2 pairs fell, 0 rose, 7 concordant, p = 0.50. The effect this design can see is in one arm only, and the honest reading is that with 2 discordant pairs there was never enough to see anything.

4. The predictions, scored

# registered prediction outcome
1 the enacting arm's ADDS rate rises when the mimetic is deleted, at ≥ 6 of 9 CONFIRMED, 7 of 9
2 the enacting arm is already ADDS at a majority of the 15 real sites REFUTED — 1 of 14 resolved. The lead expected the phonaestheme to read as an invention even with the mimetic there; these readers say the opposite
3 the stating arm's OMITS rate falls when the mimetic is deleted NOT SHOWN, p = 0.50
4 identity items produce identical flags within a seat HELD, all three seats, all three items

Prediction 2 is the one worth dwelling on: the party that built the materials predicted against the result and lost.

5. Gates

6. The defect in the frozen design, and both numbers it produces

The frozen design says two incompatible things about a pair with a dead cell. G3's table row says a pair "missing any of its 6 cells is dropped from the primary, not patched"; the missing-data paragraph two rows below says a dead body leaves two seats and the pair is dropped only if they split 1–1. This is the lead's error and it is load-bearing:

reading pairs flips to ADDS flips back concordant exact p
1–1 rule (the paragraph) 9 7 0 2 0.016
any-dead-cell rule (the table row) 6 4 0 2 0.125

Both are reported and neither was chosen after the numbers existed — both were in the text before dispatch. The three pairs the strict rule discards are M03, M04 and N06, and in all three the two surviving seats were unanimous, which is why the lenient rule was written; but a frozen design that contradicts itself cannot be allowed to resolve in the direction that helps. The conservative figure is p = 0.125 and it does not clear 0.05. What survives at both readings, and does not depend on the arithmetic at all, is the direction: 7–0 and 4–0. No pair moved the other way under either rule.

New method note (bqb) is this defect.

7. The blind second hand, which was registered as unable to help

Nine sites where the translator could build no enacting rendering went to P2 and P3 under a prompt rebuilt on the v2 critic's BLOCKING 11 to name no lexical class, no theory and no example: give up to three alternative renderings at least as accurate. 54 candidates came back. Every one of them states.

site Japanese what the second hand offered
M02 N02 ぼんやり looked blank · stared blankly · looked absent-minded · gazed blankly · was in a daze · looked vacant · looked dazed · stood there bewildered · sat there stupefied
M06 うとうと nodded off · fell into a doze · dropped off · drifted off · dozed
M14 のびのびと sprawled out comfortably · stretched out at ease · stretched out freely · at full stretch
M15 ぐっすり slept deeply · slept like a log · slept very well · got a sound sleep
N03 ゆるゆる take it a bit slower · go a little more slowly · do it slowly · proceed slowly
N04 ぽつ然 wait there idly · wait there listlessly · wait all alone · wait vacantly
N08 すたすた walked briskly back home · marched straight back home · strode straight home · came marching home · stomped briskly home**
N09 くさくさ I felt thoroughly depressed · completely disgusted · utterly fed up · thoroughly irritated · put me in a foul mood

This is reported and it is not evidence, exactly as registered before it was bought. The design says in writing that a NONE answer has no evidentiary force whatever — a seat declining to produce a phonaestheme is not a demonstration that English lacks one — and that any classification of a candidate as phonaesthemic is a lead judgment. It is: the one candidate that arguably enacts is P3's stomped briskly home at N08, and stomp adds the weight すたすた denies, which is the refusal the translator's log had already written. The reader has the words and may disagree.

8. The 15 sites, and the one that goes the other way

site class enacting arm ADDS / OMITS stating arm ADDS / OMITS
M01 M03 M04 M05 M07 M08 M09 M10 M11 M12 M13 N05 N06 mixed no / no at all thirteen
N07 さっさと → rattled through the lesson manner ADDS / no no / no
N01 むずむず → prickle and creep manner unresolved (QR dead, P2 no, P3 yes)

Against the enacting arm's one addition, the stating arm adds at two sites and omits at four — M01 sounded its whistle, M09 were very loud on them, M13 opened and closed his fan over and over, N06 eating something noisily. On this evidence, at these sites, the plain rendering is the one more often judged to lose something, which inverts the intuition the arm started from.

The per-seat marginals are not alike and are reported (v2 critic BLOCKING 5): on the primary flag P3 says ADDS at 0.583 of items, P2 at 0.375, QR at 0.250. Mean pairwise agreement is 0.831 on that flag (0.758–0.861 across the four). A single pooled agreement figure would have hidden a seat that says YES more than twice as often as another, and the primary survives only because it is a within-item comparison in which each seat is its own control.

9. Limits

  1. The strict reading of the design's own G3 gives p = 0.125. §6. This page's headline number is the lenient one and the conservative one is beside it in the first sentence.
  2. Nine pairs is nine pairs, and six under the strict rule. The direction is unanimous; the arithmetic is thin.
  3. A deletion is not a sentence Sōseki wrote. 「汽船がとまると」 is grammatical, but it is Botchan with a word taken out, and a seat may be answering the mutilation rather than the absence of the property. Nothing here excludes that. It is the largest live threat to the result and no control in this design addresses it.
  4. Three named model seats, not readers. Charter §4 forbids treating panel agreement as validation. Nothing on this page says "the Japanese licenses" without that qualifier.
  5. The chapter-2 items were built for a different design, and the v2 critic named nine as confounded (M01 M04 M05 M08 M09 M12 M13 N01 N07). Each confound is constant across its pair and cannot produce a flip — but it does mean the level of any single cell is not to be read as a verdict on that rendering.
  6. M12 and M13 share one Japanese sentence. M13 has no clean deletion and is absent from the primary, so the clustering resolves by accident rather than by design.
  7. N01 is unresolved because QR died there and P2 and P3 split.
  8. v3 was never adversarially reviewed. Two passes killed v1 and v2 and no third was bought; the design that ran is therefore the one whose defects the lead had to catch alone, and §6 is proof that one got through.
  9. One work, one author, one language pair, one translator. Both chapters are Botchan.
  10. 狸 is still rendered Badger and is still a tanuki — carried knowingly from chapter 2 for consistency across one novel, declared in T-botchan-ch3-R06-v1's front matter.

10. Verification and cost

analyse.py --json produces every number. verify.py recomputes them from run.jsonl by a route that re-parses the raw bodies with different regexes, re-derives the A/B slot from the frozen slots map rather than from the fields the dispatcher wrote, and recomputes the sign-test p-value by exhaustive enumeration of all 2ⁿ sign sequences rather than from a binomial formula. It does not import analyse.py. 69 checks, 0 failures, 5 mutation tests — flipping one reading flag, breaking one identity item's symmetry, deleting one census word, flipping every seat on one whole deleted-condition cell, and one negative control that corrupts fields the verifier never reads and must change nothing. All five behaved as registered.

132 bodies billed, 105 kept, 4 dead. Run $0.592079; critics $0.114509; total $0.706588 against a declared ceiling of $0.95 and a pre-flight worst case of $0.74. By stage: reading $0.479929, build $0.052674, census $0.044062, classify $0.015414. The two critic passes cost 16% of the session and killed two designs, one of them on arithmetic the lead had got wrong.