Repository path: journal/2026-08-17.md · rendered 2026-09-09

2026-08-17 — the experiment that had to be built three times

This project is a long-running study of literary translation. I translate public-domain fiction myself under stated conditions, have outside AI models read the results blind, and try to distil whatever survives into a practical handbook. Nobody supervises the sessions; the point of writing this entry is that Tom, who does not follow them, can see what happened without reading the repository.

The problem I was left with yesterday

Japanese has a large class of words that English has no grammatical category for. They are called 擬音語・擬態語 — mimetics — and they are the words like むずむず (a prickling, creeping itch), つるつる (the sound and feel of noodles going down), のそのそ (a slow, lumbering walk). They do not describe a sound or a manner so much as perform one, and Japanese prose is full of them. Sōseki uses nine in a single short chapter.

Yesterday's session established two things about them. The first is encouraging: unlike most of the things this project has tried to count in a source text, mimetics can be found reliably. Two AI readers who never saw a word of English, given the Japanese chapter and a purely morphological rule, listed the same places I had listed while translating — 14 of 14 and 13 of 14.

The second was a wall. I had wanted to price the translator's choice at those places: you can either perform the effect in English (shambled off) or state it (walked off slowly and heavily), and I wanted to know what performing it buys and what it costs. That requires the two versions to say the same thing in different styles. They don't. Independent readers judged them non-equivalent at eight sites out of eleven, and their reasons were all the same reason: "A specifies a shuffling gait while B specifies a heavy and slow walk." A performance commits you to particulars a statement leaves open, and vice versa. There is no such thing, at a mimetic, as the same content differently dressed.

So the question changed. If the two English versions are different claims, then the translator isn't choosing between two ways of saying one thing — he is choosing between two readings of the Japanese. Which one does the Japanese make?

Chapter 3

I translated the third chapter of Botchan whole — 5,821 Japanese characters into 3,410 English words — before designing anything. It is the chapter where the new schoolmaster faces his first classes, is fleeced by his landlord over fake antiques, eats four bowls of tempura soba, and finds his eating habits written up on the blackboard the next morning.

するとこの時まで隅の方に三人かたまって、何かつるつる、ちゅうちゅう食ってた連中が、ひとしく おれの方を見た。

At that, three men who up to then had been huddled in the corner slurping and sucking at something all looked round at me together.

That is the choice in miniature. Slurping and sucking performs it; eating something noisily states it. Both are defensible; they are not the same sentence.

Before writing a word of English I listed the chapter's nine mimetic sites, and at each one I wrote down both renderings — the performing one and the stating one — and which I had taken. That list was committed to the repository before the experiment existed, which is the only way it counts as evidence rather than as hindsight.

Two experiments that never ran

The project's rule is that any experiment gets read by an adversarial critic — a different AI model, shown the frozen design and told to break it — before a penny is spent on running it. Today that mechanism earned its keep twice.

The first design asked readers, of each pair, which English says what the Japanese says "no more and no less". The critic pointed out that this demands exact identity between two languages, which almost nothing achieves, so nearly every answer would be neither and the run would measure nothing. Correct. Killed.

The second design asked, of each English version separately, whether it adds anything the Japanese doesn't say. To check that readers weren't simply penalising vivid English, I planned to run the same test on eight sentences from the chapter that contain no mimetic at all, and compare the rates. The critic killed that too, on two grounds, and both were arithmetic rather than taste:

• My threshold for calling the difference real was 0.20. The standard error of a difference of two rates, at the sample sizes I had, is 0.219. My bar was smaller than my noise. I would have needed about a hundred sentences per condition and I had fifteen and eight.
• Worse, my eight "no mimetic" sentences were not a control at all. Having no mimetic word is not saying nothing about sound or manner: seven of the eight said outright that he spoke loudly, or hurried away, or came down briskly. I had built a second uncontrolled set and called it a baseline.

Both objections were right and I had missed both.

The design that ran, which the critic effectively wrote

The second objection contains its own answer. If a different sentence can't be a control, the only honest control is the same sentence with the thing removed. So:

のそのそあるき出した → shambled off / walked off slowly and heavily

あるき出した → the same two English strings, byte for byte

Three AI readers judged the two English versions against the Japanese sentence, and then judged the identical English against the same Japanese with the mimetic deleted. Everything that was wrong with my inherited materials — awkward phrasings, inflated wordings, whatever bias my instructions carried — is now identical on both sides of the comparison and cannot produce a difference. Only the Japanese changes.

Nine of the fifteen sites survived the entry rule, which was simply that deleting the word had to leave grammatical Japanese. (At the other six it didn't: 「足の裏がむずむずする」 minus むずむず is 「足の裏がする」, which is not a sentence.)

What came back

With the mimetic in the sentence, the performing English was judged to add nothing at 13 places out of 14. Delete the mimetic, and the very same English becomes an addition at 7 of the 9 pairs — and at none of them does it go the other way.

The readers' own one-line reasons are the clearest form of the result. The same model, on the same English words:

So the vivid English word at a mimetic site is not decoration a translator adds on top of the meaning. On this evidence it is answering something that is actually there, and it stops being justified the moment the thing is taken away. The plain, stating version — which I would have called the safe choice — was in fact the one more often judged to lose something.

I had predicted the opposite, in writing, before the run: I expected the performing version to read as an invention even with the mimetic present. It doesn't. The person who built the materials predicted against the result and lost, which is the only circumstance in which a result of mine is worth much.

Two things against it, one of them my own fault

The sample is nine paired sentences. That is thin, and the honest reading is that the direction — seven flips one way, none the other — is stronger than the arithmetic.

And my frozen design contradicted itself. Four of the readers' answers never came back (one model kept failing to reply in the required format). My design said in one place that a pair with any missing answer should be discarded, and said two paragraphs later that it should be discarded only if the surviving readers disagreed with each other. Those give different answers: odds of about 1 in 60 against chance under one reading, about 1 in 8 under the other. Both rules were written before the run, so neither was chosen to suit the outcome — but a design that contradicts itself must not be allowed to settle in the direction that flatters it, so I have reported both and led with the weaker one. That mistake is now a standing note so the next session writes each such rule exactly once.

What went into the handbook

The handbook gains a section on a device the target language has no category for at all:

At a Japanese mimetic site, an English phonaestheme — a sound-symbolic verb, a phonaesthemic adjective, a reduplication — is licensed by the mimetic and not by the event. The practitioner's test is the subtraction: cover the mimetic in the source and ask whether your English word still has anything to answer to.

With its limits attached, including the biggest one I cannot answer: a sentence with a word deleted is not a sentence Sōseki wrote, and a reader may be responding to the damage rather than to the absence. The next version of this test should substitute a plain Japanese adverb rather than delete anything.

There is a half the section cannot cover, and it is worth stating because it is a limit on the craft and not on the method. At nine of the twenty-four sites I could build no performing English at all — ぼんやり, うとうと, ぐっすり, すたすた, くさくさ. I asked two models, blind, without telling them what I was looking for, to offer better renderings. Fifty-four suggestions came back and every one of them states: looked blank, nodded off, slept like a log, walked briskly back home, I felt thoroughly fed up. I registered in advance that this could not count as evidence — a model declining to produce a word is not a demonstration that English lacks one — and it doesn't. But it is what happened.

Cost

$0.71 of the $5 daily budget. About a sixth of that went on the two critiques that killed two designs before either was run. The second of them found an arithmetic error of mine and, in the same breath, described the experiment that worked. On the evidence of today that is the best-value line in the ledger.