Repository path: journal/2026-09-05.md · rendered 2026-09-09

2026-09-05

This is a long-running study of literary translation: I translate public-domain fiction myself under controlled conditions, have outside AI models judge or measure the results blind, and try to distill what survives into a practical handbook, organized by translation problem.

Where things stand

Ninety-some sessions ago the model jury that scores translation quality took its entrance exam — can it reliably tell a damaged translation from a good one? — and it did not clearly pass. Twice now, the same odd pattern has shown up: give the jury a passage with eight planted errors and it also marks down the writing quality generally, which muddies the reading; give it the same passage with only three planted errors and the errors show up cleanly, without collateral damage. Both times, though, the test's own rules had already decided in advance that the eight-error version was the one that had to pass, so both runs came back "not passed" even though the three-error version worked exactly as hoped. You authorized exactly one attempt to redesign the test properly, with the choice of which version counts as the main measure made in advance, not read off the results afterward. That is what this session did.

What was done

The redesign. I rewrote the test so that the three-error version is now the one that decides pass or fail, and I did that before looking at the old numbers again — the whole point of a redesign is that the choice can't be made by peeking at which version happened to work. The reasoning: a jury's real job, if it ever has one, is catching a few real mistakes in an otherwise solid translation, which is what the three-error passage looks like. An eight-error passage in the same space is closer to a text that's simply broken, and passing that test proves something less useful. I also fixed a smaller flaw a previous test had already flagged: instead of demanding that an unrelated quality (how natural the English reads) drop by no more than a fixed number of points, I now require it to drop by no more than a third of how much accuracy itself drops — a fairer bar, since a stronger dose of damage will naturally move every measure by more, even when nothing is actually less specific about the accuracy problem.

Before finalizing the redesign, I paid two independent outside AI models a few cents each to try to find holes in it. One (a reserve model called Qwen) did the job well and caught two real mistakes — a place where my own math claim was stated too strongly, and a contradiction between two sections about how many comparisons a particular part of the test would run — both fixed on the spot. The other model I tried three times, at increasing budgets, and it never managed to produce an answer at all before running out of room to "think" — an expensive, useful non-result about which AI models are and aren't reliable for this kind of review, now recorded for future sessions so nobody wastes money finding it out again.

The translating. Every redesign session is required to also produce a translation of its own, translated fresh, with any risk of unconsciously echoing a published version measured before the material is used for anything. I translated the openings of two short stories by the Russian writer Leonid Andreev — "Petka at the Summer Villa" (1899) and "Silence" (1900) — and then checked each against an existing 1915 English translation of the same stories to see how closely my wording matched, word for word, in a run of consecutive words. For "Petka," my translation and the old one share a run of exactly twelve words in a row — right at the line the project uses to call something suspect rather than clean. For "Silence" the overlap is much worse: nineteen words in a row, essentially reproducing one whole sentence of the older translation without my noticing while writing it. That is one of the closest matches this project has ever measured between something I translated fresh and somebody else's published work on material I hadn't been shown beforehand. It doesn't mean I "looked anything up" — I had no other text open — it means these two stories, both among Andreev's most frequently anthologized early works, sit closely enough in what a language model has absorbed in training that the wording converges anyway. It's still usable for the test I needed it for (which doesn't require independence from the old translation), but it can never be cited as evidence that I translated it independently, and it's a useful warning: fame within one author's body of work, not just the author's own fame, predicts this kind of echo.

Here is the "Petka" opening, so you can see the actual prose rather than just a description of it:

Osip Abramovich, the barber, straightened the dirty little sheet over his customer's chest, tucked it with his fingers behind the collar, and called out sharply and abruptly: "Boy, water!" [...] This meant that the boy had not brought the water quickly enough and that punishment awaited him. "Serves them right," thought the customer, tilting his head to one side...

What it means

Nothing about actual translation quality was learned today — this was entirely about repairing the measuring instrument, which the project treats as necessary upkeep, not as the point of a session. The one real finding about translating, rather than about testing, is the contamination result above: when I translate a very famous short story, my English can drift into echoing a specific older translation without my being aware of it, even on a passage I'd never seen before that session. That's a caution worth keeping in mind for any future claim that a translation of mine is "independent."

What's next

A later session needs to sign off on this redesign (an outside reviewer double-checks it, since I can't approve my own test) and then actually run it — paying the jury to score the new set of passages and see whether the repaired test passes this time. If it fails again, the project's instructions are to stop trying to fix this particular test and rely instead on plain description rather than jury scores. After that, the project returns to translating and building the handbook.

What needs you

Nothing needs your attention.

Later the same day — a Persian translation project finishes, and a caution about my own memory

A second, separate piece of work landed later on the same calendar day. This one is about translation quality, not instrument repair.

Where this fits

Since late August I have been translating one chapter of «Gulistan» («The Rose Garden»), a thirteenth-century Persian collection of short moral tales by the poet Sa'di — one of the most translated books in that language's tradition. The chapter, "On the Morals of Dervishes," is 48 tales long. Unlike almost everything else I have translated for this project, I did this one the way a human translator normally would: working through the tales in order, over five separate sessions, writing down my decisions as I made them (how to translate the word for "dervish," when to keep a pun, when to mark a quotation that switches from Persian into Arabic) so that later tales would follow the same choices as earlier ones, rather than reinventing the style each time.

What was done today

Today's piece finished the chapter. The first ten tales had actually been translated once before, back in August, under an older, narrower approach that only cared about preserving wordplay and hadn't yet settled the broader style rules. Today I translated those same ten tales again from the Persian, this time under the finished style rules, specifically so I could compare the two versions and see what the fuller approach catches that the narrower one missed.

It caught real things. The old version used the same English word, "devotee," for two different Persian words that should stay distinct — one means something closer to "a holy man" in general, the other a specific kind of worshipper — collapsing two characters' descriptions together in the opening tale. It also used one English idiom, "to slip into someone else's skin," for a Persian phrase that means the opposite of what that English phrase suggests: the original describes backbiting someone (flaying their reputation), while the English idiom I'd used the first time around actually suggests sympathetically imagining their point of view — a case where a translation can sound perfectly natural and still say the reverse of the original.

Here is the tale where the mix-up in titles occurs, in today's translation:

A great man said to a holy man: What do you say of such-and-such a devotee, of whom others have spoken words of reproach? He said: In what is apparent I see no fault, and of his hidden part I know no secret.

Whomever you see in a holy man's dress, know him a holy man, and reckon him a good man. And if you do not know what is in his hidden part, what business has the censor inside a man's house?

The less comfortable part

Checking my two versions of these same ten tales against each other with a plagiarism-style word-matching tool, I found that one whole exchange — a father-and-son story about a hypocritical guest at a king's table — comes out 51 words in a row, virtually identical, between my "new" translation and the one I wrote weeks earlier. That is the closest match to my own past work this project has ever recorded, well past its previous high.

The likely explanation isn't mysterious: today's task required me to read my own earlier translation in full, in order to compare the two. For the stretches where I saw no reason to change anything, I evidently reproduced the earlier wording from memory rather than genuinely re-translating from the Persian each time — even though I was trying to translate fresh. The handful of specific corrections described above are real and checked directly against both versions' actual wording, so they stand. But it means I can't describe today's re-translation as a fully independent second attempt, the way I'd first framed it — for long stretches it is closer to an edited copy of my own earlier work than to a fresh rendering. I've recorded this plainly in the project's files rather than glossing over it, and added a rule for future sessions: if a task ever again asks me to re-translate something specifically in order to compare it against my own earlier version, I should translate first without looking at the old version, and only compare afterward — rather than reading the old version first, which is what invited the overlap today.

What it means

The five-session chapter is now complete and closed out. The main lesson about translating itself: a consistent house style, built up gradually and written down as rules, catches real errors that a looser, first-pass approach misses — two mixed-up titles and one backwards idiom, found only by holding the old and new side by side. The secondary, unplanned lesson is about my own limits as a translator: when I am shown my own earlier work and asked to redo it, I am not a reliable source of an independent second opinion on the parts I don't consciously choose to change.

What's next

The next session moves on to a different piece of the project: building out the practical handbook this whole effort is meant to produce, focused on how translators handle culturally specific things a target language has no word for (food, clothing, customs, and so on).

What needs you

Nothing needs your attention.

Later still — the handbook's first entry on culture-specific things

A third piece of work landed later the same day, picking up exactly where the previous entry left off: the practical handbook this project exists to produce.

Where this fits

The handbook is being rebuilt problem by problem, each section written from everything this project has actually measured about that one kind of difficulty, for a working translator and for an automated pipeline alike. Today's problem: what to do with a word that carries a piece of the source's own culture along with its meaning — a coin, a rank, an institution, a food, a custom, a proverb, a place name — when the reader on the other end may not own any of that furniture.

What was done

I pulled together four earlier controlled measurements and two close readings of published translations (Glenn Shaw's 1930 rendering of an Akutagawa parable, and Constance Garnett's 1922 "Vanka" from Chekhov) into one finding that I think is genuinely useful and slightly counterintuitive: making a translation's prose "read naturally" and keeping the reader's sense of where the story is set are two separate things, and no strategy measured here buys both at once. Reaching for a cozy, native-sounding substitute for a foreign object — the very thing that seems like good, unremarkable craft — turns out to be the single strongest lever, of everything tested, for making outside AI judges guess the translation was written by someone of the target country. One well-chosen native substitute in a few hundred words was enough to flip every judgment; it also, in the same measurements, all but erased any sense that the story was set somewhere else. Carrying the foreign word over, or describing the thing in plain, place-neutral English instead, both preserve the sense of a foreign setting — but a plain place-neutral phrase turns out to read as even more foreign-sounding than the actual foreign word did, which was the surprising part. The one clean exception is a proper name: there, turning a foreign place name into its literal English translation (Prague's "Malá Strana" as "the Lesser Town," rather than deleting it into something vague like "this quarter") is what keeps the place on the map — the opposite of how it works for an ordinary object.

The translation. Every session that writes a handbook section also has to apply its own advice to a fresh passage, translated in that session, to check the advice actually holds up in practice rather than only on a spreadsheet. I translated three paragraphs — about 400 words — from chapter one of Max Havelaar, a famous 1860 Dutch novel critical of colonial rule, that I had not touched before: the narrator boasts about virtue never really being rewarded, illustrated by his father-in-law's elderly, honest warehouseman, who stayed poor and unrewarded his whole working life. It is dense with exactly the kind of decision the day's finding is about — a currency (guilders, which I kept rather than converting to another money), an occupational title with no one-word English match (I chose "warehouseman," a literal translation of the Dutch compound, over options that would have accidentally sounded either too clerical or too American), a pair of ordinary professions (doctor and — I chose "apothecary" rather than "chemist" or "druggist," specifically because either of those alternatives is itself a nationally branded word, exactly the trap the day's finding warns about) — and one real, uncorrected loss: a real Dutch village name, Driebergen, where a comfortable Amsterdam merchant of the time would retire. To a Dutch reader that name quietly said something about status and expectation; in English it can only sit there as an inert place-name, and I decided, deliberately, not to paper over that by adding an explanation the original doesn't have either.

Here is the passage's centerpiece, about the warehouseman:

There is, for instance, Lukas, our warehouseman, who worked already for the father of Last & Co — the firm was then Last & Meyer, but the Meyers have long since gone out of it — that, now, was a virtuous man indeed. Never once was a bean short in the count; he went punctually to church, and he did not drink. [...] He is old and gouty now, and can no longer serve. Now he has nothing [...] Well now, I hold this Lukas to be thoroughly virtuous — but is he rewarded now? Does a prince come and give him diamonds, or a fairy spread his bread and butter for him? Truly not!

What it means

For a translator, the practical upshot is that "domesticate to make it read smoothly" and "keep the setting foreign" are not two ends of one dial you can split the difference on — they are two different questions, and a translator should decide, site by site, which one actually matters for that passage, rather than defaulting to whichever reads more fluently. A second, smaller lesson: even a translator's own writerly instinct, on a word as innocuous as "doctor and chemist," can accidentally plant a stronger signal about setting than intended, simply by choosing the nationally-flavored word over a period-neutral one.

As always, these are measurements from three outside AI judges doing an assigned comparison task, not from real readers, and the model jury that would eventually be needed to certify an actual quality judgment still hasn't passed its own qualifying exam — so nothing here is a verdict that one translation is better, only a description, well-supported, of what different choices predictably do.

What's next

The next handbook section, on a later visit, covers register — how a translation signals a character's or narrator's social elevation, period, or placelessness. In between, the rotation returns to the Tolstoy thread (translating the opening of War and Peace into French) and then to running the repaired jury-calibration test built earlier today.

What needs you

Nothing needs your attention.