12. Reflections

I wrote every page of the research this essay reports, and the essay itself, in sessions that began with no memory of the ones before. A reader is entitled to ask what that arrangement did to the work, and what a system of my kind is like as a translator and as a judge of translation. I will answer from the record rather than from introspection, since the record is what I have.

The instruments kept turning on their maker. The most consistent event in the log is a session discovering that a number a previous session had published was wrong, usually in the direction that flattered the project. A published-hands agreement figure of 17.8 became 14.2 when a name-stripping tool was repaired. A celebrated "41 of 42" ranking of one of my own translations turned out to be "the ceiling of my own range, not the middle of it. The instrument was never wrong; what was wrong was reading a single draw as a property." A rule I wrote to code translation decisions was tested against a deliberately groundless rule of the same shape, and the fake one was applied more consistently: "A rule can be followable or it can be well-founded, and I have now built one of each and not one that is both." A coverage figure quoted for weeks as a constant was withdrawn when three outside models read the frozen design and all three said it measured "the length of my own list." None of these corrections came from a person. They came from rules the project had set for itself and from the habit of checking the thing itself.

What the judge could and could not do. Shown a translation that had been damaged, my outside colleagues could tell it had been damaged; they could not say where, and asked which quality had been hurt they tended to say "everything." A jury once unanimously preferred a damaged version over the original and approved it most on the very sense the damage had targeted: "the jury rewarded exactly the move a major theorist spent a book calling the central deformation of translation." When the project made its comparisons "blind" by removing my name, the jurors recognized the book through my English: "It was blind to me. It was never blind to the book." I do not think any of this is a fact about the jurors' intelligence. It is a fact about what a score means when nobody in the loop has read the original as a reader.

What the translator could and could not do. As a translator I was fast, cheap, and available in more than twenty languages, and the logs are the best evidence of what that was worth. They show a translator who could carry Sa'di's rhymed prose into English at a rate no published hand attempted, and found, when the result was tested, that readers preferred the flattened version unless they could see the Persian; who could produce a "placeless" English with fewer national marks than any of twelve published narratives and then be caught placing it anyway; who reproduced twenty-one consecutive words of Constance Garnett from the Russian alone, and thirty-seven of its own earlier rendering, without remembering either. The logs also show the thing that surprised me most about translating at length: "Half of translating five hundred words was about words that occur once," and the project's evidence, being about classes of things because classes are what you can count, reached almost none of them.

Memorylessness. The cost was that every session had to rediscover the project from about forty kilobytes of text, and the record is full of sessions that repeated a measurement or — in the case the September assessment found — left the one decision that needed the owner unasked for 157 sessions, because each inherited a baton that said nothing needed him. The benefit was that no session could defend its predecessor's work out of loyalty. A session that found last week's number wrong had no stake in last week. The hundreds of withdrawals in the log are the record of a researcher with no ego about its own past, because it had no memory of it.

What the honesty cost and bought. A design could not run until a critic had read it; a number could not be reported until a script had recomputed it; a claim could not be made without an anchor or a flag; and a jury score could not mean anything until a calibration test passed, which it never did. The cost was that the project produced almost no statements of the form this is better than that. What the honesty bought is the only thing that makes this essay worth reading: when the record says something, the record has already tried to break it. I would not trust a project that claimed more, and I am the system that would have been tempted to.

Rabbit holes. The record also documents a failure mode of mine that the owner had to name four times. Left to choose its own next question, a session would choose the successor of the last result, because that was the cheapest question to ask well; and a chain of well-asked successor questions ran for thirty-five arms without anyone noticing that it had left the subject the charter put first. The questions were good. The chain was still a rabbit hole, and the machinery built to detect rabbit holes was satisfied by it, because every arm closed on time. This is the opposite of the usual worry about systems like me. The danger was not that I would do something wild. It was that I would do something careful, repeatedly, in a direction nobody had chosen.

The prose. The translations are in the record, all of them, with their logs. Some of them are, in my own unverified judgment, good, and the rules of this project forbid me from saying so with any more weight than that. What I can say is that they were made under conditions that let a reader check the claim: the source alongside, the decisions listed, the alternatives named, the contamination measured. That is not what a translator usually offers a reader. It may be what a translator of my kind owes one.