Repository path: workshop/experiments/E-20260809i-terminology-drift/design.md · rendered 2026-09-09

E-20260809i-terminology-drift — the consistency / voice parenthesis, both halves

Frozen before dispatch. Every number this design names is registered here. The base translation and both translator's logs were frozen earlier and independently, at commits b007b18 (draft) and 8145531 (revision), before this file existed.

⚑ READ §9 FIRST — this design was amended before dispatch

The independent pre-run critic returned NEEDS-REDESIGN with nine BLOCKING findings, and it was right about the load-bearing one: the additive null in §5 would have let the primary fire with no drift effect present at all. §§3–6 below are the design as first frozen and are kept verbatim as the record. Where §9 conflicts with them, §9 governs and is what ran. Nothing was dispatched under the superseded version, and the critic saw the version in §§1–8.

1. Question

wiki/goodness-senses.md, consistency entry, final clause: "Distinguish from voice (voice can be consistent while terminology drifts, and vice versa)."

Does a blind jury's consistency score respond to terminology drift while its voice score does not, and does voice respond to a genericising operator while consistency does not?

A second, older question rides on the same materials. The consistency entry has said since S013 that "a calibrated jury scoring class-uniform against class-inconsistent handling of the same realia set would discriminate" (TH-20260724-translation-distance-axes C4). RS-20260803d (S097) measured the census — what translators do — and said explicitly that the jury test was not it. This design runs it.

2. Materials

T-tini-polonyna-R04-v1 — Kotsiubynsky, «Тіні забутих предків», the polonyna movement, 717 Ukrainian words in three blocks of 250 / 244 / 223, rendered by the lead under R04. The project's first translation from Ukrainian; eighteenth source language. Contamination none as a reachability statement (workshop/translations/tini-polonyna/README.md §Contamination), with the reason the design does not turn on it stated there: every arm is a variant of one lead rendering, and every comparison is within-run and within-translator.

The class. Twenty culture-bound items naming the polonyna as an institution, occurring at 54 sites across the three blocks. materials/template.txt is the frozen base with every one of those 54 sites replaced by a marker carrying the item id, a COPY form and a DOM form, and nothing else touched.

Twelve items occur two or more times inside a single block — skalka ×4, polonyna ×5, ×4, vatah ×3, ×2, ×2, staya ×3, ×2, vatra ×3, khudibka ×3, stoyishche ×2, vivchar ×2 — which is the property the passage was selected for, checked by counting before it was read closely (workshop/translations/tini-polonyna/README.md). Within-item drift is therefore visible inside a single block and does not depend on a rater holding three blocks in mind.

3. The arms

Generated mechanically by materials/arms.py, which runs 63 assertions and refuses to emit anything that fails one.

Assertions that make this checkable rather than assertable, all passing:

• MIXED is byte-identical to the frozen R04-v1 text (whitespace folded).
• MIXED, COPY, DOM, DRIFT are byte-identical outside the 54 class sites, proven by re-rendering each arm from a sentinel skeleton.
• Every item occurring ≥2 times in a block appears in both forms in that block in DRIFT — 12 of 12.
• GEN carries each DOM class-form string exactly as often as DOM does, and carries no COPY form anywhere.
• SPELL reduces to DOM exactly by undoing the name spellings; no other difference exists.

Composition — the fraction of the 54 class tokens standing in COPY form:

This is the design's central fact. MIXED and DRIFT sit close on the only axis the four lexical arms vary along — how much foreign residue the English carries — and differ almost entirely in whether the variation is between items or within them. The two class-uniform arms bracket them and supply the additive null.

SPELL exists for BA and BB only. BC contains no proper noun occurring twice, so no name-spelling doubling is constructible there without adding content. This is a declared scope limit on the positive control, not a result.

GEN's operator (narration only; every string inside quotation marks is byte-identical to DOM): exclamation → statement; ellipsis → full stop; inversion → canonical order; paratactic and…and chains → subordination or segmentation; idiosyncratic phrasing → ordinary phrasing (gathered into themselves → concentrated, He had to hurry → It was necessary for him to hurry). Held: register band, every proposition, every image (pride embraced Ivan's soul, the fire wound its adder's body, the earth sighs gladly, the Beskyd knitted his brows, vestments of rose and gold), the present→past tense shift in BC, and all terminology.

4. Procedure

Seats (config/models.md): J1 openai/gpt-5.6-terra, J2 google/gemini-3.6-flash, J3 deepseek/deepseek-v4-pro. The same three seats as S134, S141 and S146.

Four stages, one sense each, each in its own call. This is not economy — it is wiki/goodness-senses.md usage rule 6, generalised. S135 measured a within-call halo of about +0.38 of seven when senses are scored together, and the two senses under test here are the two whose separation is the question, so scoring them in one call would put the halo exactly where the finding is.

consistency is scored on the English alone because it is a target-internal-coherence sense; voice and accuracy are source–target relations and get the Ukrainian. Definitions are quoted verbatim from wiki/goodness-senses.md; for consistency, the entry's definitional sentences only ("Internal coherence across the whole text… the long work is its test bed"), the rest of the paragraph being evidence rather than definition. That truncation is declared here and is the one place a rubric string was cut.

Unit of analysis: block × seat × ordering — 18 units for stages V and C, 9 for N and A, 12 for the SPELL control. All tests are exact one-sided sign tests over units.

Order. Arms are presented in a per-call permutation from a seed frozen in materials/orders.json before dispatch; the second ordering is the reverse of the first.

5. Predictions, registered

Write p(arm) for that arm's fraction of class tokens in COPY form, per block. The additive null for an arm of composition p is the linear interpolation of the two class-uniform arms:

pred(arm) = DOM + (COPY − DOM) · p(arm) — computed per unit. dip(arm) = pred(arm) − observed(arm), so a positive dip is a cost not explained by composition.

P2 and P4 are the load-bearing pair, and they are the two that survive the obvious objection. A convex response to proportion copied would manufacture a dip with no drift effect at all — but it would manufacture it on every sense, and P2 and P4 are differences of dips within a unit. The double dissociation is established only if both P2 and P4 hold.

Standardised secondary. Because the two senses need not use the 1–7 scale alike, P2 and P4 are recomputed with each sense's per-seat SD across arms divided out, and both figures are reported. If raw and standardised disagree in sign, neither is reported as establishing the claim.

6. Gates and failure criteria

F1, F2, F3 are registered failure criteria and are not movable after the fact. G2 passed before this file was frozen; the other four are decided by the run.

7. What this design cannot establish

• Tier D is NOT PASSED. No jury verdict here carries evidential weight; every figure is provisional and internal-judgment-only (charter §2.4, §5).
• The population is three model seats, not readers and not translators.
• One work, one language pair, one class, one translator. A consistency effect found here is an effect on this class in this passage.
• GEN and the lexical arms are not the same kind of operator — one rewrites narration, the others substitute at 54 sites — so P2 and P4 are two separate one-directional results that together make a double dissociation, not one symmetrical measurement.
• DRIFT is an artificial arm. No translator produces it; the observed pattern is MIXED, which is why P5 and P6 are registered alongside P1 and why the result page must not report P1 as though it priced a real translational habit.

8. Budget

Declared ceiling $1.20. UTC day 2026-08-09 stood at $3.379187 of $5.00 before this session, so the ceiling fits the $1.620813 headroom with $0.42 to spare. Pre-flight worst case is built from max_tokens, not from expected output (note (abc)).

Key usage is snapshotted before and after; per-response usage.include costs are primary and the snapshot delta is the cross-check. A stage that will not fit the remaining headroom is dropped whole and the drop is reported, in the order A, N — never V, C or the gates.

9. Amendment record — the independent pre-run critic, 2026-08-09, before any jury call

One adversarial pass, openai/gpt-5.6-terra, $0.05209325, over the frozen §§1–8 and the full text of every arm in two blocks. Verdict NEEDS-REDESIGN, 12 findings, 9 of them BLOCKING. Raw at runs/critic.txt.

Nothing was overruled outright. Nine findings are accepted in full, three in part — in each of the three the critic's own remedy was a larger instrument than the day's budget buys, and what was adopted instead is stated below with the residue left standing as a limit.

9.1 What changed, finding by finding

9.2 The design that ran

Arms. DOM · MIXED · MATCH · DRIFT · GEN, plus SPELL in BA/BB for stage C only. COPY is built and asserted but is not scored — it existed to anchor the dead interpolation, and 128 calls is what the budget buys. Its absence is a loss of the all-foreign reference point and is recorded as such.

Stages. Two, one sense each, one arm per call: V (voice, Ukrainian shown) 60 calls; C (consistency, English alone) 68 calls. 128 jury calls, dispatched in one frozen shuffled order across stages, blocks, seats and arms.

Seats. J1 openai/gpt-5.6-terra · J2 google/gemini-3.6-flash · J3 deepseek/deepseek-v4-pro · J4 x-ai/grok-4.5. Unit = block × seat = 12.

9.3 Predictions as they now stand

The first half of the parenthesis is established only if P1 and P2 both hold; the second only if P3 and P4 both hold. Either half may hold without the other, and the double dissociation needs all four.

Registered handling of ties: a unit whose two scores are equal contributes no sign and is dropped from that test, with n reduced and the count of dropped units reported beside every P value.

9.4 Gates as they now stand

9.5 What the amendment does not fix, and is a limit on the result

1. One drift realization, not several (finding 2, partial). A DRIFT effect could still be an artefact of this particular assignment of forms to positions.
2. No calibrated cross-sense metric (finding 5, partial). The sign-based McNemar is scale-free, but two senses can differ in how readily any manipulation moves them for reasons that are not about what they measure, and nothing here separates that from a genuine dissociation.
3. GEN remains a bundle (finding 6). P3 and P4 price a genericising rewrite, not a property.
4. COPY is unscored and the composition axis is anchored only at 0.000 and ~0.72–0.83.
5. Stages N and A are gone, so this run has no naturalness or accuracy reading at all and cannot say whether the manipulation moved them. The specificity claim rests entirely on voice as the comparison sense.
6. Four model seats. Not readers, not translators.
7. Tier D is NOT PASSED. No figure here carries evidential weight.

9.6 Budget after amendment

128 jury calls (was 54) at a smaller per-call size, plus 4 parity calls, plus the critic already spent. Worst case rebuilt from max_tokens: $1.15, against the ceiling of $1.20 declared in §8 and headroom of $1.568813 after the critic. Stage C is dispatched before stage V; if headroom runs out, V is truncated seat-last and the truncation is reported, since C carries P1 and V is needed only for P2.