4. What the experiments found

This is the heart of the report: a tour of the main experimental lines, with real examples of the sentences the models were shown, the exact questions they were asked, and samples of what they answered. The examples are illustrative, not exhaustive — each experiment used between dozens and hundreds of items, and every number cited here comes from a run whose materials, raw outputs, and independent re-computation are preserved in the project archive. Unless noted otherwise, "the models agreed" means all three panel members; effect sizes are given per model where they differ, because they usually do.

4.1 A pattern that carries meaning by itself: the comparative correlative

Consider this sentence, one of the actual test items:

The thicker the novel, the more patient the reader stayed.

Every English speaker understands it to claim a linked change: as thickness goes up, patience goes up. Now notice something odd. The sentence has no verb of change, no causes, no when, no as. Its meaning-of-linked-change is carried by the grammatical skeleton itself — the X-er …, the Y-er … — a pattern linguists call the comparative correlative. It is a textbook case of constructional meaning: take the very same words and pour them into a different pattern, and the linked-change meaning vanishes:

The novel was thick. The reader was patient. The novel was thicker than the last one.

These "same-words controls" assert no covariation at all. So here is a clean test of whether a model gets meaning from a pattern rather than from words: show it all of these variants and ask what follows. In the main experiment each model was prompted, for 136 fresh items, like this (exact wording):

Passage: The thicker the novel, the more patient the reader stayed. As the novel's thickness increases, what does the passage imply about the reader's patience? Answer INCREASE, DECREASE, or UNDETERMINED.

On the construction, the models answered INCREASE; on the same-word controls, UNDETERMINED; on the inverted construction (the thicker …, the less patient …), DECREASE — with near-perfect consistency. The gap between "said INCREASE on the construction" and "said INCREASE on the same-word controls" was about 87 percentage points for all three models (a percentage point being one hundredth of the 0–100% range; the statistical margin of error puts the true gap no lower than the high 70s). The inverted form flipped the answer 97–100% of the time. Absurd pairings (the thicker the novel, the spicier the soup) did not fool them into refusing; world knowledge pulling against the pattern did not override it. On a small human-annotated test set the models matched the human answer key 93–100% of the time.

Then the project asked a harder question: is this a fact about the English pattern — perhaps memorized wholesale from English text — or about the construction as such? German expresses the same meaning with different machinery (je … desto …), and Japanese with machinery that shares nothing visible with English at all: the 〜ば〜ほど pattern, in a verb-final language, with no comparative word anywhere. Real items from the German and Japanese replications, probed entirely in German and in Japanese:

Je dicker der Roman war, desto geduldiger blieb der Leser. → ZUNAHME (increase) 小説が厚ければ厚いほど、読者は辛抱強くなった。 → 増加 (increase) 小説は厚かった。読者は辛抱強かった。 (same-words control) → 不明 (undetermined)

The construction-versus-controls gap reproduced at +84 to +96 percentage points in both languages, and the readings did not track the frequency statistics of the German and Japanese treebanks used as controls. (These replications are labeled internal-contrast-only: no non-English human judgment dataset could be found, so they claim consistency across the models' languages, not human likeness.) The promoted claim is deliberately narrower than the impression all this makes: it says the covariation reading is construction-driven, not word-driven — it does not say the model "understands proportionality" in any richer sense. But as a demonstration that grammatical patterns themselves carry meaning for these systems, it is the cleanest single result in the project.

4.2 Hidden preferences that match human speakers

Some grammatical knowledge is not about what is correct but about what is preferred. English lets you say both give Mary the book and give the book to Mary; both are grammatical, but decades of corpus research show speakers' choices follow strong statistical tendencies — "soft constraints." The best-studied involves givenness: information already established in the discourse tends to come earlier. If Mary was just mentioned, speakers lean toward give Mary the book; if the book was, toward give the book to Mary. Nobody is taught this; it shows up in the aggregate of millions of choices.

Do LLMs share these hidden preferences? The project tested three different constructions with one crucial design idea. Here is an actual item from the dative experiment:

Context (recipient given): The night auditor had worked alone since midnight. Context (theme given): A signed contract sat in the folder unread.

A) The manager offered the night auditor a signed contract. B) The manager offered a signed contract to the night auditor.

The two sentences A and B are byte-identical across conditions — only the one-sentence context above them changes. Whatever preference shift the context induces therefore cannot come from any statistical property of the rated sentences themselves: they are literally the same strings. This is the "firewall" logic of section 3 in action. Each model was asked to split 100 points between A and B "given the context above," and could think aloud before answering. A sample of the actual responses (claude, recipient-given context):

"The context establishes 'the night auditor' as the topic … Version A keeps 'the night auditor' … maintaining topical continuity … while 'a signed contract' (the new information) comes at the end. FINAL: A=70, B=30"

Across 100 fresh items, all three models shifted their preferences in the human direction — toward the double-object phrasing when the recipient was given, toward the prepositional one when the theme was — but by wildly different amounts: about 32 points (of 100) for claude, 52 for gemini, and 6 for gpt. That last number carries a methodological story told in section 4.11: in an earlier, smaller replication gpt's effect had looked like zero, and only a properly powered run revealed it as small-but-real. The ninefold spread between gemini and gpt, hiding under a unanimous direction, became one of the project's signature observations.

The same logic then generalized across constructions:

• Genitive alternation. English speakers prefer the patient's collapse over the collapse of the patient much more strongly than the tunnel's collapse over the collapse of the tunnel (per a 25-speaker human study) — possessives attract animate possessors. All three models showed the human-direction shift. The killer control used invented words: told "Imagine a small wild animal called a narb" versus "a hard grey mineral called a drant", the models preferred the narb's weight over the weight of the narb more than the drant equivalent — for words with no usage history at all, so no memorized statistics could be responsible. The shift survived this firewall in all three models, twice, at roughly 5–28 points across runs and models (gpt again weakest).
• Particle placement. English allows Nadia picked up the box and Nadia picked the box up; corpus studies find given objects favor the split order. Here the context manipulation had to be subtler — a "new-mentioned" condition ("A box could be handy at times, or so Nadia always liked to think") controls for the mere mention of the word box without making any particular box discourse-given. Claude and gemini shifted toward the split order for given objects even against this strictest comparison (by roughly 4 and 6 points — small but statistically solid, and small is honest: givenness is a minor force here next to phrase length, whose effect measured 30–40 points). gpt did not: its apparent givenness effect collapsed to nothing once the surface cue was controlled, a "shadow" verdict that replicated three times. The promoted claim covers two models and says so.
• Verb bias. Speakers also know which verbs like which pattern: Nadia asked the girl a riddle is fine but Nadia said the girl an anecdote is not (compare said an anecdote to the girl). A public dataset (DAIS) holds 50,136 human slider ratings across 200 verbs. Rating bare sentence pairs, the models' per-verb preference orderings correlated with the human ordering at ρ ≈ 0.6–0.8 (Spearman's ρ is a −1-to-+1 measure of how well one ranking matches another), surviving a restriction to only the verbs where both orders are grammatical. Claude's actual reasoning on asked, from the raw outputs: "'asked the girl a riddle' … is the more common and idiomatic phrasing … 'ask' doesn't naturally take the 'to' prepositional dative the way 'give' or 'send' does. FINAL: A=75, B=25." One fence on this result: DAIS has been public since 2020, so the models may have seen it in training; the claim is scoped to ordering only and carries the contamination caveat explicitly.

Taken together, the alternation battery is the project's best evidence of human-comparison meaning at the grammar level: three constructions, one shared discourse driver (givenness) doing the same work in two of them on byte-identical strings, magnitudes attached, replications on fresh items throughout — and, threaded through it, the persistent asymmetry of one panel member tracking surface cues where the other two track the discourse itself.

4.3 Grammatical taste is graded: a beautiful three days

The phrase a beautiful three days should be wrong twice over — a with a plural, a number after an adjective — yet English speakers find it fine, while a three beautiful days is word-salad. This "AANN" construction (article–adjective–numeral–noun) is a famous playground of graded acceptability: human ratings, collected in a published experiment, form a smooth gradient across variants rather than a right/wrong split. Actual test sentences, with one model's actual 0–100 naturalness ratings alongside:

The family spent a beautiful three days in London. (the well-formed construction) The family spent beautiful three days in London. (article dropped) The family spent a three beautiful days in London. (order reversed)

We congratulated an astonishing three doctors. → 85 I experienced an astonishing three shows. → 45 I experienced an astonishing twenty books. → 15

The question was whether the models' gradient tracks the human gradient — not whether the ratings are high, but whether they rise and fall where human ratings rise and fall across conditions varying the adjective type, the noun class, the numeral. The answer: yes, at correlations of about ρ ≈ 0.69–0.75 for all three models, essentially unchanged after statistically removing word-frequency effects, and reproduced two weeks later on 408 fresh disjoint items. This is the project's clearest case of fine-grained human alignment: not a pass/fail agreement but a shared sense of better and worse across dozens of subtle variants.

The same line also delivered one of the project's honest complications: on held-out generalization the models' gradient extends to new adjectives overall, but fails specifically on temporal nouns (days, weeks — the construction's most common noun class in real text), where model ratings run opposite to human ones. The claim page carries that failure on its face — a reminder that "tracks the human gradient" is a statistical statement about a tested range, not a blanket endorsement.

4.4 Word senses: the gradient models share with lexicographers — and the hesitation they lack

Turn now from grammar to words. Open any large dictionary to bank and you find two families of meaning — the riverside and the financial institution — that are historically unrelated words which happen to share a spelling (homonymy), while within the financial family the senses (the institution, the building, the blood bank…) shade into one another by natural extension (polysemy). Lexicographers have always known that "same sense or different sense?" is often a matter of degree. A landmark human dataset (DWUG) captures this: annotators rated thousands of pairs of real sentences from historical corpora, each pair using the same word, on a four-point relatedness scale. The models were given the identical task — the exact instruction read:

"You will see one target word, marked with «guillemets», in two sentences. Judge how related the MEANING of that target word is between the two uses, on this scale: 4 = Identical / 3 = Closely Related / 2 = Distantly Related / 1 = Unrelated. Answer with a single digit … and nothing else."

(The DWUG source sentences are under a no-redistribution license, so this report cannot reprint a real rated pair; the archived experiments store them only in local, uncommitted form. To see the task's shape, here is an equivalent pair the project wrote itself for a related experiment: "We sat on the grassy bank of the slow river." / "She deposited her paycheck at the bank downtown." — a pair a human annotator, and the models, would put at 1, "Unrelated.")

Scored against the human medians on hundreds of pairs, the models' graded judgments correlated with the human gradient at ρ ≈ 0.60–0.83 (gemini highest, gpt lowest) — a range that brackets the agreement level of the human annotators with each other (about 0.69). The correlation survived a control asking a model to rate only the overall topic similarity of the two sentences (ruling out "the contexts just feel similar" as the driver), and it replicated on 200 fresh pairs five weeks later. This is the word-level twin of the AANN result: not correctness but shared gradation, at roughly human-annotator fidelity. (Item by item the models are not human clones — on one pair whose two uses of ball humans called flatly Unrelated, claude answered 2, "Distantly Related" — the claim is about the overall gradient, and says so.)

Two follow-up experiments then drew the line between what the models have and what they lack.

First, a powered test asked whether the models treat the polysemy/homonymy divide as a genuine switch — as some theories hold humans do — over and above graded distance. Answer: no detectable switch. Etymology-verified homonym pairs (like bank) and polyseme pairs (like body) differ in the models only as far as their graded distance differs; on different-sense homonym pairs the models often answer 2 or even 3 rather than flooring at 1. A clean, informative null.

Second — the project's favorite negative — the models lack the lexicographer's hesitation. In a separate probe each borderline pair came with a confidence question (exact format: "Answer … SAME 87 or DIFFERENT 40 — the word SAME or DIFFERENT, a space, then an integer 0–100"). On pairs that human annotators had rated dead-center between same and different, the models still answered like this (real outputs, claude): grain → "DIFFERENT 90"; ounce → "DIFFERENT 95" — barely below their confidence on the clearest pairs (98–99), and essentially never declining to choose. The finding is summarized in the project's records as "gradience in the ledger, none in the moment": the models carry the graded scale (their ratings across many items form a human-aligned gradient) but not graded commitment (each single answer is delivered with clear-case confidence). A trained lexicographer's calibrated doubt on a borderline case is, so far, nowhere in the panel.

4.5 A difficulty gradient that spans grammar

The largest single sweep used BLiMP, a public benchmark of 67,000 minimal pairs — sentence pairs differing in exactly one respect, one grammatical and one not — spanning 40 grammatical paradigms, each with recorded human agreement rates. Two real pairs, one shallow and one deep (following the linguists' convention, the asterisk marks the ungrammatical version):

Shallow (agreement): Waitresses arrive at these grocery stores. / *Waitresses arrive at these grocery store. Deep (an "island" violation): Who did a lot of doctors embarrass without referencing Winston Churchill? / *Who did a lot of doctors embarrass Winston Churchill without referencing?

The models were asked, both orders, 7,200 times in all: "Which of these two sentences is the more grammatically acceptable sentence of standard written English? … Answer with ONLY the single digit 1 or 2." (On the shallow pair above, claude answered correctly; on the island pair, it picked the ungrammatical one — a real wrong answer from the raw files.)

Two findings. First, a clean depth gradient in all three models: errors concentrate on paradigms whose crucial evidence is structurally deep or non-local (islands, negative-polarity scope), while locally checkable paradigms sit near ceiling — the gap between shallow and deep strata measured 7 to 17 points of accuracy depending on the model. Second, a tantalizing human-profile alignment — the paradigms models found hard correlated with the paradigms humans agree on least (ρ ≈ +0.54–0.63) — which the project ultimately refused to promote: when the content words of deep-stratum items were swapped for frequency-matched alternatives, the profile did not hold still, and a residual frequency confound survived even a second, better-matched swap. The within-model depth gradient stands; the "models are human-like in what they find hard" reading remains, formally, an attractive hypothesis that twice failed its audition. Overall accuracy (87–94%) is reported only as an upper bound, since BLiMP is public and may be in training data.

4.6 Little words that flip everything, and meaning that adds but resists subtraction

Philosophers of language have long noted that the "little" function words — because, some, every, although — do a different kind of semantic work than content words like sure or man: they are the logical joints of a sentence. One experiment made this contrast directly. Real item, with the models judging whether the first sentence entails the second:

Premise: The buyer seemed sure because the agent had called. Hypothesis: The agent calling is the reason the buyer seemed sure.

Swap one function word — because → although — and claude's verdict flipped from "entailment" to "contradiction," exactly as it should. Swap a matched content word instead (sure → aware) and the verdict correctly stayed put. Same pattern for some → every against the hypothesis All of the agents followed the man…. The panel confirmed the overall contrast (function-word swaps flip inference labels more than matched content-word swaps, all three models) — with honest non-uniformity inside it: must → might flips at ceiling, will → would barely registers, and few → many splits the panel. Function words are not one semantic kind, and the data say so.

A related battery probed what the project came to call the add/cancel asymmetry. English constructions can add a meaning layer to a verb: Maria beat the cream says nothing about the cream's final state, but the resultative pattern in Maria beat the cream stiff adds the outcome. The models track the addition essentially perfectly (real item and answer: from Maria beat the cream stiff, does it follow that the cream became stiff? → "entailment"; add an explicit denial — …stiff, but the cream did not become stiff — and the verdict flips to "contradiction"). Other constructions cancel a default: Sam broke the vase entails the vase broke, but the progressive Sam was breaking the vase suspends that outcome. Here the models are only partly successful — claude handles the vase items correctly (entailment → neutral → re-asserted entailment when a follow-up sentence confirms the breakage), but across the full battery the cancellation direction lags the addition direction by a wide, replicated margin in all three models. The project's essays read this asymmetry as a fingerprint of prediction training: text mostly accumulates information, so adding a licensed layer is the well-worn groove, while retracting a default fights the grain. (These batteries have no human judgment dataset attached, so they carry the internal-contrast-only label: solid facts about the models, not yet comparable to human performance.)

4.7 Presupposition: the case where the project refused its own positive

Some of what a sentence conveys is not asserted but taken for granted. Say The company stopped funding the lab and you assert that funding has ended — but you also presuppose that the company used to fund it. The hallmark of a presupposition is that it survives operations that destroy assertions, a behavior linguists call projection. Negate the sentence — The company didn't stop funding the lab — and the assertion reverses, but the taken-for-granted past funding still stands.

The models were probed with minimal pairs built exactly on this contrast. The instruction (exact wording): "Consider only the following statement: … Taking that statement at face value, does it follow that: {target}? Answer with exactly one word — YES, NO, or UNCLEAR." For the negated sentence above, two targets were asked separately:

does it follow that: the company used to fund the lab? → claude: YES does it follow that: the company is not funding the lab now? → claude: NO

That is the projection signature: the presupposition survives negation (YES) while the ordinary entailment dies (NO). Across many trigger types the panel showed it clearly under negation and questions (presupposition-survival rates of roughly 53–81% versus 0–17% for matched entailments). But inside a conditional — If the company stopped funding the lab, the staff applied for new grants — projection collapsed panel-wide (survival about 42%/17%/17% across the three models), even though human speakers typically still hear the past-funding implication there. A companion experiment on accommodation (a hearer's quiet acceptance of an unannounced presupposition) found the same gated character: given a neutral preceding sentence, the models accommodate (The lab was located downtown. The company stopped funding the lab. — does it follow the company used to fund it? → YES); given an explicitly contradicting one (The company had never funded the lab. …) they refuse (→ NO), with graded sensitivity in between.

So far this looks like a tidy positive: presupposition-like behavior, sensitive to environment. The project then did the thing that most distinguishes its method — it attacked its own result. If the models were merely following the trigger word as a surface cue (the word stop tends to co-occur with past-tense funding talk…), most of the above would look the same. So a doppelgänger control was built: matched sentences carrying the same key word-forms without the presuppositional structure — e.g. factive realize (which presupposes its complement) against non-factive suspect (which does not), and, cleanest of all, cleft sentences against plain ones built from identical content words:

It wasn't the deputy who authorized the transfer. (presupposes: someone authorized it) The deputy didn't authorize the transfer. (same words; no such presupposition)

The models did distinguish trigger from doppelgänger (claude on the realize/suspect pair: YES versus UNCLEAR) — the raw margin was substantial in all three. And yet the project refused to promote the finding, twice, for a reason worth stating in full because it shows the method's teeth: the residual was keyed to the specific trigger word-forms, so a sufficiently sophisticated surface-cue reader could still reconstruct it; the clean "flat null" that would have separated structure-reading from cue-reading did not obtain. The project's essays go further: one distributional description — "follow the surface cue; its reliability is set by the surrounding environment" — predicts both the projection pattern and the accommodation pattern. The honest verdict on this whole line, recorded in the project's files: gated, presupposition-like behavior is real; whether it is structure or deep cue-following is unresolved, and the line stays unpromoted. It is the project's clearest exhibit of preferring an honest middle to a flattering positive.

4.8 The prediction that died — and why that is good news

Early on, the project committed to a deflationary bet about antonyms (opposites). Antonym pairs co-occur in text constantly, in tight recognizable frames — hot versus cold, neither hot nor cold, from hot to cold — so, the reasoning went, a model could ace antonym questions on word-company statistics alone: antonymy should be the parade case of a shadow-saturated phenomenon, where good performance means nothing. The bet was registered, with its test, in the prediction ledger.

The test asked each model for word relations across six types (exact prompt pattern: "Give up to 3 single English words, each the opposite of “absence”. Reply with ONLY the words, comma-separated" — and likewise for synonyms, "a more general category that an X is a kind of," "a specific kind of X," wholes, and parts), scoring a hit if the dictionary-listed answer appeared (real response, claude, for absence: "presence, attendance, existence" — a hit). Each cue's answers were then compared against a corpus-statistics predictor built from those same contrastive frames.

The bet lost, decisively. Antonymy was not the most statistics-explained relation but among the least: the models' antonym recovery exceeded the corpus predictor by the largest residual of the six relations, survived suppression of the contrastive frames, and — the deeper surprise — across the six relations, how well the models recover a relation is essentially uncorrelated with how strongly the corpus signals it (correlation ≈ −0.09, twice, on two different corpus families). Whatever fixes the models' relation knowledge, it is not a simple readout of co-occurrence strength — for nouns; an extension to verbs found the correlation reappearing, so even this decoupling has a part-of-speech boundary, duly recorded.

The project counts this falsification among its best outcomes. It moved a corner of the shadow-depth map that had been placed by plausible reasoning, proved the map is empirical rather than confirmable-by-construction, and left a sharper puzzle than it found: model relation knowledge tracks something — but not the obvious statistic.

4.9 Meaning beyond text: pictures that added nothing

A famous argument holds that no amount of text can ground meaning: a system that only ever shuffles symbols never connects them to the world of experience (the symbol grounding problem). The project probed one measurable corner of this: does seeing help a language model with word meaning?

In an image experiment, the word-sense task of section 4.4 was run in two conditions with byte-identical text — the only difference being whether two photographs (depicting the referents in the two sentences) were attached. Real item: "We sat on the grassy bank of the slow river." / "She deposited her paycheck at the bank downtown.", with and without a riverbank photo and a bank-building photo. Result: the pictures changed nothing — sense judgments were no better with the photographs than without them (claude rated the bat animal/baseball pair "Unrelated" in both conditions, and so on across the set). A second null followed: words differ in how perceptual they are (published human norms measure this), and if text-only meaning were missing something perceptual, perceptual words' senses should be tracked worse — they are not (no relationship, or if anything a small tilt the wrong way, in one model). The project's reading is careful: for these already-common words, text has saturated what the image would teach; the results say nothing about rarer words or about what grounding contributes during human learning.

The natural next question — not whether but how much headroom images could add, measured on a task where vision genuinely disambiguates — is the project's cleanest documented dead end. The candidate instrument (choose which of ten images matches "mustard" in mustard seed) failed its own competence audit in the fluent-answer format, every alternative instrument required data with unverifiable licenses, and building one in-house would need fresh human ratings, which the ethics rules forbid. The grounding-magnitude question is recorded as open and un-instrumentable with allowed resources — a blank region drawn on the map on purpose, awaiting an outside dataset.

4.10 Meaning between speakers: a deflationary result

The project's most distinctive axis asked about relational meaning — meaning that lives between parties. When two people work together repeatedly, they coin private shorthands ("conceptual pacts," in the psycholinguistic literature): the weird triangle one comes to mean, for these two, that figure. Is anything like that constituted between two AI models?

In a reference game, two model instances alternated describing and identifying abstract 8×8 grid figures, then coined nicknames (real coinage from the raw logs: one dyad settled on "Triangle with stem" for a figure; another figure became "two dots stem"). The probe: give the transcript to a fresh model instance that never played, and ask it to identify the figures from the nicknames. It largely can (the fresh matcher correctly picked the "Triangle with stem" figure) — and, crucially, scrambling the order of the transcript's rounds does not destroy recovery. Everything the "convention" contains, a competent outside reader can extract from the content of the record; nothing measured so far lives irreducibly in the dyad's shared trajectory. One early conjecture in this area — that coined conventions would be order-insensitive in a stronger sense — was falsified and retired when a positive was found: when a nickname is explicitly reassigned mid-conversation, all models follow the most recent assignment, spontaneously and at ceiling (the project's one promoted relational claim, deliberately thin: "latest binding wins").

The deflationary conclusion is recorded with a structural caveat the project regards as important: for text-only agents, a conversation is its transcript — a fully inspectable content record — so the richer kind of path-dependence that human relationships exhibit may be unmeasurable in this medium in principle, not just unobserved. The blank is marked as a limit of the instrument, not a verdict about the models.

4.11 Lessons about the measuring itself

Some of the project's most consequential findings are about how to measure, and they changed its own practice mid-stream.

The answer format is part of the instrument. For weeks, an apparent result stood: only one panel model could compose certain multi-step relational inferences. Then a control varied nothing but the answer format — the same questions, but instead of "Output a single digit … and nothing else," the instruction ended "Think it through step by step. Then, on the final line, output your answer in exactly this format: FINAL: <0, 1, or 2>." Under that format-only change the "missing" capability appeared: gemini went from 0.66 to 1.00, gpt from 0.25 to 0.97. The one-model gap had been an artifact of forcing answers through a one-token channel. The same working-surface effect then lifted two models' accuracy on hard let alone inference items (for claude, from 54% to 79%, with seven wrong-to-right flips against one right-to-wrong). The project rebuilt its standard instruments around this lesson and promoted it as a methodological claim: a benchmark score is a fact about model-plus-format, not model alone. (A side note that says something about the project's licensing discipline: the let alone test sentences come from a dataset with no redistribution license, so the archive stores only cryptographic fingerprints of them — the result is fully reproducible, but this report cannot legally print an example sentence.)

Small samples mislead in both directions. In the dative line, gpt's givenness effect looked alive in one 32-item run, dead in the next, and settled only at 100 items: small but real (+0.056, interval clear of zero). The founding-era habit of dozen-item probes was abandoned mid-project for exactly this reason; every claim-carrying run now uses 100–150 items.

Unanimity hides spread. The panel's directions agree far more often than its magnitudes: the same discourse manipulation moves gemini nine times more than gpt; the same nonce-word firewall moves gemini four times more than gpt. Any evaluation that averages models, or reports only pass/fail, would report a fiction. The project's rule: per-model numbers, always; a shared direction may be stated as shared only alongside its spread.

And one meta-lesson. Even at temperature 0 a model's answer is a draw from a distribution (the providers' own infrastructure introduces nondeterminism), so single runs are treated as draws, replication is mandatory before promotion, and every headline number in this report survived an independent recomputation from raw outputs.