Report, 2026-10-25: newer OpenRouter models
Why this session ran. The owner asked which OpenRouter models the workflows call. They then asked whether newer models at the same price or less would do the work as well, and in particular whether Gemini 3.8 Flash reviews as well as Gemini 3.1 Pro, today's reviewer B. This was a test, not a scheduled run. No entry and no setting changed.
What OpenRouter lists now.
- Two models are newer than the ones in use and cost no more:
- Google's Gemini 4 Argon, at the same input price as Gemini 3.1 Pro and a lower output price;
- OpenAI's GPT-6.1 Sol, at the same input price as GPT-5.6 Terra and a lower output price.
- Two prices in
config/models.mdare out of date: DeepSeek V4 Pro is now about an eighth of its listed price, and Gemini 3.8 Flash costs 10 percent more.
How the reviewers were tested. I reused the 28 test entries from this morning's reviewer experiment. Twenty have errors planted on purpose, and eight are fresh drafts. Each new model got exactly the prompt the pipeline uses. Every issue from every reviewer was then pooled and shuffled, and a Claude Opus call, which did not know which reviewer raised which issue, judged each one right or wrong.
What I found.
- Gemini 3.8 Flash:
- It caught the same number of planted errors as Gemini 3.1 Pro (49 of 50).
- It was just as reliable when it called something blocking.
- On its own it found a few fewer real problems.
- Paired with reviewer A, the panel found every real blocking problem either way.
- It costs about a quarter as much per entry and answers more than twice as fast.
- Gemini 4 Argon found the most: all 50 planted errors and the most real problems overall. It costs about a third less than Gemini 3.1 Pro.
- GPT-6 Luna, the cheapest model tried, was wrong in half its blocking calls, so it should not review.
- Pronunciation: GPT-6.1 Sol transcribed slightly better than GPT-5.6 Terra, the first pronunciation panel member, at about the same cost.
What needs the owner.
- Whether to switch reviewer B to Gemini 3.8 Flash (cheapest) or Gemini 4 Argon (most thorough).
- Whether to move the first pronunciation panel member to GPT-6.1 Sol.
The full figures are in experiments/model-refresh-v1/README.md. The test cost US$1.55, recorded in the budget ledger.
Follow-up: the switch
The owner chose Gemini 4 Argon for reviewer B and GPT-6.1 Sol for the first pronunciation panel member, and asked for the prices in the model table to be brought up to date. All three are done.
- Model table. Every model in the table was checked again against OpenRouter's list today.
- Low-effort reasoning. GPT-6.1 Sol refuses calls with reasoning switched off. The tool that calls OpenRouter now runs it, and the Google models, at low effort.
- Word-list panel. It keeps GPT-5.6 Terra for now. Its band votes should be compared with GPT-6.1 Sol's before it changes.
- Switched-off families. Reviewer B had three kinds of comment switched off because its older models were usually wrong about them: example rules, explanation fields and inflections. Those switch-offs measured other models. I followed the rule the owner set this morning when reviewer A changed model, so they are on again and will be measured afresh. The owner can ask for them to be switched off again.
- Check. One short test call through each new role answered correctly.