The results

Gemini 3.5 Flash-Lite's fact-recall score comes from an automated keyword scan rather than a human read.

Both drawing tasks carry two grades: a machine count of the elements or rules a picture actually satisfies, and my own rank for how it looks. The strict drawing shows why both are needed — twelve of the thirteen models satisfy all ten rules in every run, and I still rank them from first to last. Identical scores, visibly different pictures. A drawing can satisfy every requirement and still be the worse one.

Reading the table. The headline number is a model's average across four separate runs, with its worst run beside it. Averaging is what makes columns comparable — it does not drift as you run a model more times, and it still works where a column has three runs rather than four. The worst run is the number to read if you plan to take an answer unchecked. Claude Opus 5 leads on both, averaging 123.2 of 124 and never dropping below 121; DeepSeek and Gemini 3.5 Flash-Lite give up the most ground — almost always by repeating one small mistake across runs rather than by failing broadly. Counting whole tasks a model kept clean in every run, out of 18: Opus 5 17, Sonnet 17, Qwen 17, GPT-5.6 Sol 17, Opus 4.8 16, Fable 16, Gemini 3.6 Flash 16, DeepSeek V4-Flash 16, Grok 15, DeepSeek V4 15, GPT-5.6 Terra 14, GPT-5.6 Luna 13, Gemini 3.5 Flash-Lite 12. But the newest and strongest column still fails the sentence-word-count question in every single run: twelve of the thirteen models answer it wrong all four times, and the one model that ever gets it right is the weakest overall. Every model finds all four planted contradictions with zero false alarms except Gemini 3.5 Flash-Lite, which never finds more than three, and one GPT-5.6 Terra run that found three of four. Fifteen of the twenty-four machine-graded tasks carry at least one of these gaps — the run-by-run record shows them run by run. None of it is about which model reasons hardest; it's about where each one's carefulness runs out, and how consistently.

How to read the ordering

The table is ordered by the average run, not the worst. That is a deliberate change. A worst-run score is the minimum of however many times I ran the model, so it keeps sliding downward the more carefully a model is measured — scoring this same data as three runs instead of four lifts DeepSeek by nearly three points. A number that moves with the sample size is not a property of the model, and it cannot be compared against a column run a different number of times.

The worst run is still worth publishing, and it is still in every cell, because it answers a different question: not how good is this model but how bad does it get. Set the two side by side and the distance between them is the interesting part:

The gap is the column worth staring at: it is the distance between a model's typical work and its bad day. DeepSeek loses eleven points to it and Grok 4.5 seven and a half — both do markedly better work than their worst run implies, and are markedly less dependable. Sonnet 5 and Qwen lose barely one: what they manage once, they manage every time. Claude Opus 4.8 is the clearest case — third on average, sixth on its worst run. The same model, six places apart depending on which question you ask. Ordered by worst run, as this table was until 2026-07-29, it read as the weaker model. It is not weaker; it is streakier.

Two more things before quoting any of this. Four runs is enough to catch a coin-flip but not enough to separate models a few points apart — treat the leaders as a group rather than a ranking; the gap between first and fourth here is roughly one task. And every model ran at its own out-of-the-box setting, which is not the same setting for everyone: Sonnet 5 did this at Medium effort while Fable 5 and Opus 4.8 ran at High.