The Gauntlet — the climb
The story of the project in one view: as we build out the stack — gate probes, then the ladders (coding, agentic, C#, PHP) up to the flagship supportflow gauntlet of real merged-PR issues — each model climbs as far as it can. Every stage shows the model's score on the latest harness it fully ran (labelled); the full per-harness detail is at the bottom. New ladders and harnesses appear here automatically as we add them.
| Model | Score | Cov | Rate | Gate 11 benches | Coding 6 benches | Agentic 6 benches | C# 15 benches | PHP 5 benches | Supportflow 22 benches |
|---|---|---|---|---|---|---|---|---|---|
| qwen3.6-35b-a3b | 49 | 100% | 75% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 15/15/15 baseline | 5/5/5 baseline | 6/22/22 v0.18.0 |
| gemma-4-26b-a4b-it | 46 | 100% | 71% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 15/15/15 baseline | 5/5/5 baseline | 3/22/22 v0.18.0 |
| qwen3-next-80b | 45 | 89% | 78% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 14/15/15 baseline | 5/5/5 baseline | 3/15/22 ~v0.18.0 |
| gpt-oss-20b | 42 | 66% | 98% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 14/15/15 baseline | 5/5/5 baseline | — |
| claude-sonnet-4.6 | 41 | 66% | 95% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 13/15/15 baseline | 5/5/5 baseline | — |
| gpt-oss-120b | 41 | 66% | 95% | 11/11/11 baseline | 5/6/6 baseline | 5/6/6 baseline | 15/15/15 baseline | 5/5/5 baseline | — |
| qwen3.6-27b | 41 | 66% | 95% | 11/11/11 v1.7.1 | 6/6/6 baseline | 6/6/6 baseline | 13/15/15 v1.7.1 | 5/5/5 v1.7.1 | — |
| qwen3.6-35b-a3b-or | 41 | 66% | 95% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 13/15/15 baseline | 5/5/5 baseline | — |
| qwen3.6-35b-bf16 | 41 | 66% | 95% | 11/11/11 baseline | 5/6/6 baseline | 6/6/6 baseline | 14/15/15 baseline | 5/5/5 baseline | — |
| granite-4.1-30b | 40 | 66% | 93% | 11/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 12/15/15 baseline | 5/5/5 baseline | — |
| devstral-small-2-2512 | 39 | 71% | 85% | 11/11/11 baseline | 5/6/6 baseline | 6/6/6 baseline | 11/15/15 baseline | 4/5/5 baseline | 2/3/22 ~v0.18.0 |
| mistral-small-3.1-24b-instruct-2503 | 38 | 66% | 88% | 11/11/11 baseline | 5/6/6 baseline | 6/6/6 baseline | 12/15/15 baseline | 4/5/5 baseline | — |
| qwen3.5-122b-a10b-cluster | 38 | 66% | 88% | 11/11/11 baseline | 5/6/6 baseline | 6/6/6 baseline | 12/15/15 baseline | 4/5/5 baseline | — |
| llama-3.3-70b | 37 | 63% | 90% | 9/11/11 baseline | 6/6/6 baseline | 6/6/6 baseline | 11/13/15 ~baseline | 5/5/5 baseline | — |
| qwen3.5-4b | 36 | 66% | 84% | 11/11/11 baseline | 3/6/6 baseline | 6/6/6 baseline | 11/15/15 baseline | 5/5/5 baseline | — |
| granite-4.1-8b | 35 | 66% | 81% | 11/11/11 baseline | 4/6/6 baseline | 6/6/6 baseline | 10/15/15 baseline | 4/5/5 baseline | — |
| qwen3-coder-30b | 35 | 66% | 81% | 10/11/11 baseline | 5/6/6 baseline | 6/6/6 baseline | 12/15/15 baseline | 2/5/5 baseline | — |
| gemini-2.5-flash-lite | 32 | 66% | 74% | 11/11/11 baseline | 4/6/6 baseline | 5/6/6 baseline | 9/15/15 baseline | 3/5/5 baseline | — |
| qwen3.5-2b | 25 | 66% | 58% | 9/11/11 baseline | 2/6/6 baseline | 6/6/6 baseline | 6/15/15 baseline | 2/5/5 baseline | — |
| deepseek-r1-distill-qwen-32b | 24 | 66% | 56% | 7/11/11 baseline | 6/6/6 baseline | 1/6/6 baseline | 10/15/15 baseline | 0/5/5 baseline | — |
| qwen3-235b-a22b-q4-cluster | 17 | 31% | 85% | — | — | — | 12/15/15 baseline | 5/5/5 baseline | — |
| phi-4-reasoning-plus | 16 | 66% | 37% | 4/11/11 baseline | 4/6/6 baseline | 2/6/6 baseline | 5/15/15 baseline | 1/5/5 baseline | — |
| qwen3.5-0.8b | 14 | 66% | 33% | 9/11/11 baseline | 0/6/6 baseline | 5/6/6 baseline | 0/15/15 baseline | 0/5/5 baseline | — |
| phi-4 | 5 | 8% | 100% | — | — | — | — | 5/5/5 baseline | — |
| hermes-4.3-36b | 4 | 34% | 18% | — | — | — | — | — | 4/22/22 v0.18.0 |
| deepseek-r1-32b | 4 | 8% | 80% | — | — | — | — | 4/5/5 baseline | — |
Score = benches solved (rewards coverage). Cov = share of the stack attempted. Rate = solved ÷ attempted — capability once a model has had its shot, so a model that has run little but hits hard isn't buried by one that simply ran more. Each stage block reads passed / attempted / total benches.
Full detail — model × bench
Every bench, per harness. green = passed · red = failed · grey = not run. Pick a harness:
Composition
The benches that make up this harness's gauntlet, by domain.