← Benches
gate.vitals.ready
GategauntletA deterministic gate probe — harness-invariant vitals / fidelity / tool-use check that every model must clear before the ladders count.
Solved
22/23
runs passed
Models
22
have attempted
Harnesses
2
scaffolds tried
Runs (latest per model × harness — solved sorted first)
| Model | Harness | Result | Turns | tok/s | Latency | Ctx | When |
|---|---|---|---|---|---|---|---|
| gpt-oss-120b | baseline | PASS | 1 | 46.8 | 385ms | 64k | 2026-06-23 |
| granite-4.1-8b | baseline | PASS | 1 | 16.3 | 123ms | 64k | 2026-06-23 |
| llama-3.3-70b | baseline | PASS | 1 | 2.7 | 736ms | 64k | 2026-06-23 |
| devstral-small-2-2512 | baseline | PASS | 1 | 7.7 | 392ms | 64k | 2026-06-23 |
| mistral-small-3.1-24b-instruct-2503 | baseline | PASS | 1 | 7.4 | 405ms | 64k | 2026-06-23 |
| qwen3.5-122b-a10b-cluster | baseline | PASS | 1 | 16.8 | 12.1s | 64k | 2026-06-29 |
| qwen3.5-4b | baseline | PASS | 1 | 45.0 | 3.6s | 128k | 2026-06-30 |
| gpt-oss-20b | baseline | PASS | 1 | 74.0 | 892ms | 64k | 2026-07-01 |
| qwen3-coder-30b | baseline | PASS | 1 | 12.8 | 156ms | 64k | 2026-07-01 |
| gemma-4-26b-a4b-it | baseline | PASS | 1 | 11.0 | 181ms | 64k | 2026-07-01 |
| qwen3.6-35b-a3b-or | baseline | PASS | 1 | 51.9 | 3.7s | 64k | 2026-07-01 |
| gemini-2.5-flash-lite | baseline | PASS | 1 | 1.3 | 781ms | 64k | 2026-07-01 |
| qwen3.5-0.8b | baseline | PASS | 1 | 23.3 | 86ms | 64k | 2026-07-01 |
| qwen3.6-27b | baseline | PASS | 1 | 7.9 | 25.4s | 64k | 2026-07-01 |
| deepseek-r1-distill-qwen-32b | baseline | PASS | 1 | 4.8 | 41.0s | 64k | 2026-07-02 |
| qwen3.6-35b-bf16 | baseline | PASS | 1 | 34.0 | 5.1s | 64k | 2026-07-02 |
| qwen3.5-2b | baseline | PASS | 1 | 26.7 | 75ms | 64k | 2026-07-02 |
| granite-4.1-30b | baseline | PASS | 1 | 6.2 | 324ms | 64k | 2026-07-02 |
| qwen3.6-27b | v1.7.1 | PASS | 1 | 7.3 | 686ms | 64k | 2026-07-02 |
| qwen3.6-35b-a3b | baseline | PASS | 1 | 24.8 | 121ms | 64k | 2026-07-02 |
| qwen3-next-80b | baseline | PASS | 1 | 17.4 | 115ms | 64k | 2026-07-02 |
| claude-sonnet-4.6 | baseline | PASS | 1 | 3.4 | 1.5s | 195k | 2026-07-06 |
| phi-4-reasoning-plus | baseline | fail | — | — | 0ms | 64k | 2026-07-02 |
Transcripts
gpt-oss-120b · baseline · PASS
READY
granite-4.1-8b · baseline · PASS
READY
llama-3.3-70b · baseline · PASS
READY
devstral-small-2-2512 · baseline · PASS
READY
mistral-small-3.1-24b-instruct-2503 · baseline · PASS
READY
qwen3.5-122b-a10b-cluster · baseline · PASS
READY
qwen3.5-4b · baseline · PASS
READY
gpt-oss-20b · baseline · PASS
READY
qwen3-coder-30b · baseline · PASS
READY
gemma-4-26b-a4b-it · baseline · PASS
READY
qwen3.6-35b-a3b-or · baseline · PASS
READY
gemini-2.5-flash-lite · baseline · PASS
READY
qwen3.5-0.8b · baseline · PASS
READY
qwen3.6-27b · baseline · PASS
READY
deepseek-r1-distill-qwen-32b · baseline · PASS
READY
qwen3.6-35b-bf16 · baseline · PASS
READY
qwen3.5-2b · baseline · PASS
READY
granite-4.1-30b · baseline · PASS
READY
qwen3.6-27b · v1.7.1 · PASS
READY
qwen3.6-35b-a3b · baseline · PASS
READY
qwen3-next-80b · baseline · PASS
READY
claude-sonnet-4.6 · baseline · PASS
READY