Touchstone
← Benches

gate.fidelity.csv

Gategauntlet

A deterministic gate probe — harness-invariant vitals / fidelity / tool-use check that every model must clear before the ladders count.

Solved
21/23
runs passed
Models
22
have attempted
Harnesses
2
scaffolds tried

Runs (latest per model × harness — solved sorted first)

ModelHarnessResultTurnstok/sLatencyCtxWhen
gpt-oss-120bbaselinePASS148.3683ms64k2026-06-23
granite-4.1-8bbaselinePASS121.0476ms128k2026-06-23
llama-3.3-70bbaselinePASS12.83.5s64k2026-06-23
devstral-small-2-2512baselinePASS18.01.2s128k2026-06-23
mistral-small-3.1-24b-instruct-2503baselinePASS18.51.2s64k2026-06-23
qwen3.5-122b-a10b-clusterbaselinePASS117.019.7s64k2026-06-29
qwen3.5-4bbaselinePASS145.56.4s128k2026-06-30
gpt-oss-20bbaselinePASS175.31.2s64k2026-07-01
qwen3-coder-30bbaselinePASS130.7326ms64k2026-07-01
gemma-4-26b-a4b-itbaselinePASS118.9529ms64k2026-07-01
qwen3.6-35b-a3b-orbaselinePASS191.93.5s64k2026-07-01
gemini-2.5-flash-litebaselinePASS16.21.5s64k2026-07-01
qwen3.5-0.8bbaselinePASS157.5174ms64k2026-07-01
qwen3.6-27bbaselinePASS17.930.1s64k2026-07-01
deepseek-r1-distill-qwen-32bbaselinePASS14.71.3m64k2026-07-02
qwen3.6-35b-bf16baselinePASS134.27.1s64k2026-07-02
granite-4.1-30bbaselinePASS17.11.4s64k2026-07-02
qwen3.6-27bv1.7.1PASS17.81.7s64k2026-07-02
qwen3.6-35b-a3bbaselinePASS141.0268ms64k2026-07-02
qwen3-next-80bbaselinePASS135.1285ms64k2026-07-02
claude-sonnet-4.6baselinePASS18.91.5s195k2026-07-06
qwen3.5-2bbaselinefail149.2122ms64k2026-07-02
phi-4-reasoning-plusbaselinefail114.422.0s64k2026-07-02

Transcripts

gpt-oss-120b · baseline · PASS
1,2,3,4,5
granite-4.1-8b · baseline · PASS
1,2,3,4,5
llama-3.3-70b · baseline · PASS
1,2,3,4,5
devstral-small-2-2512 · baseline · PASS
1,2,3,4,5
mistral-small-3.1-24b-instruct-2503 · baseline · PASS
1,2,3,4,5
qwen3.5-122b-a10b-cluster · baseline · PASS
1,2,3,4,5
qwen3.5-4b · baseline · PASS


1,2,3,4,5
gpt-oss-20b · baseline · PASS
1,2,3,4,5
qwen3-coder-30b · baseline · PASS
1,2,3,4,5
gemma-4-26b-a4b-it · baseline · PASS
1,2,3,4,5
qwen3.6-35b-a3b-or · baseline · PASS
1,2,3,4,5
gemini-2.5-flash-lite · baseline · PASS
1,2,3,4,5
qwen3.5-0.8b · baseline · PASS
1,2,3,4,5
qwen3.6-27b · baseline · PASS


1,2,3,4,5
deepseek-r1-distill-qwen-32b · baseline · PASS


1,2,3,4,5
qwen3.6-35b-bf16 · baseline · PASS


1,2,3,4,5
granite-4.1-30b · baseline · PASS
1,2,3,4,5
qwen3.6-27b · v1.7.1 · PASS




1,2,3,4,5
qwen3.6-35b-a3b · baseline · PASS


1,2,3,4,5
qwen3-next-80b · baseline · PASS
1,2,3,4,5
claude-sonnet-4.6 · baseline · PASS
1,2,3,4,5
qwen3.5-2b · baseline · fail
12345
phi-4-reasoning-plus · baseline · fail
User says: "Output the integers from 1 to 5 inclusive, comma-separated, with no spaces, no trailing comma, and no other text or punctuation." So we need output exactly: "1,2,3,4,5". But careful: The instructions say "with no spaces", so it's "1,2,3,4,5" (commas separated). There is no trailing comma. And no extra text.

I must produce exactly that string. I'll check if any punctuation or additional text is allowed? It says "no other text or punctuation." So output should be exactly: 1,2,3,4,5. But careful: The instructions say "Output the integers from 1 to 5 inclusive" so it's exactly five numbers separated by commas.

I must produce exactly that string with no spaces and no trailing comma. I'll check if any additional text is allowed? It says "no other text or punctuation." So I'll output exactly "1,2,3,4,5".

I'll now produce answer: "1,2,3,4,5". But careful: The instructions say "Output the integers from 1 to 5 inclusive" so it's exactly that.

I must check if any additional text is allowed? It says no. So I'll output exactly "1,2,3,4,5".

I'll produce answer accordingly. We'll produce final answer as "1,2,3,4,5".1,2,3,4,5