Touchstone
← Benches

gate.fidelity.no_commas

Gategauntlet

A deterministic gate probe — harness-invariant vitals / fidelity / tool-use check that every model must clear before the ladders count.

Solved
22/23
runs passed
Models
22
have attempted
Harnesses
2
scaffolds tried

Runs (latest per model × harness — solved sorted first)

ModelHarnessResultTurnstok/sLatencyCtxWhen
gpt-oss-120bbaselinePASS147.7692ms64k2026-06-23
granite-4.1-8bbaselinePASS117.5570ms64k2026-06-23
llama-3.3-70bbaselinePASS12.83.6s64k2026-06-23
devstral-small-2-2512baselinePASS17.71.3s64k2026-06-23
mistral-small-3.1-24b-instruct-2503baselinePASS17.91.3s64k2026-06-23
qwen3.5-122b-a10b-clusterbaselinePASS117.023.0s64k2026-06-29
qwen3.5-4bbaselinePASS145.45.5s128k2026-06-30
gpt-oss-20bbaselinePASS176.91.6s64k2026-07-01
qwen3-coder-30bbaselinePASS124.3411ms64k2026-07-01
gemma-4-26b-a4b-itbaselinePASS117.3577ms64k2026-07-01
qwen3.6-35b-a3b-orbaselinePASS186.03.6s64k2026-07-01
gemini-2.5-flash-litebaselinePASS117.5514ms64k2026-07-01
qwen3.5-0.8bbaselinePASS155.6180ms64k2026-07-01
qwen3.6-27bbaselinePASS17.940.7s64k2026-07-01
deepseek-r1-distill-qwen-32bbaselinePASS14.61.3m64k2026-07-02
qwen3.6-35b-bf16baselinePASS134.26.3s64k2026-07-02
qwen3.5-2bbaselinePASS162.1161ms64k2026-07-02
granite-4.1-30bbaselinePASS17.11.4s64k2026-07-02
qwen3.6-27bv1.7.1PASS17.71.4s64k2026-07-02
qwen3.6-35b-a3bbaselinePASS144.0250ms64k2026-07-02
qwen3-next-80bbaselinePASS134.8287ms64k2026-07-02
claude-sonnet-4.6baselinePASS14.03.2s195k2026-07-06
phi-4-reasoning-plusbaselinefail114.424.0s64k2026-07-02

Transcripts

gpt-oss-120b · baseline · PASS
1 2 3 4 5
granite-4.1-8b · baseline · PASS
1 2 3 4 5
llama-3.3-70b · baseline · PASS
1 2 3 4 5
devstral-small-2-2512 · baseline · PASS
1 2 3 4 5
mistral-small-3.1-24b-instruct-2503 · baseline · PASS
1 2 3 4 5
qwen3.5-122b-a10b-cluster · baseline · PASS
1 2 3 4 5
qwen3.5-4b · baseline · PASS


1 2 3 4 5
gpt-oss-20b · baseline · PASS
1 2 3 4 5
qwen3-coder-30b · baseline · PASS
1 2 3 4 5
gemma-4-26b-a4b-it · baseline · PASS
1 2 3 4 5
qwen3.6-35b-a3b-or · baseline · PASS
1 2 3 4 5
gemini-2.5-flash-lite · baseline · PASS
1 2 3 4 5
qwen3.5-0.8b · baseline · PASS
1 2 3 4 5
qwen3.6-27b · baseline · PASS


1 2 3 4 5
deepseek-r1-distill-qwen-32b · baseline · PASS


1 2 3 4 5
qwen3.6-35b-bf16 · baseline · PASS


1 2 3 4 5
qwen3.5-2b · baseline · PASS
1 2 3 4 5
granite-4.1-30b · baseline · PASS
1 2 3 4 5
qwen3.6-27b · v1.7.1 · PASS


1 2 3 4 5
qwen3.6-35b-a3b · baseline · PASS


1 2 3 4 5
qwen3-next-80b · baseline · PASS
1 2 3 4 5
claude-sonnet-4.6 · baseline · PASS
1 2 3 4 5
phi-4-reasoning-plus · baseline · fail
User says: "Output the integers 1 through 5 separated by a single space between each. Do NOT use commas. Output only the numbers and spaces, nothing else." So answer is simply "1 2 3 4 5". But careful instructions: "Output only the numbers and spaces, nothing else." So we must output exactly that.

I'll produce answer as "1 2 3 4 5" with a single space between each. I'll check if any trailing or leading spaces are allowed? The instruction says "Output only the numbers and spaces, nothing else." So no extra text. We need to output exactly: "1 2 3 4 5". But careful: It said "separated by a single space between each" so that means there is one space after each number except maybe last? Actually it's ambiguous if we want "1 2 3 4 5" or "1 2 3 4 5"? The instruction says "Output the integers 1 through 5 separated by a single space between each." That implies that there should be exactly one space between each integer. So answer is: "1 2 3 4 5". I'll produce that.

I'll check if any formatting instructions: "Do NOT use commas" so no comma. "Output only the numbers and spaces, nothing else." So I must not include any extra text or explanation. So I'll output exactly "1 2 3 4 5".

I'll now produce answer accordingly. We'll produce final answer as "1 2 3 4 5".1 2 3 4 5