Domains

Each domain is a slice of the gauntlet — what it tests, how it grades, and which models stand out.

Gate

Prove a model can be trusted with the basics before harder tiers count.

Deterministic single- and multi-step prompts graded mechanically (vitals, tools, fidelity).

Can the model write and fix code that actually runs.

Generated code executed against fixed test cases; pass = tests green.

Can the model drive a multi-step tool loop to a goal.

Agent runs against scripted tool environments, graded on task completion.

Calibration against the public SWE-bench style.

FAIL->PASS: apply a fix, the repo's own tests must go green. Calibration sidebar.

The flagship: senior-dev work on a live production app.

FAIL->PASS graded by the app's own PHPUnit suite -- fix real issues without breaking anything else.

Domain ladder: real .NET work.

Graded by dotnet test.

Domain ladder: real PHP work.

Graded by phpunit.

Uncategorised / legacy benches.

Various.