hermes

The open-source Hermes coding agent (v0.18.0) runs each SupportFlow bench end-to-end: it reads the issue, explores the cloned repo, edits files and runs its tools inside a Docker container, then the gold test grades it.

why it exists: A real third-party agent scaffold on our local models — measures how far a genuine autonomous agent (not our localharness loop) gets on the same real-repo PRs. External-agent calibration alongside openhands/mini-swe-agent.

Models on this harness (gauntlet solved / attempted)