IMR R&D · project journal

Teaching small local models to do senior-dev work

No bigger GPUs. No cloud model. Just disciplined engineering wrapped around a weak model running on hardware we already own — and a real production codebase to prove it on.

The thesis

The bet is simple and slightly heretical: a small, locally-hosted model can produce work approaching a frontier system — if you build the right scaffolding around it. Automate the ceremony of a task, gate every change behind hard validation, and let the harness keep a weak model on rails. The north star is sonnet-level output from a weak local model through engineering, not a bigger bill.

The proving ground

We don't grade on toy problems. The bench is supportflow — a live production Laravel ticketing app — and the grader is the app's own PHPUnit suite: take a real bug, run the failing test, and the model has to make it pass without breaking anything else. 47 of these fail→pass benches, each a real merged fix. Inference runs on our own GX10; a frontier model sits in as the ceiling control. The only variable that moves is the model and the harness around it.

The turning point: a rock vs a scalpel

Early results looked like “weak model syndrome” — the models just couldn't do it. That conclusion was wrong, and catching it was the breakthrough. The bottleneck wasn't the model; it was the tools we'd handed it. Our harness truncated every file read at 4,000 characters — the model literally never saw code past the top of a real file — and its only way to change anything was to rewrite the entire file. Ask a weak model to regurgitate a 300-line file to change one line and it will drop something every time.

Frontier coding agents carry a laser-guided scalpel — targeted edits, ranged reads, grep. We'd given our models a crumbling rock through a pinhole and blamed them for missing.

So we rebuilt the toolkit: grep to find code without reading the tree, a surgical edit that swaps an exact string (and refuses if it isn't unique), and full, line-numbered reads. The result was immediate and a little startling: a 4-billion-parameter model performed a real production fix in six steps. The same surgical bug that the “rock” couldn't crack was solved by the 4B, the 35B and the 80B alike. The floor moved decisively — it was never the models.

The cap that was hiding the wins

Next jump came from an unglamorous bug: the output limit was pinned at 8K tokens, quietly guillotining any fix mid-write. Lifting it to 16K took our lead model from 9/47 to 29/47 solved in one shot — same model, same prompts. A reminder that with weak models, the plumbing matters as much as the prompt.

Context engineering & keeping a weak model on rails

Small models have small, fragile context windows, so the harness now treats context as a managed resource: it uses the model's actual verified window rather than a hopeful config number, splits each turn's budget dynamically between input and output, compacts history only when near the limit, and bails out with a clear warning if the window is simply too small to work in — no wasted runs.

On top of that sit the rails that stop a weak model from defeating itself: a forced-commit nudge so it stops exploring and starts editing, a circling fast-fail that kills repeat-action loops in seconds, an edit-over-rewrite guard, rich test feedback so it sees why it failed, and best-of-N retries that let a test oracle pick the winning attempt.

Where it stands today

Our lead model (a lean 35B mixture-of-experts) sits at 29/47 one-shot and 33/47 with best-of-3 at a 128K window. There's a clean model-size signal — the 4B lands around 10/47 on the same harness — which is exactly what we want to see: the scaffolding lifts everyone, and capability still scales with the model.

And an honest limit: the hardest benches cluster in one gnarly multi-file subsystem, and there the lead model stalls even when handed the exact failing test. We read the full transcripts to be sure — it explores forever rather than committing to a fix it can't reason out. That's a genuine capability wall, not a harness gap. No prompt tweak makes a 35B write a fix it can't figure out; that corner needs a stronger model or planning / sub-agent scaffolding. We'd rather report that plainly than dress it up.

Where it's going

Coding is the entry point, not the destination — it's familiar territory to prove the approach. The real goal is a multi-role agent harness: running our own local models as proper agents across roles. The pieces are already converging. localharness is the capability inside the worker; the Hermes × Paperclip adapter is the employment layer that runs that worker as a managed employee in a company — lifecycle, identity, cost, delivery. Senior-dev output today; a small local workforce next.

The picture

Project journal · updated as the work moves.