Same harness, two models
In five lines.
- Two models of similar size and context length, Laguna xs 2.1 and MiniMax M2.5, on the same harness, the same prompt, the same tools, the same 78 SWE-bench Verified problems and the same limits. Laguna is 5× cheaper per uncached input token and 10× cheaper per output token, and the same price per cached input token.
- They resolved the same number of problems: 58 and 59 of 78 on OpenCode, 55 and 58 on mini-swe-agent, intervals overlapping in both pairings.
- Laguna cost 4.4× more per resolved instance on OpenCode and 3.6× more on mini-swe-agent.
- The reason is in the ledger: Laguna made 2.6× the calls and carried 2.9× the context on each, 7.5× the prompt tokens in total, and almost all of it cache reads. Cost is context times calls, and the price sheet prices neither.
- The trajectories say why: Laguna reads whole files before searching, re-reads unchanged files three times as often, and keeps testing after its tests are green. Three of those habits respond to a prompt; the stopping habit does not.
Cost per resolved instance, by model and harness
four runs · each model on each harness · same 78 instances, same limits, no prompt preset · lower is better
Inference spend only, from the gateway's own metering (compute cost is reported separately on the Cost page). Sorted cheapest first. A missing bar is an unmeasured run, not a free one.
The grid
| OpenCode | mini-swe-agent | |
|---|---|---|
| Laguna xs 2.1 | $15.45 · 58 of 78 · $0.266 per resolved | $15.55 · 55 of 78 · $0.283 per resolved |
| MiniMax M2.5 | $3.49 · 59 of 78 · $0.059 per resolved | $4.37 · 58 of 78 · $0.075 per resolved |
Reading across a row: swapping the harness moves cost by 1% to 25% and calls by a quarter. Reading down a column: swapping the model moves cost by a factor of four at a resolve rate inside the noise. The second harness is the check that the finding is not an OpenCode artefact; it reproduced.
Where the money goes
| per 78 attempts, OpenCode, plain prompt | Laguna xs 2.1 | MiniMax M2.5 | ratio |
|---|---|---|---|
| cost | $15.45 | $3.49 | 4.4× |
| LLM calls | 6,514 | 2,527 | 2.6× |
| prompt tokens per call | 76.9K | 26.4K | 2.9× |
| prompt tokens in total | 501M | 67M | 7.5× |
| cached share of prompt tokens | 98.5% | 96.1% | — |
| whole-file reads | 503 | 97 | 5.2× |
| re-reads of an unchanged file | 728 | 245 | 3.0× |
| redundant test runs | 38 | 3 | 12.7× |
| call at which the first edit happened, median | 15 | 6 | 2.5× |
| spend after own tests were green | $4.39 | $1.45 | 3.0× |
Cache reads cost the same on both models, $0.03 per million, and they are 96% of Laguna's bill against about half of MiniMax's. The 5× and 10× discounts apply only to the four percent of Laguna's spend that is new text. There are seven and a half times as many cache reads, and each one is a whole conversation re-sent. The behaviour is the model's: it appears on both harnesses, and the same counters computed the same way on both trajectories agree with the judge's independent attribution (repeated reads, then redundant tests, then probing without searching).
What a prompt bought, and what it did not
Five working rules appended to the same harness cut Laguna's cost by 36% and MiniMax's by 28% on the same 78 problems, with no measurable change in resolve rate. The rules the models obeyed were the reading rules: whole-file reads fell by half, context per call fell, the call count barely moved. The rule both ignored was to verify once and stop; spend after the last edit halved but stayed at a third to a half of the bill. That is the argument for enforcing the stop in the harness rather than asking the model. Details on What a prompt can buy.
What this does and does not show
It shows that on this subset, under these harnesses, the price sheet is a poor predictor of the bill, and that resolve rate cannot separate two models whose behaviour differs by a factor of seven in tokens. It does not rank the models in general: two harnesses, one prompt each, 78 django-heavy instances, one run per cell. Both models resolve the same problems and fail the same problems; the hard core is the same instances in every pairing.
Full tables and provenance: Cheaper per token is not cheaper per task and Model behaviour. Runs A, C, E and F of the 78-instance experiment; raw exports on Downloads.