Cost
Per 500-instance run
| harness | inference | per resolved | compute (estimated) | wall-clock |
|---|---|---|---|---|
| OpenCode | $26.29 | $0.068 | $4.91 | 1.5 h |
| custom minimal agent | $33.98 | $0.088 | $7.03 | 4.1 h |
| Claude Code | $39.50 | $0.103 | $6.57 | 4.0 h |
| Codex CLI | $30.63 | $0.080 | $9.50 | 5.5 h |
| mini-swe-agent | $32.72 | $0.086 | $7.95 | 5.5 h |
Inference cost is the gateway's metered figure from the provider's own usage, at the provider's list price at the time of the call, including cached-token discounts. Compute is an estimate from recorded usage at list prices, method below; it is not reconciled against the bill, and the per-resolved column is inference only.
Compute, estimated per run
Three ingredients, each recorded during the run, priced at list:
- Harness tasks on Fargate. Every attempt's export carries the seconds ECS billed its task. A harness task is 1 vCPU and 3 GB, which is $0.05382 per task-hour in us-west-2.
- Grading hosts on Spot EC2. The fleet series records the grading slots in use every 30 seconds; 4 grades pack per c5d.2xlarge, priced at $0.17 per host-hour, the middle of the $0.14 to $0.21 Spot range seen in the period.
- The fixed tier. The control-plane services cost about $6 a day while the UI and eval tiers are up; a run carries its wall-clock share.
| harness | wall-clock | Fargate task-h | harness | Spot host-h | grading | fixed | compute | inference |
|---|---|---|---|---|---|---|---|---|
| OpenCode | 1.5 h | 59 | $3.17 | 8.1 | $1.37 | $0.37 | $4.91 | $26.29 |
| custom minimal agent | 4.1 h | 59 | $3.15 | 16.8 | $2.86 | $1.02 | $7.03 | $33.98 |
| Claude Code | 4.0 h | 55 | $2.98 | 15.2 | $2.58 | $1.01 | $6.57 | $39.50 |
| Codex CLI | 5.5 h | 84 | $4.54 | 21.1 | $3.58 | $1.38 | $9.50 | $30.63 |
| mini-swe-agent | 5.5 h | 60 | $3.21 | 19.8 | $3.37 | $1.37 | $7.95 | $32.72 |
Compute is $5 to $10 per 500-instance run against $26 to $40 of inference: between a seventh and a quarter of a run's total, and reported separately so a cheaper model cannot hide a more expensive fleet. Two things the split shows. Codex's slow calls cost real task time, 84 Fargate task-hours against 55 for Claude Code on the same problems, because a task is billed while it waits on the provider. And the OpenCode run was the cheapest in compute because its fleet finished in 1.5 hours; the fixed tier and the grading hosts are paid by the hour, whatever the harness is doing.
What the estimate leaves out: NAT and data transfer, CloudWatch, Aurora's minimum and the registry are estate costs, not per run (they are in the monthly figure below). Spot price moves by the hour, so the grading column carries the range's midpoint. The planner's static cap is 145 concurrent agent tasks (the account's Fargate vCPU quota of 200, at one vCPU per task, less headroom); the 500-runs peaked between 60 and 104 in flight with up to 36 grading tasks on a Spot fleet that follows patch arrivals.
Where an instance's wall-clock goes
Where the wall-clock goes (p50 per instance)
seconds per phase, median over the run's instances
Queue wait is the time an instance waited for a harness slot under the planner's ceiling — it is the fleet size, not the agent.
Queue wait dominates every bar and is a property of the fleet, not the harness: the dispatcher ramps concurrency under the pacer and the account's task quota, so a 500-instance run is a queue by design. Image pull and grading are the same for every harness. Only the agent phase is the scaffold's, and that phase is compared on By harness.
What the whole project cost
Measured from the AWS bill and the OpenRouter account, 2026-08-09 to 2026-09-10:
| bucket | USD |
|---|---|
| AWS, everything, one month | 317 |
| of which the private registry (about 2 TB of instance and harness images) | 20 |
| OpenRouter inference, all runs and all development, all time | 651 |
| of which the five 500-instance runs on this site | 166 |
| coding-assistant subscription for the build (Claude Pro, then Max) | 247 |
| total | 1,215 |
The AWS figure is the originating account. The fresh-account validation ran in a second account whose bill is read from Cost Explorer as its days become available: $3.05 so far for the clean-pass day, 2026-09-12 (reproducibility).
The AWS figure is what one person's account paid for the whole build: NAT and data transfer $73, Fargate $72, Spot EC2 $59, CloudWatch $29, Aurora $26, VPC endpoints $20, the registry $20, load balancers $12. Idle, the estate costs about a dollar a day: the compute tiers are torn down nightly by a guard and brought up per session; what remains is the registry, Aurora at its minimum, and one small NAT instance.
What the per-run cost figures do not include
Model-provider spend for calibration probes (a few dollars per pool), the judge passes ($1.32 for the OpenCode pass, $0.63 for the Codex pass — see Judge), and re-runs after framework fixes. Each is metered under its own key and reported on the run that spent it.
- run
- 01789087148400722169-0c9303b3 · 2026-09-11
- model
- openrouter/minimax/minimax-m2.5 (alias minimax-m2.5-opencode)
- harness
- OpenCode · CLI 1.18.18 (pinned in the harness image build at this framework commit)
- framework
- 631be843b55ccabe33347f0a7de2f7ee0676ce1a
- harness image
- sha256:c8c56129b240e898fb5de452b50e36ad1aedf04fada946592ee98ca757753564
- benchmark
- SWE-bench/SWE-bench_Verified @ 78f471bf655a · swebench 5.0.2 · images SWE-bench_Verified-78f471bf655a.json
- network
- isolated
- explore
- this run in the Explorer: every instance, its trajectory and patch, every inference call, the judge’s verdicts
- download
- this run's artifacts (patches, trajectories, call ledgers, eval reports, judge passes) · preds.jsonl · all runs