Skip to main content

Storage & data model

Three stores, three lifetimes​

storeholdslifetime
Aurora Postgresruns, instance_results (one row per attempt × phase), llm_calls, judge_results, judge_sampling, capacity_snapshot, run_timeline_tick, run_events, operator_limit_edits, model_tpm_observations, pacer_cfg_seedsthe record
Valkeyinstance_progress:* (TTL 300 s), pacer buckets and 60-s counters per alias, the planner's last decision, judge live snapshots, control flagswhile it is happening
S3per attempt: patch, native and normalised trajectory, harness log, eval report, test output, run logthe evidence

Migrations are idempotent ADD COLUMN IF NOT EXISTS statements applied by the results-writer at startup, with a drift check that refuses to start on a schema older than the code.

The attempt row​

One instance_results row per (run, instance, attempt, phase). The harness row carries the patch and trajectory paths, tokens, cost and how the attempt ended; the eval row carries the verdict, the error category and the report path. A rerun adds rows; nothing is overwritten. The dashboard's per-instance bucket is the latest attempt's eval verdict once graded.

The denominators (ADR-0038)​

attempted = every instance the run launched. gradeable = attempts with a verdict plus empty patches. Infrastructure failures and invalid grades are excluded from gradeable and counted separately. Both rates are stored with the run and published together, always.

The export contract this site reads​

GET /runs/{id}/export produces the run JSON (provenance, totals with intervals, timing, integrity, one row per instance) and GET /runs/{id}/timeline the raw ticks, capacity rows, events and calls that scripts/export_run_timeline.py turns into the timeline bundle (manifest with hashes, fleet series, instance lanes, per-instance call files, discovery steps, judge verdicts and report). The exporter is allow-list pruned and deny-list scrubbed — it fails on any account id, ARN, subnet, internal hostname or key id — and the manifest records the scrub result. Everything on this site is one of those files, committed.