ROOTRACE · judges' review

Pre-audited root-cause racing, with evidence you can re-verify

Nemotron proposes hypotheses; ROOTRACE races pre-audited variants in isolated sandboxes, checks the outputs are identical, and picks the winner with a fixed statistical rule. Nothing on this page is live: it shows recorded evidence, an offline emulation, and an honest placeholder for a future live run.

OFFLINE EMULATIONagentic-f10-3-judges-replay · evaluator-v2.0

Produced by the F10.3 offline test pipeline. No network, no credential, no cost. NOT live.

Label check: this run's provenance file says origin “live_model / nebius” only because the emulation reuses the production code path. Provider fake-nemotron-endpoint, sandboxes emu-sess-NNN: this is NOT a live run.

RECORDED REPLAY · recorded data · no API calls

Showing everything. Replay re-reveals the recorded evidence stage by stage in this browser; it does not execute anything.

Recorded replay progress

Final recorded state of each stage. Nothing is executing.

1. Plan
Completed
4 hypotheses recorded; 3 mapped to a testable mechanism.
2. Approve
Completed
Simulated test-fixture approvals recorded for: live_model, nebius_sandbox. These are not real live-execution authorizations: no live model call and no Nebius sandbox run was approved or made.
3. Execute
Completed
3 of 4 hypotheses executed; 1 skipped (not applicable).
4. Verify
Completed
Output compared with the baseline: 3 identical, 0 differ.
5. Decide
Completed
Recorded winner: H1.
6. Clean up
Completed
Ledger: 6 created, 6 deleted, 0 still open.

“Not recorded” means this frozen record has no entry for the stage — it is not a claim that the stage did or did not happen. “Not run” means the record shows it did not run.

Latency: baseline vs measured candidates

Lower is better — shorter bar = faster

Baseline
unmodified workload
596 ms
H1 ★
recorded winner
225 ms2.65× faster
H2
falsified
592 ms1.01× faster
H3
falsified
588 ms1.01× faster
H4
not run — not applicable
not run

Bars show the median latency of the timed trials. Median of 8 baseline trials and 8 trials per candidate (the same samples feed the ratio and the p-values). Timings come from the build machine's local DuckDB, not from Nebius hardware.

Why a recorded number differs from the bar (4)

Baseline: the recorded p50_ms (583.9 ms) is the lower of the two middle trials; the median shown (595.701 ms) averages them (n = 8, even). Checked against the raw samples.
H1: the recorded p50_ms (222.041 ms) is the lower of the two middle trials; the median shown (224.871 ms) averages them (n = 8, even). Checked against the raw samples.
H2: the recorded p50_ms (588.788 ms) is the lower of the two middle trials; the median shown (591.708 ms) averages them (n = 8, even). Checked against the raw samples.
H3: the recorded p50_ms (579.991 ms) is the lower of the two middle trials; the median shown (588.495 ms) averages them (n = 8, even). Checked against the raw samples.

Nothing was replaced: the bars, the speedup and the statistics all use the same definition; the recorded values above stay in the evidence file.

Result

Winner · H1

OFFLINE EMULATION

Selected by the deterministic evaluator rule.

Mechanism: customer_summary_slow issues a separate query for each of the 501 customers, resulting in 501 queries per call and heavy client-side loop overhead
Speedup: 2.65× vs baseline (median)
Correctness: Output identical to the baseline (primary, held-out and every timed trial).
Statistical eligibility: Eligible: Holm-adjusted p = 0.00282 (< α 0.01) and ratio 2.65× ≥ 2× threshold.

FUTURE LIVE · NOT RUN Future live run: Not run

Audit detail — expand any section

AWhat this evidence iswhat it is and is not, executor, recorded p50

What it is. A real execution of this repository's plan -> approve -> supervised execute -> evaluate -> publish -> clean-up code, with only the two external services replaced: Nemotron by a scripted fixture response, Nebius/ConTree by an emulator that runs the audited workloads locally. Statistics, correctness gate, winner selection, hashes and session ownership are produced by the real code.

What it is not. Not a Nemotron output and not a Nebius measurement. Latencies are the audited DuckDB workloads timed on the build machine, not Nebius timings. Its provenance file states origin 'live_model' / 'nebius' because the emulation reuses the production code path with a non-stub client; that field is NOT evidence of a live run.

recorded baseline p50_ms 584 ms · executor emulated provider · recorded H1 p50_ms 222 ms

HISTORICAL · LIVE NEBIUS
Recorded evidence from a real Nebius sandbox race. Frozen; not running now.
HISTORICAL · LOCAL BACKEND
Recorded evidence from the official race on the local process backend. Frozen; NOT run on Nebius.
OFFLINE EMULATION
Real ROOTRACE code with Nemotron and Nebius replaced by test doubles. No network, no cost. Not live.
FUTURE LIVE · NOT RUN
No live run has been made for this workflow. Placeholder only.

1Nemotron planninghypotheses → pre-audited mechanisms

ROOTRACE selects and measures PRE-AUDITED variants. A model proposal is mapped to one of these mechanisms only if the text supports it; anything else is rejected. patch_diff text from a model is hashed and never executed. ROOTRACE does not write or run arbitrary code.

Model: Scripted response (tests/agents/agentic_fixtures/model_response_mixed.json); provider 'fake-nemotron-endpoint'. The mapping below is that scripted response run through the real mapper.

Model said	Declared mechanism	Outcome	Audited slot	Variant	Test type	Model confidence
H1	per_row_query_fanout	mapped	H1	h1_groupby	direct	0.9
H2	missing_index_lookup	mapped	H3	h3_indexed	direct	0.4
H3	concurrent_dispatch	mapped	H4	none	precondition	0.2
H4	unsupported_other	rejected (unsupported mechanism)	—	none	—	0.1

H1 · direct test
Excessive per-call query fanout due to per-customer fetches
Tested: query fanout eliminated via a single GROUP BY query (customer_summary_fast)
H2 · proxy test
High JSON serialization and network overhead per query
Tested: per-row Python object construction cost, with query count held fixed at the observed 501 queries per call
H3 · direct test
Missing index causing slow query execution on the database side
Tested: per-query execution cost with an index on orders.customer_id, fanout unchanged
H4 · precondition test
Thread pool saturation from launching many concurrent queries
Tested: concurrent/thread-pooled query dispatch

The audited mechanism catalogue (4 entries)

Source: agents/mechanism_vocab.py (closed vocabulary) + agents/variant_catalog.py (hand-audited variants)

per_row_query_fanout → h1_groupby (direct) — one query issued per row/customer (N+1 fanout); fix: a single aggregate query
python_object_marshalling → h2_reduced_marshalling (proxy) — client-side per-row Python object construction / serialisation cost
missing_index_lookup → h3_indexed (direct) — per-query execution cost due to a missing index on the lookup column
concurrent_dispatch → no variant (precondition) — thread-pool / concurrent query dispatch (only applicable if the baseline has concurrency)

2Correctness, statistics, winnerverdict table, raw p, Holm p

Hyp.	Verdict	Output vs baseline	Ratio	Raw p	Holm p	Winner-eligible
H1 ★	supported · significant	identical (primary + held-out + every timed trial)	2.65×	0.000939	0.00282	yes
H2	falsified	identical (primary + held-out + every timed trial)	1.01×	0.958	1.00	no
H3	falsified	identical (primary + held-out + every timed trial)	1.01×	0.958	1.00	no
H4	not applicable	not recorded	—	—	—	—

Evaluator v2: a candidate is winner-eligible only if its output is identical to the baseline's and its Holm-adjusted p is below alpha (family-wise 0.01) and its ratio meets the committed 2.0× threshold. The winner is chosen by this deterministic rule; the model's stated confidence is never used.

H2 is a proxy test: it measures the closest real analogue of the claim; a significant result supports the proxy only.

H4 is a precondition test: its precondition (a thread pool) is absent from the incident, so it was classed not applicable and never executed.

Recorded winner: H1 — selected by the deterministic evaluator, not by the model's confidence.

3Execution timeline and sandboxessession lifetimes, clean-up

session lifetimes · 0 ms35074 ms (build host clock) · peak 4 open at once

#1 7557f60fd0 · checkpoint provisioning (pip install, runner bundle, verify)
#2 b25e5b4110 · checkpoint dogfood verification
#3 9488516458 · baseline fork
#4 867b731bea · fork H1
#5 1e3cbb19c1 · fork H2
#6 e041dcc50e · fork H3

Lane	Sandbox	Status	Clean-up
baseline	emu-sess-003	measured	ok
H1	emu-sess-004	complete	ok
H2	emu-sess-005	complete	ok
H3	emu-sess-006	complete	ok
H4	—	not applicable	no sandbox created

Session ownership ledger: 6 created, 6 deleted, 0 still open, hash chain valid. Sessions are identified by a short digest of their key; raw keys are never published. Lane roles marked “derived” are inferred from creation order, not recorded.

4Provenance and SHA-256 verificationhashes, expected values

Verification runs in your browser (WebCrypto SHA-256, plus the statistics recomputed from the raw samples). Nothing is sent anywhere.

File	SHA-256 (raw bytes)	Size	Verify your copy
replay/offline_emulation/agentic-f10-3-judges-replay/agentic-f10-3-judges-replay.plan.json Plan record (the single model call, parsed and mapped)	171c2482161ba84f96b820ba19347084632c626c13688079f06a8fb943f3b1b0	7,533 B	choose file…
replay/offline_emulation/agentic-f10-3-judges-replay/agentic-f10-3-judges-replay.execution-claim.json Single-use execution claim (bound configuration)	8133593518867d53feef7c4bab0f83c360e4b052bace840622b4d67649116e89	4,009 B	choose file…
replay/offline_emulation/agentic-f10-3-judges-replay/agentic-f10-3-judges-replay.race-evidence.json Race evidence (v2)	c608a6e09d70c578dc2c3c9d88fdabfc7ab0a8ffcfbc4537d18b1d02544e234d	22,954 B	choose file…
replay/offline_emulation/agentic-f10-3-judges-replay/agentic-f10-3-judges-replay.provenance.json Provenance chain	e2050623ed4e2298e758bff1a00da6fb8ad6979d7480e9e18aa0a45caf129e52	8,339 B	choose file…

Computing…

Verified when the bundle was built (Python, not re-run here)

PASS agents.two_phase_loop.verify_two_phase (plan -> claim -> provenance -> evidence chain digests) — evidence_sha256=c608a6e09d70c578dc2c3c9d88fdabfc7ab0a8ffcfbc4537d18b1d02544e234d

# re-verify from the repository (no network)
python scripts/verify_frozen_evidence.py
python scripts/build_judges_bundle.py check

5Limitations of this evidence4 items — read before quoting a number

Provider behaviour (billing, kill semantics, real pip install, real verify.py constants) is NOT exercised by an emulator.
Timings come from the build machine's local DuckDB, not from Nebius hardware.
No spend ceiling exists in code (spend_ceiling_enforced is false in the recorded configuration).
The recorded run configuration has spend_ceiling_enforced: false: no control in this repository bounds money spent on a live run.
The dashboard reads only the files listed under “Provenance”. It starts no run and calls no model or sandbox API.

6What has not happened: future live runnot run

FUTURE LIVE · NOT RUN

Future live run: not run

No live Nemotron call and no live Nebius sandbox run has been made for this version of the workflow. This card is a placeholder: it shows no measurements, no winner and no status because none exist.

python scripts/agentic_plan_execute.py preflight --help (offline readiness checks and blockers)
A live model call and a sandbox run each need their own explicit, hash-bound approval file.
Money is not bounded by any control in this repository: use a provider-side spend alert.