Rerun a run from its receipt

This cookbook reruns one run from its receipt, on the code it ran on, and records the rerun as a check of that run.

Weeks later, someone asks whether seed 2 still gives the same numbers, and the code has changed since. The run’s receipt says exactly what ran and on which commit, so the rerun is a lookup, not a dig through history.

Paste this into your agent

Paste this into your agentCopy

Rerun seed 2 from its receipt, on the code it ran on, and check it.

Rerun seed 2 of int8-ptq from its Rabbit Brain receipt, and tell me if it still gives the same numbers.

- Run `rb context --model <your model id>` first.
- Read the run's receipt with `rb show <run id>`: the command, and the commit it ran on.
- Check out that commit in a worktree inside this repository (`git worktree add .worktrees/rerun <commit>`), and run the receipt's command there through `rb journal run`.
- Record the new run as a check of the old one: `--repeats <run id> --again --why "rerun from its receipt"`, with `--commit`.

Finish with `rb show` of the criterion exactly as printed, and what rb says of the repeat.

The repository: the INT8 ResNet-20 evaluation, recorded in an earlier session with each seed run through rb journal run. One commit later, eval.py builds its output path differently.

What you’ll see

$ rb show top1-drop-budget
Criterion top1-drop-budget on int8-ptq (primary, own): INT8 loses at most 0.5 points of top-1 against FP32
  top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
          criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
          reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    loss: the interval clear of 0 on the worse side (higher is better)
          rule:      criterion met: estimate -0.08 ≥ -0.5
          timing:    declared before the evidence (by production times)
          units:     3 seeds from 4 runs, 1 repeat
          integrity: ok
          · runddrk2x repeats runo6q2kt: the same number, a check that holds
  runfgc6vp: difference -0.11 (file) · unit seed=1 · seed=1
  runo6q2kt: difference -0.08 (file) · unit seed=2 · seed=2
  runifsbew: difference -0.05 (file) · unit seed=3 · seed=3
  runddrk2x: difference -0.08 (file) · unit seed=2 · repeats runo6q2kt · seed=2

top1-drop-budget is the criterion’s id: its name in every command. The rerun joins seed 2’s unit and adds no unit of its own: the criterion still rests on 3 seeds, now “from 4 runs, 1 repeat”. rb says the check holds: the same number, a check that holds.

What rb did, step by step

1. Read what it takes to run the experiment again

   $ rb show int8-ptq
   Experiment int8-ptq: INT8 post-training quantization · open · under q-int8
     design not confirmed · list it to confirm: rb confirm int8-ptq
     hypothesis or purpose: not recorded
     why: not recorded
     variants: fp32 (baseline) · int8 (candidate)
     varies: precision, calib_images
     spec 85201d31eae8 · frozen 2026-11-06T20:25:37.417+00:00 (85201d31eae8) · note: before the runs · not confirmed
     settings:
       optim.lr                 0.1            verified · configs/train.yaml#optim.lr
       optim.momentum           0.9            verified · configs/train.yaml#optim.momentum
       data.batch_size          128            verified · configs/train.yaml#data.batch_size
       eval.batch_size          256            verified · configs/train.yaml#eval.batch_size
       seed                     (each run)     per-run · no source
       fp32.precision           fp32           verified · configs/fp32.yaml#precision
       int8.precision           int8           verified · configs/int8.yaml#precision
       int8.calib_images        512            verified · configs/int8.yaml#calib_images
     criteria:
     top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
             criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
             reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
             result:    loss: the interval clear of 0 on the worse side (higher is better)
             rule:      criterion met: estimate -0.08 ≥ -0.5
             timing:    declared before the evidence (by production times)
             units:     3 seeds
             integrity: ok
     runs: runfgc6vp, runo6q2kt, runifsbew
     to run it again:
       code         ok       commit 12ec61a726
       environment  ok       python 3.14.4
       files        ok       all 3 files the runs named are here unchanged

2. Read the run’s receipt

   The command, which rb watched make these numbers, and the commit the code was at:

   $ rb show runo6q2kt
   Run runo6q2kt on int8-ptq (file)
     produced 2026-11-06 20:25 UTC (its file's time) · recorded 2026-11-06 20:25 UTC
     fp32.top1 = 91.66
     int8.top1 = 91.58
     per-run: seed=2
     command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-06)
     code that made it: 12ec61a726fe (as given)
     repository when recorded: 12ec61a726fe
     config results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
     file results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here

3. See what changed since

   $ git diff 12ec61a -- eval.py
   diff --git a/eval.py b/eval.py
   index 35a1cb3..901a798 100644
   --- a/eval.py
   +++ b/eval.py
   @@ -7,7 +7,7 @@ p = argparse.ArgumentParser()
    p.add_argument("--seed", type=int, required=True)
    a = p.parse_args()
    fp32, int8 = TOP1[a.seed]
   -out = f"results/int8-ptq/seed{a.seed}"
   +out = os.path.join("results", "int8-ptq", f"seed{a.seed}")
    os.makedirs(out, exist_ok=True)
    with open(f"{out}/metrics.json", "w") as f:
        json.dump({"fp32": {"top1": fp32}, "int8": {"top1": int8}}, f, indent=2)

4. Check out the receipt’s commit in a worktree

   A worktree inside the repository, with no record of its own, writes to the repository’s record, and each run there keeps the worktree’s code on its receipt.

   $ git worktree add -q .worktrees/rerun 12ec61a

   Then, in .worktrees/rerun, the briefing says so first:

   $ rb context
   # The record: INT8 ResNet-20 on CIFAR-10
   
   This worktree (detached at 12ec61a726fe · ~/int8-rerun/.worktrees/rerun) has no record of its own: rb writes to the record in main · ~/int8-rerun, and each run keeps this worktree's code.
   
   … (94 more lines)

5. Run the receipt’s command

   $ rb journal run -- python eval.py --seed 2
   {"metrics": {"fp32.top1": 91.66, "int8.top1": 91.58}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: fp32.top1=91.66, int8.top1=91.58. rb reconcile proposes no new run: a recorded run of int8-ptq holds these numbers.
   
   Next: rb reconcile

6. Record it as a check of the run

   The output is the same, byte for byte, as the first run’s, so rb would read it as the same run. --again --why records it as a second execution, and --repeats makes it a check of seed 2, not a new seed.

   $ rb run add int8-ptq --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml --command "python eval.py --seed 2" --commit 12ec61a726feda8c90dbba424b07ee7ec0b9d32c --repeats runo6q2kt --again --why "rerun from its receipt"
   runddrk2x on int8-ptq: fp32.top1=91.66, int8.top1=91.58 · seed=2
     Ran with the spec: 5 of its settings found in .worktrees/rerun/results/int8-ptq/seed2/config.yaml, all matching.
     It repeats runo6q2kt: a check of that run, in its unit, never a new one.
     Produced: 2026-11-06 20:25:40 (file)
     top1-drop-budget criterion met · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
   
   Next: rb show top1-drop-budget

   $ rb show runddrk2x
   Run runddrk2x on int8-ptq (file)
     produced 2026-11-06 20:25 UTC (its file's time) · recorded 2026-11-06 20:25 UTC
     fp32.top1 = 91.66
     int8.top1 = 91.58
     per-run: seed=2
     command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-06)
     code that made it: 12ec61a726fe (as given)
     repository when recorded: 12ec61a726fe
     config .worktrees/rerun/results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
     file .worktrees/rerun/results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here
     note: recorded again: rerun from its receipt. · not confirmed

Why it matters

A receipt turns “can you rerun that?” into one command on one commit, and the rerun lands as a check of the run, not as an extra seed.