Start tracking a project already under way

This cookbook records where a repository’s work stands, from its own files, in one agent turn.

You have weeks of work in a repository: configs, a plan, a results folder with a report, and no record of any of it. You want the record to start from what the files say, not from what anyone remembers.

Paste this into your agent

After rb init "<the work's theme>" in your own terminal, paste the line it prints. rb does the rest of the briefing:

Paste this into your agent

Record where this repository's work stands now, from its own files.

This is the line rb init prints, word for word.

Use Rabbit Brain to track this: read rb context, then record where the work stands.

The repository we ran it on holds an INT8 post-training quantization of a ResNet-20 on CIFAR-10: configs/ for training, FP32 and INT8; a plan in docs/plan.md (“INT8 ships if it loses at most 0.5 points of top-1 against FP32.”); and results/int8-ptq/ with three seeds and a report. It has two commits, both from before the record.

What you’ll see

rb status in your own terminal, once the agent is done:

$ rb status
INT8 ResNet-20 on CIFAR-10 · int8-resnet-20-on-cifar-10
Kept with rb since 2026-11-06. All 3 runs were recorded afterwards, not as they happened.
Last reconcile, 2026-11-06 20:25: 1 re-indexed (int8-ptq) · 1 result quoted · 1 next step quoted

Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10? (open)
  INT8 loses at most 0.5 points of top-1 against FP32. change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb); loss: the interval clear of 0 on the worse side (higher is better); criterion met: estimate -0.08 ≥ -0.5; rule and question link declared after the evidence existed: all 3 units were produced before it was declared. units: 3 seeds.
  experiment context: int8-ptq · INT8 post-training quantization
          result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
          next: "Try per-channel calibration with 128 images." (README.md:11)
          note neam7zl: The question is docs/plan.md:3 turned into a question: "INT8 ships if it loses at most 0.5 points of top-1 against FP32." · not confirmed
          git clean

Rules: 1 met

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - int8-ptq: 1 criterion was declared after its evidence existed

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-06 20:25 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
          criterion met · loss · change.top1 ≥ -0.5
          reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    loss: the interval clear of 0 on the worse side (higher is better)
          rule:      criterion met: estimate -0.08 ≥ -0.5
          timing:    declared after the evidence existed: all 3 units were produced before it was declared
          units:     3 seeds

Experiments
  int8-ptq INT8 post-training quantization · open · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · not frozen · settings 7 verified · 0 provisional · 0 unknown · 1 per-run · runs 3
          to run again: code unknown · environment unknown · files ok

Runs: 3 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm

The line under the title says when the record began, and that all three runs were recorded afterwards, not as they happened.
The question is the plan’s line turned into a question, and a note on the experiment says so.
top1-drop-budget is the criterion’s id: its name in every command.
The criterion is the plan’s. INT8 is 0.08 points of top-1 below FP32, the 95% interval runs from about 0.15 to 0.005 points below, over 3 seeds, and the rule (change.top1 ≥ -0.5) is met.
Its timing reads declared after the evidence existed: the three runs were made before the record began, and rb never reads them as if the criterion came first.
The report’s own result and next step are quoted word for word, with file and line.
“to run again: code unknown · environment unknown”: these runs were made before rb was watching, so their receipts cannot say which code or machine made them.

The experiment’s design is the agent’s until you confirm it:

$ rb confirm
Designs waiting for a confirmation (1):
  int8-ptq: INT8 post-training quantization
    questions: q-int8
    hypothesis or purpose: not recorded
    why: not recorded
    variants: fp32 (baseline), int8 (candidate) · varies precision, calib_images
    setting optim.lr: 0.1 from configs/train.yaml#optim.lr
    setting optim.momentum: 0.9 from configs/train.yaml#optim.momentum
    setting data.batch_size: 128 from configs/train.yaml#data.batch_size
    setting eval.batch_size: 256 from configs/train.yaml#eval.batch_size
    setting seed: per run
    setting fp32.precision: "fp32" from configs/fp32.yaml#precision
    setting int8.precision: "int8" from configs/int8.yaml#precision
    setting int8.calib_images: 512 from configs/int8.yaml#calib_images
    criterion top1-drop-budget (primary): change.top1 ≥ -0.5 · for q-int8
    confirm: rb confirm int8-ptq@775110fdf0ef6a94a6d1aa30ed23b790454a0151cf4aa4f991f701e30af6773c
    note neam7zl on int8-ptq stays not confirmed: a note is not part of the design. To say one as yours, write it in your own terminal: rb note add "<its words>" -e int8-ptq, then rb retract <note> --why "<reason>".

$ rb confirm int8-ptq@775110fdf0ef6a94a6d1aa30ed23b790454a0151cf4aa4f991f701e30af6773c
int8-ptq's design is confirmed.
  note neam7zl on int8-ptq stays not confirmed: a note is not part of the design. To say one as yours, write it in your own terminal: rb note add "<its words>" -e int8-ptq, then rb retract <note> --why "<reason>".

Next: rb status

What rb did, step by step

Read the briefing

rb context opens with what to do now, and points at rb docs --start, the protocol for a project already under way:

$ rb context
# The record: INT8 ResNet-20 on CIFAR-10

## First, record where the work stands

Nothing is recorded yet, and this project holds work already done: 2 commits and 1 results folder. Record where the work stands now, in this turn, from the files, and ask the researcher nothing first. How: rb docs --start.

… (53 more lines)

Read what the repository already says

rb reconcile looks for results in folders named for them (results/, eval/, metrics/, reports/, artifacts/ and the like), down to four levels from the top. It reads each results folder that has a report, and quotes it word for word. Under “Not read” it names a folder it passed over that holds results files of its own. Where each run keeps a folder with its own config, it says what each config records (the command, the commit, dirty: true, the settings that differ between folders) and prints the rb run add line for each. It names each results file in the top folder too: a table (results.tsv, results.csv, results.jsonl and the like) with the --where line that reads a row, and a report in prose (RESULTS.md, report.md), from which rb reads no run. It writes nothing.
Terminal window
```
$ rb reconcile
The repository at 93d9471: 1 results folder.

int8-ptq  results/int8-ptq/  new, not in the record
  title: "INT8 post-training quantization" (README.md:1)
  says: "Three seeds of FP32 and INT8, evaluated with `python eval.py --seed <N>`." (README.md:3)
  result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
  next: "Try per-channel calibration with 128 images." (README.md:11)
  files: 1 report · 6 results files
  git: all 7 report and results files are tracked · folder unchanged since the last commit

rb reconcile --apply keeps this in the record: 1 experiment to add.
```

Record the question and the experiment

$ rb question add "Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10?" --id q-int8
q-int8: Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10?

Next: rb experiment add "<what would answer it>" -q q-int8 --hypothesis "<what you expect>"

$ rb experiment add "INT8 post-training quantization" --id int8-ptq -q q-int8 --baseline fp32 --candidate int8 --varies precision,calib_images
Experiment int8-ptq: INT8 post-training quantization · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · under q-int8
  Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
  Made 2 variants: fp32 (baseline), int8 (candidate); a setting of one is rb spec set int8-ptq fp32.<setting> <value>.
  int8-ptq declares it varies precision, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.precision <value> --source <file#key>
  int8-ptq declares it varies calib_images, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.calib_images <value> --source <file#key>

Next: rb spec set int8-ptq --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e int8-ptq -q q-int8 --metric <name> --at-most <x>

Cite every setting to the file that states it

$ rb spec set int8-ptq --from configs/train.yaml --keys "optim.*,data.*,eval.*"
int8-ptq: 4 settings from configs/train.yaml @93d9471 · 4 verified

$ rb spec set int8-ptq fp32.precision fp32 --source "configs/fp32.yaml#precision"
int8-ptq/fp32.precision = fp32 · verified in configs/fp32.yaml#precision

$ rb spec set int8-ptq int8.precision int8 --source "configs/int8.yaml#precision"
int8-ptq/int8.precision = int8 · verified in configs/int8.yaml#precision

$ rb spec set int8-ptq int8.calib_images 512 --source "configs/int8.yaml#calib_images"
int8-ptq/int8.calib_images = 512 · verified in configs/int8.yaml#calib_images

$ rb spec set int8-ptq seed --per-run
int8-ptq/seed · per-run: each run gives its own, with --set seed=<value> on rb run add

Record the plan’s criterion, cited to its line

rb checks that docs/plan.md:3 is there and states the criterion’s threshold.

$ rb metric add top1 --unit % --maximize
Metric top1 (%): higher is better

$ rb criterion add "INT8 loses at most 0.5 points of top-1 against FP32" -e int8-ptq -q q-int8 --metric change.top1 --at-least -0.5 --why docs/plan.md:3 --id top1-drop-budget
top1-drop-budget: INT8 loses at most 0.5 points of top-1 against FP32
  change.top1 ≥ -0.5 · primary · on int8-ptq
  declared; no evidence recorded yet

Next: rb run add int8-ptq --from <the results file a run wrote> --set seed=<value>

Record each run from its own file

$ rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml
runobcr7a on int8-ptq: fp32.top1=91.71, int8.top1=91.6 · seed=1
  Ran with the spec: 5 of its settings found in results/int8-ptq/seed1/config.yaml, all matching.
  Produced: 2026-10-20 10:00:00 (file)
  top1-drop-budget estimate meets criterion · point only: change.top1: -0.11 % · paired difference, 1 seed (rb)
  rb put the code in git at refs/rb/snapshots/49a2f7fb6a172a9a46a4624957355aec6732c343, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/49a2f7fb6a172a9a46a4624957355aec6732c343 removes it.

Next: rb show top1-drop-budget

$ rb run add int8-ptq --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml
runejfzuk on int8-ptq: fp32.top1=91.66, int8.top1=91.58 · seed=2
  Ran with the spec: 5 of its settings found in results/int8-ptq/seed2/config.yaml, all matching.
  Produced: 2026-10-20 10:00:00 (file)
  top1-drop-budget estimate meets criterion · point only: change.top1: -0.095 % · paired difference, 2 seeds (rb)

Next: rb show top1-drop-budget

$ rb run add int8-ptq --from results/int8-ptq/seed3/metrics.json --set seed=3 --config results/int8-ptq/seed3/config.yaml
runqkvoca on int8-ptq: fp32.top1=91.74, int8.top1=91.69 · seed=3
  Ran with the spec: 5 of its settings found in results/int8-ptq/seed3/config.yaml, all matching.
  Produced: 2026-10-20 10:00:00 (file)
  top1-drop-budget criterion met · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)

Next: rb show top1-drop-budget

Keep what the report says

$ rb reconcile --apply
The repository at 93d9471: 1 results folder.

Kept in the record: 1 re-indexed (int8-ptq) · 1 result quoted · 1 next step quoted.

int8-ptq  results/int8-ptq/  in the record
  title: "INT8 post-training quantization" (README.md:1)
  says: "Three seeds of FP32 and INT8, evaluated with `python eval.py --seed <N>`." (README.md:3)
  result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
  next: "Try per-channel calibration with 128 images." (README.md:11)
  files: 1 report · 6 results files
  git: all 7 report and results files are tracked · folder unchanged since the last commit

Runs kept another way

Each run in a folder of its own, with its config and metrics: Record a folder per run.
Each run a row of one table, such as results.tsv: Record one table of many runs.

Why it matters

The record starts from what the files say, cited to the line, and every result made before it says so.

Rabbit Brainrabbitbrain.ai · GitHub · PyPI · Privacy · Apache-2.0