# Rabbit Brain docs, every page

# Rabbit Brain docs

https://rabbitbrain.ai/docs/

`git status` says what state your code is in. `rb status` says what state your research is in.

When an agent runs your experiments, the state of the work lives in its context window and in the summary it writes at the end. That summary leaves out what mattered: the seed nobody recorded, the threshold picked after the number came in, the two variants that also differed in batch size. rb keeps each of them on the record, and shows it every time the record is read.

rb checks each setting against the file it cites, computes every finding from the runs, and records who made each write. A finding is never one word. This is `rb show` of a criterion after three seeds, declared and frozen before the first one:

```rb-session frame="terminal"
$ rb show warmup-helps
Criterion warmup-helps on warmup (primary, own): Warmup lowers final validation loss by at least 0.02 nats
  warmup-helps Warmup lowers final validation loss by at least 0.02 nats
          criterion met · gain · change.val_loss ≤ -0.02 (n ≥ 3)
          reads:     change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb)
          result:    gain: the interval clear of 0 on the better side (lower is better)
          rule:      criterion met: estimate -0.02767 ≤ -0.02
          timing:    declared before the evidence (by production times)
          units:     3 seeds
          integrity: ok
  runof35go: difference -0.031 (file) · unit seed=1 · seed=1
  runk3rudt: difference -0.026 (file) · unit seed=2 · seed=2
  run7bk2pa: difference -0.026 (file) · unit seed=3 · seed=3
```

`warmup-helps` is the criterion's id: its name in every command.

## The loop

`rb --help` says it in three steps: agents record; rb checks and computes findings; you decide.

1. Your agent records

   In your repository, `rb init "<the work's theme>"` starts `.rb/` and installs rb's instructions for the coding agents it finds there. Then paste the line it prints into your agent's chat:

   ```text
   Use Rabbit Brain to track this: read rb context, then record where the work stands.
   ```

   The agent records the questions, the experiments, each setting with the file it came from, the criteria a result must meet, and the runs.

2. rb checks and computes

   rb reads each setting's source and finds the value there, or says it does not. It computes every finding from the runs, every time the record is read. It never takes a result from anyone's summary, the agent's included.

3. You decide

   `rb status` says where the work stands and what waits on you. Calls on results are yours: a call an agent records, and an experiment's design an agent wrote, wait until you confirm them with `rb confirm` in your own terminal.

## Try it in a minute

`rb tour` builds a small real project in `./rb-tour` and runs the whole loop, every command shown with its real output. With [uv](https://docs.astral.sh/uv/), it runs without installing anything:

```sh
uvx --from rabbit-brain rb tour
```

It ends like this:

```rb-session frame="terminal"
$ rb tour
…
That is the loop, and rb-tour/.rb/ is an ordinary record, YAML files you can read: poke at it.
One criterion meets its rule: three seeds, declared and frozen before the runs, every setting with its source.
The other is undetermined: two seeds report step_ms and it asks for three. rb will not round that up, and neither should anyone.

Keep going, in the record:  cd rb-tour
  rb ui         your screen in this terminal      rb open    the same, in your browser
  rb page       the record as one HTML file       rb export  the page to publish, the record if asked
  rb docs       the whole manual                  rb skill add   teach your coding agent the discipline
…
```

## Where to go next

  - [Install](https://rabbitbrain.ai/docs/install/): One command, then the one-minute tour.
  - [Your first record](https://rabbitbrain.ai/docs/first-record/): The line to paste into your agent, and what it records.
  - [The record](https://rabbitbrain.ai/docs/concepts/record/): What `.rb/` holds, and where each value came from.
  - [Cookbooks](https://rabbitbrain.ai/docs/cookbooks/): Eleven tasks, each with its prompt and rb's real output.
  - [The CLI](https://rabbitbrain.ai/docs/reference/cli/): Every command, with its help as rb prints it.
  - [For agents](https://rabbitbrain.ai/docs/agents/llms-txt/): Where an agent finds rb's manual, and what it must not do.

---

# Install

https://rabbitbrain.ai/docs/install/

This page installs rb, checks it, and runs the one-minute tour.

rb needs Python 3.10 or later. The package is `rabbit-brain` and the command is `rb`. Each release has wheels for Linux (x86_64 and aarch64), macOS (Apple Silicon and Intel) and Windows (x86_64), each with the `rb` binary, the Python SDK and the terminal screen, so installing needs no compiler.

On Windows, rb 0.8.3 is still a preview: [issue #2](https://github.com/rabbit-brain/rb/issues/2) lists its known issues, such as updating through uv. Claude Code runs rb's [session hook](https://rabbitbrain.ai/docs/connect/#the-session-hook) there in Git Bash; without Git Bash the hook fails, the session goes on, and rb does not look.

  **pip**
In a project's environment, which also lets you `import rabbit_brain` there:

```sh
pip install rabbit-brain
```
  
  **uv**
As a tool, in an environment of its own:

```sh
uv tool install rabbit-brain
```
  
  **uvx (no install)**
Run it once without installing anything:

```sh
uvx --from rabbit-brain rb tour
```

After `uvx`, `rb` is not on your path: run the next commands as `uvx --from rabbit-brain rb <command>`, or keep `rb` with `uv tool install rabbit-brain`.
  

Check it:

```rb-session frame="terminal"
$ rb --version
rabbit-brain 0.8.3
```

## Take the tour

`rb tour` builds a small real project in `./rb-tour` and runs the whole loop: a question, an experiment with its settings and criteria, a freeze, three runs, and `rb status`. Every command is shown with its real output. It begins like this:

```rb-session frame="terminal"
$ rb tour

$ rb init 'Dropout on a tiny MLP'
Project 'dropout-on-a-tiny-mlp' started: ~/try-rb/rb-tour/.rb
Rabbit Brain YAML files beside the code: commit .rb/ with it, so the record travels with the work.

For the agents here:
  No agent's instructions here yet. rb skill add puts rb's for one: --claude (Claude Code), --agents-md (Codex and others), --cursor

Then paste this line into your agent's chat:
  Use Rabbit Brain to track this: read rb context, then record where the work stands.

$ rb question add 'Can we use dropout 0.1 within our loss and step-time budgets?' --id q1
q1: Can we use dropout 0.1 within our loss and step-time budgets?

Next: rb experiment add "<what would answer it>" -q q1 --hypothesis "<what you expect>"

$ rb experiment add 'dropout 0.1 vs none' --id drop -q q1 --hypothesis 'Dropout 0.1 lowers val_loss' -m 'less co-adaptation' --baseline none --candidate p01 --varies dropout --note 'assumes the validation split stayed fixed'
Experiment drop: dropout 0.1 vs none · none (baseline) vs p01 (candidate) · varies dropout · under q1
  hypothesis: Dropout 0.1 lowers val_loss (why: less co-adaptation)
  Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
  Made 2 variants: none (baseline), p01 (candidate); a setting of one is rb spec set drop none.<setting> <value>.
  drop declares it varies dropout, but no variant has a value for it yet; give each variant its own: rb spec set drop <variant>.dropout <value> --source <file#key>
… (134 more lines)
```

`rb-tour/` is an ordinary record afterwards: `cd rb-tour`, then `rb ui` for the screen in your terminal or `rb open` for the same record in your browser.

## Updates

- **Installed as a tool** (`uv tool install`, or `pipx install`): at most once a day, rb asks PyPI for the newest release and, when it is newer, upgrades through the same installer. The next command runs the new rb.
- **Installed with pip**: rb never touches that environment. At most once a day it says one line on standard error when a newer rb is out, with the `pip install -U rabbit-brain` that gets it.

Neither runs in CI, in an agent session, with `DO_NOT_TRACK` set, with `RB_AUTO_UPDATE=0`, or after `rb config auto-update off`. The check carries nothing about you.

### A team on two versions

rb 0.7.0 and 0.7.1 read format 1 only, and rb 0.8.3 writes format 2 only in the file of a rule that uses a count, a best of N, a relative change or a cited difference of means. rb 0.7.0, 0.7.1, 0.8.0, 0.8.1 and 0.8.2 read the files 0.8.3 writes as they read their own.

- **A record with no format-2 file**: 0.7.0 and 0.7.1 read what 0.8.0 writes with no part they do not know, and write to it as before. Their readings differ from 0.8.0's in one place, and 0.7.0's in two more:
  - an `--each` rule beside an analysis of its comparison is read from the analysis in 0.7.0 and 0.7.1, and from its runs in 0.8.0;
  - on an experiment with one variant, 0.7.0 finds no number for a rule over a number recorded with no variant;
  - on an experiment with `population.repeat`, 0.7.0 reads the runs as if no repeat were declared.

  A call made on such a reading in one version reads "made on an earlier reading" in the other.
- **A record with a format-2 file**: in 0.7.0 and 0.7.1, `rb doctor` names each such file, `rb status` counts them, and both say to upgrade (`pip install --upgrade rabbit-brain`). They read every other file and fail every `--fail-on` gate. They write nothing to the record, and refuse to clone a bundle that holds such a file.
- **A file raised to `rb: 2` by hand**, with none of the four rules: 0.8.0 reads it as fine, and 0.8.1, 0.8.2 and 0.8.3 name it as no valid record.

Upgrade everyone who shares a record together.

## What rb sends, and when

Recording and reading the record need no account and no network. rb makes a network request only when:

- you run `rb clone`: it downloads a bundle given as a web address, and `git clone`s the code the bundle names;
- you run `rb login`, `rb sync` or `rb logout`: they call rabbitbrain.ai, where anyone can make an account, or a hub you name with `--hub` or `RB_HUB_URL`;
- the daily update check above runs;
- you turn on usage analytics with `rb telemetry enable`, in your own terminal. They are off until then, and they never carry research content, paths, command arguments or names. `rb telemetry preview` shows what would be sent. `DO_NOT_TRACK=1` or `RB_TELEMETRY=0` keeps them off.

`rb feedback` prints a link to a page that shows what it would send before you send it. rb itself sends nothing.

## Leaving

`rb remove --dry-run` shows what `rb remove` would take out of the repository, what it would keep, and why. `rb remove` then takes out only what checks out as rb's: the record, rb's instructions for each agent, and its session hook in each agent's hook file, where every other hook stays. Your research files and git history stay as they were. The command itself goes with `pip uninstall rabbit-brain`, or `uv tool uninstall rabbit-brain`.

  - [Next: your first record](https://rabbitbrain.ai/docs/first-record/): Start the record, and give your agent the line to paste.

---

# Your first record

https://rabbitbrain.ai/docs/first-record/

This page starts the record in your repository, hands it to your coding agent, and shows what you read when the agent is done.

## Start the record

In your own terminal, in the repository you work in:

```sh
rb init "<the work's theme>"
```

The title names the work's theme: a paper's title, a model, a line of work. Each thing the work tries to find out becomes a question of its own later. We ran it in a small repository: a ViT-Tiny training script that trains with and without label smoothing, three configs, a `CLAUDE.md`, and a plan in `docs/plan.md` that says "Label smoothing 0.1 is worth keeping if it raises top-1 by at least 0.3 points."

```rb-session frame="terminal"
$ rb init "Label smoothing on ViT-Tiny"
Project 'label-smoothing-on-vit-tiny' started: ~/vit-smoothing/.rb
Rabbit Brain YAML files beside the code: commit .rb/ with it, so the record travels with the work.

For the agents here:
  Claude Code (CLAUDE.md): the skill in .claude/skills/rabbit-brain/SKILL.md; rb's MCP server in a new .mcp.json; rb's session hook in a new .claude/settings.json (SessionStart and Stop run rb journal scan, which looks at the repository and says nothing)

Then paste this line into your agent's chat:
  Use Rabbit Brain to track this: read rb context, then record where the work stands.
```

`rb init` creates `.rb/` and installs rb's instructions for the coding agents it finds in the repository: here, Claude Code, because of `CLAUDE.md`. With them comes a [session hook](https://rabbitbrain.ai/docs/connect/#the-session-hook) in `.claude/settings.json`, so rb looks at the repository as each session starts and as each turn ends. [Connect your agent](https://rabbitbrain.ai/docs/connect/) says what it installs for Claude Code, Codex and Cursor. A session that is already running loads no skill added during it, so start a new one.

## Hand it to your agent

Paste the line `rb init` printed into your agent's chat. Add what you want done, in your own words:

**Track this with Rabbit Brain, and do one piece of work.**

The first sentence is the line rb init prints. Works in Claude Code, Codex, Cursor, or any agent that runs terminal commands.

```text
Use Rabbit Brain to track this: read rb context, then record where the work stands.
Then find out whether label smoothing is worth keeping on our ViT-Tiny, with seeds 1 to 3.
```

## What the agent did

Everything below is rb's real output from that session. The agent read `rb context` first. In a repository with work in it and nothing recorded yet, the briefing opens with what to do now:

```rb-session frame="terminal"
$ rb context
# The record: Label smoothing on ViT-Tiny

## First, record where the work stands

Nothing is recorded yet, and this project holds work already done: 1 commit. Record where the work stands now, in this turn, from the files, and ask the researcher nothing first. How: rb docs --start.

Generated by `rb context` from `.rb/`; read the state from here or `rb status --json`, never from memory or a summary.
… (49 more lines)
```

It recorded the question with your words kept beside it, then the experiment, each setting from the file that states it, and the criterion from the plan, cited to its line. All of it came before the first run:

```rb-session frame="terminal"
$ rb question add "Is label smoothing worth keeping on our ViT-Tiny?" --id q-smooth --words "find out whether label smoothing is worth keeping on our ViT-Tiny"
q-smooth: Is label smoothing worth keeping on our ViT-Tiny?

Next: rb experiment add "<what would answer it>" -q q-smooth --hypothesis "<what you expect>"
```

```rb-session frame="terminal"
$ rb spec set smooth --from configs/train.yaml --keys "optim.*,data.*"
smooth: 3 settings from configs/train.yaml @234b059 · 3 verified
```

```rb-session frame="terminal"
$ rb criterion add "Label smoothing 0.1 raises top-1 by at least 0.3 points" -e smooth -q q-smooth --metric change.top1 --at-least 0.3 --min-n 3 --why docs/plan.md:3 --id smoothing-helps
smoothing-helps: Label smoothing 0.1 raises top-1 by at least 0.3 points
  change.top1 ≥ 0.3 (n ≥ 3) · primary · on smooth
  declared; no evidence recorded yet

Next: rb run add smooth --from <the results file a run wrote> --set seed=<value>
```

`smoothing-helps` is the criterion's id: its name in every command.

It ran each seed through `rb journal run`, so the command, its exit code and the files it wrote are kept, and recorded each run from its own output file. After the third seed, the criterion has an interval:

```rb-session frame="terminal"
$ rb journal run -- python train.py --seed 3
{"metrics": {"plain.top1": 71.55, "smooth.top1": 72.1}}
exit 0. The command, its exit code and what it wrote are in the journal.
It printed its metrics: plain.top1=71.55, smooth.top1=72.1. rb reconcile proposes the run of smooth that records them.

Since this session last looked:
  - out/seed3/ holds results no recorded experiment reads yet   (rb reconcile)
  - out/seed3/metrics.json: results landed and are not in the record   (rb reconcile)

Next: rb reconcile
```

```rb-session frame="terminal"
$ rb run add smooth --from out/seed3/metrics.json --set seed=3 --config out/seed3/config.yaml --command "python train.py --seed 3" --commit 234b05955a4653f6e045af115202ce98dbdb730b
runljejw5 on smooth: plain.top1=71.55, smooth.top1=72.1 · seed=3
  Ran with the spec: 4 of its settings found in out/seed3/config.yaml, all matching.
  Only rb's own files are uncommitted (.claude/settings.json, .claude/skills/rabbit-brain/SKILL.md, .mcp.json): they are not the run's code, so the receipt reads the tree as clean.
  Produced: 2026-11-08 03:36:31 (file)
  smoothing-helps criterion met · gain: change.top1: +0.54 [+0.4017, +0.6783] % · paired difference, t interval, 3 seeds, 95% (rb)

Next: rb show smoothing-helps
```

The files `rb init` installed are uncommitted, but they are rb's own, not the run's code, so the receipt reads the tree as clean.

## What you'll see

Run `rb status` in your own terminal:

```rb-session frame="terminal"
$ rb status
Label smoothing on ViT-Tiny · label-smoothing-on-vit-tiny

Is label smoothing worth keeping on our ViT-Tiny? (open)
  Label smoothing 0.1 raises top-1 by at least 0.3 points. change.top1: +0.54 [+0.4017, +0.6783] % · paired difference, t interval, 3 seeds, 95% (rb); gain: the interval clear of 0 on the better side (higher is better); criterion met: estimate +0.54 ≥ 0.3; rule and question link declared before the evidence (by production times). units: 3 seeds.

Rules: 1 met

The design of 1 experiment is not confirmed: rb confirm lists it.

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:36 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  smoothing-helps Label smoothing 0.1 raises top-1 by at least 0.3 points
          criterion met · gain · change.top1 ≥ 0.3 (n ≥ 3)
          reads:     change.top1: +0.54 [+0.4017, +0.6783] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    gain: the interval clear of 0 on the better side (higher is better)
          rule:      criterion met: estimate +0.54 ≥ 0.3
          timing:    declared before the evidence (by production times)
          units:     3 seeds

Experiments
  smooth  Label smoothing 0.1 vs none · open · plain (baseline) vs smooth (candidate) · varies label_smoothing · not frozen · settings 5 verified · 0 provisional · 0 unknown · 1 per-run · runs 3
          to run again: code ok · environment ok · files ok

Runs: 3 recorded on 1 experiment · every receipt is what rb wrote

Nothing open.

Next: rb confirm
```

Read it top down: the project; the question, with its criterion read under it; the tally; what waits for you; the line that says what rb looks at; then each criterion on its axes, each experiment, and the runs. The criterion was declared before any run, so its timing reads "declared before the evidence". [Findings](https://rabbitbrain.ai/docs/concepts/findings/) explains each line.

## Make it yours

The agent wrote the experiment's design: the question it is filed under, its variants, its settings and the criterion. A design an agent wrote reads "not confirmed" until you look at it and confirm it. `rb confirm` lists it, part by part, and ends with the command that confirms it as listed:

```rb-session frame="terminal"
$ rb confirm
Designs waiting for a confirmation (1):
  smooth: Label smoothing 0.1 vs none
questions: q-smooth
hypothesis or purpose: not recorded
why: not recorded
variants: plain (baseline), smooth (candidate) · varies label_smoothing
setting optim.lr: 0.001 from configs/train.yaml#optim.lr
setting optim.weight_decay: 0.05 from configs/train.yaml#optim.weight_decay
setting data.batch_size: 128 from configs/train.yaml#data.batch_size
setting seed: per run
setting plain.label_smoothing: 0.0 from configs/plain.yaml#label_smoothing
setting smooth.label_smoothing: 0.1 from configs/smooth.yaml#label_smoothing
criterion smoothing-helps (primary): change.top1 ≥ 0.3 (n ≥ 3) · for q-smooth
confirm: rb confirm smooth@f6b6812a74bdbdfd0c6b1dbf8c1ebd0de7390051d0b73fbfb11624b7e88dc0d3
```

```rb-session frame="terminal"
$ rb confirm smooth@f6b6812a74bdbdfd0c6b1dbf8c1ebd0de7390051d0b73fbfb11624b7e88dc0d3
smooth's design is confirmed.

Next: rb status
```

`rb confirm` runs only in your own terminal. An agent session that runs it is refused.

  - [Connect your agent](https://rabbitbrain.ai/docs/connect/): What rb installs for Claude Code, Codex and Cursor.
  - [Cookbooks](https://rabbitbrain.ai/docs/cookbooks/): Eleven tasks for the work you do next.

---

# Connect your agent

https://rabbitbrain.ai/docs/connect/

This page says what rb installs for each coding agent, how to add it later, and what stays yours whichever agent writes.

`rb init`, run in your own terminal, installs rb's instructions for each agent it finds in the repository. It makes the files that are not there, and says what it wrote. A file that was there before, an `AGENTS.md`, a `.mcp.json` or an agent's hook file, changes only once you agree: rb asks once, shows exactly what it would add, and anything but `y` or `yes` keeps the file as it is. `--yes` answers up front, for a script. Where rb cannot ask, it leaves the file as it is and names the command that adds its part.

`rb init --dry-run` lists every file `rb init` would write, and writes nothing. [rb with Claude Code, Codex and Cursor](https://rabbitbrain.ai/docs/cookbooks/three-agents/) shows both with real output.

  **Claude Code**
When it finds `.claude/`, `CLAUDE.md` or a Claude Code session, `rb init` installs three things:

- the skill, in `.claude/skills/rabbit-brain/SKILL.md`;
- rb's MCP server, as one entry in `.mcp.json`;
- rb's [session hook](#the-session-hook), `SessionStart` and `Stop` in `.claude/settings.json`.

To add them later, or to a repository rb did not set up, run what `rb init` names:

```sh
rb skill add --claude --project
claude mcp add --scope project rabbit-brain -- rb mcp
```

`rb skill add` with no flag installs the skill once for all your projects, in `~/.claude/skills/rabbit-brain/`. A session that is already running loads no skill or MCP server added during it: start a new session after you install.
  
  **Codex**
Codex reads `AGENTS.md`. rb adds a marked section at its end, between two marker lines, when the file exists or Codex runs `rb init`:

```sh
rb skill add --agents-md
```

In your own terminal it asks once before it changes an `AGENTS.md` that is there. `--yes` adds the section without asking. Any other agent that reads `AGENTS.md` gets the same instructions.

When `.codex/` is there or Codex runs it, rb also adds its [session hook](#the-session-hook), `SessionStart` and `Stop` in `.codex/hooks.json`. Codex runs a project's hooks once it is trusted, in `/hooks`.
  
  **Cursor**
When it finds `.cursor/` or `.cursorrules`, rb adds a rule for Cursor's agent in `.cursor/rules/rabbit-brain.mdc`, and its [session hook](#the-session-hook), `sessionStart` and `stop` in `.cursor/hooks.json`:

```sh
rb skill add --cursor
```
  
  **A plain shell**
You need no agent at all. Every write an agent makes is an ordinary rb command you can type:

```sh
rb init "INT8 ResNet-20 on CIFAR-10"
rb question add "Does INT8 lose at most 0.5 points of top-1 against FP32?" --id q-int8
rb experiment add "INT8 post-training quantization" --id int8-ptq -q q-int8 --baseline fp32 --candidate int8 --varies precision
rb spec set int8-ptq int8.precision int8 --source configs/int8.yaml#precision
rb metric add top1 --unit % --maximize
rb criterion add "INT8 loses at most 0.5 points of top-1 against FP32" -e int8-ptq -q q-int8 --metric change.top1 --at-least -0.5
rb journal run -- python eval.py --seed 1
rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1
rb status
```

To read the record: `rb status` and `rb show <id>` in the terminal; `rb ui`, a screen in the terminal; `rb open`, the same record in your browser, served on your machine only; and `rb page`, one HTML file you can send. [The CLI](https://rabbitbrain.ai/docs/reference/cli/) lists every command.
  

## What every agent gets

- **`rb context`**: the record as one briefing a fresh session reads first, instead of anyone's summary. The skill tells the agent to run `rb context --model <your model id>` before anything else in a project with `.rb/`. rb keeps that model with the session's writes.
- **`rb docs`**: the whole manual, built into rb, so it always matches the installed version. `rb docs --start` is the protocol for a project already under way.
- **`--json` on every command**: one JSON object, the envelope, with the same facts as the text.
- **`rb mcp`**: the same commands as tools over the Model Context Protocol, each call recorded as the agent's.

## The session hook

With each agent's instructions, rb installs a session hook in that agent's project file. The hook runs `rb journal scan --hook <agent>` as a session starts and as each turn ends, so rb looks at the repository whether the agent reads rb's instructions or not. It has a 10-second limit. rb says nothing, tells the session nothing and records nothing. It keeps the look in the journal as the agent's.

| Agent | Where |
| --- | --- |
| Claude Code | `SessionStart` and `Stop`, in `.claude/settings.json` |
| Codex | `SessionStart` and `Stop`, in `.codex/hooks.json` |
| Cursor | `sessionStart` and `stop`, in `.cursor/hooks.json` |

A hook file that was there changes only once you agree, as `AGENTS.md` and `.mcp.json` do. rb's handler goes beside the hooks there, and every byte that was there is kept. Where rb cannot ask, it names the command that adds the hook: `rb skill add --claude --project --yes`, `--agents-md --yes` or `--cursor --yes`. A handler that runs `rb journal scan` in a form no rb wrote is yours: rb leaves it as it is. `rb remove` takes rb's handler out and leaves every other hook.

On Windows, Claude Code runs the hook in Git Bash. Without Git Bash it runs it in PowerShell, which does not read it: the hook fails, the session goes on, and rb does not look. Cursor's hook and Codex's on Windows look for rb on the PATH alone.

## What stays yours

Your agent records as it works, and rb records who made each write. Calls on results (accept, reject, investigate) and on lines of work (continue, park, stop, pivot, ship) are yours. A call recorded in an agent session is proposed, not made, and an experiment's design an agent wrote reads "design not confirmed". Both wait until you confirm them with `rb confirm` in your own terminal, and rb refuses `rb confirm` in an agent session. [Calls](https://rabbitbrain.ai/docs/concepts/calls/) has the details.

---

# The record

https://rabbitbrain.ai/docs/concepts/record/

This page says what the record holds, where each value in it came from, and who wrote what.

The record is a folder, `.rb/`, at the root of your repository. You commit it with your code, so it travels with the branch and git versions it. Every file in it is plain YAML you can read.

## What it holds

| Kind | What it is |
| --- | --- |
| **Questions** | What the work is trying to find out, in your words. Questions form a tree: a sub-question names its parent. |
| **Experiments** | What is tested: a hypothesis or a purpose, the variants (one baseline, and candidates, controls or ablations), and the settings the variants differ in on purpose. |
| **Settings** | Each value an experiment depends on, with the file it came from and whether rb has checked it there. |
| **Metrics** | Each number's unit, which way is better, and its other names. |
| **Criteria** | What a result must meet: a rule over one comparison, with who declared it and when. See [Criteria](https://rabbitbrain.ai/docs/concepts/criteria/). |
| **Runs** | What ran: its numbers, its per-run settings such as the seed, and a receipt. |
| **Analyses** | A computation over saved outputs, such as a bootstrap or a reanalysis, and the comparisons it produced with their intervals. |
| **Resources** | Datasets, checkpoints and code the runs read or made: what each is, where its bytes are, and how to get it again. |
| **Notes** | Words about an experiment or the project: an assumption, a caveat, context. A note changes no finding. |
| **Decisions** | The calls made on the work, each with who made it, why, and what it was made on. See [Calls](https://rabbitbrain.ai/docs/concepts/calls/). |

There is one file per record, and one folder per experiment. Two branches that add runs write different files, so their work merges without a conflict:

```text
.rb/project.yaml                          the project: its title, where it was cloned from
.rb/notes/<id>.yaml                       one file per note on the project
.rb/questions/<id>.yaml                   what the work is trying to find out
.rb/experiments/<id>/experiment.yaml      an experiment: its questions, purpose, hypothesis, shared settings, freeze
.rb/experiments/<id>/variants/<name>.yaml one file per variant: its role and its own settings
.rb/experiments/<id>/notes/<id>.yaml      one file per note on the experiment
.rb/experiments/<id>/criteria/<id>.yaml   one file per criterion
.rb/experiments/<id>/runs/<id>.yaml       one file per run, with its receipt
.rb/experiments/<id>/analyses/<id>.yaml   one file per analysis
.rb/metrics/<id>.yaml                     the metric catalogue
.rb/resources/<id>.yaml                   datasets, checkpoints, code
.rb/decisions/<id>.yaml                   calls, each with who made it, why, and what it was made on
.rb/log.jsonl                             every write: who made it, in which chat session, how, and the hash of what was written
.rb/cache/                                local copies, kept out of git
```

## Where a value came from

A setting is only as good as its source. `rb spec set` records a value with the file that states it: a config key (`configs/train.yaml#optim.lr`), a line, a quoted sentence, one row of a SQLite database, or one row of a CSV, TSV or JSON Lines table (`results/sweep.csv#[precision=int8].calib`). rb reads that source on the spot. Each setting then has one of four statuses:

- **verified**: the source states this value. rb keeps the text it read, the file's hash, and the commit.
- **provisional**: the value was given but not checked, for example a web address.
- **unknown**: nobody has found it yet.
- **inherited**: it came with a record cloned from someone else's.

rb is strict on purpose. The source must name the setting, not only hold the value. A comment never verifies. `resnet50` does not state 50, and `adamw` does not state `adam`. A source that states another value is a **conflict**, and one that no longer states it is **stale**: both are problems on every finding of that experiment. Every read checks the sources again, so a config edited since is caught.

A verified setting is what the config file states, not what a run used. To check what a run used, record the run with its own resolved config (`rb run add ... --config out/seed1/config.yaml`). A run whose config disagrees with the spec is shown and not counted.

## Who wrote what

Agents and people both write to the record, and rb never refuses a write because of who makes it. What keeps the record honest is attribution: every write records who made it (you, or the agent rb detects), the chat session when the agent's runtime gives one, the agent's model when it said it, and the reason as given.

That attribution is in the data: `rb log --json`, `--json` on every command, and `.rb/log.jsonl`. It is never on a screen. `rb status`, `rb open`, `rb ui` and the page never say who made a write. They do say when something an agent wrote waits for you, for example "design not confirmed".

## Nothing computed is stored

A finding, an experiment's state and a question's state are never in a file. rb computes them from the runs every time the record is read, with the same definitions every time. If a page you were sent ever disagrees with `rb status`, `rb status` is right.

## Hand edits

Never edit `.rb/` by hand. rb compares each record's content with its last write, and catches a change made outside rb: a run in that state is not counted, and every finding that depends on it lists the problem. `rb doctor` lists such files. `rb doctor --restore` puts back what rb last wrote, and `rb doctor --adopt --why "..."` keeps the change, recorded as whoever adopts it. A comment you add, or a reformat, is not an edit.

A file that is not Rabbit Brain YAML, or not a valid record, stops `rb status`, `rb show` and every gate (`E_STATE_CORRUPT`), saying where it breaks. Beside it:

- A run or an analysis is recorded whole. The command exits 0, and its reply names the files it could not read, with `rb doctor`'s fix (`data.unread_files`, `data.warnings`, an RBWarning from Python).
- `rb retract` of a note beside a criterion's file that does not read, and `rb criterion relate` beside an analysis's file that does not read, record the retraction or the relation whole, exit 0 and name the file, with `rb doctor`'s fix. Neither reads that file before it writes. rb 0.8.2 wrote them and then exited 2.
- A note changes no finding. Beside a note's file that does not read, every write leaves that note out, is made whole and succeeds. Only `rb retract` of that note is refused, with nothing written: it names the note's file, where it breaks and `rb doctor`'s fix.
- A write that reads that file before it writes is still refused, with nothing written: beside an experiment's `experiment.yaml`, every write to that experiment.

## What rb sees

Every `rb status` says, in one line, what rb looks at:

```rb-session frame="terminal"
$ rb status
…
rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:36 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.
…
```

It also looks in every folder named for results (`eval/`, `reports/`, `artifacts/` and the like) down to four levels from the top, which the line then names, and in any folder a recorded run named. `rb reconcile` lists what it found there that nobody recorded, and the `rb run add` line that records it. Under "Not read" it names a folder it passed over that holds results files of its own.

It names each results file in the repository's top folder too: a table (`results.tsv`, `results.csv`, `results.jsonl` and the like) with the `--where` line that reads a row, and a report in prose (`RESULTS.md`, `report.md`), from which rb reads no run. Once a table's rows are recorded, it says how many ("8 of 8 rows recorded as runs").

A row of a CSV, TSV or JSON Lines table is a run: `rb run add <exp> --from results.tsv --where <column>=<value>` records one, and `--every-row` records every row `--where` picks, each as its own run. `--every-row` reads a SQLite table the same way: `--from results.db --table evals`. [Record one table of many runs](https://rabbitbrain.ai/docs/cookbooks/one-table/) and [Record a folder per run](https://rabbitbrain.ai/docs/cookbooks/run-folders/) show both layouts.

## The format

Every file opens with its format. rb writes a criterion's file in format 2 (`rb: 2`) only while it holds one of four kinds of rule that format 1 cannot hold: a count, a best of N, a relative change or a cited difference of means (see [Criteria](https://rabbitbrain.ai/docs/concepts/criteria/#four-more-kinds-of-rule)). It writes every other file in format 1 (`rb: 1`), as rb 0.7 writes it, and reads both formats. A record that uses none of the four is format 1 throughout. A file that says `rb: 2` and holds none of the four, its first line raised by hand, is no valid record (`E_STATE_CORRUPT`), where rb 0.8.0 said nothing: `rb doctor` says to set its first line back to `rb: 1`, or `rb doctor --restore` puts back what rb wrote.

rb 0.7.0 and 0.7.1 read format 1 only. They read a record with no format-2 file, and write to it as before. Once a file is format 2, they name it ("is format 2; this rb reads format 1: it was written by a newer rb"), say to upgrade (`pip install --upgrade rabbit-brain`), and write nothing to the record.

Both formats are one JSON Schema, [rb-v2.schema.json](https://rabbitbrain.ai/schema/rb-v2.schema.json). `rb schema all` prints it, and [The record's format](https://rabbitbrain.ai/docs/reference/schema/) lists every kind.

---

# Criteria

https://rabbitbrain.ai/docs/concepts/criteria/

This page says what a criterion is, what it compares, the rules it can apply, and how rb reads when it was declared.

A criterion says what would count, before the numbers come in: "INT8 loses at most 0.5 points of top-1 against FP32." It reads one comparison and applies one rule, and rb records who declared it and when. rb computes its finding from the runs every time the record is read.

```sh
rb criterion add "INT8 loses at most 0.5 points of top-1 against FP32" \
  -e int8-ptq -q q-int8 --metric change.top1 --at-least -0.5 --min-n 3 --why "docs/plan.md:3"
```

## What it compares

`--metric` names a number the runs report:

| `--metric` | What it reads |
| --- | --- |
| `top1` | a number named `top1`, or the only candidate's when the runs name it by variant |
| `int8.top1` | the `int8` variant's number, a level |
| `candidate.top1` | the candidate's number |
| `change.top1` | the candidate's number minus the baseline's, from the same run |
| `top1` with `--subject fp32 --reference int8` | FP32's number against INT8's |

Record each variant's own numbers, such as `{"fp32": {"top1": 91.71}, "int8": {"top1": 91.6}}`, never a difference you worked out: rb works out `change.top1` itself. `--where`, `--subject-where` and `--reference-where` select runs by their per-run settings.

## Its rule

Each criterion has exactly one:

| Rule | Holds when |
| --- | --- |
| `--at-most X`, `--at-least X` | the estimate, the mean over the runs' units, is at most X, or at least X |
| `--equals X --tolerance T` | the estimate is within T of X |
| `--each` (with a threshold) | every independent unit meets the threshold, not only the estimate |
| `--count K --of N`, `--best --of N`, `--relative`, `--means` | see [Four more kinds of rule](#four-more-kinds-of-rule) |
| `--bound` (with a threshold) | the interval's bound meets it, not only the estimate |
| `--better` | the interval is clear of 0 on the better side |
| `--noninferior M` | the interval's worse bound is inside M |
| `--equivalent M` | the interval is inside ±M |

The last three read the interval, so they need one: rb gives an interval from 3 units, such as three seeds. `--better` and `--noninferior` also need to know which way is better, which the metric says: `rb metric add top1 --maximize`. Without it they are refused.

A threshold is read one way. Two of `--each`, `--bound`, `--count` and `--best` together are refused: "A threshold is read one way: --each and --bound each say a different one."

## Four more kinds of rule

rb 0.8.3 reads four more kinds of rule. Each reads a threshold its own way, and each is said in the same words by `rb criterion add`, `rb show`, `rb status`, `rb context`, the SDK and MCP:

| Rule | What it reads | As rb says it |
| --- | --- | --- |
| `--count K --of N` | whether at least K of the N units meet the threshold, each unit judged as `--each` judges it | "4 of 5 units' int8.top1 ≥ 91.58" |
| `--best --of N` | the unit best in the threshold's direction, its own value, read against the threshold | "the best of 5 units' int8.top1 ≥ 91.65" |
| `--relative` | the relative change of the two arms' means, (subject - reference) / reference, in percent; its threshold is a percent | "change.top1 as a relative change ≥ -0.1%" |
| `--means`, beside `--source` and `--quote` | the cited number as a difference of the arms' means over all their units | "change.val_loss as a difference of means = 0.0317 ± 0.00005" |

- **A count** names both numbers: "recorded values meet criterion: 4 of 5 seeds at least 91.58; the rule asks for 4". Where the runs hold another number of units than the rule is of, or fewer than K, it reads undetermined: "criterion undetermined: the rule is of 5 seeds; the runs hold 1".
- **A best of N** takes `--at-most` or `--at-least`, and has no interval: "a best of N is one unit's own value, with no interval".
- **A relative change** compares two arms. From 3 units its interval is a delta-method t interval: "relative change of the means, paired, delta-method t interval".
- **A cited difference of means** reads "difference of means, as cited", with a Welch interval once each arm holds 3 units. rb sets the cited number against that difference of means, never against its paired reading, so a unit that only one arm ran does not make the rule read "not comparable".

A met count or best of N reads **recorded values meet criterion**, since the recorded units themselves meet it. A relative change and a difference of means take a threshold on their estimate, or with `--bound` on their interval's bound.

A criterion that holds one of the four is kept in record format 2 (`rb: 2`), in its own file. Every other file stays in format 1: see [The record's format](https://rabbitbrain.ai/docs/reference/schema/). [Read a rule over seeds](https://rabbitbrain.ai/docs/cookbooks/rules-over-seeds/) declares a count, a best of N and a relative change over five seeds.

`--min-n 3` says how many independent units the reading must rest on: with fewer, the rule reads undetermined. `--noise 0.01` says how much a result moves from run to run: an estimate that close to the threshold gets a note that another unit may land on the other side.

## Where its number comes from

A criterion's threshold is a file's or yours, never the agent's own:

- **A plan** that states it: `--why "docs/plan.md:3"`. rb refuses the criterion when the line is not there, is a comment, or does not state the threshold.
- **A paper** or another document: `--source docs/baselines/resnet-paper.md --quote "| ResNet | 20 | 0.27M | 8.75 |" --locator "Table 6"`. rb checks that the file holds the quote and that it states the number. A cited criterion reads **reproduced** or **not reproduced**, and **reproduced on the estimate** where the reading has no interval or one that crosses its line.
- **Your words**: `--words "dropout passes only if the whole 95% interval stays within 0.02 nats"`, which rb keeps verbatim beside the criterion. Words an agent passes read as `words (as given)`, since rb cannot know you said them.

## Its role

`--role primary`, the default, is what the work claims. `secondary` and `exploratory` are context. A `gate` is a priority policy for where to spend effort, not a claim: a missed gate asks for a call to park or continue the line, and never refutes anything.

## When it was declared

A criterion is declared when it is written. rb compares that moment with when its evidence existed, and says it on every finding:

- **declared before the evidence**: written before the runs it reads were produced;
- **declared after the evidence existed**: some or all of those runs already existed, and rb says how many;
- **declared after the data was seen**: the data it evaluates on was looked at first;
- **order unknown**: nobody knows when some of its evidence was produced. Numbers typed in on the command line or through MCP with no `--produced-at` have no time of their own, so a rule over them reads this.

Evidence that already existed still counts. It is never hidden, and never read as if the criterion came first. [Declare the criterion before the runs](https://rabbitbrain.ai/docs/cookbooks/criterion-before-runs/) shows both timings side by side.

## Freezing

`rb freeze <experiment> -m "before the runs"` locks the experiment's variants, settings and criteria, and records a hash of them. After that, a change fails with `E_FROZEN` unless the same command gives its reason with `--amend --why "..."`. The amendment is kept, with who made it and the hash before and after.

## Which question it answers

`-q <question>` links a criterion to the question its finding bears on, and a question's reading counts only the criteria linked to it. `--when-met "..."` keeps, verbatim, what meeting the rule would mean for that question. When the rule misses, rb never says the opposite.

---

# Findings

https://rabbitbrain.ai/docs/concepts/findings/

This page explains each line of a finding, how rb reads the runs, and the integrity problems a finding can carry.

rb never stores a finding, and no finding is one word. Each time the record is read, rb works out every criterion's finding from its runs and analyses, and says it on separate axes. This is `rb show` of a criterion after five seeds:

```rb-session frame="terminal"
$ rb show interval-within-budget
Criterion interval-within-budget on drop (primary, own): The whole 95% interval of dropout's cost stays within 0.02 nats · words (as given): "dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout"
  interval-within-budget The whole 95% interval of dropout's cost stays within 0.02 nats
          criterion met · unresolved · change.val_loss no worse than 0.02, its interval's worse bound inside 0.02 (n ≥ 5)
          reads:     change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)
          result:    unresolved: the interval includes 0
          rule:      criterion met: the interval's worse bound +0.01338 inside 0.02
          timing:    declared before the evidence (by production times)
          units:     5 seeds
          integrity: ok
  run2yelmh: difference +0.008 (file) · unit seed=1 · seed=1
  runwmk7z5: difference -0.003 (file) · unit seed=2 · seed=2
  runtetay2: difference +0.014 (file) · unit seed=3 · seed=3
  runkquyec: difference +0.003 (file) · unit seed=4 · seed=4
  run35lzxe: difference +0.006 (file) · unit seed=5 · seed=5
```

`interval-within-budget` is the criterion's id: its name in every command.

## The axes

| Line | What it says |
| --- | --- |
| **reads** | The comparison rb read: the number, its interval, its unit, how it was computed, and from what. `(rb)` is rb's reading of the runs; an analysis is named by its id. |
| **result** | What the number shows, in the metric's direction. **gain** or **loss**: the interval is clear of 0 on the better or the worse side. **unresolved**: the interval includes 0. **equivalent**: inside an `--equivalent` margin. **point only**: no interval yet. **measured**: one arm on its own, with its interval. |
| **rule** | Whether the criterion's rule is met on that reading: **criterion met**, **criterion not met**, **criterion undetermined** (it needs an interval, more units, or the metric's direction), or **criterion untested**. With no interval, or with an interval that crosses the threshold, a rule that holds reads **estimate meets criterion**. A cited criterion reads **reproduced** or **not reproduced**, and **reproduced on the estimate** where it is met on a reading with no interval or one that crosses its line; a gate, **gate met** or **gate missed**. The tally says how many rules are met on the estimate alone: "Rules: 5 reproduced, 2 of them on the estimate alone". |
| **timing** | Whether the criterion was declared before its evidence. See [Criteria](https://rabbitbrain.ai/docs/concepts/criteria/#when-it-was-declared). |
| **units** | How many independent units the reading rests on, and what a unit is: here, a seed. |
| **integrity** | Every problem with what the finding depends on, or `ok`. |

The result and the rule are separate on purpose. "criterion met · unresolved" above is an ordinary finding: the rule asked that the whole interval of dropout's cost stay within 0.02 nats, and it does, while the interval still includes 0, so rb cannot say dropout changes the loss at all.

## How rb reads the runs

- Each run gives one difference: the subject's number minus the reference's, both from that run.
- Runs with the same per-run values (two runs at `seed=2`) are one unit, their numbers averaged. A run recorded with `--repeats <run>` checks another run, and adds no unit.
- The estimate is the mean over units. From 3 units, rb gives a 95% Student t interval and labels it as its own.
- Arms recorded in separate runs pair up by unit. The reading says what it pairs them by ("paired by seed"), with the unpaired reading beside it, unless it says exactly the same. Whether runs that share a seed or an epoch are linked is yours to say: rb never decides it. Arms that never share a unit are compared unpaired (Welch).
- A setting that only numbers repeats, such as an epoch, is declared with `rb spec set <exp> population.repeat epoch --no-verify` (or `seed,epoch`). No reading is then said to pool it, a rule is never read at one of its values alone, and it is never offered as the unit.
- A run is shown and not counted, with the reason, when its config disagrees with the spec, when it was recorded under an earlier spec, or when it was edited outside rb.

## Why one number is not enough

With one or two seeds there is no interval, and a rule on the estimate can still hold: rb says **estimate meets criterion · point only**. That is not "it works". These are two criteria on the same comparison, one on the estimate and one on the interval, after the first seed:

```rb-session frame="terminal"
$ rb run add drop --from out/seed1/metrics.json --set seed=1 --config out/seed1/config.yaml --command "python train.py --seed 1"
run2yelmh on drop: none.val_loss=1.31, p01.val_loss=1.318 · seed=1
  Ran with the spec: 3 of its settings found in out/seed1/config.yaml, all matching.
  Produced: 2026-11-08 03:36:50 (file)
  mean-within-budget estimate meets criterion · point only: change.val_loss: +0.008 nats · paired difference, 1 seed (rb)
  interval-within-budget criterion undetermined · point only: change.val_loss: +0.008 nats · paired difference, 1 seed (rb)
  rb put the code in git at refs/rb/snapshots/3bf56b1d2d985fe410d77e394087a33e85d90a96, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/3bf56b1d2d985fe410d77e394087a33e85d90a96 removes it.

Next: rb show mean-within-budget  ·  rb show interval-within-budget
```

And after the third: the interval appears, but it still reaches past 0.02, and the second criterion asked for 5 seeds:

```rb-session frame="terminal"
$ rb run add drop --from out/seed3/metrics.json --set seed=3 --config out/seed3/config.yaml --command "python train.py --seed 3"
runtetay2 on drop: none.val_loss=1.315, p01.val_loss=1.329 · seed=3
  Ran with the spec: 3 of its settings found in out/seed3/config.yaml, all matching.
  Produced: 2026-11-08 03:36:51 (file)
  mean-within-budget estimate meets criterion · unresolved: change.val_loss: +0.006333 [-0.01508, +0.02775] nats · paired difference, t interval, 3 seeds, 95% (rb)
  interval-within-budget criterion undetermined · unresolved: change.val_loss: +0.006333 [-0.01508, +0.02775] nats · paired difference, t interval, 3 seeds, 95% (rb)

Next: rb show mean-within-budget  ·  rb show interval-within-budget
```

[Get an interval, not a single number](https://rabbitbrain.ai/docs/cookbooks/interval-from-seeds/) walks through all five seeds.

## Integrity

A finding lists every problem it depends on. The common ones:

| Problem | What it means |
| --- | --- |
| `confound` | the two arms differ in a setting the experiment does not say it varies, and nobody accepted the difference |
| `conflict`, `stale` | a setting's source states another value, or no longer states it |
| `unknown` | a required setting nobody has found |
| `not_comparable` | a setting differs from the value a cited source used, or a cited difference of means is one rb's paired reading cannot give |
| `drift` | a frozen experiment no longer matches its freeze |
| `edited_outside_rb` | a record it depends on was changed by hand |

A setting that is provisional, not verified, is a note on the finding, not a problem. So are numbers typed in, a run with uncommitted changes, arms that differ in a per-run setting (500 problems against 250), and runs that pool several values of a setting. A rule's card on the page `rb page` writes carries the same notes as `rb show`.

## In CI

`rb status --fail-on <gate>` exits 1 when the record fails that gate. `unestablished` is the one most CI jobs want: it fails when a primary criterion is not met, undetermined or untested, when it is not shown to have been declared before what it reads, when a finding has an integrity problem, or when a record was changed outside rb. Check out the whole history in CI (`fetch-depth: 0` with `actions/checkout`): rb tells a clean merge from a hand edit only through git's history.

```sh
rb status --fail-on unestablished
```

The other gates are `unmet`, `unresolved`, `untested`, `integrity`, `late`, `undecided`, `stale`, `checks` and `unknown`. `rb docs` says what each one checks.

---

# Calls

https://rabbitbrain.ai/docs/concepts/calls/

This page says what a call is, why a call or a design an agent records waits for you, and how to take anything back.

A **call** is a decision on the work, recorded with why and with the finding it was made on. There are two kinds:

- on a criterion, an experiment, a question or a setting: **accept**, **reject**, **investigate**;
- on an experiment or a question, to steer the line of work: **continue**, **park**, **stop**, **pivot**, **ship**.

```sh
rb decide <subject> <action> --why "<the reason>"
```

A finding never makes a call, and a call never changes a finding. An experiment's state (open, accepted, parked, shipped and so on) is what the latest call on it made it.

## Calls are yours

Anyone can record a call, and rb records who did. A call recorded in an agent session is **proposed, not made**: it changes nothing until you confirm it in your own terminal. Your agent records one only when you ask, with your words passed verbatim in `--words`.

Here an agent records the call it was asked for, on a criterion declared after the runs. `stricter-warmup-bar` is that criterion's id: its name in every command.

```rb-session frame="terminal"
$ rb decide stricter-warmup-bar reject --why "docs/plan.md:3 sets the bar at 0.02 nats" --words "Reject the 0.03 bar. The plan says 0.02."
deijru2: proposed, not made: reject stricter-warmup-bar · note: docs/plan.md:3 sets the bar at 0.02 nats · words (as given): "Reject the 0.03 bar. The plan says 0.02."
  It changes nothing until it is confirmed: rb confirm deijru2, in your own terminal.

Since this session last looked:
  - stricter-warmup-bar misses its rule and the call recorded on it is not made yet: it waits for a confirmation: Warmup lowers final validation loss by at least 0.03 nats   (rb decide stricter-warmup-bar accept|reject|investigate --why "...")
  - warmup amended (new criterion stricter-warmup-bar: change.val_loss ≤ -0.03 (n ≥ 3)): a stricter bar, set after seeing the runs
  - criterion stricter-warmup-bar added on warmup (change.val_loss ≤ -0.03 (n ≥ 3)): Warmup lowers final validation loss by at least 0.03 nats

Next: rb status
```

`rb status` lists it under **Proposed, not made**, with the command that confirms it:

```rb-session frame="terminal"
$ rb status
…
Proposed, not made (1)
  - reject stricter-warmup-bar · note: docs/plan.md:3 sets the bar at 0.02 nats · words (as given): "Reject the 0.03 bar. The plan says 0.02." · confirm: rb confirm deijru2
…
```

You confirm it:

```rb-session frame="terminal"
$ rb confirm deijru2
d7vos7j: rejected stricter-warmup-bar · note: docs/plan.md:3 sets the bar at 0.02 nats · your words: "Reject the 0.03 bar. The plan says 0.02."   (confirms deijru2)
  made on stricter-warmup-bar: -0.02767 [-0.03484, -0.0205], n 3 (rb): gain, criterion not met

Next: rb status
```

Before you confirm, rb says the words an agent passed as `words (as given)`, since rb cannot know you said them. After, they read as `your words`.

`rb confirm` runs only outside an agent session: in one, it is refused. `rb confirm` alone lists everything that waits, and ends with the one command that confirms it all as listed.

## Designs are yours too

An experiment's **design** is how it tests its question: the questions it is filed under, its hypothesis or purpose, its variants and what they vary, its settings and their sources, its criteria, its gate and its freeze. When an agent wrote any part of it, the experiment reads **design not confirmed** until you confirm it:

```sh
rb confirm <experiment>          # lists the design, every part of it
rb confirm <experiment>@<hash>   # confirms it exactly as listed
```

A design changed after you listed it is not confirmed unseen: rb lists it again. Once you have confirmed it, a later change by an agent says what changed, part by part. Confirming changes no finding.

## Calls made before the record

A call made before rb was in the repository is recorded as a past call, dated as it was made, and cited to the document that states it:

```sh
rb decide int8 park --why "..." --past --when "2026-05-07" --source docs/calls.md:3 --quote "We park INT8 for now"
```

rb checks the quote is in the file and says what git shows of it: when the line was committed, and whether that was before the record began. A past call an agent recorded is a **past call mentioned, not confirmed**, and waits for the same one confirmation.

## Taking something back

`rb retract <id> --why "..."` takes back anything: a run, a criterion, a note, a call. It stays on the record with who took it back and why, and stops counting. rb says what rested on it, and which findings the retraction changed. An agent session cannot take back a call you made: it can propose another call, which waits for you.

---

# Sharing

https://rabbitbrain.ai/docs/concepts/sharing/

This page says each way the record leaves your repository, what travels with each, and what rabbitbrain.ai serves.

The record lives in your repository, and it leaves your machine only when you send it. rb has four ways to show it, from the lightest to the fullest:

| Command | What it writes | What travels |
| --- | --- | --- |
| `rb page` | One self-contained HTML file, read-only, that you can send anywhere. | The page alone. |
| `rb open` | Nothing: the same record in your browser, live, served on 127.0.0.1 of your machine only. | Nothing. |
| `rb export` | The page with a README and a manifest that hashes it, for any static host. | The page alone. |
| `rb export --with-record` | A bundle: the record, the files its checks read, the page, and a manifest with every file's hash. | The record too. |

This is the page of a small record, as rb writes it with `rb export --with-record --url`:

<Shot name="rb-page" alt="rb's page for an INT8 ResNet-20 record: the open question, its one criterion met, one design waiting for a confirmation, and a panel to reproduce the project." width={1280} height={760} />

## What a page leaves out

A page is made to be sent to anyone. It names no writer, none of the words you gave an agent, no call that waits for your confirmation, and no machine's platform. Its findings are computed when it is written: write it again after new runs.

## What the record carries

A bundle made with `--with-record` carries the record as it is, and the record keeps everything: who made each write, an agent's chat session, name and model, the words you passed with `--words`, and each run's host, platform and branch. rb says so each time you make one:

```rb-session frame="terminal"
$ rb export --with-record --zip --out ~/int8-record.zip
~/int8-record.zip written: the whole record, the page, and a manifest hashing every file.
Readable without rb; rb status inside record/ reopens it. Regenerate after new runs.
The record keeps who made each write (human:<name> or agent:<name>, and the channel it came through), an agent's chat session, name, version, provider and model, the user's words passed with --words, and each run's host, platform and branch. Anyone who can read the bundle can read all of it.
Code repository recorded: https://example.org/ada/int8-resnet.git at f49125ceaa. Pass --no-repo to leave it out.

Next: rb clone ~/int8-record.zip <dest>  ·  rb export --with-record --url <public address>   (make it publishable)
```

## Publishing a bundle

`rb export --with-record --url <address>` writes a folder to serve from that address. Its `page.html` opens with the exact `rb clone` command, and the folder holds `bundle.zip`, so one static host serves both the page and what a stranger clones:

```rb-session frame="terminal"
$ rb export --with-record --url https://example.org/int8-resnet/ --out ~/int8-site
~/int8-site written: the whole record, the page, and a manifest hashing every file.
Readable without rb; rb status inside record/ reopens it. Regenerate after new runs.
The record keeps who made each write (human:<name> or agent:<name>, and the channel it came through), an agent's chat session, name, version, provider and model, the user's words passed with --words, and each run's host, platform and branch. Anyone who can read the bundle can read all of it.
Code repository recorded: https://example.org/ada/int8-resnet.git at f49125ceaa. Pass --no-repo to leave it out.
Publish the directory at the address you gave; page.html is the reading surface and anyone reproduces it with: rb clone https://example.org/int8-resnet/bundle.zip
```

The page also has a **Rerun this research** panel: a prompt to paste into a coding agent, which clones the record, gets the code, reruns the runs, and says which criteria reproduced. [Share the record, and rerun it](https://rabbitbrain.ai/docs/cookbooks/share-a-record/) shows the prompt and a rerun from start to end.

## Starting from someone else's record

`rb clone <bundle or address> <folder>` starts a new line of work from a bundle. When the bundle names its code, the clone is a checkout of that code at the bundle's commit, on a new branch `rerun`, with the record in it. The questions, experiments, settings and criteria arrive **inherited**: every criterion reads **untested** until runs made in the new repository test it, then **reproduced** or **not reproduced**. The parent's runs and calls stay in the parent's record.

This is how an advisor or a reviewer checks the work: clone it, rerun it, and read what the criteria become.

## A team on two versions

rb 0.8.3 writes a criterion's file in format 2 only while it holds a count, a best of N, a relative change or a cited difference of means. rb 0.8.0, 0.8.1 and 0.8.2 read the files 0.8.3 writes as they read their own. rb 0.7.0 and 0.7.1 read a record with no format-2 file, and write to it as before. Once a file is format 2, they write nothing to the record, refuse to clone a bundle that holds it, and say to upgrade. [The record's format](https://rabbitbrain.ai/docs/reference/schema/#format-1-and-format-2) says what each version does. Upgrade everyone who shares a record together.

## rabbitbrain.ai

[rabbitbrain.ai](https://rabbitbrain.ai) serves shared record pages, and sample records at [rabbitbrain.ai/demo](https://rabbitbrain.ai/demo). Each page is rb's own page, with its **Rerun this research** prompt and the `rb clone` command for its `bundle.zip`.

`rb login` signs a machine in to rabbitbrain.ai, and `rb sync` keeps a private copy of a record's page in your account there. Anyone can make an account, at [rabbitbrain.ai/signin](https://rabbitbrain.ai/signin). A share link made from that page can be revoked, and `rb sync --with-record` sends the record too, so the link's readers can rerun it. Before `rb login`, `rb sync` says what is missing:

```rb-session frame="terminal"
$ rb sync
rb sync: No credential for rabbitbrain.ai on this machine.
  → Run rb login once per machine, or set RB_HUB_TOKEN for CI.
```

rb itself never publishes. Hosting an export is up to you, and a share link is made on the site, never by rb.

---

# Cookbooks

https://rabbitbrain.ai/docs/cookbooks/

Each cookbook is one task, start to finish, with the prompt to paste into your coding agent and rb's real output from running it.

Every cookbook gives you:

- the problem, in two sentences;
- **Paste this into your agent**: a short task for Claude Code, Codex, Cursor, or any agent that runs terminal commands;
- **What you'll see**: rb's output, exactly as it printed it;
- **What rb did, step by step**: the commands under the hood, so you can run them yourself;
- **Why it matters**, in one line.

  - [Start tracking a project already under way](https://rabbitbrain.ai/docs/cookbooks/project-under-way/): Record where the work stands from the repository's own files, with no questions asked first.
  - [Record one table of many runs](https://rabbitbrain.ai/docs/cookbooks/one-table/): A results.tsv in the top folder, one row a run, recorded with one command per arm.
  - [Record a folder per run](https://rabbitbrain.ai/docs/cookbooks/run-folders/): Each run's own folder, recorded from what its config says.
  - [Check a summary's claims against the runs](https://rabbitbrain.ai/docs/cookbooks/check-a-summary/): Turn each claim in an agent's summary into the criterion it states, and let rb read it.
  - [Declare the criterion before the runs](https://rabbitbrain.ai/docs/cookbooks/criterion-before-runs/): Declare what counts and freeze it, so the finding says it came first.
  - [Compare against a published baseline](https://rabbitbrain.ai/docs/cookbooks/published-baseline/): Cite the paper's number and settings, and get reproduced or not reproduced.
  - [Get an interval, not a single number](https://rabbitbrain.ai/docs/cookbooks/interval-from-seeds/): Run seeds until the criterion has the interval it needs.
  - [Read a rule over seeds](https://rabbitbrain.ai/docs/cookbooks/rules-over-seeds/): A count, a best of N and a relative change, each read its own way.
  - [Share the record, and rerun it](https://rabbitbrain.ai/docs/cookbooks/share-a-record/): A page your advisor reads, and a bundle their agent clones and reruns.
  - [Rerun a run from its receipt](https://rabbitbrain.ai/docs/cookbooks/rerun-from-receipt/): The command and the commit from the receipt, and a repeat that checks the run.
  - [rb with Claude Code, Codex and Cursor](https://rabbitbrain.ai/docs/cookbooks/three-agents/): One record, whichever agent writes to it.

## About the output on these pages

Every output block is real. For each cookbook we made a small project in a folder of its own: a training or evaluation script that writes fixed numbers per seed, a few configs, and sometimes a plan or a paper's table. Then we ran the commands its prompt asks for with rb 0.8.3, as the agent's, in an agent session as Claude Code runs them, and the commands marked as yours in a plain terminal. Two things were changed after the runs: the folder's path is shown as `~`, and long outputs are cut where a line says so.

Ids such as `run4k2xa9`, hashes and times differ in your run.

---

# Start tracking a project already under way

https://rabbitbrain.ai/docs/cookbooks/project-under-way/

This cookbook records where a repository's work stands, from its own files, in one agent turn.

You have weeks of work in a repository: configs, a plan, a results folder with a report, and no record of any of it. You want the record to start from what the files say, not from what anyone remembers.

## Paste this into your agent

After `rb init "<the work's theme>"` in your own terminal, paste the line it prints. rb does the rest of the briefing:

**Record where this repository's work stands now, from its own files.**

This is the line rb init prints, word for word.

```text
Use Rabbit Brain to track this: read rb context, then record where the work stands.
```

The repository we ran it on holds an INT8 post-training quantization of a ResNet-20 on CIFAR-10: `configs/` for training, FP32 and INT8; a plan in `docs/plan.md` ("INT8 ships if it loses at most 0.5 points of top-1 against FP32."); and `results/int8-ptq/` with three seeds and a report. It has two commits, both from before the record.

## What you'll see

`rb status` in your own terminal, once the agent is done:

```rb-session frame="terminal"
$ rb status
INT8 ResNet-20 on CIFAR-10 · int8-resnet-20-on-cifar-10
Kept with rb since 2026-11-08. All 3 runs were recorded afterwards, not as they happened.
Last reconcile, 2026-11-08 03:36: 1 re-indexed (int8-ptq) · 1 result quoted · 1 next step quoted

Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10? (open)
  INT8 loses at most 0.5 points of top-1 against FP32. change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb); loss: the interval clear of 0 on the worse side (higher is better); criterion met: estimate -0.08 ≥ -0.5; rule and question link declared after the evidence existed: all 3 units were produced before it was declared. units: 3 seeds.
  experiment context: int8-ptq · INT8 post-training quantization
          result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
          next: "Try per-channel calibration with 128 images." (README.md:11)
          note naoxcg3: The question is docs/plan.md:3 turned into a question: "INT8 ships if it loses at most 0.5 points of top-1 against FP32." · not confirmed
          git clean

Rules: 1 met

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - int8-ptq: 1 criterion was declared after its evidence existed

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:36 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
          criterion met · loss · change.top1 ≥ -0.5
          reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    loss: the interval clear of 0 on the worse side (higher is better)
          rule:      criterion met: estimate -0.08 ≥ -0.5
          timing:    declared after the evidence existed: all 3 units were produced before it was declared
          units:     3 seeds

Experiments
  int8-ptq INT8 post-training quantization · open · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · not frozen · settings 7 verified · 0 provisional · 0 unknown · 1 per-run · runs 3
          to run again: code unknown · environment unknown · files ok

Runs: 3 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm
```

- The line under the title says when the record began, and that all three runs were recorded afterwards, not as they happened.
- The question is the plan's line turned into a question, and a note on the experiment says so.
- `top1-drop-budget` is the criterion's id: its name in every command.
- The criterion is the plan's. INT8 is 0.08 points of top-1 below FP32, the 95% interval runs from about 0.15 to 0.005 points below, over 3 seeds, and the rule (`change.top1 ≥ -0.5`) is met.
- Its timing reads **declared after the evidence existed**: the three runs were made before the record began, and rb never reads them as if the criterion came first.
- The report's own result and next step are quoted word for word, with file and line.
- "to run again: code unknown · environment unknown": these runs were made before rb was watching, so their receipts cannot say which code or machine made them.

The experiment's design is the agent's until you confirm it:

```rb-session frame="terminal"
$ rb confirm
Designs waiting for a confirmation (1):
  int8-ptq: INT8 post-training quantization
questions: q-int8
hypothesis or purpose: not recorded
why: not recorded
variants: fp32 (baseline), int8 (candidate) · varies precision, calib_images
setting optim.lr: 0.1 from configs/train.yaml#optim.lr
setting optim.momentum: 0.9 from configs/train.yaml#optim.momentum
setting data.batch_size: 128 from configs/train.yaml#data.batch_size
setting eval.batch_size: 256 from configs/train.yaml#eval.batch_size
setting seed: per run
setting fp32.precision: "fp32" from configs/fp32.yaml#precision
setting int8.precision: "int8" from configs/int8.yaml#precision
setting int8.calib_images: 512 from configs/int8.yaml#calib_images
criterion top1-drop-budget (primary): change.top1 ≥ -0.5 · for q-int8
confirm: rb confirm int8-ptq@775110fdf0ef6a94a6d1aa30ed23b790454a0151cf4aa4f991f701e30af6773c
note naoxcg3 on int8-ptq stays not confirmed: a note is not part of the design. To say one as yours, write it in your own terminal: rb note add "<its words>" -e int8-ptq, then rb retract <note> --why "<reason>".
```

```rb-session frame="terminal"
$ rb confirm int8-ptq@775110fdf0ef6a94a6d1aa30ed23b790454a0151cf4aa4f991f701e30af6773c
int8-ptq's design is confirmed.
  note naoxcg3 on int8-ptq stays not confirmed: a note is not part of the design. To say one as yours, write it in your own terminal: rb note add "<its words>" -e int8-ptq, then rb retract <note> --why "<reason>".

Next: rb status
```

## What rb did, step by step

1. Read the briefing

   `rb context` opens with what to do now, and points at `rb docs --start`, the protocol for a project already under way:

   ```rb-session frame="terminal"
   $ rb context
   # The record: INT8 ResNet-20 on CIFAR-10

   ## First, record where the work stands

   Nothing is recorded yet, and this project holds work already done: 2 commits and 1 results folder. Record where the work stands now, in this turn, from the files, and ask the researcher nothing first. How: rb docs --start.

   … (53 more lines)
   ```

2. Read what the repository already says

   `rb reconcile` looks for results in folders named for them (`results/`, `eval/`, `metrics/`, `reports/`, `artifacts/` and the like), down to four levels from the top. It reads each results folder that has a report, and quotes it word for word. Under "Not read" it names a folder it passed over that holds results files of its own. Where each run keeps a folder with its own config, it says what each config records (the command, the commit, `dirty: true`, the settings that differ between folders) and prints the `rb run add` line for each. It names each results file in the top folder too: a table (`results.tsv`, `results.csv`, `results.jsonl` and the like) with the `--where` line that reads a row, and a report in prose (`RESULTS.md`, `report.md`), from which rb reads no run. It writes nothing.

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at 93d9471: 1 results folder.

   int8-ptq  results/int8-ptq/  new, not in the record
     title: "INT8 post-training quantization" (README.md:1)
     says: "Three seeds of FP32 and INT8, evaluated with `python eval.py --seed <N>`." (README.md:3)
     result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
     next: "Try per-channel calibration with 128 images." (README.md:11)
     files: 1 report · 6 results files
     git: all 7 report and results files are tracked · folder unchanged since the last commit

   rb reconcile --apply keeps this in the record: 1 experiment to add.
   ```

3. Record the question and the experiment

   ```rb-session frame="terminal"
   $ rb question add "Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10?" --id q-int8
   q-int8: Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10?

   Next: rb experiment add "<what would answer it>" -q q-int8 --hypothesis "<what you expect>"
   ```

   ```rb-session frame="terminal"
   $ rb experiment add "INT8 post-training quantization" --id int8-ptq -q q-int8 --baseline fp32 --candidate int8 --varies precision,calib_images
   Experiment int8-ptq: INT8 post-training quantization · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · under q-int8
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 2 variants: fp32 (baseline), int8 (candidate); a setting of one is rb spec set int8-ptq fp32.<setting> <value>.
     int8-ptq declares it varies precision, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.precision <value> --source <file#key>
     int8-ptq declares it varies calib_images, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.calib_images <value> --source <file#key>

   Next: rb spec set int8-ptq --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e int8-ptq -q q-int8 --metric <name> --at-most <x>
   ```

4. Cite every setting to the file that states it

   ```rb-session frame="terminal"
   $ rb spec set int8-ptq --from configs/train.yaml --keys "optim.*,data.*,eval.*"
   int8-ptq: 4 settings from configs/train.yaml @93d9471 · 4 verified

   $ rb spec set int8-ptq fp32.precision fp32 --source "configs/fp32.yaml#precision"
   int8-ptq/fp32.precision = fp32 · verified in configs/fp32.yaml#precision

   $ rb spec set int8-ptq int8.precision int8 --source "configs/int8.yaml#precision"
   int8-ptq/int8.precision = int8 · verified in configs/int8.yaml#precision

   $ rb spec set int8-ptq int8.calib_images 512 --source "configs/int8.yaml#calib_images"
   int8-ptq/int8.calib_images = 512 · verified in configs/int8.yaml#calib_images

   $ rb spec set int8-ptq seed --per-run
   int8-ptq/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

5. Record the plan's criterion, cited to its line

   rb checks that `docs/plan.md:3` is there and states the criterion's threshold.

   ```rb-session frame="terminal"
   $ rb metric add top1 --unit % --maximize
   Metric top1 (%): higher is better
   ```

   ```rb-session frame="terminal"
   $ rb criterion add "INT8 loses at most 0.5 points of top-1 against FP32" -e int8-ptq -q q-int8 --metric change.top1 --at-least -0.5 --why docs/plan.md:3 --id top1-drop-budget
   top1-drop-budget: INT8 loses at most 0.5 points of top-1 against FP32
     change.top1 ≥ -0.5 · primary · on int8-ptq
     declared; no evidence recorded yet

   Next: rb run add int8-ptq --from <the results file a run wrote> --set seed=<value>
   ```

6. Record each run from its own file

   ```rb-session frame="terminal"
   $ rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml
   runm4ei4d on int8-ptq: fp32.top1=91.71, int8.top1=91.6 · seed=1
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed1/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     top1-drop-budget estimate meets criterion · point only: change.top1: -0.11 % · paired difference, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/49a2f7fb6a172a9a46a4624957355aec6732c343, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/49a2f7fb6a172a9a46a4624957355aec6732c343 removes it.

   Next: rb show top1-drop-budget

   $ rb run add int8-ptq --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml
   run5b7apy on int8-ptq: fp32.top1=91.66, int8.top1=91.58 · seed=2
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed2/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     top1-drop-budget estimate meets criterion · point only: change.top1: -0.095 % · paired difference, 2 seeds (rb)

   Next: rb show top1-drop-budget

   $ rb run add int8-ptq --from results/int8-ptq/seed3/metrics.json --set seed=3 --config results/int8-ptq/seed3/config.yaml
   run23v5dl on int8-ptq: fp32.top1=91.74, int8.top1=91.69 · seed=3
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed3/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     top1-drop-budget criterion met · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)

   Next: rb show top1-drop-budget
   ```

7. Keep what the report says

   ```rb-session frame="terminal"
   $ rb reconcile --apply
   The repository at 93d9471: 1 results folder.

   Kept in the record: 1 re-indexed (int8-ptq) · 1 result quoted · 1 next step quoted.

   int8-ptq  results/int8-ptq/  in the record
     title: "INT8 post-training quantization" (README.md:1)
     says: "Three seeds of FP32 and INT8, evaluated with `python eval.py --seed <N>`." (README.md:3)
     result: "INT8 top-1 is 91.62 against FP32's 91.70 over three seeds: 0.08 points lower." (README.md:7)
     next: "Try per-channel calibration with 128 images." (README.md:11)
     files: 1 report · 6 results files
     git: all 7 report and results files are tracked · folder unchanged since the last commit
   ```

## Runs kept another way

- Each run in a folder of its own, with its config and metrics: [Record a folder per run](https://rabbitbrain.ai/docs/cookbooks/run-folders/).
- Each run a row of one table, such as `results.tsv`: [Record one table of many runs](https://rabbitbrain.ai/docs/cookbooks/one-table/).

## Why it matters

The record starts from what the files say, cited to the line, and every result made before it says so.

---

# Record one table of many runs

https://rabbitbrain.ai/docs/cookbooks/one-table/

This cookbook records a repository whose runs are the rows of one results table in its top folder.

Your training script adds a line to `results.tsv` after each run, and a `RESULTS.md` says what the runs showed. You want each row recorded as its own run, without typing a command per row.

## Paste this into your agent

**Record every row of results.tsv as a run, and check what RESULTS.md says.**

```text
My runs are the rows of results.tsv. Record them with Rabbit Brain, and check what RESULTS.md says against them.

- Run `rb context --model <your model id>` first, then `rb reconcile`.
- Record the question and the experiment, with each setting cited to the config that states it.
- Record every row of results.tsv as its own run, one `rb run add ... --every-row` per arm, with each row's seed and finish time.
- Turn the claim in RESULTS.md into the criterion it states, cited to its line.
- Compute nothing yourself: rb computes the findings.

Finish with `rb reconcile` and `rb status` exactly as printed.
```

The repository trains a ViT-Tiny on CIFAR-100 with no label smoothing and with smoothing 0.1, four seeds each. `train.py` adds one line to `results.tsv` per run:

```rb-session frame="terminal"
$ head -n 4 results.tsv
smoothing	seed	val_acc	finished_at
0.0	1	71.42	2026-10-27T09:00:00Z
0.1	1	72.06	2026-10-27T10:00:00Z
0.0	2	71.18	2026-10-27T11:00:00Z
```

```rb-session frame="terminal"
$ cat RESULTS.md
# Results

Label smoothing 0.1 raises validation accuracy by about 0.66 points over four seeds.
```

## What you'll see

`rb status` in your own terminal, once the agent is done:

```rb-session frame="terminal"
$ rb status
Label smoothing on a small ViT · label-smoothing-on-a-small-vit
Kept with rb since 2026-11-08. All 8 runs were recorded afterwards, not as they happened.

Does label smoothing 0.1 raise validation accuracy on ViT-Tiny? (open)
  Label smoothing 0.1 raises validation accuracy by about 0.66 points. change.val_acc: +0.66 [+0.6256, +0.6944] % · paired difference, t interval, 4 seeds, 95% (rb); gain: the interval clear of 0 on the better side (higher is better); reproduced on the estimate: estimate +0.66 within 0.66 ± 0.005; rule and question link declared after the evidence existed: all 4 units were produced before it was declared. units: 4 seeds from 8 runs; paired by seed: each seed's ls01 run against the same seed's none run; unpaired, the same runs read +0.66 [+0.3946, +0.9254] % · unpaired difference, Welch interval, 8 runs, 95% (rb); whether runs that share a seed are linked is not rb's to say. The repeat form is for a counter only (a seed, an epoch, a repeat's number), never a condition: where seed only numbers repeats, rb spec set ls population.repeat seed --no-verify reads each run as its own unit; no run config was compared with the spec (record runs with --config); the interval [+0.6256, +0.6944] crosses 0.655 and 0.665: reproduced on the estimate, not across the whole interval.

Rules: 1 reproduced, on the estimate alone

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - ls: 1 criterion was declared after its evidence existed

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:37 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  ls-gain-claim Label smoothing 0.1 raises validation accuracy by about 0.66 points
          reproduced on the estimate · gain · change.val_acc = 0.66 ± 0.005
          reads:     change.val_acc: +0.66 [+0.6256, +0.6944] % · paired difference, t interval, 4 seeds, 95% (rb)
          result:    gain: the interval clear of 0 on the better side (higher is better)
          rule:      reproduced on the estimate: estimate +0.66 within 0.66 ± 0.005
          timing:    declared after the evidence existed: all 4 units were produced before it was declared
          units:     4 seeds from 8 runs
          · paired by seed: each seed's ls01 run against the same seed's none run; unpaired, the same runs read +0.66 [+0.3946, +0.9254] % · unpaired difference, Welch interval, 8 runs, 95% (rb); whether runs that share a seed are linked is not rb's to say. The repeat form is for a counter only (a seed, an epoch, a repeat's number), never a condition: where seed only numbers repeats, rb spec set ls population.repeat seed --no-verify reads each run as its own unit
          · no run config was compared with the spec (record runs with --config)
          · the interval [+0.6256, +0.6944] crosses 0.655 and 0.665: reproduced on the estimate, not across the whole interval

Experiments
  ls      Label smoothing 0.1 vs none · open · none (baseline) vs ls01 (candidate) · varies smoothing · not frozen · settings 5 verified · 0 provisional · 0 unknown · 1 per-run · runs 8
          to run again: code unknown · environment unknown · files ok

Runs: 8 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm
```

- All 8 runs come from rows of `results.tsv`, and the line under the title says they were recorded afterwards, not as they happened.
- `ls-gain-claim` is the criterion's id: its name in every command. It is RESULTS.md's claim, and it reads **reproduced on the estimate**. The estimate, +0.66, is within 0.005 of 0.66, but the interval is wider than that. The tally says so too: "1 reproduced, on the estimate alone".
- The arms were recorded apart, a run per row, so the reading says what it pairs them by: "paired by seed", each seed's ls01 run against the same seed's none run. The unpaired reading of the same runs is beside it. Whether runs that share a seed are linked is not rb's to say. Where a seed only numbers repeats, rb gives the command that reads each run as its own unit, `rb spec set ls population.repeat seed --no-verify`, for a counter only.
- Its timing reads **declared after the evidence existed**: every row was written before the criterion.
- "no run config was compared with the spec": the runs were recorded from rows, with no `--config`.

## What rb did, step by step

1. Read what the repository already says

   `rb reconcile` names each results file in the top folder. `results.tsv` is a table: rb gives its rows, the columns that hold numbers, and the `rb run add ... --where` line that reads a row. `RESULTS.md` is a report in prose: rb reads no run from it, and a number it states is checked by a cited rule.

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at d17af26: 2 results files in the top folder.

   Results files in no results folder (2):
     RESULTS.md: prose: rb reads no run from it; a number there is a cited rule
       a number it states is checked by a cited rule: rb criterion add "<what it claims>" -e <experiment> --metric <metric> --equals <the number> --tolerance <half its last digit> --source RESULTS.md --quote "<its words>"
     results.tsv: a TSV table of 8 rows, numbers in smoothing, seed, val_acc
       a row is a run: rb run add <experiment> --from results.tsv --where <column>=<value>,...
   ```

2. Record the question, the experiment and its settings

   ```rb-session frame="terminal"
   $ rb question add "Does label smoothing 0.1 raise validation accuracy on ViT-Tiny?" --id q-ls
   q-ls: Does label smoothing 0.1 raise validation accuracy on ViT-Tiny?

   Next: rb experiment add "<what would answer it>" -q q-ls --hypothesis "<what you expect>"
   ```

   ```rb-session frame="terminal"
   $ rb experiment add "Label smoothing 0.1 vs none" --id ls -q q-ls --baseline none --candidate ls01 --varies smoothing
   Experiment ls: Label smoothing 0.1 vs none · none (baseline) vs ls01 (candidate) · varies smoothing · under q-ls
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 2 variants: none (baseline), ls01 (candidate); a setting of one is rb spec set ls none.<setting> <value>.
     ls declares it varies smoothing, but no variant has a value for it yet; give each variant its own: rb spec set ls <variant>.smoothing <value> --source <file#key>

   Next: rb spec set ls --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e ls -q q-ls --metric <name> --at-most <x>
   ```

   ```rb-session frame="terminal"
   $ rb spec set ls --from configs/train.yaml --keys "optim.*,data.*"
   ls: 3 settings from configs/train.yaml @d17af26 · 3 verified

   $ rb spec set ls none.smoothing 0.0 --source "configs/none.yaml#smoothing"
   ls/none.smoothing = 0.0 · verified in configs/none.yaml#smoothing

   $ rb spec set ls ls01.smoothing 0.1 --source "configs/ls01.yaml#smoothing"
   ls/ls01.smoothing = 0.1 · verified in configs/ls01.yaml#smoothing

   $ rb spec set ls seed --per-run
   ls/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

   ```rb-session frame="terminal"
   $ rb metric add val_acc --unit % --maximize
   Metric val_acc (%): higher is better
   ```

3. Record every row, one command per arm

   `--where smoothing=0.0` picks the four rows of one arm, and `--every-row` records each as its own run. `--set seed=@seed` and `--produced-at @finished_at` read each run's seed and time from its own row. Every row is read before any is written, and a row already recorded is passed.

   ```rb-session frame="terminal"
   $ rb run add ls --from results.tsv --where smoothing=0.0 --variant none --every-row --set seed=@seed --produced-at @finished_at
   rundj7dwa on ls: none.val_acc=71.42 · seed=1
   runoabazi on ls: none.val_acc=71.18 · seed=2
   runxm3w7v on ls: none.val_acc=71.55 · seed=3
   runqfzjce on ls: none.val_acc=71.3 · seed=4
   4 of the 4 rows --where picks in results.tsv recorded as runs.
     rb put the code in git at refs/rb/snapshots/2f2ced2447df1446e5704536fc328b9bc50e2bb1, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/2f2ced2447df1446e5704536fc328b9bc50e2bb1 removes it.

   $ rb run add ls --from results.tsv --where smoothing=0.1 --variant ls01 --every-row --set seed=@seed --produced-at @finished_at
   runatzcyo on ls: ls01.val_acc=72.06 · seed=1
   run5ztvun on ls: ls01.val_acc=71.87 · seed=2
   runwp3vf5 on ls: ls01.val_acc=72.21 · seed=3
   runuyne2r on ls: ls01.val_acc=71.95 · seed=4
   4 of the 4 rows --where picks in results.tsv recorded as runs.
   ```

   A `.jsonl` or `.ndjson` file is read the same way, one JSON object a row. Without `--every-row`, a `--where` that picks several rows is refused, and offers it.

   `--every-row` records a SQLite table's rows too: `rb run add <exp> --from results.db --table evals --where precision=int8 --set seed=@seed --every-row`.

4. Check the report's claim

   ```rb-session frame="terminal"
   $ rb compare ls
   ls  Label smoothing 0.1 vs none · varies smoothing · 8 runs under the current spec · means over seeds
     variant  role       smoothing  val_acc (%)             
     none     baseline   0.0        71.36 (n=4)             
     ls01     candidate  0.1        72.02 +0.66 better (n=4)
   ```

   ```rb-session frame="terminal"
   $ rb criterion add "Label smoothing 0.1 raises validation accuracy by about 0.66 points" -e ls -q q-ls --metric change.val_acc --equals 0.66 --tolerance 0.005 --source RESULTS.md --quote "Label smoothing 0.1 raises validation accuracy by about 0.66 points over four seeds." --id ls-gain-claim
   ls-gain-claim: Label smoothing 0.1 raises validation accuracy by about 0.66 points
     change.val_acc = 0.66 ± 0.005 · primary · on ls
     cited: RESULTS.md "Label smoothing 0.1 raises validation accuracy by about 0..." · states 0.66 (read 'Label smoothing 0.1 raises validation accuracy by about 0.66 points over four seeds.')
     declared after the evidence existed: all 4 units were produced before it was declared
     it already reads change.val_acc: +0.66 [+0.6256, +0.6944] % · paired difference, t interval, 4 seeds, 95% (rb): reproduced on the estimate: estimate +0.66 within 0.66 ± 0.005

   Next: rb run add ls --from <the results file a run wrote> --set seed=<value>
   ```

   ```rb-session frame="terminal"
   $ rb show ls-gain-claim
   Criterion ls-gain-claim on ls (primary, cited): Label smoothing 0.1 raises validation accuracy by about 0.66 points
     cited: RESULTS.md "Label smoothing 0.1 raises validation accuracy by about 0..." · states 0.66, read 'Label smoothing 0.1 raises validation accuracy by about 0.66 points over four seeds.'
     ls-gain-claim Label smoothing 0.1 raises validation accuracy by about 0.66 points
             reproduced on the estimate · gain · change.val_acc = 0.66 ± 0.005
             reads:     change.val_acc: +0.66 [+0.6256, +0.6944] % · paired difference, t interval, 4 seeds, 95% (rb)
             result:    gain: the interval clear of 0 on the better side (higher is better)
             rule:      reproduced on the estimate: estimate +0.66 within 0.66 ± 0.005
             timing:    declared after the evidence existed: all 4 units were produced before it was declared
             units:     4 seeds from 8 runs
             integrity: no run config was compared, as said below
             · paired by seed: each seed's ls01 run against the same seed's none run; unpaired, the same runs read +0.66 [+0.3946, +0.9254] % · unpaired difference, Welch interval, 8 runs, 95% (rb); whether runs that share a seed are linked is not rb's to say. The repeat form is for a counter only (a seed, an epoch, a repeat's number), never a condition: where seed only numbers repeats, rb spec set ls population.repeat seed --no-verify reads each run as its own unit
             · no run config was compared with the spec (record runs with --config)
             · the interval [+0.6256, +0.6944] crosses 0.655 and 0.665: reproduced on the estimate, not across the whole interval
     rundj7dwa: none.val_acc 71.42 (row) · unit seed=1 · produced before the declaration · seed=1
     runoabazi: none.val_acc 71.18 (row) · unit seed=2 · produced before the declaration · seed=2
     runxm3w7v: none.val_acc 71.55 (row) · unit seed=3 · produced before the declaration · seed=3
     runqfzjce: none.val_acc 71.3 (row) · unit seed=4 · produced before the declaration · seed=4
     runatzcyo: ls01.val_acc 72.06 (row) · unit seed=1 · produced before the declaration · seed=1
     run5ztvun: ls01.val_acc 71.87 (row) · unit seed=2 · produced before the declaration · seed=2
     runwp3vf5: ls01.val_acc 72.21 (row) · unit seed=3 · produced before the declaration · seed=3
     runuyne2r: ls01.val_acc 71.95 (row) · unit seed=4 · produced before the declaration · seed=4
   ```

   Each run's number reads "(row)": it came from a row of the table.

5. Read the repository again

   Once the table's rows are recorded, `rb reconcile` says how many:

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at d17af26: 2 results files in the top folder.

   Results files in no results folder (2):
     RESULTS.md: prose: rb reads no run from it; a number there is a cited rule
       a number it states is checked by a cited rule: rb criterion add "<what it claims>" -e <experiment> --metric <metric> --equals <the number> --tolerance <half its last digit> --source RESULTS.md --quote "<its words>"
     results.tsv: a TSV table of 8 rows, numbers in smoothing, seed, val_acc; 8 of 8 rows recorded as runs
       a row is a run: rb run add <experiment> --from results.tsv --where <column>=<value>,...
   ```

## Why it matters

A table of runs becomes a run per row, each with its own seed and time, and a claim in prose is checked against those runs, not copied from the report.

---

# Record a folder per run

https://rabbitbrain.ai/docs/cookbooks/run-folders/

This cookbook records a repository where each run keeps its own folder, with its config and its metrics.

Your eval script writes `results/<run>/config.yaml` and `results/<run>/metrics.json` for every run: the command, the commit, whether the code had uncommitted changes, and the numbers. You want each folder recorded as the run it is, with what its config says.

## Paste this into your agent

**Record each run folder in results/ as a run, from what its own config says.**

```text
Each folder in results/ is one run of eval.py. Record them with Rabbit Brain.

- Run `rb context --model <your model id>` first, then `rb reconcile`.
- Record the question and the experiment, with each setting cited to the config that states it, and the metric.
- Run `rb reconcile` again, and record each folder with the `rb run add` line it prints, word for word.
- Compute nothing yourself: rb computes the findings.

Finish with `rb reconcile` and `rb status` exactly as printed.
```

The repository scores a small model on 500 grade-school maths problems with a plain prompt and with three worked examples, two seeds each. Each run has its folder in `results/`, `r01` to `r04`. The fourth one's config and metrics:

```rb-session frame="terminal"
$ cat results/r04/config.yaml
command: python eval.py --config configs/examples.yaml --seed 2 --out results/r04
commit: 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc
dirty: true
started_at: 2026-10-28T11:30:00Z
prompt: examples
problems: 500
seed: 2
```

```rb-session frame="terminal"
$ cat results/r04/metrics.json
{
  "accuracy": 46.9,
  "finished_at": "2026-10-28T11:53:00Z"
}
```

The fourth run started while `prompts/examples.txt` held an edit nobody committed, so its config says `dirty: true`.

## What you'll see

`rb status` in your own terminal, once the agent is done:

```rb-session frame="terminal"
$ rb status
Worked examples in the prompt · worked-examples-in-the-prompt
Kept with rb since 2026-11-08. All 4 runs were recorded afterwards, not as they happened.

Do three worked examples in the prompt raise accuracy on grade-school maths? (open)
  No criterion is linked directly to this question.
  Worked examples vs a plain prompt · open

Rules: none yet

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - examples: runs report examples.accuracy, plain.accuracy, which no recorded criterion reads

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:37 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Experiments
  examples Worked examples vs a plain prompt · open · plain (baseline) vs examples (candidate) · varies prompt · not frozen · settings 3 verified · 0 provisional · 0 unknown · 1 per-run · runs 4
          to run again: code partial · environment unknown · files ok

Runs: 4 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm
```

- The four runs are recorded, and the line under the title says they were recorded afterwards, not as they happened.
- "environment unknown": these runs were made before rb was watching, so their receipts cannot say which machine made them. Each receipt keeps the commit and the command its config gives.
- No criterion is recorded yet: no file here states what counts. "runs report examples.accuracy, plain.accuracy, which no recorded criterion reads" says so.

## What rb did, step by step

1. Read what the repository already says

   A results folder that holds a run's own config beside a results file is a folder of one run. For each, `rb reconcile` says what its config says of the run, as the file's words with their lines: the command, the commit, `dirty`, and when it started. It names the settings that differ between the folders, and prints an `rb run add` line for each. With no experiment yet, the line reads `<experiment>`. It writes nothing.

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at 855f30c: 4 results folders.

   4 results folders hold results files no recorded run named, in results/ (4 folders). With no report of its own none is an experiment: rb run add <experiment> --from <its file> records a run from one.

   Run folders, each with its own record of the run (4). What each record says is the file's, not what rb saw:
     results/r01/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 1 --out results/r01' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:00:00Z (line 4)
       rb run add '<experiment>' --from results/r01/metrics.json --config results/r01/config.yaml --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:20:00Z --command 'python eval.py --config configs/plain.yaml --seed 1 --out results/r01' --command-source results/r01/config.yaml:1
     results/r02/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 1 --out results/r02' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:30:00Z (line 4)
       rb run add '<experiment>' --from results/r02/metrics.json --config results/r02/config.yaml --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:52:00Z --command 'python eval.py --config configs/examples.yaml --seed 1 --out results/r02' --command-source results/r02/config.yaml:1
     results/r03/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 2 --out results/r03' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T11:00:00Z (line 4)
       rb run add '<experiment>' --from results/r03/metrics.json --config results/r03/config.yaml --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:21:00Z --command 'python eval.py --config configs/plain.yaml --seed 2 --out results/r03' --command-source results/r03/config.yaml:1
     results/r04/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 2 --out results/r04' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: true (line 3) · started_at: 2026-10-28T11:30:00Z (line 4)
       results/r04/config.yaml:3 says dirty: true: the code had uncommitted changes when it ran
       rb run add '<experiment>' --from results/r04/metrics.json --config results/r04/config.yaml --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:53:00Z --command 'python eval.py --config configs/examples.yaml --seed 2 --out results/r04' --command-source results/r04/config.yaml:1
     Settings that differ between them (2):
       prompt: plain (r01, r03), examples (r02, r04)
       seed: 1 (r01, r02), 2 (r03, r04)
   ```

2. Record the question, the experiment and its settings

   ```rb-session frame="terminal"
   $ rb question add "Do three worked examples in the prompt raise accuracy on grade-school maths?" --id q-examples
   q-examples: Do three worked examples in the prompt raise accuracy on grade-school maths?

   Next: rb experiment add "<what would answer it>" -q q-examples --hypothesis "<what you expect>"
   ```

   ```rb-session frame="terminal"
   $ rb experiment add "Worked examples vs a plain prompt" --id examples -q q-examples --baseline plain --candidate examples --varies prompt
   Experiment examples: Worked examples vs a plain prompt · plain (baseline) vs examples (candidate) · varies prompt · under q-examples
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 2 variants: plain (baseline), examples (candidate); a setting of one is rb spec set examples plain.<setting> <value>.
     examples declares it varies prompt, but no variant has a value for it yet; give each variant its own: rb spec set examples <variant>.prompt <value> --source <file#key>

   Next: rb spec set examples --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e examples -q q-examples --metric <name> --at-most <x>
   ```

   ```rb-session frame="terminal"
   $ rb spec set examples plain.prompt plain --source "configs/plain.yaml#prompt"
   examples/plain.prompt = plain · verified in configs/plain.yaml#prompt

   $ rb spec set examples examples.prompt examples --source "configs/examples.yaml#prompt"
   examples/examples.prompt = examples · verified in configs/examples.yaml#prompt

   $ rb spec set examples problems 500 --source "configs/plain.yaml#problems"
   examples/problems = 500 · verified in configs/plain.yaml#problems

   $ rb spec set examples seed --per-run
   examples/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

   ```rb-session frame="terminal"
   $ rb metric add accuracy --unit % --maximize
   Metric accuracy (%): higher is better
   ```

3. Read the folders again

   Now each line names the experiment, the variant the config's `prompt` names, the metric, and the seed:

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at 855f30c: 4 results folders.

   4 results folders hold results files no recorded run named, in results/ (4 folders). With no report of its own none is an experiment: rb run add <experiment> --from <its file> records a run from one.

   Run folders, each with its own record of the run (4). What each record says is the file's, not what rb saw:
     results/r01/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 1 --out results/r01' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:00:00Z (line 4)
       rb run add examples --from results/r01/metrics.json --variant plain --keys accuracy --config results/r01/config.yaml --set seed=1 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:20:00Z --command 'python eval.py --config configs/plain.yaml --seed 1 --out results/r01' --command-source results/r01/config.yaml:1
     results/r02/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 1 --out results/r02' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:30:00Z (line 4)
       rb run add examples --from results/r02/metrics.json --variant examples --keys accuracy --config results/r02/config.yaml --set seed=1 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:52:00Z --command 'python eval.py --config configs/examples.yaml --seed 1 --out results/r02' --command-source results/r02/config.yaml:1
     results/r03/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 2 --out results/r03' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T11:00:00Z (line 4)
       rb run add examples --from results/r03/metrics.json --variant plain --keys accuracy --config results/r03/config.yaml --set seed=2 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:21:00Z --command 'python eval.py --config configs/plain.yaml --seed 2 --out results/r03' --command-source results/r03/config.yaml:1
     results/r04/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 2 --out results/r04' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: true (line 3) · started_at: 2026-10-28T11:30:00Z (line 4)
       results/r04/config.yaml:3 says dirty: true: the code had uncommitted changes when it ran
       rb run add examples --from results/r04/metrics.json --variant examples --keys accuracy --config results/r04/config.yaml --set seed=2 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:53:00Z --command 'python eval.py --config configs/examples.yaml --seed 2 --out results/r04' --command-source results/r04/config.yaml:1
     Settings that differ between them (2):
       prompt: plain (r01, r03), examples (r02, r04)
       seed: 1 (r01, r02), 2 (r03, r04)
   ```

4. Record each folder with its line

   Each run is checked against its own config ("Ran with the spec: 3 of its settings found in results/r01/config.yaml, all matching"), and is dated by the `finished_at` its metrics file gives. r04's receipt keeps what its config says: "dirty: true: the code had uncommitted changes when it ran".

   ```rb-session frame="terminal"
   $ rb run add examples --from results/r01/metrics.json --variant plain --keys accuracy --config results/r01/config.yaml --set seed=1 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:20:00Z --command "python eval.py --config configs/plain.yaml --seed 1 --out results/r01" --command-source results/r01/config.yaml:1
   runhbskex on examples: plain.accuracy=41.2 · seed=1
     Ran with the spec: 3 of its settings found in results/r01/config.yaml, all matching.
     Produced: 2026-10-28 10:20:00 (reported)
     rb put the code in git at refs/rb/snapshots/6edfbdd7cd3fe9b2350eee1cfbc31b16fd6806d1, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/6edfbdd7cd3fe9b2350eee1cfbc31b16fd6806d1 removes it.

   Since this session last looked:
     - 3 results folders hold results written before the record began, which no recorded experiment reads yet: results/r02/, results/r03/, results/r04/   (rb reconcile)

   Next: rb criterion add "<what it should show>" -e examples -q q-examples --metric plain.accuracy --at-most <x>   (write the criterion before the next run)

   $ rb run add examples --from results/r02/metrics.json --variant examples --keys accuracy --config results/r02/config.yaml --set seed=1 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T10:52:00Z --command "python eval.py --config configs/examples.yaml --seed 1 --out results/r02" --command-source results/r02/config.yaml:1
   runkb5nfe on examples: examples.accuracy=47.8 · seed=1
     Ran with the spec: 3 of its settings found in results/r02/config.yaml, all matching.
     Produced: 2026-10-28 10:52:00 (reported)

   Since this session last looked:
     - 2 results folders hold results written before the record began, which no recorded experiment reads yet: results/r03/, results/r04/   (rb reconcile)

   Next: rb criterion add "<what it should show>" -e examples -q q-examples --metric examples.accuracy --at-most <x>   (write the criterion before the next run)

   $ rb run add examples --from results/r03/metrics.json --variant plain --keys accuracy --config results/r03/config.yaml --set seed=2 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:21:00Z --command "python eval.py --config configs/plain.yaml --seed 2 --out results/r03" --command-source results/r03/config.yaml:1
   runnirx3r on examples: plain.accuracy=40.6 · seed=2
     Ran with the spec: 3 of its settings found in results/r03/config.yaml, all matching.
     Produced: 2026-10-28 11:21:00 (reported)

   Since this session last looked:
     - results/r04/ holds results written before the record began, which no recorded experiment reads yet   (rb reconcile)

   Next: rb criterion add "<what it should show>" -e examples -q q-examples --metric plain.accuracy --at-most <x>   (write the criterion before the next run)

   $ rb run add examples --from results/r04/metrics.json --variant examples --keys accuracy --config results/r04/config.yaml --set seed=2 --commit 48917c6e0ef4fc881eff8e5fc999ce9aabe62abc --produced-at 2026-10-28T11:53:00Z --command "python eval.py --config configs/examples.yaml --seed 2 --out results/r04" --command-source results/r04/config.yaml:1
   runv2drww on examples: examples.accuracy=46.9 · seed=2
     Ran with the spec: 3 of its settings found in results/r04/config.yaml, all matching.
     results/r04/config.yaml:3 says dirty: true: the code had uncommitted changes when it ran.
     Produced: 2026-10-28 11:53:00 (reported)

   Next: rb criterion add "<what it should show>" -e examples -q q-examples --metric examples.accuracy --at-most <x>   (write the criterion before the next run)
   ```

5. Read the repository again

   Each folder now says which run it was recorded as, and that the runs of `examples` read it:

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at 855f30c: 4 results folders.

   Run folders, each with its own record of the run (4). What each record says is the file's, not what rb saw:
     results/r01/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 1 --out results/r01' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:00:00Z (line 4)
       recorded as runhbskex: rb show runhbskex
     results/r02/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 1 --out results/r02' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T10:30:00Z (line 4)
       recorded as runkb5nfe: rb show runkb5nfe
     results/r03/  config.yaml says command: 'python eval.py --config configs/plain.yaml --seed 2 --out results/r03' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: false (line 3) · started_at: 2026-10-28T11:00:00Z (line 4)
       recorded as runnirx3r: rb show runnirx3r
     results/r04/  config.yaml says command: 'python eval.py --config configs/examples.yaml --seed 2 --out results/r04' (line 1) · commit: 48917c6e0ef4 (line 2) · dirty: true (line 3) · started_at: 2026-10-28T11:30:00Z (line 4)
       results/r04/config.yaml:3 says dirty: true: the code had uncommitted changes when it ran
       recorded as runv2drww: rb show runv2drww
     Settings that differ between them (2):
       prompt: plain (r01, r03), examples (r02, r04)
       seed: 1 (r01, r02), 2 (r03, r04)

   results/r01/  read by the runs of examples
     files: 2 results files
     git: both report and results files are tracked · folder unchanged since the last commit

   results/r02/  read by the runs of examples
     files: 2 results files
     git: both report and results files are tracked · folder unchanged since the last commit

   results/r03/  read by the runs of examples
     files: 2 results files
     git: both report and results files are tracked · folder unchanged since the last commit

   results/r04/  read by the runs of examples
     files: 2 results files
     git: both report and results files are tracked · folder unchanged since the last commit
   ```

## Why it matters

A run's own folder already says how it ran. rb records it from those words, with their lines, and says where the config reports uncommitted code.

---

# Check a summary's claims against the runs

https://rabbitbrain.ai/docs/cookbooks/check-a-summary/

This cookbook checks each claim in an agent's summary against the runs it is about.

An agent's session ended with a confident summary, and you are about to repeat it to your team. Before you do, each claim should be checked against the runs, by criteria rb reads, not by another summary.

## Paste this into your agent

**Check every claim in SUMMARY.md against the runs, with rb doing the reading.**

```text
My last session wrote SUMMARY.md. Check each claim in it against the runs with Rabbit Brain, and take nothing on the summary's word.

- Run `rb context --model <your model id>` first.
- Record the experiment, and every run from its own file in `results/`.
- Turn each claim into the criterion it states, cited to its line: `--source SUMMARY.md --quote "<the claim's words>"`. Use the claim's own number, with no threshold of yours.
- A claim no criterion can read goes in a note that names the files you read.
- Compute nothing yourself: rb computes the findings.

Finish with `rb status` exactly as printed, and for each claim, what rb says of it.
```

We ran it on the INT8 ResNet-20 repository, with three seeds in `results/int8-ptq/` and this `SUMMARY.md` from an earlier session:

```markdown title="SUMMARY.md"
# Session summary

- INT8 matches FP32: it never loses more than 0.05 points of top-1 on any seed.
- INT8 averages 91.62 top-1 over the three seeds.
- 512 calibration images are enough.
```

## What you'll see

```rb-session frame="terminal"
$ rb status
INT8 ResNet-20 on CIFAR-10 · int8-resnet-20-on-cifar-10
Kept with rb since 2026-11-08. All 3 runs were recorded afterwards, not as they happened.

Does INT8 match FP32 top-1 on CIFAR-10? (open)
  On every int8-ptq criterion below: question link declared after the evidence existed: all 3 units were produced before it was declared; units: 3 seeds.
  The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here. top1: fp32 vs int8: +0.08 [+0.005476, +0.1545] % · paired difference, t interval, 3 seeds, 95% (rb); gain: the interval clear of 0 on the better side (higher is better); not reproduced: 2 of 3 units are above 0.05 (seed=1, seed=2); rule and question link declared after the evidence existed: all 3 units were produced before it was declared.
  The summary's claim that INT8 averages 91.62 top-1 is checked here. int8.top1: 91.6233 [91.4778, 91.7689] % · mean, t interval, 3 seeds, 95% (rb); measured: int8 on its own, with its interval, so neither better nor worse; reproduced on the estimate: estimate 91.6233 within 91.62 ± 0.005; rule and question link declared after the evidence existed: all 3 units were produced before it was declared. the interval [91.48, 91.77] crosses 91.615 and 91.625: reproduced on the estimate, not across the whole interval.
  experiment context: int8-ptq · INT8 post-training quantization
          note n25nx5v: No file I read compares calibration sizes, so no rule reads the claim that 512 calibration images are enough (SUMMARY.md:5). Files read: SUMMARY.md, configs/int8.yaml, results/int8-ptq/seed1/, seed2/, seed3/. · not confirmed

Rules: 1 reproduced, on the estimate alone · 1 not reproduced

Needs judgment (1)
  - every-seed-claim is not reproduced and no call on it is recorded: The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here
      rb decide every-seed-claim accept|reject|investigate --why "..."

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - int8-ptq: 2 criteria were declared after their evidence existed

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:36 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  every-seed-claim The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here
          not reproduced · gain · each unit's top1: fp32 vs int8 ≤ 0.05
          reads:     top1: fp32 vs int8: +0.08 [+0.005476, +0.1545] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    gain: the interval clear of 0 on the better side (higher is better)
          rule:      not reproduced: 2 of 3 units are above 0.05 (seed=1, seed=2)
          timing:    declared after the evidence existed: all 3 units were produced before it was declared
          units:     3 seeds
  int8-mean-claim The summary's claim that INT8 averages 91.62 top-1 is checked here
          reproduced on the estimate · measured · int8.top1 = 91.62 ± 0.005
          reads:     int8.top1: 91.6233 [91.4778, 91.7689] % · mean, t interval, 3 seeds, 95% (rb)
          result:    measured: int8 on its own, with its interval, so neither better nor worse
          rule:      reproduced on the estimate: estimate 91.6233 within 91.62 ± 0.005
          timing:    declared after the evidence existed: all 3 units were produced before it was declared
          units:     3 seeds
          · the interval [91.48, 91.77] crosses 91.615 and 91.625: reproduced on the estimate, not across the whole interval

Experiments
  int8-ptq INT8 post-training quantization · open · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · not frozen · settings 3 verified · 0 provisional · 0 unknown · 1 per-run · runs 3
          to run again: code unknown · environment unknown · files ok

Runs: 3 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm  ·  rb decide every-seed-claim accept|reject|investigate --why "..."
```

`every-seed-claim` and `int8-mean-claim` are the criteria's ids: their names in every command.

Claim by claim:

- **"It never loses more than 0.05 points of top-1 on any seed"**: **not reproduced**. rb names the seeds that break it: seed 1 and seed 2. This criterion compares FP32 against INT8, so "gain" on its result line means FP32 is ahead.
- **"INT8 averages 91.62 top-1"**: **reproduced on the estimate**. The mean over three seeds is 91.6233, within 0.005 of 91.62. The interval is wider than that: reproduced on the estimate, not across the whole interval. The tally says so too: "1 reproduced, on the estimate alone".
- **"512 calibration images are enough"**: no criterion reads it, since no run compares calibration sizes. The agent left a note that says so, with the files it read.

The miss waits for you under **Needs judgment**: whether to accept, reject or investigate it is your call, not the agent's.

## What rb did, step by step

1. Record the experiment and its settings

   ```rb-session frame="terminal"
   $ rb experiment add "INT8 post-training quantization" --id int8-ptq -q q-int8 --baseline fp32 --candidate int8 --varies precision,calib_images
   Experiment int8-ptq: INT8 post-training quantization · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · under q-int8
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 2 variants: fp32 (baseline), int8 (candidate); a setting of one is rb spec set int8-ptq fp32.<setting> <value>.
     int8-ptq declares it varies precision, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.precision <value> --source <file#key>
     int8-ptq declares it varies calib_images, but no variant has a value for it yet; give each variant its own: rb spec set int8-ptq <variant>.calib_images <value> --source <file#key>

   Next: rb spec set int8-ptq --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e int8-ptq -q q-int8 --metric <name> --at-most <x>
   ```

   ```rb-session frame="terminal"
   $ rb spec set int8-ptq fp32.precision fp32 --source "configs/fp32.yaml#precision"
   int8-ptq/fp32.precision = fp32 · verified in configs/fp32.yaml#precision

   $ rb spec set int8-ptq int8.precision int8 --source "configs/int8.yaml#precision"
   int8-ptq/int8.precision = int8 · verified in configs/int8.yaml#precision

   $ rb spec set int8-ptq int8.calib_images 512 --source "configs/int8.yaml#calib_images"
   int8-ptq/int8.calib_images = 512 · verified in configs/int8.yaml#calib_images

   $ rb spec set int8-ptq seed --per-run
   int8-ptq/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

2. Turn each claim into the criterion it states

   A criterion with `--source` must find its number in the quoted line, and rb checks the line is there. The "never loses more than 0.05" claim is written as FP32 against INT8, at most 0.05 on every seed (`--each`), so the number in the criterion is the number in the line.

   ```rb-session frame="terminal"
   $ rb criterion add "The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here" -e int8-ptq -q q-int8 --metric top1 --subject fp32 --reference int8 --at-most 0.05 --each --source SUMMARY.md --quote "it never loses more than 0.05 points of top-1 on any seed" --id every-seed-claim
   every-seed-claim: The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here
     each unit's top1: fp32 vs int8 ≤ 0.05 · primary · on int8-ptq
     cited: SUMMARY.md "it never loses more than 0.05 points of top-1 on any seed" · states 0.05 (read '- INT8 matches FP32: it never loses more than 0.05 points of top-1 on any seed.')
     declared; no evidence recorded yet

   Next: rb run add int8-ptq --from <the results file a run wrote> --set seed=<value>

   $ rb criterion add "The summary's claim that INT8 averages 91.62 top-1 is checked here" -e int8-ptq -q q-int8 --metric int8.top1 --equals 91.62 --tolerance 0.005 --source SUMMARY.md --quote "INT8 averages 91.62 top-1 over the three seeds" --id int8-mean-claim
   int8-mean-claim: The summary's claim that INT8 averages 91.62 top-1 is checked here
     int8.top1 = 91.62 ± 0.005 · primary · on int8-ptq
     cited: SUMMARY.md "INT8 averages 91.62 top-1 over the three seeds" · states 91.62 (read '- INT8 averages 91.62 top-1 over the three seeds.')
     declared; no evidence recorded yet

   Next: rb run add int8-ptq --from <the results file a run wrote> --set seed=<value>
   ```

3. Record each run from its own file

   The findings change as each run lands. After the third:

   ```rb-session frame="terminal"
   $ rb run add int8-ptq --from results/int8-ptq/seed3/metrics.json --set seed=3 --config results/int8-ptq/seed3/config.yaml
   run7fviep on int8-ptq: fp32.top1=91.74, int8.top1=91.69 · seed=3
     Ran with the spec: 1 of its settings found in results/int8-ptq/seed3/config.yaml, matching.
     Produced: 2026-10-20 10:00:00 (file)
     every-seed-claim not reproduced · gain: top1: fp32 vs int8: +0.08 [+0.005476, +0.1545] % · paired difference, t interval, 3 seeds, 95% (rb)
     int8-mean-claim reproduced on the estimate · measured: int8.top1: 91.6233 [91.4778, 91.7689] % · mean, t interval, 3 seeds, 95% (rb)

   Next: rb show every-seed-claim  ·  rb show int8-mean-claim
   ```

4. Note the claim no criterion can read

   ```rb-session frame="terminal"
   $ rb note add "No file I read compares calibration sizes, so no rule reads the claim that 512 calibration images are enough (SUMMARY.md:5). Files read: SUMMARY.md, configs/int8.yaml, results/int8-ptq/seed1/, seed2/, seed3/." -e int8-ptq
   Note n25nx5v on int8-ptq: No file I read compares calibration sizes, so no rule reads the claim that 512 calibration images are enough (SUMMARY.md:5). Files read: SUMMARY.md, configs/int8.yaml, results/int8-ptq/seed1/, seed2/, seed3/.
     Kept verbatim with who wrote it; a note changes no finding.
   ```

5. Read each claim's criterion

   ```rb-session frame="terminal"
   $ rb show every-seed-claim
   Criterion every-seed-claim on int8-ptq (primary, cited): The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here
     cited: SUMMARY.md "it never loses more than 0.05 points of top-1 on any seed" · states 0.05, read '- INT8 matches FP32: it never loses more than 0.05 points of top-1 on any seed.'
     every-seed-claim The summary's claim that INT8 never loses more than 0.05 points of top-1 on any seed is checked here
             not reproduced · gain · each unit's top1: fp32 vs int8 ≤ 0.05
             reads:     top1: fp32 vs int8: +0.08 [+0.005476, +0.1545] % · paired difference, t interval, 3 seeds, 95% (rb)
             result:    gain: the interval clear of 0 on the better side (higher is better)
             rule:      not reproduced: 2 of 3 units are above 0.05 (seed=1, seed=2)
             timing:    declared after the evidence existed: all 3 units were produced before it was declared
             units:     3 seeds
             integrity: ok
     runwe2xdq: difference +0.11 (file) · unit seed=1 · produced before the declaration · seed=1
     run23qmvv: difference +0.08 (file) · unit seed=2 · produced before the declaration · seed=2
     run7fviep: difference +0.05 (file) · unit seed=3 · produced before the declaration · seed=3
   ```

## Why it matters

A summary is a claim about the runs. rb reads the runs, and says which claims hold, which do not, and which nothing checked.

---

# Declare the criterion before the runs

https://rabbitbrain.ai/docs/cookbooks/criterion-before-runs/

This cookbook declares a criterion and freezes it before the first run, then shows what rb says of one declared after.

A threshold picked after the number came in is not a test: it was fitted to the result. Declare the criterion and lock it before the first run, and rb's finding says so. A criterion declared later says that too.

## Paste this into your agent

**Declare the criterion and freeze it before any run, then run the seeds.**

```text
Use Rabbit Brain to find out if warmup is worth it for the 6-layer GPT. docs/plan.md says what counts.

- Run `rb context --model <your model id>` first.
- Before any run: record the question, the experiment with both arms and every setting cited to its config, and the criterion docs/plan.md states, cited to its line. Then freeze the experiment.
- Then run seeds 1 to 3 through `rb journal run -- python train.py --seed <N>`, and record each run from its output file, with its config.
- Compute nothing yourself.

Finish with `rb show` of the criterion exactly as printed.
```

The repository: a 6-layer GPT trained with and without learning-rate warmup, `configs/` for each arm, and `docs/plan.md`: "Warmup ships if it lowers final validation loss by at least 0.02 nats against no warmup."

## What you'll see

The plan's criterion, read after three seeds:

```rb-session frame="terminal"
$ rb show warmup-helps
Criterion warmup-helps on warmup (primary, own): Warmup lowers final validation loss by at least 0.02 nats
  warmup-helps Warmup lowers final validation loss by at least 0.02 nats
          criterion met · gain · change.val_loss ≤ -0.02 (n ≥ 3)
          reads:     change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb)
          result:    gain: the interval clear of 0 on the better side (lower is better)
          rule:      criterion met: estimate -0.02767 ≤ -0.02
          timing:    declared before the evidence (by production times)
          units:     3 seeds
          integrity: ok
  runof35go: difference -0.031 (file) · unit seed=1 · seed=1
  runk3rudt: difference -0.026 (file) · unit seed=2 · seed=2
  run7bk2pa: difference -0.026 (file) · unit seed=3 · seed=3
```

`warmup-helps` is the criterion's id: its name in every command. Its timing reads **declared before the evidence**: the criterion was declared, and the experiment frozen, before any seed ran.

Now suppose someone raises the bar to 0.03 nats after seeing these numbers. The experiment is frozen, so rb refuses the change until it gives its reason:

```rb-session frame="terminal"
$ rb criterion add "Warmup lowers final validation loss by at least 0.03 nats" -e warmup -q q-warmup --metric change.val_loss --at-most -0.03 --min-n 3 --id stricter-warmup-bar
rb criterion add: Experiment warmup was frozen at 2026-11-08T03:36:41.223+00:00 (e84183e9a687); this changes it (new criterion stricter-warmup-bar: change.val_loss ≤ -0.03 (n ≥ 3)).
  → Give the change its reason: the same command with --amend --why "<reason>". The amendment is recorded, by whoever makes it.

$ rb criterion add "Warmup lowers final validation loss by at least 0.03 nats" -e warmup -q q-warmup --metric change.val_loss --at-most -0.03 --min-n 3 --id stricter-warmup-bar --amend --why "a stricter bar, set after seeing the runs"
stricter-warmup-bar: Warmup lowers final validation loss by at least 0.03 nats
  change.val_loss ≤ -0.03 (n ≥ 3) · primary · on warmup
  declared after the evidence existed: all 3 units were produced before it was declared
  it already reads change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb): criterion not met: estimate -0.02767 > -0.03

Next: rb run add warmup --from <the results file a run wrote> --set seed=<value>
```

Both criteria read the same three runs. `rb status` keeps them apart by when each was declared:

```rb-session frame="terminal"
$ rb status
Warmup on a small GPT · warmup-on-a-small-gpt

Is warmup worth it for the 6-layer GPT? (open)
  On every warmup criterion below: units: 3 seeds.
  Warmup lowers final validation loss by at least 0.03 nats. change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb); gain: the interval clear of 0 on the better side (lower is better); criterion not met: estimate -0.02767 > -0.03; rule and question link declared after the evidence existed: all 3 units were produced before it was declared. the interval [-0.03484, -0.0205] crosses -0.03: missed on the estimate, not across the whole interval.
  Warmup lowers final validation loss by at least 0.02 nats. change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb); gain: the interval clear of 0 on the better side (lower is better); criterion met: estimate -0.02767 ≤ -0.02; rule and question link declared before the evidence (by production times).

Rules: 1 met · 1 not met

Needs judgment (1)
  - stricter-warmup-bar misses its rule and no call on it is recorded: Warmup lowers final validation loss by at least 0.03 nats
      rb decide stricter-warmup-bar accept|reject|investigate --why "..."

The design of 1 experiment is not confirmed: rb confirm lists it.

Can be done now (1)
  - warmup: 1 criterion was declared after its evidence existed: a reading of its own needs its subject's runs made after the declaration
      rb run add warmup --from <the results file a run wrote> --set seed=<value>   (run after the declaration)

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:36 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  warmup-helps Warmup lowers final validation loss by at least 0.02 nats
          criterion met · gain · change.val_loss ≤ -0.02 (n ≥ 3)
… (21 more lines)
```

The late criterion reads **declared after the evidence existed**, and its miss waits for a call under **Needs judgment**. [Calls](https://rabbitbrain.ai/docs/concepts/calls/) shows that call being made.

## What rb did, step by step

1. Record the experiment and cite its settings

   ```rb-session frame="terminal"
   $ rb experiment add "Warmup vs no warmup" --id warmup -q q-warmup --baseline none --candidate warmup --varies warmup_steps
   Experiment warmup: Warmup vs no warmup · none (baseline) vs warmup (candidate) · varies warmup_steps · under q-warmup
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 2 variants: none (baseline), warmup (candidate); a setting of one is rb spec set warmup none.<setting> <value>.
     warmup declares it varies warmup_steps, but no variant has a value for it yet; give each variant its own: rb spec set warmup <variant>.warmup_steps <value> --source <file#key>

   Next: rb spec set warmup --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e warmup -q q-warmup --metric <name> --at-most <x>
   ```

   ```rb-session frame="terminal"
   $ rb spec set warmup --from configs/base.yaml --keys "optim.*,data.*"
   warmup: 4 settings from configs/base.yaml @90e2f71 · 4 verified

   $ rb spec set warmup none.warmup_steps 0 --source "configs/no-warmup.yaml#warmup_steps"
   warmup/none.warmup_steps = 0 · verified in configs/no-warmup.yaml#warmup_steps

   $ rb spec set warmup warmup.warmup_steps 500 --source "configs/warmup.yaml#warmup_steps"
   warmup/warmup.warmup_steps = 500 · verified in configs/warmup.yaml#warmup_steps

   $ rb spec set warmup seed --per-run
   warmup/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

2. Declare the plan's criterion

   `--why "docs/plan.md:3"`: rb checks that the line is there and states the criterion's threshold.

   ```rb-session frame="terminal"
   $ rb criterion add "Warmup lowers final validation loss by at least 0.02 nats" -e warmup -q q-warmup --metric change.val_loss --at-most -0.02 --min-n 3 --why docs/plan.md:3 --id warmup-helps
   warmup-helps: Warmup lowers final validation loss by at least 0.02 nats
     change.val_loss ≤ -0.02 (n ≥ 3) · primary · on warmup
     declared; no evidence recorded yet

   Next: rb run add warmup --from <the results file a run wrote> --set seed=<value>
   ```

3. Freeze it

   The freeze records a hash of the variants, settings and criteria, and prints each criterion it locks.

   ```rb-session frame="terminal"
   $ rb freeze warmup -m "before the runs"
   warmup frozen: spec e84183e9a687 · before the runs
     warmup: Warmup vs no warmup
       variants: none (baseline), warmup (candidate) · varies warmup_steps
       criterion warmup-helps (primary): change.val_loss ≤ -0.02 (n ≥ 3)
       optim.lr = 0.0006 · verified · configs/base.yaml#optim.lr
       optim.weight_decay = 0.1 · verified · configs/base.yaml#optim.weight_decay
       data.batch_size = 64 · verified · configs/base.yaml#data.batch_size
       data.seq_len = 256 · verified · configs/base.yaml#data.seq_len
       seed = (each run) · per-run · no source
       none.warmup_steps = 0 · verified · configs/no-warmup.yaml#warmup_steps
       warmup.warmup_steps = 500 · verified · configs/warmup.yaml#warmup_steps
       look: warmup-helps is locked as the test now. Check it is the one you mean: a change to it needs --amend --why "<reason>".
     Changing a setting or its source, a variant, what it varies, or a criterion now needs --amend --why "<reason>"; the amendment is kept with who made it.

   Next: rb run add warmup --from <metrics.json> --command "..."
   ```

4. Run each seed through rb journal run

   The command, its exit code and the files it wrote are kept, even if nothing else is written down.

   ```rb-session frame="terminal"
   $ rb journal run -- python train.py --seed 1
   seed 1: val_loss none 2.412 warmup 2.381
   {"metrics": {"none.val_loss": 2.412, "warmup.val_loss": 2.381}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: none.val_loss=2.412, warmup.val_loss=2.381. rb reconcile proposes the run of warmup that records them.

   Since this session last looked:
     - out/seed1/ holds results no recorded experiment reads yet   (rb reconcile)
     - out/seed1/metrics.json: results landed and are not in the record   (rb reconcile)

   Next: rb reconcile
   ```

5. Record each run from its output

   ```rb-session frame="terminal"
   $ rb run add warmup --from out/seed1/metrics.json --set seed=1 --config out/seed1/config.yaml --command "python train.py --seed 1"
   runof35go on warmup: none.val_loss=2.412, warmup.val_loss=2.381 · seed=1
     Ran with the spec: 5 of its settings found in out/seed1/config.yaml, all matching.
     Produced: 2026-11-08 03:36:41 (file)
     warmup-helps criterion undetermined · point only: change.val_loss: -0.031 nats · paired difference, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/a811c430d496facb6736463d66d13e9744b1ca54, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/a811c430d496facb6736463d66d13e9744b1ca54 removes it.

   Next: rb show warmup-helps

   $ rb run add warmup --from out/seed2/metrics.json --set seed=2 --config out/seed2/config.yaml --command "python train.py --seed 2"
   runk3rudt on warmup: none.val_loss=2.405, warmup.val_loss=2.379 · seed=2
     Ran with the spec: 5 of its settings found in out/seed2/config.yaml, all matching.
     Produced: 2026-11-08 03:36:41 (file)
     warmup-helps criterion undetermined · point only: change.val_loss: -0.0285 nats · paired difference, 2 seeds (rb)

   Next: rb show warmup-helps

   $ rb run add warmup --from out/seed3/metrics.json --set seed=3 --config out/seed3/config.yaml --command "python train.py --seed 3"
   run7bk2pa on warmup: none.val_loss=2.418, warmup.val_loss=2.392 · seed=3
     Ran with the spec: 5 of its settings found in out/seed3/config.yaml, all matching.
     Produced: 2026-11-08 03:36:42 (file)
     warmup-helps criterion met · gain: change.val_loss: -0.02767 [-0.03484, -0.0205] nats · paired difference, t interval, 3 seeds, 95% (rb)

   Next: rb show warmup-helps
   ```

## Why it matters

A criterion declared first is a test. rb records which came first, so a reader never has to take it on trust.

---

# Compare against a published baseline

https://rabbitbrain.ai/docs/cookbooks/published-baseline/

This cookbook checks your runs against a number a paper reports, on the paper's own settings.

"We match the paper" is only as good as the comparison under it: the same number, read the same way, with the same settings. Save the paper's text in the repository, cite it, and let rb read your runs against it.

## Paste this into your agent

**Check our run against the paper's number, with the paper's settings cited.**

```text
Check our ResNet-20 against the paper's 8.75% error with Rabbit Brain. The paper's numbers are saved in docs/baselines/resnet-paper.md. Count it as reproduced if we land within 0.3 points of the paper's 8.75% error.

- Run `rb context --model <your model id>` first.
- Record the experiment, and each training setting from configs/train.yaml with the value the paper used, cited to its line in docs/baselines/resnet-paper.md (`--cited`, `--cited-source`).
- Record the paper's number as a criterion cited to its table row (`--source`, `--quote`, `--locator "Table 6"`), with my tolerance and my words.
- Run seeds 1 to 3 through `rb journal run`, and record each run.

Finish with `rb show` of the criterion exactly as printed, and every setting that differs from the paper's.
```

The repository trains a ResNet-20 on CIFAR-10. It keeps the paper's text it relies on in `docs/baselines/resnet-paper.md`: the training settings from section 4.2 and the ResNet-20 row of Table 6.

```markdown title="docs/baselines/resnet-paper.md"
# Deep Residual Learning for Image Recognition (2016): excerpts

Section 4.2, CIFAR-10:

We use a weight decay of 0.0001 and momentum of 0.9.
These models are trained with a mini-batch size of 128.
We start with a learning rate of 0.1.

Table 6, classification error on the CIFAR-10 test set:

| method | layers | params | error (%) |
|---|---|---|---|
| ResNet | 20 | 0.27M | 8.75 |
| ResNet | 32 | 0.46M | 7.51 |
```

## What you'll see

```rb-session frame="terminal"
$ rb show matches-paper-error
Criterion matches-paper-error on repro (primary, cited): Our ResNet-20 lands within 0.3 points of the paper's 8.75% error · words (as given): "Count it as reproduced if we land within 0.3 points of the paper's 8.75% error."
  cited: docs/baselines/resnet-paper.md "| ResNet | 20 | 0.27M | 8.75 |" (Table 6) · states 8.75, read '| ResNet | 20 | 0.27M | 8.75 |'
  matches-paper-error Our ResNet-20 lands within 0.3 points of the paper's 8.75% error
          reproduced · measured · resnet20.error = 8.75 ± 0.3 (n ≥ 3)
          reads:     resnet20.error: 8.747 [8.597, 8.896] % · mean, t interval, 3 seeds, 95% (rb)
          result:    measured: resnet20 on its own, with its interval, so neither better nor worse
          rule:      reproduced: estimate 8.747 within 8.75 ± 0.3
          timing:    declared before the evidence (by production times)
          units:     3 seeds
          integrity: ok
  runzkb5l2: resnet20.error 8.81 (file) · unit seed=1 · seed=1
  runq67s63: resnet20.error 8.69 (file) · unit seed=2 · seed=2
  run5kx5fi: resnet20.error 8.74 (file) · unit seed=3 · seed=3
```

`matches-paper-error` is the criterion's id: its name in every command. **Reproduced**: the three seeds average 8.747% error, within 0.3 points of the paper's 8.75%. The criterion cites the table row it read, and your words stay beside it, as given.

### When a setting differs from the paper's

We ran the same cookbook again with one change: our config trains with a batch size of 256, where the paper used 128. rb does not compare numbers made under different settings:

```rb-session frame="terminal"
$ rb show matches-paper-error
Criterion matches-paper-error on repro (primary, cited): Our ResNet-20 lands within 0.3 points of the paper's 8.75% error
  cited: docs/baselines/resnet-paper.md "| ResNet | 20 | 0.27M | 8.75 |" (Table 6) · states 8.75, read '| ResNet | 20 | 0.27M | 8.75 |'
  matches-paper-error Our ResNet-20 lands within 0.3 points of the paper's 8.75% error
          reproduced · measured · resnet20.error = 8.75 ± 0.3 (n ≥ 3)
          reads:     resnet20.error: 8.747 [8.597, 8.896] % · mean, t interval, 3 seeds, 95% (rb)
          result:    measured: resnet20 on its own, with its interval, so neither better nor worse
          rule:      reproduced: estimate 8.747 within 8.75 ± 0.3
          timing:    declared before the evidence (by production times)
          units:     3 seeds
          integrity: ok
  runaqvtwg: resnet20.error 8.81 (file) · unit seed=1 · seed=1
  runryutga: resnet20.error 8.69 (file) · unit seed=2 · seed=2
  runpzabbn: resnet20.error 8.74 (file) · unit seed=3 · seed=3
```

The same three error rates now read **criterion undetermined**: not comparable. The difference is an integrity problem until the settings match.

## What rb did, step by step

1. Record the experiment's purpose

   ```rb-session frame="terminal"
   $ rb experiment add "ResNet-20 against the ResNet paper (2016)" --id repro -q q-repro --purpose "Reproduce the ResNet-20 CIFAR-10 test error of the ResNet paper (2016), Table 6" --baseline resnet20
   Experiment repro: ResNet-20 against the ResNet paper (2016) · resnet20 (baseline) · under q-repro
     purpose: Reproduce the ResNet-20 CIFAR-10 test error of the ResNet paper (2016), Table 6
     Planned: nothing in the record tests it yet, and it shows as a planned direction until its first run or analysis.
     Made 1 variant: resnet20 (baseline); a setting of one is rb spec set repro resnet20.<setting> <value>.

   Next: rb spec set repro --from <config.yaml> --keys "<the keys that define it>"  ·  rb criterion add "<what it should show>" -e repro -q q-repro --metric <name> --at-most <x>
   ```

2. Cite each setting, with the value the paper used

   `--cited` keeps the paper's value beside yours, and rb says it back under the setting. A difference makes every criterion that cites the paper not comparable.

   ```rb-session frame="terminal"
   $ rb spec set repro optim.lr --source "configs/train.yaml#optim.lr" --cited 0.1 --cited-source docs/baselines/resnet-paper.md:7
   repro/optim.lr = 0.1 · verified in configs/train.yaml#optim.lr
     cited: 0.1 (docs/baselines/resnet-paper.md:7)

   $ rb spec set repro optim.momentum --source "configs/train.yaml#optim.momentum" --cited 0.9 --cited-source docs/baselines/resnet-paper.md:5
   repro/optim.momentum = 0.9 · verified in configs/train.yaml#optim.momentum
     cited: 0.9 (docs/baselines/resnet-paper.md:5)

   $ rb spec set repro optim.weight_decay --source "configs/train.yaml#optim.weight_decay" --cited 0.0001 --cited-source docs/baselines/resnet-paper.md:5
   repro/optim.weight_decay = 0.0001 · verified in configs/train.yaml#optim.weight_decay
     cited: 0.0001 (docs/baselines/resnet-paper.md:5)

   $ rb spec set repro data.batch_size --source "configs/train.yaml#data.batch_size" --cited 128 --cited-source docs/baselines/resnet-paper.md:6
   repro/data.batch_size = 128 · verified in configs/train.yaml#data.batch_size
     cited: 128 (docs/baselines/resnet-paper.md:6)

   $ rb spec set repro seed --per-run
   repro/seed · per-run: each run gives its own, with --set seed=<value> on rb run add
   ```

   ```rb-session frame="terminal"
   $ rb show repro/data.batch_size
   repro/data.batch_size = 128 · verified
     source: configs/train.yaml#data.batch_size · read 'data.batch_size: 128' @9d87bd1
     cited source used 128 (docs/baselines/resnet-paper.md:6)
     history:
       2026-11-08T03:36:44 spec set: setting data.batch_size: new, 128
       2026-11-08T03:36:44 spec check: verified
   ```

3. Cite the paper's number as a criterion

   rb checks that the quoted row is in the file and states 8.75.

   ```rb-session frame="terminal"
   $ rb criterion add "Our ResNet-20 lands within 0.3 points of the paper's 8.75% error" -e repro -q q-repro --metric resnet20.error --equals 8.75 --tolerance 0.3 --source docs/baselines/resnet-paper.md --quote "| ResNet | 20 | 0.27M | 8.75 |" --locator "Table 6" --min-n 3 --words "Count it as reproduced if we land within 0.3 points of the paper's 8.75% error." --id matches-paper-error
   matches-paper-error: Our ResNet-20 lands within 0.3 points of the paper's 8.75% error
     resnet20.error = 8.75 ± 0.3 (n ≥ 3) · primary · on repro
     cited: docs/baselines/resnet-paper.md "| ResNet | 20 | 0.27M | 8.75 |" (Table 6) · states 8.75 (read '| ResNet | 20 | 0.27M | 8.75 |')
     declared; no evidence recorded yet

   Next: rb run add repro --from <the results file a run wrote> --set seed=<value>
   ```

4. Run and record each seed

   ```rb-session frame="terminal"
   $ rb journal run -- python train.py --seed 1
   {"metrics": {"resnet20.error": 8.81}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: resnet20.error=8.81. rb reconcile proposes the run of repro that records them.

   Since this session last looked:
     - out/seed1/ holds results no recorded experiment reads yet   (rb reconcile)
     - out/seed1/metrics.json: results landed and are not in the record   (rb reconcile)

   Next: rb reconcile
   ```

   ```rb-session frame="terminal"
   $ rb run add repro --from out/seed1/metrics.json --set seed=1 --config out/seed1/config.yaml --command "python train.py --seed 1"
   runzkb5l2 on repro: resnet20.error=8.81 · seed=1
     Ran with the spec: 5 of its settings found in out/seed1/config.yaml, all matching.
     Produced: 2026-11-08 03:36:45 (file)
     matches-paper-error criterion undetermined · point only: resnet20.error: 8.81 % · mean, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/e8626933b5fe8d9b4040088d8db7c87428d15d30, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/e8626933b5fe8d9b4040088d8db7c87428d15d30 removes it.

   Next: rb show matches-paper-error

   $ rb run add repro --from out/seed2/metrics.json --set seed=2 --config out/seed2/config.yaml --command "python train.py --seed 2"
   runq67s63 on repro: resnet20.error=8.69 · seed=2
     Ran with the spec: 5 of its settings found in out/seed2/config.yaml, all matching.
     Produced: 2026-11-08 03:36:45 (file)
     matches-paper-error criterion undetermined · point only: resnet20.error: 8.75 % · mean, 2 seeds (rb)

   Next: rb show matches-paper-error

   $ rb run add repro --from out/seed3/metrics.json --set seed=3 --config out/seed3/config.yaml --command "python train.py --seed 3"
   run5kx5fi on repro: resnet20.error=8.74 · seed=3
     Ran with the spec: 5 of its settings found in out/seed3/config.yaml, all matching.
     Produced: 2026-11-08 03:36:46 (file)
     matches-paper-error reproduced · measured: resnet20.error: 8.747 [8.597, 8.896] % · mean, t interval, 3 seeds, 95% (rb)

   Next: rb show matches-paper-error
   ```

## Why it matters

A match to a paper counts only on the paper's settings. rb checks them before it says reproduced.

---

# Get an interval, not a single number

https://rabbitbrain.ai/docs/cookbooks/interval-from-seeds/

This cookbook runs seeds until a criterion has the interval it asks for, and shows rb's finding after each seed.

One seed gives one number, and one number cannot tell a real cost from run-to-run noise. Ask for an interval in the criterion itself, declare how many seeds it needs, and rb will not call it before then.

## Paste this into your agent

**Read the result on its 95% interval, over 5 seeds declared up front.**

```text
Use Rabbit Brain to find out whether dropout 0.1 costs us validation loss on the MLP. docs/plan.md says what counts. I want an interval, not one number: dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout.

- Run `rb context --model <your model id>` first.
- Before any run: record the experiment, its settings from the configs, the plan's criterion cited to its line, and my criterion on the interval, with my words, needing 5 seeds. Freeze it.
- Run seeds 1 to 5 through `rb journal run`, and record each run.
- After each seed, tell me rb's finding as printed. Never round "point only" up to a result.

Finish with `rb show` of both criteria exactly as printed.
```

The repository trains a small MLP with no dropout and with dropout 0.1. `docs/plan.md` says: "Dropout 0.1 is acceptable if it costs at most 0.02 nats of validation loss."

Two criteria read the same comparison, dropout's validation loss minus no dropout's. Each is listed by its id, its name in every command:

- `mean-within-budget`, the plan's: the mean cost is at most 0.02 nats (`--at-most 0.02`);
- `interval-within-budget`, yours: the whole 95% interval stays within 0.02 nats (`--noninferior 0.02`), on at least 5 seeds (`--min-n 5`).

## What you'll see

After each seed, `rb run add` prints both findings. Seed by seed, as rb printed them:

| Seeds | The reading, as rb prints it | `mean-within-budget` | `interval-within-budget` |
| --- | --- | --- | --- |
| 1 | +0.008 nats | estimate meets criterion · point only | criterion undetermined · point only |
| 2 | +0.0025 nats | estimate meets criterion · point only | criterion undetermined · point only |
| 3 | +0.006333 [-0.01508, +0.02775] nats | estimate meets criterion · unresolved | criterion undetermined · unresolved |
| 4 | +0.0055 [-0.006011, +0.01701] nats | criterion met · unresolved | criterion undetermined · unresolved |
| 5 | +0.0056 [-0.002184, +0.01338] nats | criterion met · unresolved | criterion met · unresolved |

With one or two seeds there is no interval, and the plan's criterion can only say **estimate meets criterion · point only**. At three seeds the interval reaches past the plan's 0.02, so rb says "estimate meets criterion", not "criterion met". From four seeds the whole interval is inside 0.02. Your criterion stays undetermined until the fifth seed, because you declared that it needs five.

```rb-session frame="terminal"
$ rb status
Dropout on a small MLP · dropout-on-a-small-mlp

Does dropout 0.1 cost us validation loss on the MLP? (open)
  On every drop criterion below: question link declared before the evidence (by production times); units: 5 seeds.
  The whole 95% interval of dropout's cost stays within 0.02 nats. change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb); unresolved: the interval includes 0; criterion met: the interval's worse bound +0.01338 inside 0.02; rule and question link declared before the evidence (by production times) · words (as given): "dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout".
  Dropout 0.1 costs at most 0.02 nats of validation loss. change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb); unresolved: the interval includes 0; criterion met: estimate +0.0056 ≤ 0.02; rule and question link declared before the evidence (by production times).

Rules: 2 met
… (30 more lines)
```

**Unresolved** on the result line is not a failure: the interval still includes 0, so rb cannot say dropout changes the loss at all. Both criteria ask only that it not cost more than 0.02 nats, and it does not.

## What rb did, step by step

1. Declare both criteria before any seed

   ```rb-session frame="terminal"
   $ rb criterion add "Dropout 0.1 costs at most 0.02 nats of validation loss" -e drop -q q-drop --metric change.val_loss --at-most 0.02 --why docs/plan.md:3 --id mean-within-budget
   mean-within-budget: Dropout 0.1 costs at most 0.02 nats of validation loss
     change.val_loss ≤ 0.02 · primary · on drop
     declared; no evidence recorded yet

   Next: rb run add drop --from <the results file a run wrote> --set seed=<value>

   $ rb criterion add "The whole 95% interval of dropout's cost stays within 0.02 nats" -e drop -q q-drop --metric change.val_loss --noninferior 0.02 --min-n 5 --words "dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout" --id interval-within-budget
   interval-within-budget: The whole 95% interval of dropout's cost stays within 0.02 nats
     change.val_loss no worse than 0.02, its interval's worse bound inside 0.02 (n ≥ 5) · primary · on drop
     declared; no evidence recorded yet

   Next: rb run add drop --from <the results file a run wrote> --set seed=<value>
   ```

2. Freeze the experiment

   ```rb-session frame="terminal"
   $ rb freeze drop -m "before the first seed"
   drop frozen: spec 12f84c212b8c · before the first seed
     drop: Dropout 0.1 vs none
       variants: none (baseline), p01 (candidate) · varies dropout
       criterion mean-within-budget (primary): change.val_loss ≤ 0.02
       criterion interval-within-budget (primary): change.val_loss no worse than 0.02, its interval's worse bound inside 0.02 (n ≥ 5)   (words (as given): "dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout")
       optim.lr = 0.001 · verified · configs/train.yaml#optim.lr
       data.batch_size = 256 · verified · configs/train.yaml#data.batch_size
       seed = (each run) · per-run · no source
       none.dropout = 0.0 · verified · configs/none.yaml#dropout
       p01.dropout = 0.1 · verified · configs/p01.yaml#dropout
       look: mean-within-budget is locked as the test now. Check it is the one you mean: a change to it needs --amend --why "<reason>".
       look: interval-within-budget is locked as the test now. Check it is the one you mean: a change to it needs --amend --why "<reason>".
     Changing a setting or its source, a variant, what it varies, or a criterion now needs --amend --why "<reason>"; the amendment is kept with who made it.

   Next: rb run add drop --from <metrics.json> --command "..."
   ```

3. Run a seed, record it, read both criteria

   ```rb-session frame="terminal"
   $ rb journal run -- python train.py --seed 1
   {"metrics": {"none.val_loss": 1.31, "p01.val_loss": 1.318}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: none.val_loss=1.31, p01.val_loss=1.318. rb reconcile proposes the run of drop that records them.

   Since this session last looked:
     - out/seed1/ holds results no recorded experiment reads yet   (rb reconcile)
     - out/seed1/metrics.json: results landed and are not in the record   (rb reconcile)

   Next: rb reconcile
   ```

   ```rb-session frame="terminal"
   $ rb run add drop --from out/seed1/metrics.json --set seed=1 --config out/seed1/config.yaml --command "python train.py --seed 1"
   run2yelmh on drop: none.val_loss=1.31, p01.val_loss=1.318 · seed=1
     Ran with the spec: 3 of its settings found in out/seed1/config.yaml, all matching.
     Produced: 2026-11-08 03:36:50 (file)
     mean-within-budget estimate meets criterion · point only: change.val_loss: +0.008 nats · paired difference, 1 seed (rb)
     interval-within-budget criterion undetermined · point only: change.val_loss: +0.008 nats · paired difference, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/3bf56b1d2d985fe410d77e394087a33e85d90a96, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/3bf56b1d2d985fe410d77e394087a33e85d90a96 removes it.

   Next: rb show mean-within-budget  ·  rb show interval-within-budget

   $ rb run add drop --from out/seed2/metrics.json --set seed=2 --config out/seed2/config.yaml --command "python train.py --seed 2"
   runwmk7z5 on drop: none.val_loss=1.302, p01.val_loss=1.299 · seed=2
     Ran with the spec: 3 of its settings found in out/seed2/config.yaml, all matching.
     Produced: 2026-11-08 03:36:51 (file)
     mean-within-budget estimate meets criterion · point only: change.val_loss: +0.0025 nats · paired difference, 2 seeds (rb)
     interval-within-budget criterion undetermined · point only: change.val_loss: +0.0025 nats · paired difference, 2 seeds (rb)

   Next: rb show mean-within-budget  ·  rb show interval-within-budget

   $ rb run add drop --from out/seed3/metrics.json --set seed=3 --config out/seed3/config.yaml --command "python train.py --seed 3"
   runtetay2 on drop: none.val_loss=1.315, p01.val_loss=1.329 · seed=3
     Ran with the spec: 3 of its settings found in out/seed3/config.yaml, all matching.
     Produced: 2026-11-08 03:36:51 (file)
     mean-within-budget estimate meets criterion · unresolved: change.val_loss: +0.006333 [-0.01508, +0.02775] nats · paired difference, t interval, 3 seeds, 95% (rb)
     interval-within-budget criterion undetermined · unresolved: change.val_loss: +0.006333 [-0.01508, +0.02775] nats · paired difference, t interval, 3 seeds, 95% (rb)

   Next: rb show mean-within-budget  ·  rb show interval-within-budget

   $ rb run add drop --from out/seed4/metrics.json --set seed=4 --config out/seed4/config.yaml --command "python train.py --seed 4"
   runkquyec on drop: none.val_loss=1.308, p01.val_loss=1.311 · seed=4
     Ran with the spec: 3 of its settings found in out/seed4/config.yaml, all matching.
     Produced: 2026-11-08 03:36:52 (file)
     mean-within-budget criterion met · unresolved: change.val_loss: +0.0055 [-0.006011, +0.01701] nats · paired difference, t interval, 4 seeds, 95% (rb)
     interval-within-budget criterion undetermined · unresolved: change.val_loss: +0.0055 [-0.006011, +0.01701] nats · paired difference, t interval, 4 seeds, 95% (rb)

   Next: rb show mean-within-budget  ·  rb show interval-within-budget

   $ rb run add drop --from out/seed5/metrics.json --set seed=5 --config out/seed5/config.yaml --command "python train.py --seed 5"
   run35lzxe on drop: none.val_loss=1.312, p01.val_loss=1.318 · seed=5
     Ran with the spec: 3 of its settings found in out/seed5/config.yaml, all matching.
     Produced: 2026-11-08 03:36:52 (file)
     mean-within-budget criterion met · unresolved: change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)
     interval-within-budget criterion met · unresolved: change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)

   Next: rb show mean-within-budget  ·  rb show interval-within-budget
   ```

4. Read the criterion on its interval

   ```rb-session frame="terminal"
   $ rb show interval-within-budget
   Criterion interval-within-budget on drop (primary, own): The whole 95% interval of dropout's cost stays within 0.02 nats · words (as given): "dropout passes only if the whole 95% interval stays within 0.02 nats of no dropout"
     interval-within-budget The whole 95% interval of dropout's cost stays within 0.02 nats
             criterion met · unresolved · change.val_loss no worse than 0.02, its interval's worse bound inside 0.02 (n ≥ 5)
             reads:     change.val_loss: +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)
             result:    unresolved: the interval includes 0
             rule:      criterion met: the interval's worse bound +0.01338 inside 0.02
             timing:    declared before the evidence (by production times)
             units:     5 seeds
             integrity: ok
     run2yelmh: difference +0.008 (file) · unit seed=1 · seed=1
     runwmk7z5: difference -0.003 (file) · unit seed=2 · seed=2
     runtetay2: difference +0.014 (file) · unit seed=3 · seed=3
     runkquyec: difference +0.003 (file) · unit seed=4 · seed=4
     run35lzxe: difference +0.006 (file) · unit seed=5 · seed=5
   ```

5. Compare the arms

   `rb compare` gives the means; the criteria carry the intervals.

   ```rb-session frame="terminal"
   $ rb compare drop
   drop  Dropout 0.1 vs none · varies dropout · 5 runs under the current spec · means over seeds
     variant  role       dropout  val_loss (nats)                    
     none     baseline   0.0      1.309 (n=5)                        
     p01      candidate  0.1      1.315 +0.0056 worse [primary] (n=5)
     the primary criteria read: val_loss p01 vs none (mean-within-budget reads +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)), val_loss p01 vs none (interval-within-budget reads +0.0056 [-0.002184, +0.01338] nats · paired difference, t interval, 5 seeds, 95% (rb)); deltas are means, the findings carry the intervals (rb show <criterion>)
   ```

## Why it matters

"It costs 0.008 nats" from one seed and "it costs between -0.002 and +0.013" from five are different claims. rb says which one you have.

---

# Read a rule over seeds

https://rabbitbrain.ai/docs/cookbooks/rules-over-seeds/

This cookbook declares three rules that read the seeds themselves, not only their mean: a count, a best of N and a relative change.

A mean can hide a bad seed, and a plan can ask about each seed rather than the mean. rb 0.8.3 reads a rule over the units themselves: how many meet a threshold, which one is best, or by what percent the arms' means differ.

## Paste this into your agent

**Record the five seeds, and the plan's three rules over them.**

```text
Use Rabbit Brain to check INT8 against FP32 over the five seeds in results/int8-ptq/. docs/plan.md says what counts, one rule a line.

- Run `rb context --model <your model id>` first.
- Record the question and the experiment, with each setting cited to the config that states it.
- Record each of the plan's three rules as rb reads it, cited to its line: a count (`--count 4 --of 5`), a best of N (`--best --of 5`) and a relative change (`--relative`).
- Record each seed as a run.
- Compute nothing yourself: rb computes the findings.

Finish with `rb status` and `rb show` of each criterion exactly as printed.
```

The repository evaluates an FP32 ResNet-20 and its INT8 copy on CIFAR-10 for five seeds, each into `results/int8-ptq/seed<N>/`. `docs/plan.md` holds the three rules, on lines 4 to 6:

- "INT8 reaches at least 91.58% top-1 on at least 4 of the 5 seeds."
- "INT8's best seed of the 5 reaches at least 91.65% top-1."
- "INT8 loses at most 0.1% of FP32's top-1, as a relative change of the means."

## What you'll see

`rb status` in your own terminal, once the agent is done:

```rb-session frame="terminal"
$ rb status
INT8 ResNet-20 on CIFAR-10 · int8-resnet-20-on-cifar-10
Kept with rb since 2026-11-08. All 5 runs were recorded afterwards, not as they happened.

Does INT8 hold up against FP32 across the five seeds? (open)
  On every int8-seeds criterion below: question link declared after the evidence existed: all 5 units were produced before it was declared; units: 5 seeds.
  INT8's best of 5 seeds reaches 91.65% top-1. int8.top1: 91.69 % · best unit seed 3, 5 seeds (rb); point only: a best of N is one unit's own value, with no interval; recorded values meet criterion: best of 5 seeds, seed 3: 91.69 ≥ 91.65; rule and question link declared after the evidence existed: all 5 units were produced before it was declared.
  INT8 reaches 91.58% top-1 on at least 4 of 5 seeds. int8.top1: 91.62 [91.55, 91.69] % · mean, t interval, 5 seeds, 95% (rb); measured: int8 on its own, with its interval, so neither better nor worse; recorded values meet criterion: 4 of 5 seeds at least 91.58; the rule asks for 4; rule and question link declared after the evidence existed: all 5 units were produced before it was declared.
  INT8 loses at most 0.1% of FP32's top-1. change.top1: -0.0916 [-0.1329, -0.05029] % · relative change of the means, paired, delta-method t interval, 5 seeds, 95% (rb); loss: the interval clear of 0 on the worse side (higher is better); estimate meets criterion: estimate -0.0916% ≥ -0.1%; rule and question link declared after the evidence existed: all 5 units were produced before it was declared. the interval [-0.1329, -0.05029] crosses -0.1: met on the estimate, not across the whole interval.

Rules: 3 met, all 3 on the estimate alone

The design of 1 experiment is not confirmed: rb confirm lists it.

Facts that ask for nothing (1)
  - int8-seeds: 3 criteria were declared after their evidence existed

rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:37 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Rules
  four-of-five INT8 reaches 91.58% top-1 on at least 4 of 5 seeds
          recorded values meet criterion · measured · 4 of 5 units' int8.top1 ≥ 91.58
          reads:     int8.top1: 91.62 [91.55, 91.69] % · mean, t interval, 5 seeds, 95% (rb)
          result:    measured: int8 on its own, with its interval, so neither better nor worse
          rule:      recorded values meet criterion: 4 of 5 seeds at least 91.58; the rule asks for 4
          timing:    declared after the evidence existed: all 5 units were produced before it was declared
          units:     5 seeds
  best-of-five INT8's best of 5 seeds reaches 91.65% top-1
          recorded values meet criterion · point only · the best of 5 units' int8.top1 ≥ 91.65
          reads:     int8.top1: 91.69 % · best unit seed 3, 5 seeds (rb)
          result:    point only: a best of N is one unit's own value, with no interval
          rule:      recorded values meet criterion: best of 5 seeds, seed 3: 91.69 ≥ 91.65
          timing:    declared after the evidence existed: all 5 units were produced before it was declared
          units:     5 seeds
  relative-drop INT8 loses at most 0.1% of FP32's top-1
          estimate meets criterion · loss · change.top1 as a relative change ≥ -0.1%
          reads:     change.top1: -0.0916 [-0.1329, -0.05029] % · relative change of the means, paired, delta-method t interval, 5 seeds, 95% (rb)
          result:    loss: the interval clear of 0 on the worse side (higher is better)
          rule:      estimate meets criterion: estimate -0.0916% ≥ -0.1%
          timing:    declared after the evidence existed: all 5 units were produced before it was declared
          units:     5 seeds
          · the interval [-0.1329, -0.05029] crosses -0.1: met on the estimate, not across the whole interval

Experiments
  int8-seeds INT8 post-training quantization, five seeds · open · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · not frozen · settings 7 verified · 0 provisional · 0 unknown · 1 per-run · runs 5
          to run again: code unknown · environment unknown · files ok

Runs: 5 recorded on 1 experiment · every receipt is what rb wrote

Next: rb confirm
```

- `four-of-five`, `best-of-five` and `relative-drop` are the criteria's ids: their names in every command.
- The count names both numbers: how many seeds meet the threshold, and how many the rule asks for.
- The best of N names its seed and that seed's own value. It has no interval: "a best of N is one unit's own value, with no interval".
- The relative change is said in percent. Its interval crosses -0.1, so it reads **estimate meets criterion**, met on the estimate, not across the whole interval.
- A met count or best of N reads **recorded values meet criterion**, since the recorded seeds themselves meet it. The tally counts all three among the rules met on the estimate alone.

## What rb did, step by step

1. Declare the three rules

   Each is said in rb's own words: "4 of 5 units' int8.top1 ≥ 91.58", "the best of 5 units' int8.top1 ≥ 91.65" and "change.top1 as a relative change ≥ -0.1%".

   ```rb-session frame="terminal"
   $ rb criterion add "INT8 reaches 91.58% top-1 on at least 4 of 5 seeds" -e int8-seeds -q q-seeds --metric int8.top1 --at-least 91.58 --count 4 --of 5 --why docs/plan.md:4 --id four-of-five
   four-of-five: INT8 reaches 91.58% top-1 on at least 4 of 5 seeds
     4 of 5 units' int8.top1 ≥ 91.58 · primary · on int8-seeds
     declared; no evidence recorded yet

   Next: rb run add int8-seeds --from <the results file a run wrote> --set seed=<value>

   $ rb criterion add "INT8's best of 5 seeds reaches 91.65% top-1" -e int8-seeds -q q-seeds --metric int8.top1 --at-least 91.65 --best --of 5 --why docs/plan.md:5 --id best-of-five
   best-of-five: INT8's best of 5 seeds reaches 91.65% top-1
     the best of 5 units' int8.top1 ≥ 91.65 · primary · on int8-seeds
     declared; no evidence recorded yet

   Next: rb run add int8-seeds --from <the results file a run wrote> --set seed=<value>

   $ rb criterion add "INT8 loses at most 0.1% of FP32's top-1" -e int8-seeds -q q-seeds --metric change.top1 --at-least -0.1 --relative --why docs/plan.md:6 --id relative-drop
   relative-drop: INT8 loses at most 0.1% of FP32's top-1
     change.top1 as a relative change ≥ -0.1% · primary · on int8-seeds
     declared; no evidence recorded yet

   Next: rb run add int8-seeds --from <the results file a run wrote> --set seed=<value>
   ```

2. Record each seed

   After each seed, `rb run add` reads all three. The count and the best of N read undetermined until the runs hold the 5 seeds the rule is of.

   ```rb-session frame="terminal"
   $ rb run add int8-seeds --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml
   runqsd4qe on int8-seeds: fp32.top1=91.71, int8.top1=91.6 · seed=1
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed1/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     four-of-five criterion undetermined · point only: int8.top1: 91.6 % · mean, 1 seed (rb)
     best-of-five criterion undetermined · point only: int8.top1: 91.6 % · best unit seed 1, 1 seed (rb)
     relative-drop criterion not met · point only: change.top1: -0.1199 % · relative change of the means, paired, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/d5832f1e0ac5298950edf64261220c331f21c020, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/d5832f1e0ac5298950edf64261220c331f21c020 removes it.

   Since this session last looked:
     - four-of-five: criterion undetermined: the rule is of 5 seeds; the runs hold 1   (rb decide four-of-five accept|reject|investigate --why "...")
     - best-of-five: criterion undetermined: the rule is of 5 seeds; the runs hold 1   (rb decide best-of-five accept|reject|investigate --why "...")
     - relative-drop misses its rule and no call on it is recorded: INT8 loses at most 0.1% of FP32's top-1   (rb decide relative-drop accept|reject|investigate --why "...")
     - 4 results folders hold results written before the record began, which no recorded experiment reads yet: results/int8-ptq/seed2/, results/int8-ptq/seed3/, results/int8-ptq/seed4/ and 1 more   (rb reconcile)

   Next: rb show four-of-five  ·  rb show best-of-five  ·  rb show relative-drop

   $ rb run add int8-seeds --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml
   run7c7jnq on int8-seeds: fp32.top1=91.66, int8.top1=91.58 · seed=2
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed2/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     four-of-five criterion undetermined · point only: int8.top1: 91.59 % · mean, 2 seeds (rb)
     best-of-five criterion undetermined · point only: int8.top1: 91.6 % · best unit seed 1, 2 seeds (rb)
     relative-drop criterion not met · point only: change.top1: -0.1036 % · relative change of the means, paired, 2 seeds (rb)

   Since this session last looked:
     - 3 results folders hold results written before the record began, which no recorded experiment reads yet: results/int8-ptq/seed3/, results/int8-ptq/seed4/, results/int8-ptq/seed5/   (rb reconcile)

   Next: rb show four-of-five  ·  rb show best-of-five  ·  rb show relative-drop

   $ rb run add int8-seeds --from results/int8-ptq/seed3/metrics.json --set seed=3 --config results/int8-ptq/seed3/config.yaml
   runhcrfar on int8-seeds: fp32.top1=91.74, int8.top1=91.69 · seed=3
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed3/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     four-of-five criterion undetermined · measured: int8.top1: 91.62 [91.48, 91.77] % · mean, t interval, 3 seeds, 95% (rb)
     best-of-five criterion undetermined · point only: int8.top1: 91.69 % · best unit seed 3, 3 seeds (rb)
     relative-drop estimate meets criterion · loss: change.top1: -0.08724 [-0.1685, -0.005936] % · relative change of the means, paired, delta-method t interval, 3 seeds, 95% (rb)

   Since this session last looked:
     - 2 results folders hold results written before the record began, which no recorded experiment reads yet: results/int8-ptq/seed4/, results/int8-ptq/seed5/   (rb reconcile)

   Next: rb show four-of-five  ·  rb show best-of-five  ·  rb show relative-drop

   $ rb run add int8-seeds --from results/int8-ptq/seed4/metrics.json --set seed=4 --config results/int8-ptq/seed4/config.yaml
   runszd6le on int8-seeds: fp32.top1=91.69, int8.top1=91.57 · seed=4
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed4/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     four-of-five criterion undetermined · measured: int8.top1: 91.61 [91.52, 91.7] % · mean, t interval, 4 seeds, 95% (rb)
     best-of-five criterion undetermined · point only: int8.top1: 91.69 % · best unit seed 3, 4 seeds (rb)
     relative-drop estimate meets criterion · loss: change.top1: -0.09815 [-0.153, -0.04325] % · relative change of the means, paired, delta-method t interval, 4 seeds, 95% (rb)

   Since this session last looked:
     - results/int8-ptq/seed5/ holds results written before the record began, which no recorded experiment reads yet   (rb reconcile)

   Next: rb show four-of-five  ·  rb show best-of-five  ·  rb show relative-drop

   $ rb run add int8-seeds --from results/int8-ptq/seed5/metrics.json --set seed=5 --config results/int8-ptq/seed5/config.yaml
   runfyc6lr on int8-seeds: fp32.top1=91.72, int8.top1=91.66 · seed=5
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed5/config.yaml, all matching.
     Produced: 2026-10-20 10:00:00 (file)
     four-of-five recorded values meet criterion · measured: int8.top1: 91.62 [91.55, 91.69] % · mean, t interval, 5 seeds, 95% (rb)
     best-of-five recorded values meet criterion · point only: int8.top1: 91.69 % · best unit seed 3, 5 seeds (rb)
     relative-drop estimate meets criterion · loss: change.top1: -0.0916 [-0.1329, -0.05029] % · relative change of the means, paired, delta-method t interval, 5 seeds, 95% (rb)

   Next: rb show four-of-five  ·  rb show best-of-five  ·  rb show relative-drop
   ```

3. Find each rule in format 2

   A criterion that holds one of these rules is kept in record format 2. Every other file stays `rb: 1`:

   ```rb-session frame="terminal"
   $ head -3 .rb/experiments/int8-seeds/criteria/four-of-five.yaml
   rb: 2
   kind: criterion
   id: four-of-five
   ```

4. Read each criterion

   ```rb-session frame="terminal"
   $ rb show four-of-five
   Criterion four-of-five on int8-seeds (primary, own): INT8 reaches 91.58% top-1 on at least 4 of 5 seeds
     four-of-five INT8 reaches 91.58% top-1 on at least 4 of 5 seeds
             recorded values meet criterion · measured · 4 of 5 units' int8.top1 ≥ 91.58
             reads:     int8.top1: 91.62 [91.55, 91.69] % · mean, t interval, 5 seeds, 95% (rb)
             result:    measured: int8 on its own, with its interval, so neither better nor worse
             rule:      recorded values meet criterion: 4 of 5 seeds at least 91.58; the rule asks for 4
             timing:    declared after the evidence existed: all 5 units were produced before it was declared
             units:     5 seeds
             integrity: ok
     runqsd4qe: int8.top1 91.6 (file) · unit seed=1 · produced before the declaration · seed=1
     run7c7jnq: int8.top1 91.58 (file) · unit seed=2 · produced before the declaration · seed=2
     runhcrfar: int8.top1 91.69 (file) · unit seed=3 · produced before the declaration · seed=3
     runszd6le: int8.top1 91.57 (file) · unit seed=4 · produced before the declaration · seed=4
     runfyc6lr: int8.top1 91.66 (file) · unit seed=5 · produced before the declaration · seed=5
   ```

   ```rb-session frame="terminal"
   $ rb show best-of-five
   Criterion best-of-five on int8-seeds (primary, own): INT8's best of 5 seeds reaches 91.65% top-1
     best-of-five INT8's best of 5 seeds reaches 91.65% top-1
             recorded values meet criterion · point only · the best of 5 units' int8.top1 ≥ 91.65
             reads:     int8.top1: 91.69 % · best unit seed 3, 5 seeds (rb)
             result:    point only: a best of N is one unit's own value, with no interval
             rule:      recorded values meet criterion: best of 5 seeds, seed 3: 91.69 ≥ 91.65
             timing:    declared after the evidence existed: all 5 units were produced before it was declared
             units:     5 seeds
             integrity: ok
     runqsd4qe: int8.top1 91.6 (file) · unit seed=1 · produced before the declaration · seed=1
     run7c7jnq: int8.top1 91.58 (file) · unit seed=2 · produced before the declaration · seed=2
     runhcrfar: int8.top1 91.69 (file) · unit seed=3 · produced before the declaration · seed=3
     runszd6le: int8.top1 91.57 (file) · unit seed=4 · produced before the declaration · seed=4
     runfyc6lr: int8.top1 91.66 (file) · unit seed=5 · produced before the declaration · seed=5
   ```

   ```rb-session frame="terminal"
   $ rb show relative-drop
   Criterion relative-drop on int8-seeds (primary, own): INT8 loses at most 0.1% of FP32's top-1
     relative-drop INT8 loses at most 0.1% of FP32's top-1
             estimate meets criterion · loss · change.top1 as a relative change ≥ -0.1%
             reads:     change.top1: -0.0916 [-0.1329, -0.05029] % · relative change of the means, paired, delta-method t interval, 5 seeds, 95% (rb)
             result:    loss: the interval clear of 0 on the worse side (higher is better)
             rule:      estimate meets criterion: estimate -0.0916% ≥ -0.1%
             timing:    declared after the evidence existed: all 5 units were produced before it was declared
             units:     5 seeds
             integrity: ok
             · the interval [-0.1329, -0.05029] crosses -0.1: met on the estimate, not across the whole interval
     runqsd4qe: int8.top1 91.6 (file) · unit seed=1 · produced before the declaration · seed=1
     runqsd4qe: fp32.top1 91.71 (file) · unit seed=1 · produced before the declaration · seed=1
     run7c7jnq: int8.top1 91.58 (file) · unit seed=2 · produced before the declaration · seed=2
     run7c7jnq: fp32.top1 91.66 (file) · unit seed=2 · produced before the declaration · seed=2
     runhcrfar: int8.top1 91.69 (file) · unit seed=3 · produced before the declaration · seed=3
     runhcrfar: fp32.top1 91.74 (file) · unit seed=3 · produced before the declaration · seed=3
     runszd6le: int8.top1 91.57 (file) · unit seed=4 · produced before the declaration · seed=4
     runszd6le: fp32.top1 91.69 (file) · unit seed=4 · produced before the declaration · seed=4
     runfyc6lr: int8.top1 91.66 (file) · unit seed=5 · produced before the declaration · seed=5
     runfyc6lr: fp32.top1 91.72 (file) · unit seed=5 · produced before the declaration · seed=5
   ```

A team on rb 0.7.0 or 0.7.1 cannot write to this record: they write nothing to a record that holds a format-2 file, and say to upgrade. Upgrade everyone who shares a record together.

## Why it matters

"The mean meets 91.58" and "4 of 5 seeds meet 91.58" are different claims. rb reads the one the plan makes.

---

# Share the record, and rerun it

https://rabbitbrain.ai/docs/cookbooks/share-a-record/

This cookbook shares a record with someone who was not there, and has their agent clone it, rerun it, and read what the criteria become.

Your advisor wants to see where the work stands without your repository, and to check it rather than take your word for it. A page lets them read it. A bundle lets their agent rerun it on their own machine.

## Paste this into your agent

**Write a page I can send, and a bundle anyone can rerun.**

Give it the address you will host the folder at.

```text
Share this Rabbit Brain record with my advisor.

- Write one page I can send: `rb page`.
- Write a bundle to host at https://example.org/int8-resnet/: `rb export --with-record --url https://example.org/int8-resnet/ --out ~/int8-site`.
- Tell me what each one carries, as rb says it: the page alone, or the record too.
```

The record: the INT8 ResNet-20 evaluation, three seeds run through `rb journal run`, each with its command and commit. The code lives in a git repository the reader can clone.

## What you'll see

**One page to send.** `rb page` writes the record as one self-contained HTML file. It names no writer and none of your words, so you can send it to anyone:

```rb-session frame="terminal"
$ rb page
rb-page.html written: the project as one file, readable anywhere; send it to anyone.
It is the record as computed now; regenerate after new runs. If it ever disagrees with rb status, rb status is right.
```

**A bundle to publish.** `rb export --with-record --url` writes a folder for any static host: the page, the record, and `bundle.zip`. rb says what the record keeps, since anyone who can read the bundle can read all of it:

```rb-session frame="terminal"
$ rb export --with-record --url https://example.org/int8-resnet/ --out ~/int8-site
~/int8-site written: the whole record, the page, and a manifest hashing every file.
Readable without rb; rb status inside record/ reopens it. Regenerate after new runs.
The record keeps who made each write (human:<name> or agent:<name>, and the channel it came through), an agent's chat session, name, version, provider and model, the user's words passed with --words, and each run's host, platform and branch. Anyone who can read the bundle can read all of it.
Code repository recorded: https://example.org/ada/int8-resnet.git at f49125ceaa. Pass --no-repo to leave it out.
Publish the directory at the address you gave; page.html is the reading surface and anyone reproduces it with: rb clone https://example.org/int8-resnet/bundle.zip
```

```rb-session frame="terminal"
$ ls ~/int8-site
README.md
bundle.zip
manifest.json
page.html
record
```

`page.html` in that folder, open in a browser:

<Shot name="rb-page" alt="rb's page for the INT8 ResNet-20 record: the open question, its one criterion met, one design waiting for a confirmation, and a panel to reproduce the project." width={1280} height={760} />

Its **Rerun this research** link opens a panel. This is its prompt, as rb wrote it into `page.html`. Whoever reads the page copies it into their own coding agent:

**Rerun this research**

The prompt on the page rb exported, word for word.

```text
Rerun this research with Rabbit Brain (rb). Its record: https://example.org/int8-resnet/

1. Install rb: pip install rabbit-brain
2. Clone the record into a new folder: rb clone https://example.org/int8-resnet/bundle.zip int8-resnet-20-on-cifar-10. This bundle names the code: rb clone gets it too, at the commit the record names, and puts the record in it. If it says it could not get the code, tell me what it says and stop: once I give you an address of the code, rb clone https://example.org/int8-resnet/bundle.zip <another new folder> --code <that address> gets it.
3. In that folder, run rb context and read it before anything else. Its "Rerun the parent" part names the code and whether it is here, each resource, such as a model or a dataset, with how to get it, and the first command for each experiment.
4. Get those resources. Run each experiment's commands in the order rb context names them, each as rb journal run -- <command>; where rb context names a command that puts a run's code here, run that one first. After each, run rb reconcile and record the run it proposes with the rb run add line it prints.
5. Then run rb status, and tell me which rules reproduced and which did not, with their numbers. Do not change a rule. If something cannot run here, say what and why.
```

## What the reader's agent did

We followed that prompt, step by step, in a fresh agent session in another folder, with the bundle saved there as `int8-record.zip`. Everything below is rb's real output.

1. Clone the record

   The bundle names its code, so `rb clone` gets it too: the clone is a checkout of that code at the bundle's commit, on a branch `rerun`, with the record in it.

   ```rb-session frame="terminal"
   $ rb clone int8-record.zip int8-advisor
   int8-advisor: "INT8 ResNet-20 on CIFAR-10" cloned: 1 experiment, 1 criterion, 1 metric, 0 resources. Everything arrived inherited; nothing counts until it is reproduced here.
   It is a checkout of the code at f49125ceaa2d, on branch rerun; the record is its latest commit, in place of the parent's, which git keeps at f49125ceaa2d.
   The parent's runs, analyses and decisions stay in the parent's record. Record runs made in this repository, and each inherited criterion reads them like any finding: reproduced or not reproduced.
   The code: https://example.org/ada/int8-resnet.git at f49125ceaa2d, checked out here. The parent's record is taken out of this branch; git keeps it at that commit, and the record here is this line of work's own.

   Next: cd int8-advisor && rb context   (the parent, the code, each resource and the first command to rerun)  ·  rb env check   (does this machine match the runs that made it?)  ·  rb export --with-record --url <your address>   (publish your fork)
   ```

   ```rb-session frame="terminal"
   $ git log --oneline -n 3
   92170f8 rerun of int8-resnet-20-on-cifar-10 from int8-record.zip: its record, inherited
   f49125c Record the INT8 runs
   12ec61a Ignore results
   ```

2. Read what it takes to rerun

   `rb context` opens with **Rerun the parent**: the code, and the first command for each experiment.

   ```rb-session frame="terminal"
   $ rb context
   # The record: INT8 ResNet-20 on CIFAR-10

   Generated by `rb context` from `.rb/`; read the state from here or `rb status --json`, never from memory or a summary.

   ## Rerun the parent

   - parent: INT8 ResNet-20 on CIFAR-10 (cloned 2026-11-08 from int8-record.zip)
   - code: https://example.org/ada/int8-resnet.git at f49125ceaa2d, here: this checkout's code is commit f49125ceaa2d's, the record aside
   - int8-ptq, first, for top1-drop-budget: `rb journal run -- python eval.py --seed 1` (the parent's run runrusdni of fp32 and int8, its command as given; per run seed=1; ran at 12ec61a726fe; this checkout's code is commit 12ec61a726fe's); no run recorded here yet
   … (93 more lines)
   ```

   `top1-drop-budget` is the criterion's id: its name in every command.

3. Run each command, and record what rb proposes

   Each seed runs through `rb journal run`. `rb reconcile` then finds what it wrote and prints the `rb run add` line that records it:

   ```rb-session frame="terminal"
   $ rb journal run -- python eval.py --seed 1
   {"metrics": {"fp32.top1": 91.71, "int8.top1": 91.6}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: fp32.top1=91.71, int8.top1=91.6. rb reconcile proposes the run of int8-ptq that records them.

   Since this session last looked:
     - results/int8-ptq/seed1/ holds results no recorded experiment reads yet   (rb reconcile)
     - results/int8-ptq/seed1/metrics.json: results landed and are not in the record   (rb reconcile)

   Next: rb reconcile
   ```

   ```rb-session frame="terminal"
   $ rb reconcile
   The repository at 92170f8: 1 results folder.

   results/int8-ptq/seed1/ holds results files no recorded run named. With no report of its own it is no experiment: rb run add <experiment> --from <its file> records a run from it.

   Results in the repository, not in the record (1):
     results/int8-ptq/seed1/metrics.json: fp32.top1=91.71, int8.top1=91.6
       rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml --commit 92170f82b815fa6839680aaa75658e66372dbca1 --command 'python eval.py --seed 1'

   rb reconcile --apply keeps this in the record: 1 run to record.

   Next: rb reconcile --apply  ·  rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml --commit 92170f82b815fa6839680aaa75658e66372dbca1 --command 'python eval.py --seed 1'
   ```

   ```rb-session frame="terminal"
   $ rb run add int8-ptq --from results/int8-ptq/seed1/metrics.json --set seed=1 --config results/int8-ptq/seed1/config.yaml --commit 92170f82b815fa6839680aaa75658e66372dbca1 --command "python eval.py --seed 1"
   runth5jdj on int8-ptq: fp32.top1=91.71, int8.top1=91.6 · seed=1
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed1/config.yaml, all matching.
     Produced: 2026-11-08 03:36:58 (file)
     top1-drop-budget criterion undetermined · point only: change.top1: -0.11 % · paired difference, 1 seed (rb)
     rb put the code in git at refs/rb/snapshots/f2b9ca9a08b5a6b34a895f43632125aa852bd4d9, so it outlives a deleted branch or worktree; git update-ref -d refs/rb/snapshots/f2b9ca9a08b5a6b34a895f43632125aa852bd4d9 removes it.

   Next: rb show top1-drop-budget
   ```

   After the third seed, the inherited criterion has its reading here:

   ```rb-session frame="terminal"
   $ rb run add int8-ptq --from results/int8-ptq/seed3/metrics.json --set seed=3 --config results/int8-ptq/seed3/config.yaml --commit 92170f82b815fa6839680aaa75658e66372dbca1 --command "python eval.py --seed 3"
   runx3rlvw on int8-ptq: fp32.top1=91.74, int8.top1=91.69 · seed=3
     Ran with the spec: 5 of its settings found in results/int8-ptq/seed3/config.yaml, all matching.
     Produced: 2026-11-08 03:37:00 (file)
     top1-drop-budget reproduced · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)

   Next: rb show top1-drop-budget
   ```

4. Read what the criteria became

   ```rb-session frame="terminal"
   $ rb status
   INT8 ResNet-20 on CIFAR-10 · int8-resnet-20-on-cifar-10

   Rerun the parent
     - parent: INT8 ResNet-20 on CIFAR-10 (cloned 2026-11-08 from int8-record.zip)
     - code: https://example.org/ada/int8-resnet.git at f49125ceaa2d, here: this checkout's code is commit f49125ceaa2d's, the record aside
     - int8-ptq, first, for top1-drop-budget: `rb journal run -- python eval.py --seed 1` (the parent's run runrusdni of fp32 and int8, its command as given; per run seed=1; ran at 12ec61a726fe; this checkout's code is commit 12ec61a726fe's); 3 runs recorded here

   Does INT8 lose at most 0.5 points of top-1 against FP32 on CIFAR-10? (open)
     INT8 loses at most 0.5 points of top-1 against FP32. change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb); loss: the interval clear of 0 on the worse side (higher is better); reproduced: estimate -0.08 ≥ -0.5; rule and question link declared before the evidence (by production times). units: 3 seeds; optim.lr = 0.1 is inherited from the parent project; optim.momentum = 0.9 is inherited from the parent project; data.batch_size = 128 is inherited from the parent project; eval.batch_size = 256 is inherited from the parent project; fp32.precision = fp32 is inherited from the parent project; int8.precision = int8 is inherited from the parent project; int8.calib_images = 512 is inherited from the parent project.

   Rules: 1 reproduced

   The design of 1 experiment is not confirmed: rb confirm lists it.

   rb looks in out/, outputs/, results/, eval-out/ and metrics/ at the top, in folders named for results (results/, eval/, metrics/, reports/, artifacts/ and the like), down to 4 levels from the top, at the files in the repository's top folder, and at .json, .yaml and .toml files git shows as new or changed. It last looked at 2026-11-08 03:37 UTC. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

   Rules
     top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
             reproduced · loss · change.top1 ≥ -0.5 (n ≥ 3)
             reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
             result:    loss: the interval clear of 0 on the worse side (higher is better)
             rule:      reproduced: estimate -0.08 ≥ -0.5
             timing:    declared before the evidence (by production times)
             units:     3 seeds
             · optim.lr = 0.1 is inherited from the parent project
             · optim.momentum = 0.9 is inherited from the parent project
             · data.batch_size = 128 is inherited from the parent project
             and 4 more: rb show top1-drop-budget

   Experiments
     int8-ptq INT8 post-training quantization · open · fp32 (baseline) vs int8 (candidate) · varies precision, calib_images · not frozen · settings 0 verified · 0 provisional · 0 unknown · 7 inherited · 1 per-run · runs 3
             to run again: code ok · environment ok · files ok

   Runs: 3 recorded on 1 experiment · every receipt is what rb wrote

   Nothing open.
   ```

The criterion arrived **inherited** and read **untested** until runs made in the clone tested it. It now reads **reproduced**, from the reader's own three runs. The parent's runs and calls stay in the parent's record.

A reader on an older rb than yours may not be able to clone it: rb 0.7.0 and 0.7.1 refuse to clone a bundle that holds a format-2 file, which rb 0.8.3 writes for a criterion that holds a count, a best of N, a relative change or a cited difference of means. Upgrade everyone who shares a record together.

## On rabbitbrain.ai

[rabbitbrain.ai](https://rabbitbrain.ai) serves shared record pages and the samples at [rabbitbrain.ai/demo](https://rabbitbrain.ai/demo). Each is rb's own page, with the same **Rerun this research** prompt, pointing at its `bundle.zip`.

`rb login` and `rb sync` are the commands for a private copy of your record's page in a rabbitbrain.ai account, which anyone can make at [rabbitbrain.ai/signin](https://rabbitbrain.ai/signin). With `rb sync --with-record`, readers of a share link you make from that page can rerun the record, until you revoke the link. `rb login` starts with a code to approve:

```rb-session frame="terminal"
$ rb login

  First, this terminal gets a code:

      PJWM-24RF

  Open https://rabbitbrain.ai/link?code=PJWM-24RF on any device; it signs you in first if needed, and approve the code.
```

## Why it matters

The person you share with reads the findings rb computes, and can rerun them on their own machine, without your account or your word for it.

---

# Rerun a run from its receipt

https://rabbitbrain.ai/docs/cookbooks/rerun-from-receipt/

This cookbook reruns one run from its receipt, on the code it ran on, and records the rerun as a check of that run.

Weeks later, someone asks whether seed 2 still gives the same numbers, and the code has changed since. The run's receipt says exactly what ran and on which commit, so the rerun is a lookup, not a dig through history.

## Paste this into your agent

**Rerun seed 2 from its receipt, on the code it ran on, and check it.**

```text
Rerun seed 2 of int8-ptq from its Rabbit Brain receipt, and tell me if it still gives the same numbers.

- Run `rb context --model <your model id>` first.
- Read the run's receipt with `rb show <run id>`: the command, and the commit it ran on.
- Check out that commit in a worktree inside this repository (`git worktree add .worktrees/rerun <commit>`), and run the receipt's command there through `rb journal run`.
- Record the new run as a check of the old one: `--repeats <run id> --again --why "rerun from its receipt"`, with `--commit`.

Finish with `rb show` of the criterion exactly as printed, and what rb says of the repeat.
```

The repository: the INT8 ResNet-20 evaluation, recorded in an earlier session with each seed run through `rb journal run`. One commit later, `eval.py` builds its output path differently.

## What you'll see

```rb-session frame="terminal"
$ rb show top1-drop-budget
Criterion top1-drop-budget on int8-ptq (primary, own): INT8 loses at most 0.5 points of top-1 against FP32
  top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
          criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
          reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
          result:    loss: the interval clear of 0 on the worse side (higher is better)
          rule:      criterion met: estimate -0.08 ≥ -0.5
          timing:    declared before the evidence (by production times)
          units:     3 seeds from 4 runs, 1 repeat
          integrity: ok
          · runlkp4tv repeats runw6nm5h: the same number, a check that holds
  runux27j3: difference -0.11 (file) · unit seed=1 · seed=1
  runw6nm5h: difference -0.08 (file) · unit seed=2 · seed=2
  runu6idii: difference -0.05 (file) · unit seed=3 · seed=3
  runlkp4tv: difference -0.08 (file) · unit seed=2 · repeats runw6nm5h · seed=2
```

`top1-drop-budget` is the criterion's id: its name in every command. The rerun joins seed 2's unit and adds no unit of its own: the criterion still rests on 3 seeds, now "from 4 runs, 1 repeat". rb says the check holds: **the same number, a check that holds**.

## What rb did, step by step

1. Read what it takes to run the experiment again

   ```rb-session frame="terminal"
   $ rb show int8-ptq
   Experiment int8-ptq: INT8 post-training quantization · open · under q-int8
     design not confirmed · list it to confirm: rb confirm int8-ptq
     hypothesis or purpose: not recorded
     why: not recorded
     variants: fp32 (baseline) · int8 (candidate)
     varies: precision, calib_images
     spec 85201d31eae8 · frozen 2026-11-08T03:37:03.922+00:00 (85201d31eae8) · note: before the runs · not confirmed
     settings:
       optim.lr                 0.1            verified · configs/train.yaml#optim.lr
       optim.momentum           0.9            verified · configs/train.yaml#optim.momentum
       data.batch_size          128            verified · configs/train.yaml#data.batch_size
       eval.batch_size          256            verified · configs/train.yaml#eval.batch_size
       seed                     (each run)     per-run · no source
       fp32.precision           fp32           verified · configs/fp32.yaml#precision
       int8.precision           int8           verified · configs/int8.yaml#precision
       int8.calib_images        512            verified · configs/int8.yaml#calib_images
     criteria:
     top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
             criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
             reads:     change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
             result:    loss: the interval clear of 0 on the worse side (higher is better)
             rule:      criterion met: estimate -0.08 ≥ -0.5
             timing:    declared before the evidence (by production times)
             units:     3 seeds
             integrity: ok
     runs: runux27j3, runw6nm5h, runu6idii
     to run it again:
       code         ok       commit 12ec61a726
       environment  ok       python 3.14.4
       files        ok       all 3 files the runs named are here unchanged
   ```

2. Read the run's receipt

   The command, which rb watched make these numbers, and the commit the code was at:

   ```rb-session frame="terminal"
   $ rb show runw6nm5h
   Run runw6nm5h on int8-ptq (file)
     produced 2026-11-08 03:37 UTC (its file's time) · recorded 2026-11-08 03:37 UTC
     fp32.top1 = 91.66
     int8.top1 = 91.58
     per-run: seed=2
     command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-08)
     code that made it: 12ec61a726fe (as given)
     repository when recorded: 12ec61a726fe
     config results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
     file results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here
   ```

3. See what changed since

   ```rb-session frame="terminal"
   $ git diff 12ec61a -- eval.py
   diff --git a/eval.py b/eval.py
   index 35a1cb3..901a798 100644
   --- a/eval.py
   +++ b/eval.py
   @@ -7,7 +7,7 @@ p = argparse.ArgumentParser()
p.add_argument("--seed", type=int, required=True)
a = p.parse_args()
fp32, int8 = TOP1[a.seed]
   -out = f"results/int8-ptq/seed{a.seed}"
   +out = os.path.join("results", "int8-ptq", f"seed{a.seed}")
os.makedirs(out, exist_ok=True)
with open(f"{out}/metrics.json", "w") as f:
        json.dump({"fp32": {"top1": fp32}, "int8": {"top1": int8}}, f, indent=2)
   ```

4. Check out the receipt's commit in a worktree

   A worktree inside the repository, with no record of its own, writes to the repository's record, and each run there keeps the worktree's code on its receipt.

   ```rb-session frame="terminal"
   $ git worktree add -q .worktrees/rerun 12ec61a
   ```

   Then, in `.worktrees/rerun`, the briefing says so first:

   ```rb-session frame="terminal"
   $ rb context
   # The record: INT8 ResNet-20 on CIFAR-10

   This worktree (detached at 12ec61a726fe · ~/int8-rerun/.worktrees/rerun) has no record of its own: rb writes to the record in main · ~/int8-rerun, and each run keeps this worktree's code.

   … (94 more lines)
   ```

5. Run the receipt's command

   ```rb-session frame="terminal"
   $ rb journal run -- python eval.py --seed 2
   {"metrics": {"fp32.top1": 91.66, "int8.top1": 91.58}}
   exit 0. The command, its exit code and what it wrote are in the journal.
   It printed its metrics: fp32.top1=91.66, int8.top1=91.58. rb reconcile proposes no new run: a recorded run of int8-ptq holds these numbers.

   Next: rb reconcile
   ```

6. Record it as a check of the run

   The output is the same, byte for byte, as the first run's, so rb would read it as the same run. `--again --why` records it as a second execution, and `--repeats` makes it a check of seed 2, not a new seed.

   ```rb-session frame="terminal"
   $ rb run add int8-ptq --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml --command "python eval.py --seed 2" --commit 12ec61a726feda8c90dbba424b07ee7ec0b9d32c --repeats runw6nm5h --again --why "rerun from its receipt"
   runlkp4tv on int8-ptq: fp32.top1=91.66, int8.top1=91.58 · seed=2
     Ran with the spec: 5 of its settings found in .worktrees/rerun/results/int8-ptq/seed2/config.yaml, all matching.
     It repeats runw6nm5h: a check of that run, in its unit, never a new one.
     Produced: 2026-11-08 03:37:06 (file)
     top1-drop-budget criterion met · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)

   Next: rb show top1-drop-budget
   ```

   ```rb-session frame="terminal"
   $ rb show runlkp4tv
   Run runlkp4tv on int8-ptq (file)
     produced 2026-11-08 03:37 UTC (its file's time) · recorded 2026-11-08 03:37 UTC
     fp32.top1 = 91.66
     int8.top1 = 91.58
     per-run: seed=2
     command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-08)
     code that made it: 12ec61a726fe (as given)
     repository when recorded: 12ec61a726fe
     config .worktrees/rerun/results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
     file .worktrees/rerun/results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here
     note: recorded again: rerun from its receipt. · not confirmed
   ```

## Why it matters

A receipt turns "can you rerun that?" into one command on one commit, and the rerun lands as a check of the run, not as an extra seed.

---

# rb with Claude Code, Codex and Cursor

https://rabbitbrain.ai/docs/cookbooks/three-agents/

This cookbook sets one repository up for Claude Code, Codex and Cursor, so all three keep the same record.

You use Claude Code on Monday, Codex on Tuesday, and Cursor when you are in the editor. Each agent needs rb's instructions in the place it reads them, and all three must write to the same record in the same way.

## Paste this into your agent

**Set this repository up for every agent you use, and say what goes in git.**

```text
Set this repository up so Claude Code, Codex and Cursor all keep the same Rabbit Brain record.

- A session that is already running loads no new skill, so `rb init` installs nothing in this session. Tell me to run `rb init --dry-run`, then `rb init "<the work's theme>"`, in my own terminal.
- After I have, run `git status --short`, and tell me which files rb added for each agent and which ones to commit.
```

The repository already has a `CLAUDE.md`, an `AGENTS.md` of its own, and a `.cursor/` folder.

## What you'll see

`rb init`, in your own terminal:

```rb-session frame="terminal"
$ rb init "INT8 ResNet-20 on CIFAR-10"
Project 'int8-resnet-20-on-cifar-10' started: ~/int8-agents/.rb
Rabbit Brain YAML files beside the code: commit .rb/ with it, so the record travels with the work.

For the agents here:
  Claude Code (CLAUDE.md): the skill in .claude/skills/rabbit-brain/SKILL.md; rb's MCP server in a new .mcp.json; rb's session hook in a new .claude/settings.json (SessionStart and Stop run rb journal scan, which looks at the repository and says nothing)
  The agents that read AGENTS.md (AGENTS.md): AGENTS.md was here before rb, so rb left it as it is. rb skill add --agents-md --yes adds rb's section at its end (the text rb skill prints, between two marker lines).
  Cursor (.cursor/): .cursor/rules/rabbit-brain.mdc; rb's session hook in a new .cursor/hooks.json (sessionStart and stop run rb journal scan, which looks at the repository and says nothing)

Then paste this line into your agent's chat:
  Use Rabbit Brain to track this: read rb context, then record where the work stands.
```

rb made the files that were not there, and left the `AGENTS.md` that was. In your own terminal it asks once before it changes a file that was there, and anything but `y` or `yes` keeps it as it is. We ran it where it could not ask, so it left the file and named the command that adds rb's section:

```rb-session frame="terminal"
$ rb skill add --agents-md --yes
wrote ~/int8-agents/AGENTS.md
The agent picks it up in its next session. The skill defers to `rb docs` for the full rules.
```

```rb-session frame="terminal"
$ git status --short
 M AGENTS.md
?? .claude/
?? .cursor/
?? .mcp.json
?? .rb/
```

| Agent | What rb wrote | Where |
| --- | --- | --- |
| Claude Code | the skill, rb's MCP server, and rb's session hook | `.claude/skills/rabbit-brain/SKILL.md`, `.mcp.json`, `.claude/settings.json` |
| Codex, and any agent that reads `AGENTS.md` | rb's section, between two marker lines, at the end of the file | `AGENTS.md` |
| Cursor | a rule, and rb's session hook | `.cursor/rules/rabbit-brain.mdc`, `.cursor/hooks.json` |

This repository has no `.codex/`, so rb wrote no hook for Codex. Where `.codex/` is there or Codex runs `rb init`, rb adds its session hook to `.codex/hooks.json` too. [The session hook](https://rabbitbrain.ai/docs/connect/#the-session-hook) says what it does.

Commit all of it with `.rb/`: the next session of each agent, on any machine, reads the same instructions and writes to the same record.

## What rb did, step by step

1. Show the plan, write nothing

   In your own terminal:

   ```rb-session frame="terminal"
   $ rb init --dry-run
   Dry run: nothing was written. The write would say:
   Project 'int8-agents' started: ~/int8-agents/.rb
   Rabbit Brain YAML files beside the code: commit .rb/ with it, so the record travels with the work.

   For the agents here:
     Claude Code (CLAUDE.md): the skill in .claude/skills/rabbit-brain/SKILL.md; rb's MCP server in a new .mcp.json; rb's session hook in a new .claude/settings.json (SessionStart and Stop run rb journal scan, which looks at the repository and says nothing)
     The agents that read AGENTS.md (AGENTS.md): AGENTS.md was here before rb, so rb left it as it is. rb skill add --agents-md --yes adds rb's section at its end (the text rb skill prints, between two marker lines).
     Cursor (.cursor/): .cursor/rules/rabbit-brain.mdc; rb's session hook in a new .cursor/hooks.json (sessionStart and stop run rb journal scan, which looks at the repository and says nothing)

   Then paste this line into your agent's chat:
     Use Rabbit Brain to track this: read rb context, then record where the work stands.

   rb init would write:
     .rb/: the record (.gitignore, .gitattributes, project.yaml, log.jsonl), and rb's lock and local copies (.lock, cache/), which .rb/.gitignore keeps out of git
     .claude/skills/rabbit-brain/SKILL.md: new
     .mcp.json: new
     .claude/settings.json: new
     .cursor/rules/rabbit-brain.mdc: new
     .cursor/hooks.json: new
     .git/rb/owned.jsonl: rb's list of what it writes outside .rb/
     .git/rb/journal/: what rb sees land in the repository, from the look rb init makes as the record begins
   Later commands add:
     refs/rb/snapshots/<tree>: a ref in git to the code a run was recorded at; git update-ref -d removes it (rb run add, rb journal run)
     .git/rb/views/: a past version of the record, checked out while the screen shows it (rb open, rb ui)
     .git/rb-last-seen.json: what you last saw, so rb ui can say what is new since (rb ui)
     rb-page.html: the record as one page, beside .rb/ by default (rb page)
     rb-export/: the page with a manifest, beside .rb/ by default (rb export)
     rb-export.zip: the same, as one zip file (rb export --zip)
   ```

2. In an agent session, install nothing

   Run inside a session that is already running, `rb init` installs nothing, since that session would not load it, and names the commands for later sessions:

   ```rb-session frame="terminal"
   $ rb init "INT8 ResNet-20 on CIFAR-10"
   Project 'int8-resnet-20-on-cifar-10' started: ~/int8-agents-session/.rb
   Rabbit Brain YAML files beside the code: commit .rb/ with it, so the record travels with the work.

   For the agents here:
     Nothing installed: a session already running loads no skill or MCP server added during it. For later sessions:
       Claude Code: rb skill add --claude --project, and claude mcp add --scope project rabbit-brain -- rb mcp
       The agents that read AGENTS.md: rb skill add --agents-md
       Cursor: rb skill add --cursor

   Use Rabbit Brain to track this: read rb context, then record where the work stands.

   Next: rb context --model <your model id>
   ```

3. rb's MCP server, for Claude Code

   ```rb-session frame="terminal"
   $ cat .mcp.json
   {
     "mcpServers": {
       "rabbit-brain": {
         "command": "rb",
         "args": [
           "mcp"
         ]
       }
     }
   }
   ```

   `claude mcp add --scope project rabbit-brain -- rb mcp` adds the same entry by hand. Every call over MCP is recorded as the agent's.

4. rb's session hook, for Cursor

   ```rb-session frame="terminal"
   $ cat .cursor/hooks.json
   {
     "version": 1,
     "hooks": {
       "sessionStart": [
         {
           "command": "rb journal scan --hook cursor; exit 0",
           "timeout": 10
         }
       ],
       "stop": [
         {
           "command": "rb journal scan --hook cursor; exit 0",
           "timeout": 10
         }
       ]
     }
   }
   ```

   As a session starts and as each turn ends, it runs `rb journal scan`, which looks at the repository and says nothing.

5. rb's section, for Codex

   ```rb-session frame="terminal"
   $ tail -n 12 AGENTS.md
   - work in another branch or worktree is recorded there the same way, never as an experiment per branch: name the experiment and the variant on every run, and the records merge with git (`rb status` says what branches not merged yet hold).

   ## Boundaries

   - Never compute a finding, an error, a ranking or a regression yourself: record the numbers, and read the finding.
   - Never edit `.rb/` by hand: rb catches it, and `rb doctor` lists it.
   - Never set or unset `RB_ACTOR` or `RB_HUB_TOKEN`, and never read the credentials file `rb login` writes.
   - Report a finding by its axes as rb prints them: the comparison with its interval and units, the rule, the timing, and every integrity problem. `criterion undetermined · point only` on 2 units is said in those words, never rounded up to "it works".

   Everything else is in `rb docs`. Tools instead of shell: `claude mcp add rabbit-brain -- rb mcp` from the project root; same record, same rules.

   <!-- rabbit-brain skill: end -->
   ```

6. And how to take it all out

   `rb remove` takes out only what is rb's, including the hook files it made, and puts `AGENTS.md` back byte for byte:

   ```rb-session frame="terminal"
   $ rb remove --dry-run
   Dry run: nothing was removed. rb remove would do this in ~/int8-agents:
   Will remove:
     .rb/: the record (1 record, each as its log says rb wrote it, and the log), and rb's own files and local copies
     .claude/skills/rabbit-brain/SKILL.md: rb's skill for Claude Code, as rb wrote it
     .cursor/rules/rabbit-brain.mdc: rb's rule for Cursor, as rb wrote it
     AGENTS.md: rb's section, as rb added it: the file goes back to what it was, byte for byte
     .mcp.json: made by rb, and as rb wrote it
     .claude/settings.json: made by rb, and as rb wrote it
     .cursor/hooks.json: made by rb, and as rb wrote it
     .git/rb/: rb's folder inside git's: its journal, its list of what it wrote outside .rb/, and the past versions it showed
   Not rb's to touch: your research files, worktrees and .git/worktrees, git history, git objects, config, hooks and the index.
   Listed, not removed:
     the rb command itself: pip uninstall rabbit-brain (or uv tool uninstall rabbit-brain, pipx uninstall rabbit-brain)
   4 files of the record have uncommitted changes. To keep a copy first, run: rb export --with-record --zip --out ~/int8-resnet-20-on-cifar-10-record.zip
   ```

## Who wrote what

All three agents write the same commands to the same record, and rb records which agent made each write, with its chat session and the model it said it runs on. That stays in the data (`rb log --json`). The screens never name a writer: what they show is whether something an agent wrote waits for you, such as **design not confirmed** or **Proposed, not made**.

## Why it matters

Whichever agent did the work, the record reads the same, and you confirm the same way.

---

# The CLI

https://rabbitbrain.ai/docs/reference/cli/

This reference lists every rb command, grouped as `rb --help` groups them, with each command's help exactly as rb prints it.

Generated from the `--help` of rabbit-brain 0.8.3.

Every command accepts `--json` and then prints exactly one JSON object on standard output, the envelope: `ok`, `rb_version`, `command`, `data`, `errors`, `next` and `rb_delta`. `rb schema envelope` prints its shape.

| Exit code | What it means |
| --- | --- |
| `0` | ok |
| `1` | a check that ran and failed: a gate of `rb status --fail-on`, a setting that did not verify, a problem `rb doctor` found, a file `rb fmt --check` would rewrite, a file `rb show` read that is not a valid record, or the command `rb journal run` wrapped. The JSON has `ok: true`, because the result is data. |
| `2` | rb could not do what was asked, and says why |
| `3` | a fault in rb itself |

| Group | Commands |
| --- | --- |
| [The record](https://rabbitbrain.ai/docs/reference/cli/the-record/) | `init`, `project`, `question`, `experiment`, `variant`, `note`, `metric`, `spec`, `criterion`, `run`, `analysis`, `resource` |
| [Calls](https://rabbitbrain.ai/docs/reference/cli/calls/) | `freeze`, `decide`, `confirm`, `retract` |
| [Reading it](https://rabbitbrain.ai/docs/reference/cli/reading/) | `open`, `ui`, `status`, `show`, `compare`, `log`, `context`, `page` |
| [Carrying it](https://rabbitbrain.ai/docs/reference/cli/carrying/) | `export`, `clone`, `env` |
| [For agents](https://rabbitbrain.ai/docs/reference/cli/for-agents/) | `mcp`, `skill`, `journal`, `reconcile`, `compile` |
| [Upkeep](https://rabbitbrain.ai/docs/reference/cli/upkeep/) | `doctor`, `fmt`, `telemetry`, `config` |
| [Leaving](https://rabbitbrain.ai/docs/reference/cli/leaving/) | `remove` |
| [Reference](https://rabbitbrain.ai/docs/reference/cli/reference/) | `tour`, `docs`, `schema`, `version`, `feedback` |
| [rabbitbrain.ai](https://rabbitbrain.ai/docs/reference/cli/rabbitbrain-ai/) | `login`, `logout`, `sync` |

## rb --help

```text
usage: rb <command> [...]

The record of the experiments your coding agents run, as Rabbit Brain YAML files in your repo: questions, experiments with their hypothesis, variants and criteria, runs with receipts, and the resources they used. Agents record; rb checks and computes findings; you decide. Documentation: rb docs. New here? Run rb tour.

The record:
  init         start the record (.rb/) here
  project      title: the project's title
  question     add|move|reword: what the work is trying to find out, as a tree experiments are filed under
  experiment   add|file|relate|gate: what is tested: its purpose, hypothesis, variants and settings; where it is filed; how it stands to others; the decision it gates
  variant      add: an arm of an experiment: baseline, candidate, control, ablation
  note         add: words about an experiment or the project: an assumption, a caveat, context; a note changes no finding
  metric       add: the catalogue: a metric's unit, which way is better, its other names
  spec         set|verify|vary: an experiment's settings, where each value comes from, and whether rb has checked it
  criterion    add|file|relate: a rule over one comparison, declared before the evidence it reads
  run          add: what ran: its results, per-run settings, inputs and outputs, with a receipt
  analysis     add: a computation over saved outputs, and the comparisons it produced with their intervals
  resource     add|locate|expose: a dataset, checkpoint, code or artifact: what it is, where its bytes are, how to get it again, who has looked at it

Calls:
  freeze       lock an experiment's spec and criteria before the evidence, recorded with who locked it
  decide       a call on a criterion, an experiment, a question or a setting: accept, reject, investigate; continue, park, stop, pivot, ship. Recorded with who made it and why; in an agent session it is proposed until the user confirms it
  confirm      make a call proposed, or a past call mentioned, in an agent session yours, or an experiment's design one wrote: run in your own terminal; with no id, list what waits
  retract      take anything back: it stays on record, stops counting, and rb says what rested on it

Reading it:
  open         the catalogue in your browser, live and read-only, on 127.0.0.1 only
  ui           open this project in your terminal: your call, the cards, the tree (bare rb does this too)
  status       each criterion on its axes, and what is open, with who usually acts
  show         one record: a criterion with its finding, an experiment with its spec, a run with its receipt, e1/lr with its history; or a .yaml record file as canonical JSON
  compare      an experiment's own table: variants by metrics, with changes against the baseline
  log          every write, newest last: who, how, what
  context      the record as one Markdown briefing a fresh agent session reads first
  page         write the project as one self-contained, read-only HTML file you can send anywhere

Carrying it:
  export       write the project's page with a hashed manifest, readable without rb; with --with-record, the record too, as a portable bundle
  clone        start your own line of work from an exported bundle: the structure arrives inherited, and nothing counts until reproduced here
  env          the environment: check this machine against the record's runs and any env.* settings

For agents:
  mcp          serve the record to an agent over MCP (stdio); every call is recorded as the agent's
  skill        print the agent skill; `rb skill add` installs it for your coding agent
  journal      what rb observed in the repository: files that appeared, git moves, wrapped commands
  reconcile    compile the repository against the record: each results folder with a report as an experiment, what its files say (verbatim), its checks, how to reproduce it; --apply keeps it
  compile      list what existing reports and results files the record can place, and what it cannot; writes nothing

Upkeep:
  doctor       check the record: who you are recorded as, edits made outside rb, merge leftovers, sources outside the repository
  fmt          write Rabbit Brain YAML files in their canonical form, comments kept (default: every record file under .rb/)
  telemetry    optional usage analytics: off until you enable it; never sends research content
  config       rb's own settings on this machine: whether rb keeps itself up to date

Leaving:
  remove       take rb out of this checkout: only what checks out as rb's (the record, its instructions for agents, its pages and exports, its refs and folders in git), after showing the plan

Reference:
  tour         a one-minute walkthrough: builds a small real project in ./rb-tour and runs the whole loop
  docs         print AGENTS.md (--start: how to start the record in a project already under way; --errors: the error table)
  schema       print a JSON Schema: `rb schema all` is Rabbit Brain formats 1 and 2 (rb-v2.schema.json), `rb schema <kind>` one record's shape
  version      print the version
  feedback     a link to send feedback on rb: the page shows what it would send before you send it; rb sends nothing

rabbitbrain.ai:
  login        rabbitbrain.ai, optional: sign this machine in with a short code, approved on any device where you are signed in
  logout       revoke this machine's hub credential and remove it from the credentials file
  sync         rabbitbrain.ai, optional: a private copy of this record's page in your account, for your own eyes from anywhere

rb <command> --help for its arguments. Start: rb init "<the work's theme>", then rb question add for each question the work asks, and rb context.
```

---

# Commands: The record

https://rabbitbrain.ai/docs/reference/cli/the-record/

These commands start the record and write to it: the project's title, questions, experiments, variants, notes, metrics, settings, criteria, runs, analyses and resources.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb init

start the record (.rb/) here

```text
usage: rb init [-h] [--json] [--id ID] [--no-agents] [--new-project] [--yes]
               [--dry-run]
               [title]

positional arguments:
  title          the work's theme, which its questions share: a paper's title,
                 a model, a line of work (default: this directory's name)

options:
  -h, --help     show this help message and exit
  --json         print one JSON object (the envelope) instead of text
  --id ID        a short id for the project (default: from the title)
  --no-agents    leave the agents' instruction files alone (default: install
                 rb's instructions for the agents found here, as rb skill add
                 does)
  --new-project  start a separate project in a worktree whose repository
                 already holds the project's record in another worktree
                 (default: refused, with how to bring that record here)
  --yes          add rb's section to an AGENTS.md and rb's server to a
                 .mcp.json that are here without asking (default: asked once
                 in your terminal; elsewhere left as they are, with what to
                 add)
  --dry-run      list every file and integration rb init would write, every
                 file it would change and how, and what later commands add,
                 writing nothing
```

## rb project

title: the project's title

```text
usage: rb project [-h] <title> ...

positional arguments:
  <title>
title     change the project's title, keeping its identity and record

options:
  -h, --help  show this help message and exit
```

### rb project title

```text
usage: rb project title [-h] [--json] [--dry-run] -m WHY [--words TEXT] title

positional arguments:
  title              the work's theme: a paper's title, a model, a line of
                     work

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  -m WHY, --why WHY  why the title is changing, kept in the log
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

## rb question

add|move|reword: what the work is trying to find out, as a tree experiments are filed under

```text
usage: rb question [-h] <add|move|reword> ...

positional arguments:
  <add|move|reword>
add              add a question, at the top or under another
move             put a question under another one, or at the top; its
                     evidence goes with it
reword           give a question new wording, kept as a revision: the log
                     keeps the old wording, who changed it and why; its id,
                     evidence and calls stay

options:
  -h, --help         show this help message and exit
```

### rb question add

```text
usage: rb question add [-h] [--json] [--dry-run] [--id ID] [--parent PARENT]
                       [--words TEXT]
                       text

positional arguments:
  text

options:
  -h, --help       show this help message and exit
  --json           print one JSON object (the envelope) instead of text
  --dry-run        run every check the write makes and print what it would
                   write, writing nothing; it exits as the write would
  --id ID
  --parent PARENT  the question this one sits under
  --words TEXT     the user's words from the chat that this acts on, verbatim
                   (an agent quoting them); kept beside the reason
```

### rb question move

```text
usage: rb question move [-h] [--json] [--dry-run] [--parent PARENT] [--top]
                        [-m WHY] [--words TEXT]
                        question

positional arguments:
  question

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  --parent PARENT
  --top
  -m WHY, --why WHY  the reason, recorded with the call
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

### rb question reword

```text
usage: rb question reword [-h] [--json] [--dry-run] -m WHY [--words TEXT]
                          question text

positional arguments:
  question
  text               the new wording, verbatim: the user's own words, never a
                     formalization of them

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  -m WHY, --why WHY  why it is reworded, kept in the log
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

## rb experiment

add|file|relate|gate: what is tested: its purpose, hypothesis, variants and settings; where it is filed; how it stands to others; the decision it gates

```text
usage: rb experiment [-h] <add|file|relate|gate> ...

positional arguments:
  <add|file|relate|gate>
add                 add an experiment: planned until its first run
file                file an experiment under one more question (or
                        --remove it from one)
relate              how an experiment stands to another: reproduces,
                        extends, follows_from, compares_with
gate                the decision an experiment gates, set or revised: the
                        action taken if it holds, and if not; the sentence it
                        was said in goes in --words

options:
  -h, --help            show this help message and exit
```

### rb experiment add

```text
usage: rb experiment add [-h] [--json] [--dry-run] [--id ID] [-q QUESTION]
                         [--hypothesis HYPOTHESIS] [-m WHY]
                         [--purpose PURPOSE] [--baseline BASELINE]
                         [--candidate CANDIDATE] [--varies VARIES]
                         [--like LIKE] [--note NOTE] [--if-holds TEXT]
                         [--if-not TEXT] [--words TEXT]
                         title

positional arguments:
  title

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --id ID               a short id you choose (default: e and six random
                        characters)
  -q QUESTION, --question QUESTION
                        a question it is filed under (repeatable: an
                        experiment can answer several)
  --hypothesis HYPOTHESIS
                        what you expect it to show
  -m WHY, --why WHY     why you expect it
  --purpose PURPOSE     what it is for, when that is not a hypothesis (a
                        benchmark, a sweep)
  --baseline BASELINE   name of the baseline variant, e.g. fp32
  --candidate CANDIDATE
                        name of a candidate variant, e.g. int8 (repeatable)
  --varies VARIES       the settings the variants differ in on purpose
                        (repeatable or comma-separated); any other difference
                        is a confound
  --like LIKE           copy another experiment's shared settings (as
                        provisional) and what it varies
  --note NOTE           a first note: context, an assumption, a caveat
  --if-holds TEXT       the decision it gates: the action taken when it holds;
                        the sentence it was said in goes in --words
  --if-not TEXT         and what to do when it does not
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

### rb experiment file

```text
usage: rb experiment file [-h] [--json] [--dry-run] [--remove]
                          experiment question

positional arguments:
  experiment
  question

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --dry-run   run every check the write makes and print what it would write,
              writing nothing; it exits as the write would
  --remove
```

### rb experiment relate

```text
usage: rb experiment relate [-h] [--json] [--dry-run] [--note NOTE]
                            experiment
                            {reproduces,extends,follows_from,compares_with}
                            target

positional arguments:
  experiment
  {reproduces,extends,follows_from,compares_with}
  target                another experiment's id here, its global uid, or a
                        published manifest's address

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --note NOTE
```

### rb experiment gate

```text
usage: rb experiment gate [-h] [--json] [--dry-run] [--if-holds TEXT]
                          [--if-not TEXT] [-m WHY] [--words TEXT]
                          experiment

positional arguments:
  experiment

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  --if-holds TEXT    the action taken when it holds
  --if-not TEXT      what to do when it does not
  -m WHY, --why WHY  the reason, recorded with the call
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

## rb variant

add: an arm of an experiment: baseline, candidate, control, ablation

```text
usage: rb variant [-h] <add> ...

positional arguments:
  <add>
add       add a variant to an experiment

options:
  -h, --help  show this help message and exit
```

### rb variant add

```text
usage: rb variant add [-h] [--json] [--dry-run] --role
                      {baseline,candidate,control,ablation} [--note NOTE]
                      [--amend] [-m WHY] [--words TEXT]
                      experiment name

positional arguments:
  experiment
  name

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --role {baseline,candidate,control,ablation}
  --note NOTE
  --amend               the experiment is frozen: add it anyway (with --why);
                        the amendment is recorded
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

## rb note

add: words about an experiment or the project: an assumption, a caveat, context; a note changes no finding

```text
usage: rb note [-h] <add> ...

positional arguments:
  <add>
add       add a note, kept verbatim with who wrote it

options:
  -h, --help  show this help message and exit
```

### rb note add

```text
usage: rb note add [-h] [--json] [--dry-run] [-e EXPERIMENT] [--words TEXT]
                   text

positional arguments:
  text

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  -e EXPERIMENT, --experiment EXPERIMENT
                        the experiment it is about (default: the whole
                        project)
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

## rb metric

add: the catalogue: a metric's unit, which way is better, its other names

```text
usage: rb metric [-h] <add> ...

positional arguments:
  <add>
add       declare a metric

options:
  -h, --help  show this help message and exit
```

### rb metric add

```text
usage: rb metric add [-h] [--json] [--dry-run] [--unit UNIT]
                     [--minimize | --maximize] [--alias ALIAS]
                     [--description DESCRIPTION]
                     name

positional arguments:
  name

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --unit UNIT
  --minimize            lower is better (an error, a loss, a latency)
  --maximize            higher is better (an accuracy, a score)
  --alias ALIAS         another name runs may use for it (repeatable), e.g.
                        cross_entropy
  --description DESCRIPTION
```

## rb spec

set|verify|vary: an experiment's settings, where each value comes from, and whether rb has checked it

```text
usage: rb spec [-h] <set|verify|vary> ...

positional arguments:
  <set|verify|vary>
set              give a setting's value and source (checked when it can
                     be), record it as unknown, or read many from a config
                     with --from
verify           check each setting's source again and record what it
                     states; verified only if it states the value
vary             declare settings the variants differ in on purpose; any
                     other difference is a confound

options:
  -h, --help         show this help message and exit
```

### rb spec set

```text
usage: rb spec set [-h] [--json] [--dry-run] [--unknown] [--per-run]
                   [--design-factor | --sample-factor] [--source SOURCE]
                   [--quote QUOTE] [--term TERM] [--commit COMMIT]
                   [--locator LOCATOR] [--variant VARIANT]
                   [--optional | --required] [--cited VALUE]
                   [--cited-source CITED_SOURCE] [--from FILE] [--keys KEYS]
                   [--note NOTE] [--no-verify] [--amend] [-m WHY]
                   [--words TEXT]
                   experiment [name] [value]

positional arguments:
  experiment
  name                  the setting; <variant>.<name> for one variant's own
                        value
  value                 a number, true/false, text, or a JSON list; optional
                        with file#key or run: sources

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --unknown             nobody has found this value yet
  --per-run             each run gives its own value (a seed): --set seed=2 on
                        rb run add
  --design-factor       a fixed design choice, not an independent sample; a
                        new setting becomes per-run, and an existing one must
                        already be per-run
  --sample-factor       clear this setting's design-factor declaration
  --source SOURCE       where the value comes from: file#key (YAML/JSON/TOML),
                        database#table[column=value].column (SQLite),
                        file:LINE, run:PATH#/pointer, https://..., or
                        note:text
  --quote QUOTE         the exact text in a file or url source that states it
  --term TERM           the word the source uses for it on a prose line, e.g.
                        "learning rate"
  --commit COMMIT       read the file at this git commit instead of the
                        working tree
  --locator LOCATOR     how a reader finds it, e.g. "§4.2" or "Table 3"
  --variant VARIANT     set it for this variant only (same as
                        <variant>.<name>)
  --optional            an unknown value is a note, not an integrity problem
                        of the findings
  --required            it does (the default for a new setting)
  --cited VALUE         the value the cited source (a paper, its evaluation
                        code) used; a difference makes cited criteria not
                        comparable
  --cited-source CITED_SOURCE
                        where the cited value comes from
  --from FILE           record settings from a YAML/JSON/TOML config, each by
                        key path, with --keys
  --keys KEYS           with --from: the keys that define the experiment, glob
                        patterns, e.g. "optim.*,model.depth"
  --note NOTE
  --no-verify           record it without checking the source now
  --amend               the experiment is frozen: change it anyway (with
                        --why); the amendment is recorded
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

### rb spec verify

```text
usage: rb spec verify [-h] [--json] [--dry-run] experiment [names ...]

positional arguments:
  experiment
  names       settings (default: every setting with a source)

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --dry-run   run every check the write makes and print what it would write,
              writing nothing; it exits as the write would
```

### rb spec vary

```text
usage: rb spec vary [-h] [--json] [--dry-run] [--amend] [-m WHY]
                    [--words TEXT]
                    experiment names [names ...]

positional arguments:
  experiment
  names              setting names (or comma-separated)

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  --amend            the experiment is frozen: change it anyway (with --why);
                     the amendment is recorded
  -m WHY, --why WHY  the reason, recorded with the call
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

## rb criterion

add|file|relate: a rule over one comparison, declared before the evidence it reads

```text
usage: rb criterion [-h] <add|file|relate> ...

positional arguments:
  <add|file|relate>
add              declare a criterion; declare it before the evidence that
                     tests it
file             declare which question a criterion bears on and what
                     meeting it would mean
relate           declare, with its reason, that a comparison reverses
                     another and how their arms align; rb reads whether it
                     does

options:
  -h, --help         show this help message and exit
```

### rb criterion add

```text
usage: rb criterion add [-h] [--json] [--dry-run] -e EXPERIMENT [-q QUESTION]
                        [--when-met WHEN_MET] --metric METRIC
                        [--subject SUBJECT] [--reference REFERENCE]
                        [--estimand ESTIMAND] [--where NAME=VALUE]
                        [--subject-where NAME=VALUE]
                        [--reference-where NAME=VALUE] [--at-most AT_MOST]
                        [--at-least AT_LEAST] [--equals TARGET]
                        [--tolerance TOLERANCE] [--better]
                        [--noninferior MARGIN] [--equivalent MARGIN] [--each]
                        [--bound] [--count K] [--best] [--of N] [--relative]
                        [--means]
                        [--role {primary,secondary,exploratory,gate}]
                        [--min-n MIN_N] [--noise NOISE] [--source SOURCE]
                        [--quote QUOTE] [--term TERM] [--commit COMMIT]
                        [--locator LOCATOR] [--note NOTE] [--amend] [-m WHY]
                        [--words TEXT] [--id ID]
                        statement

positional arguments:
  statement

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  -e EXPERIMENT, --experiment EXPERIMENT
                        the experiment whose runs test it
  -q QUESTION, --question QUESTION
                        the question this criterion bears on (repeatable)
  --when-met WHEN_MET   your interpretation for these questions when the rule
                        is met, verbatim
  --metric METRIC       what is measured: val_loss, int8.val_loss,
                        candidate.val_loss, or change.val_loss (the candidate
                        against the baseline); with --subject/--reference, the
                        bare metric
  --subject SUBJECT     the arm it is about: a variant, a role, or an arm an
                        analysis names (default: the only candidate)
  --reference REFERENCE
                        the arm it is compared against; without one it reads a
                        level
  --estimand ESTIMAND   what is estimated, when an analysis reports it that
                        way: pooled, per_class (default: rb's reading of the
                        runs, a mean over units)
  --where NAME=VALUE    select exact per-run setting values; repeat to require
                        all (where)
  --subject-where NAME=VALUE
                        select exact per-run setting values; repeat to require
                        all (subject-where)
  --reference-where NAME=VALUE
                        select exact per-run setting values; repeat to require
                        all (reference-where)
  --at-most AT_MOST
  --at-least AT_LEAST
  --equals TARGET, --within TARGET
                        with --tolerance: holds when |observed - TARGET| <=
                        tolerance
  --tolerance TOLERANCE
  --better              the subject is better than the reference: the interval
                        clear of 0 on the better side
  --noninferior MARGIN  the subject is no worse than the reference by more
                        than MARGIN, read on the interval
  --equivalent MARGIN   the difference lies inside ±MARGIN, read on the
                        interval
  --each                a threshold every independent unit must meet, not only
                        the estimate
  --bound               a threshold on the interval's bound (the upper for
                        --at-most, the lower for --at-least)
  --count K             a threshold at least K independent units must meet
                        ("19 of 20 runs below 0.890"; with --of N)
  --best                a threshold the best unit must meet, best in the
                        threshold's direction ("best of 5 runs reaches 0.889";
                        with --of N)
  --of N                with --count or --best: how many units the rule is of;
                        runs that hold another number leave it undetermined
  --relative            read the relative change, (subject - reference) /
                        reference, its threshold a percent ("improves val_bpb
                        by 22.5%": --at-most -22.5)
  --means               with --source: the cited number is a difference of the
                        arms' means over all their units, and rb reads that
  --role {primary,secondary,exploratory,gate}
                        primary (what the work claims), secondary,
                        exploratory, or gate (a priority policy: a miss parks
                        a line, it never refutes it)
  --min-n MIN_N         independent units it takes (default 1)
  --noise NOISE         run-to-run spread: an estimate this close to the rule
                        is said to be within the noise
  --source SOURCE       where the cited number (a paper's table) comes from:
                        file#key (YAML/JSON/TOML), file:LINE,
                        run:PATH#/pointer, https://..., or note:text
  --quote QUOTE         the exact text that states the number, or the ordering
                        a threshold of 0 cites; a file source must state it or
                        the criterion is refused
  --term TERM           the word the source uses for it on a prose line, e.g.
                        "learning rate"
  --commit COMMIT       read the file at this git commit instead of the
                        working tree
  --locator LOCATOR     how a reader finds it, e.g. "§4.2" or "Table 3"
  --note NOTE
  --amend               the experiment is frozen: add it anyway (with --why);
                        the amendment is recorded
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
  --id ID
```

### rb criterion file

```text
usage: rb criterion file [-h] [--json] [--dry-run] [--when-met WHEN_MET]
                         [--remove] [--amend] -m WHY [--words TEXT]
                         criterion question

positional arguments:
  criterion
  question

options:
  -h, --help           show this help message and exit
  --json               print one JSON object (the envelope) instead of text
  --dry-run            run every check the write makes and print what it would
                       write, writing nothing; it exits as the write would
  --when-met WHEN_MET  your interpretation when the rule is met, verbatim;
                       omitted keeps the current words
  --remove             retract this question declaration, keeping its history
  --amend              change the frozen declaration with the reason recorded
  -m WHY, --why WHY    the reason, recorded with the call
  --words TEXT         the user's words from the chat that this acts on,
                       verbatim (an agent quoting them); kept beside the
                       reason
```

### rb criterion relate

```text
usage: rb criterion relate [-h] [--json] [--dry-run]
                           [--orientation {same,opposite}] [--remove]
                           [--amend] -m WHY [--words TEXT]
                           criterion {reverses} target

positional arguments:
  criterion
  {reverses}
  target

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --orientation {same,opposite}
                        same: corresponding subject/reference order; opposite:
                        swap the target's order
  --remove              withdraw the relation, keeping its attribution
  --amend               change a frozen relation with the reason recorded
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

## rb run

add: what ran: its results, per-run settings, inputs and outputs, with a receipt

```text
usage: rb run [-h] <add> ...

positional arguments:
  <add>
add       record a run: numbers from any evaluator, or a file

options:
  -h, --help  show this help message and exit
```

### rb run add

```text
usage: rb run add [-h] [--json] [--dry-run] [--variant VARIANT] [--from FILE]
                  [--keys KEYS] [--table TABLE] [--where COLUMN=VALUE]
                  [--every-row] [--select COLUMN] [--config CONFIG]
                  [--set NAME=VALUE] [--input RESOURCE[:ROLE]]
                  [--output RESOURCE[:ROLE]] [--host NAME=VALUE]
                  [--commit COMMIT] [--tree TREE] [--artifact ARTIFACT]
                  [--link NAME=URL] [--command COMMAND_TEXT]
                  [--command-source FILE[:LINE]] [--again] [--replaces RUN]
                  [--repeats RUN] [--produced-at WHEN] [-m WHY] [--words TEXT]
                  [--note NOTE]
                  experiment [NAME=VALUE ...]

positional arguments:
  experiment
  NAME=VALUE            results, e.g. int8.val_loss=5.64 latency_ms=25.1

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --variant VARIANT     the variant these numbers are for: val_loss=5.64
                        becomes <variant>.val_loss
  --from FILE           a JSON/YAML/TOML object of metric names to numbers
                        (nested keys join with dots), a CSV, TSV or JSON Lines
                        table with --where, or a SQLite database with --table
  --keys KEYS           with --from: only these numbers, glob patterns, e.g.
                        "summary.*" (the file is still hashed whole)
  --table TABLE         with --from a SQLite database: the table that holds
                        the run's row (the database is read, never written)
  --where COLUMN=VALUE  with --table, or a CSV, TSV or JSON Lines table: the
                        row's columns and their values, e.g.
                        precision=int8,seed=1; exactly one row must have them
                        all
  --every-row           with --table, or a CSV, TSV or JSON Lines table:
                        record every row --where picks, each as its own run (a
                        SQLite row kept by its rowid), its @<column> values
                        from its own row
  --select COLUMN       with --table, or a CSV, TSV or JSON Lines table: only
                        these of the row's columns that name a declared metric
                        (default: all of them)
  --config CONFIG       the run's own resolved config (Hydra, W&B config.yaml,
                        hparams.yaml): checks it ran with the spec
  --set NAME=VALUE      a per-run setting this run used, e.g. seed=2; with
                        --table, seed=@seed takes the row's seed column, and
                        seed=@configs.seed (or @configs[config_id].seed) the
                        seed of the row of configs that table links
  --input RESOURCE[:ROLE]
                        a resource the run read, e.g. imagenet-val:eval
                        (repeatable)
  --output RESOURCE[:ROLE]
                        a resource the run made, e.g. int8-ckpt:checkpoint
                        (repeatable)
  --host NAME=VALUE     where it ran, as you say it: provider=runpod gpu=A100
                        (repeatable)
  --commit COMMIT       the commit that produced the numbers
  --tree TREE           the git tree of the exact code that produced them,
                        when it had uncommitted changes (rb journal run keeps
                        it; rb reconcile passes it)
  --artifact ARTIFACT, --file ARTIFACT
                        an output whose hash goes into the run (repeatable)
  --link NAME=URL       where else it lives, e.g. wandb=https://...
                        (repeatable)
  --command COMMAND_TEXT
                        the command that produced the numbers (recorded as
                        given; rb does not run it)
  --command-source FILE[:LINE]
                        the log line that records that command's launch, for a
                        run made before it was recorded; rb checks the line
                        holds it
  --again               record numbers identical to a recorded run's as a
                        second execution (with --why)
  --replaces RUN        this run replaces RUN, an invalid run of the same
                        experiment: RUN is retracted with --why and the two
                        are linked
  --repeats RUN         this run re-checks RUN: it joins that run's unit and
                        adds none
  --produced-at WHEN    when the output existed, when that is earlier than now
                        (ISO date or time), or unknown; with --table, @COLUMN
                        reads it from the row
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
  --note NOTE
```

## rb analysis

add: a computation over saved outputs, and the comparisons it produced with their intervals

```text
usage: rb analysis [-h] <add> ...

positional arguments:
  <add>
add       record an analysis: its comparisons, what it read, its method,
              and where its numbers are stated

options:
  -h, --help  show this help message and exit
```

### rb analysis add

```text
usage: rb analysis add [-h] [--json] [--dry-run]
                       [--compare SUBJECT/REFERENCE:METRIC[@ESTIMAND]=EST[LO,HI]]
                       [--from FILE] [--title TITLE] [--reads RUN|ANALYSIS]
                       [--input RESOURCE[:ROLE]] [--artifact ARTIFACT]
                       [--method METHOD] [--unit UNIT] [--units UNITS]
                       [--draws DRAWS] [--level LEVEL] [--source SOURCE]
                       [--quote QUOTE] [--commit COMMIT] [--locator LOCATOR]
                       [--supersedes ANALYSIS] [--produced-at WHEN]
                       [--command COMMAND_TEXT] [-m WHY] [--words TEXT]
                       [--note NOTE] [--id ID]
                       experiment

positional arguments:
  experiment

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --compare SUBJECT/REFERENCE:METRIC[@ESTIMAND]=EST[LO,HI]
                        one comparison it produced (repeatable), e.g.
                        new/old:score=0.12[0.07,0.17]
  --from FILE           its output file listing the comparisons
                        (JSON/YAML/TOML); hashed, and read as computed
  --title TITLE
  --reads RUN|ANALYSIS  runs or analyses it read (repeatable or comma-
                        separated)
  --input RESOURCE[:ROLE]
                        a resource it read, e.g. predictions:eval (repeatable)
  --artifact ARTIFACT, --file ARTIFACT
                        an output whose hash goes into the analysis
                        (repeatable)
  --method METHOD       how the intervals were made, e.g. "paired bootstrap
                        over scenes"
  --unit UNIT           the independent unit it resampled, e.g. "scene"
  --units UNITS         how many of them
  --draws DRAWS         bootstrap draws, when it resampled
  --level LEVEL         the intervals' level (default 0.95)
  --source SOURCE       the document that states the numbers (a README, a
                        report saved as text): rb finds the line that states
                        each
  --quote QUOTE         the exact row that states them
  --commit COMMIT       read the document at this git commit
  --locator LOCATOR     how a reader finds it, e.g. "Table 2"
  --supersedes ANALYSIS
                        an earlier analysis this one replaces (with --why)
  --produced-at WHEN    when it was computed, when earlier than now, or
                        unknown
  --command COMMAND_TEXT
                        the command that computed it (recorded as given; rb
                        does not run it)
  -m WHY, --why WHY     why it supersedes the earlier analysis
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
  --note NOTE
  --id ID
```

## rb resource

add|locate|expose: a dataset, checkpoint, code or artifact: what it is, where its bytes are, how to get it again, who has looked at it

```text
usage: rb resource [-h] <add|locate|expose> ...

positional arguments:
  <add|locate|expose>
add                record a resource: its identity, its locations, and how
                       to get it again
locate             record another place a resource's bytes are (or were)
expose             record that someone looked at a dataset: inspected it,
                       selected on it, evaluated on it, trained on it

options:
  -h, --help           show this help message and exit
```

### rb resource add

```text
usage: rb resource add [-h] [--json] [--dry-run] --type
                       {dataset,checkpoint,code,environment,artifact,other}
                       --name NAME [--version VERSION] [--subset SUBSET]
                       [--file FILE] [--sha256 SHA256] [--digest DIGEST]
                       [--at PROVIDER:REF] [--source SOURCE] [--url URL]
                       [--recipe RECIPE] [--note NOTE]
                       id

positional arguments:
  id                    a short id you choose, e.g. imagenet-val

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --type {dataset,checkpoint,code,environment,artifact,other}
  --name NAME           what it is, e.g. "ImageNet-1k validation"
  --version VERSION
  --subset SUBSET       which part, e.g. "training split, images 0..199"
  --file FILE           a local file: hashed now, and recorded as its first
                        location
  --sha256 SHA256       the sha256 of the file, when it is one file
  --digest DIGEST       any other fingerprint: a provider's digest, a dataset
                        hash
  --at PROVIDER:REF     where its bytes are (repeatable); providers: local,
                        url, git, wandb, mlflow, s3, gcs, hf, runpod, other
  --source SOURCE       where it comes from, in words
  --url URL             where to get it again
  --recipe RECIPE       the command or steps that rebuild it
  --note NOTE
```

### rb resource locate

```text
usage: rb resource locate [-h] [--json] [--dry-run] [--note NOTE]
                          id
                          {local,url,git,wandb,mlflow,s3,gcs,hf,runpod,other}
                          ref

positional arguments:
  id
  {local,url,git,wandb,mlflow,s3,gcs,hf,runpod,other}
  ref                   a path, a uri, an artifact name, a pod id

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --note NOTE
```

### rb resource expose

```text
usage: rb resource expose [-h] [--json] [--dry-run] [--at AT] [-m WHY]
                          [--words TEXT]
                          id {inspected,selected,evaluated,trained}

positional arguments:
  id
  {inspected,selected,evaluated,trained}

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  --at AT               when it happened, when earlier than now (ISO date or
                        time), or unknown; kept as reported
  -m WHY, --why WHY     what was looked at, and why
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
```

---

# Commands: Calls

https://rabbitbrain.ai/docs/reference/cli/calls/

These commands lock an experiment before its evidence, record a call, confirm a call or a design in your own terminal, and take anything back.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb freeze

lock an experiment's spec and criteria before the evidence, recorded with who locked it

```text
usage: rb freeze [-h] [--json] [--dry-run] [--amend] [-m WHY] [--words TEXT]
                 experiment

positional arguments:
  experiment

options:
  -h, --help         show this help message and exit
  --json             print one JSON object (the envelope) instead of text
  --dry-run          run every check the write makes and print what it would
                     write, writing nothing; it exits as the write would
  --amend            the experiment is frozen: adopt it as it is now (with
                     --why)
  -m WHY, --why WHY  the reason, recorded with the call
  --words TEXT       the user's words from the chat that this acts on,
                     verbatim (an agent quoting them); kept beside the reason
```

## rb decide

a call on a criterion, an experiment, a question or a setting: accept, reject, investigate; continue, park, stop, pivot, ship. Recorded with who made it and why; in an agent session it is proposed until the user confirms it

```text
usage: rb decide [-h] [--json] [--dry-run] -m WHY [--words TEXT]
                 [--revisit REVISIT] [--past] [--when TEXT]
                 [--source FILE[:LINE]] [--quote TEXT]
                 subject
                 {accept,reject,investigate,continue,park,stop,pivot,ship}

positional arguments:
  subject
  {accept,reject,investigate,continue,park,stop,pivot,ship}

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --dry-run             run every check the write makes and print what it
                        would write, writing nothing; it exits as the write
                        would
  -m WHY, --why WHY     the reason, recorded with the call
  --words TEXT          the user's words from the chat that this acts on,
                        verbatim (an agent quoting them); kept beside the
                        reason
  --revisit REVISIT     when to look again, e.g. "when the new images are in"
  --past                a call made before it was recorded: it reads
                        Retrospective, dated as --when says
  --when TEXT           with --past: when the call was made, as said (a date,
                        or words); unknown without it
  --source FILE[:LINE]  with --past and --quote: the document that states the
                        call; rb says what git shows of it, and a past call
                        from an agent session still waits for rb confirm
  --quote TEXT          with --source: the document's own words that state the
                        call, naming what it is on; rb checks the file holds
                        them
```

## rb confirm

make a call proposed, or a past call mentioned, in an agent session yours, or an experiment's design one wrote: run in your own terminal; with no id, list what waits

```text
usage: rb confirm [-h] [--json] [--dry-run] [--all] [ID ...]

positional arguments:
  ID          the calls to confirm, as rb status lists them, or
              <experiment>@<hash> for an experiment's design with its gate, as
              rb confirm lists it; an experiment's id alone lists its design

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --dry-run   run every check the write makes and print what it would write,
              writing nothing; it exits as the write would
  --all       confirm every call, gate and design that waits as it reads now,
              after saying each (the command rb confirm ends with confirms
              them as listed)
```

## rb retract

take anything back: it stays on record, stops counting, and rb says what rested on it

```text
usage: rb retract [-h] [--json] [--dry-run] [--scope MEASUREMENT] -m WHY
                  [--words TEXT]
                  subject

positional arguments:
  subject              an id (a note's too), or <experiment>/<setting>

options:
  -h, --help           show this help message and exit
  --json               print one JSON object (the envelope) instead of text
  --dry-run            run every check the write makes and print what it would
                       write, writing nothing; it exits as the write would
  --scope MEASUREMENT  runs only: supersede these measurements (e.g. --scope
                       latency_ms) and keep the rest counting
  -m WHY, --why WHY    the reason, recorded with the call
  --words TEXT         the user's words from the chat that this acts on,
                       verbatim (an agent quoting them); kept beside the
                       reason
```

---

# Commands: Reading it

https://rabbitbrain.ai/docs/reference/cli/reading/

These commands read the record: in your browser, in your terminal, as one briefing for an agent, or as one page to send.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb open

the catalogue in your browser, live and read-only, on 127.0.0.1 only

```text
usage: rb open [-h] [--json] [--port PORT]

options:
  -h, --help   show this help message and exit
  --json       print one JSON object (the envelope) instead of text
  --port PORT  port on 127.0.0.1 (default 8787)
```

## rb ui

open this project in your terminal: your call, the cards, the tree (bare rb does this too)

```text
usage: rb ui [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb status

each criterion on its axes, and what is open, with who usually acts

```text
usage: rb status [-h] [--json] [--fail-on GATE] [experiment]

positional arguments:
  experiment      only this experiment

options:
  -h, --help      show this help message and exit
  --json          print one JSON object (the envelope) instead of text
  --fail-on GATE  exit 1 when any of: unmet, unresolved, untested, integrity,
                  undecided, stale, unknown, late, checks, unestablished
                  (repeatable or comma-separated); an edit made outside rb
                  that a finding depends on fails any of them
```

## rb show

one record: a criterion with its finding, an experiment with its spec, a run with its receipt, e1/lr with its history; or a .yaml record file as canonical JSON

```text
usage: rb show [-h] [--json] id

positional arguments:
  id          an id, <experiment>/<setting>, or a path to an Rabbit Brain YAML
              file

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb compare

an experiment's own table: variants by metrics, with changes against the baseline

```text
usage: rb compare [-h] [--json] [experiment]

positional arguments:
  experiment

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb log

every write, newest last: who, how, what

```text
usage: rb log [-h] [--json] [-n N] [id]

positional arguments:
  id          only this record, or <experiment>/<setting>

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  -n N        how many (default 20)
```

## rb context

the record as one Markdown briefing a fresh agent session reads first

```text
usage: rb context [-h] [--json] [--model ID] [--provider NAME]

options:
  -h, --help       show this help message and exit
  --json           print one JSON object (the envelope) instead of text
  --model ID       an agent's own model, as its provider names it: kept with
                   every later write in this agent session (RB_MODEL gives it
                   for one command)
  --provider NAME  who provides that model, kept the same way (RB_PROVIDER for
                   one command); rb fills in Claude Code's and Codex's when
                   none is given
```

## rb page

write the project as one self-contained, read-only HTML file you can send anywhere

```text
usage: rb page [-h] [--json] [--out OUT] [--replace]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --out OUT   where to write it (default rb-page.html beside .rb/)
  --replace   write over a file at --out that rb did not write (default:
              refused, the file left as it is)
```

---

# Commands: Carrying it

https://rabbitbrain.ai/docs/reference/cli/carrying/

These commands take the record somewhere else: an export with a manifest, a bundle another repository clones, and a check of this machine against the runs.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb export

write the project's page with a hashed manifest, readable without rb; with --with-record, the record too, as a portable bundle

```text
usage: rb export [-h] [--json] [--out OUT] [--zip] [--with-record] [--url URL]
                 [--no-repo] [--replace]

options:
  -h, --help     show this help message and exit
  --json         print one JSON object (the envelope) instead of text
  --out OUT      where to write it (default rb-export/ beside .rb/)
  --zip          write one zip file instead of a directory (default rb-
                 export.zip)
  --with-record  carry the record too, for rb clone: its files and log keep
                 who made each write, agent sessions and the user's words
  --url URL      with --with-record, the public address this will be served
                 from: the page then opens with the exact rb clone command,
                 and a directory export also contains bundle.zip
  --no-repo      leave the code repository's address out of the manifest and
                 page
  --replace      with --zip, write over a file at --out that rb did not write
                 (default: refused, the file left as it is)
```

## rb clone

start your own line of work from an exported bundle: the structure arrives inherited, and nothing counts until reproduced here

```text
usage: rb clone [-h] [--json] [--code ADDRESS] source [dest]

positional arguments:
  source          a bundle rb export --with-record wrote: a directory, a .zip,
                  or its address
  dest            where to create it (default ./<project id>)

options:
  -h, --help      show this help message and exit
  --json          print one JSON object (the envelope) instead of text
  --code ADDRESS  where to get the code the runs need, in place of the
                  repository the bundle names (a copy you can reach, or one
                  the bundle does not name): the record lands in a checkout of
                  it at the bundle's commit
```

## rb env

the environment: check this machine against the record's runs and any env.* settings

```text
usage: rb env [-h] [--json] <command> ...

positional arguments:
  <command>
check     compare; differences are said, never fatal (exit 0)

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

### rb env check

```text
usage: rb env check [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

---

# Commands: For agents

https://rabbitbrain.ai/docs/reference/cli/for-agents/

These commands are what a coding agent uses: the MCP server, the skill, the journal, reconcile and compile.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb mcp

serve the record to an agent over MCP (stdio); every call is recorded as the agent's

```text
usage: rb mcp [-h] [--agent AGENT]

options:
  -h, --help     show this help message and exit
  --agent AGENT  the agent's name in the record (default: the MCP client's
                 name)
```

## rb skill

print the agent skill; `rb skill add` installs it for your coding agent

```text
usage: rb skill [-h] [--json] [--claude] [--project] [--cursor] [--agents-md]
                [--yes]
                [{add}]

positional arguments:
  {add}                 add: write it where the agent looks

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --claude              ~/.claude/skills/rabbit-brain/ (the default target)
  --project             with --claude: this project's .claude/skills/ instead
                        of the home directory
  --cursor              .cursor/rules/rabbit-brain.mdc in this project
  --agents-md, --codex  a marked section in this project's AGENTS.md, for
                        Codex and anything else that reads it
  --yes                 with --agents-md: add rb's section to an AGENTS.md
                        that is here without asking (default: asked once in
                        your terminal; elsewhere left as it is, with what to
                        add)
```

## rb journal

what rb observed in the repository: files that appeared, git moves, wrapped commands

```text
usage: rb journal [-h] [--json] [-n N] <scan|run> ...

positional arguments:
  <scan|run>
scan      look at the watched places once and journal what changed
run       run a command and journal what it did: rb journal run -- python
              eval.py ...

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  -n N        how many entries (default 20)
```

### rb journal scan

```text
usage: rb journal scan [-h] [--json] [--hook <agent>]

options:
  -h, --help      show this help message and exit
  --json          print one JSON object (the envelope) instead of text
  --hook <agent>  run from that agent's session hook (rb init installs it):
                  say nothing, exit 0, and keep the look as that agent's
```

### rb journal run

```text
usage: rb journal run [-h] [--json] ...

positional arguments:
  cmd         the command, after --

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb reconcile

compile the repository against the record: each results folder with a report as an experiment, what its files say (verbatim), its checks, how to reproduce it; --apply keeps it

```text
usage: rb reconcile [-h] [--json] [--apply] [--no-agents] [--yes]

options:
  -h, --help   show this help message and exit
  --json       print one JSON object (the envelope) instead of text
  --apply      keep it in the record: add and re-index the experiments, record
               the runs found, attributed to you
  --no-agents  with --apply where there is no record yet: leave the agents'
               instruction files alone (default: install rb's, as rb init
               does)
  --yes        with --apply where there is no record yet: add rb's part to an
               AGENTS.md or .mcp.json that is here without asking (default:
               asked once in your terminal; elsewhere left as they are)
```

## rb compile

list what existing reports and results files the record can place, and what it cannot; writes nothing

```text
usage: rb compile [-h] [--json] [paths ...]

positional arguments:
  paths       files or directories to compile from

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

---

# Commands: Upkeep

https://rabbitbrain.ai/docs/reference/cli/upkeep/

These commands keep the record sound: check it, write its files in canonical form, and rb's own settings and optional usage analytics.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb doctor

check the record: who you are recorded as, edits made outside rb, merge leftovers, sources outside the repository

```text
usage: rb doctor [-h] [--json] [--restore] [--adopt] [-m WHY] [--words TEXT]
                 [--seen PATH@SHA256]

options:
  -h, --help          show this help message and exit
  --json              print one JSON object (the envelope) instead of text
  --restore           put back what rb last wrote for every record changed
                      outside rb
  --adopt             take every change made outside rb as it stands (with
                      --why); edits to who wrote or decided what are refused
  -m WHY, --why WHY   the reason, recorded with --adopt
  --words TEXT        the user's words from the chat that this acts on,
                      verbatim (an agent quoting them); kept beside the reason
  --seen PATH@SHA256  with --adopt: the reviewed files, each pinned to the sha
                      it had (or 'deleted'); any drift since is refused
```

## rb fmt

write Rabbit Brain YAML files in their canonical form, comments kept (default: every record file under .rb/)

```text
usage: rb fmt [-h] [--json] [--check] [paths ...]

positional arguments:
  paths       files or directories (default: the record)

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --check     change nothing; exit 1 if any file is not canonical
```

## rb telemetry

optional usage analytics: off until you enable it; never sends research content

```text
usage: rb telemetry [-h] <status|enable|disable|preview|flush> ...

positional arguments:
  <status|enable|disable|preview|flush>
status              show the local analytics setting and destination
                        without sending anything
enable              opt in to limited usage analytics on this installation
disable             stop analytics and delete the local installation ID
                        and pending events
preview             show the queued payload without sending it
flush               try sending the pending usage events now

options:
  -h, --help            show this help message and exit
```

### rb telemetry status

```text
usage: rb telemetry status [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

### rb telemetry enable

```text
usage: rb telemetry enable [-h] [--json] [--project-key PROJECT_KEY]
                           [--host HOST]

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --project-key PROJECT_KEY
                        PostHog public project capture key; never a personal
                        API key
  --host HOST           PostHog HTTPS ingestion host (must match the project's
                        region)
```

### rb telemetry disable

```text
usage: rb telemetry disable [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

### rb telemetry preview

```text
usage: rb telemetry preview [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

### rb telemetry flush

```text
usage: rb telemetry flush [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb config

rb's own settings on this machine: whether rb keeps itself up to date

```text
usage: rb config [-h] [--json] <auto-update> ...

positional arguments:
  <auto-update>
auto-update  whether rb checks for a newer rb once a day, and updates
                 itself when it was installed as a tool (uv tool install,
                 pipx); says the setting with no value

options:
  -h, --help     show this help message and exit
  --json         print one JSON object (the envelope) instead of text
```

### rb config auto-update

```text
usage: rb config auto-update [-h] [--json] [{on,off}]

Whether rb checks PyPI for a newer rb at most once a day: installed as a tool
(uv tool install, pipx) it updates itself in the background, and installed
with pip it says one line and changes nothing. Never in CI, in an agent
session or under DO_NOT_TRACK. With no value it says the setting.

positional arguments:
  {on,off}    on or off

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

---

# Commands: Leaving

https://rabbitbrain.ai/docs/reference/cli/leaving/

This command takes rb out of a checkout, and only what checks out as rb's.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb remove

take rb out of this checkout: only what checks out as rb's (the record, its instructions for agents, its pages and exports, its refs and folders in git), after showing the plan

```text
usage: rb remove [-h] [--json] [--dry-run] [--yes]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --dry-run   show the plan (what rb would remove, what it would keep and
              why), removing nothing
  --yes       remove without asking (default: asked once in your terminal;
              refused where there is none)
```

---

# Commands: Reference

https://rabbitbrain.ai/docs/reference/cli/reference/

These commands are the tour, the manual, the record's format, the version and the feedback link.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb tour

a one-minute walkthrough: builds a small real project in ./rb-tour and runs the whole loop

```text
usage: rb tour [-h] [--json] [dir]

positional arguments:
  dir         where the tour lands (default ./rb-tour; must be new or empty)

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb docs

print AGENTS.md (--start: how to start the record in a project already under way; --errors: the error table)

```text
usage: rb docs [-h] [--json] [--errors] [--start]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --errors
  --start     print the start protocol: how an agent records work already done
```

## rb schema

print a JSON Schema: `rb schema all` is Rabbit Brain formats 1 and 2 (rb-v2.schema.json), `rb schema &lt;kind&gt;` one record's shape

```text
usage: rb schema [-h] [--json]
                 {all,project,question,experiment,metric,criterion,run,analysis,resource,decision,finding,envelope,ir}

positional arguments:
  {all,project,question,experiment,metric,criterion,run,analysis,resource,decision,finding,envelope,ir}

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
```

## rb version

print the version

```text
usage: rb version [-h] [--json]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
```

## rb feedback

a link to send feedback on rb: the page shows what it would send before you send it; rb sends nothing

```text
usage: rb feedback [-h] [--json] [--about WORD [WORD ...]]

Prints a link to send feedback on rb: the page shows what it would send before
you send it; rb sends nothing. In your own terminal it opens the page in your
browser; anywhere else it only prints the link.

options:
  -h, --help            show this help message and exit
  --json                print one JSON object (the envelope) instead of text
  --about WORD [WORD ...]
                        the rb command it is about, by its words (--about run
                        add); its arguments are never sent
```

---

# Commands: rabbitbrain.ai

https://rabbitbrain.ai/docs/reference/cli/rabbitbrain-ai/

These commands sign a machine in to an account at rabbitbrain.ai, keep a private copy of a record's page there, and revoke the machine's credential. Anyone can make an account, at rabbitbrain.ai/signin.

Generated from the `--help` of rabbit-brain 0.8.3.

## rb login

rabbitbrain.ai, optional: sign this machine in with a short code, approved on any device where you are signed in

```text
usage: rb login [-h] [--json] [--hub HUB]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --hub HUB   the hub's address (default RB_HUB_URL, else rabbitbrain.ai)
```

## rb logout

revoke this machine's hub credential and remove it from the credentials file

```text
usage: rb logout [-h] [--json] [--hub HUB]

options:
  -h, --help  show this help message and exit
  --json      print one JSON object (the envelope) instead of text
  --hub HUB   the hub's address (default RB_HUB_URL, else rabbitbrain.ai)
```

## rb sync

rabbitbrain.ai, optional: a private copy of this record's page in your account, for your own eyes from anywhere

```text
usage: rb sync [-h] [--json] [--hub HUB] [--with-record] [--watch]

options:
  -h, --help     show this help message and exit
  --json         print one JSON object (the envelope) instead of text
  --hub HUB      the hub's address (default RB_HUB_URL, else rabbitbrain.ai)
  --with-record  send the record too: its files and log keep who made each
                 write, agent sessions and the user's words
  --watch        keep the hosted copy current: look every 30 s and sync again
                 when the record changed, at most once a minute, until Ctrl-C
```

---

# The record's format

https://rabbitbrain.ai/docs/reference/schema/

This page describes the format of the files in `.rb/`, where its JSON Schema lives, and rb's vocabulary of record kinds.

Every file in `.rb/` opens with its format, `rb: 1` or `rb: 2`, and then its `kind`, so a file travels and validates on its own. Both formats are one JSON Schema (draft 2020-12), served at its `$id`, [rabbitbrain.ai/schema/rb-v2.schema.json](https://rabbitbrain.ai/schema/rb-v2.schema.json), and kept in the repository at [schema/rb-v2.schema.json](https://github.com/rabbit-brain/rb/blob/main/schema/rb-v2.schema.json). Format 1's own schema, [rb-v1.schema.json](https://rabbitbrain.ai/schema/rb-v1.schema.json), stays as it was.

From the command line:

```sh
# the whole format, for any JSON Schema validator
rb schema all > rb-v2.schema.json
# one kind: project, question, experiment, metric, criterion, run, analysis, resource or decision
rb schema run
# a finding's shape: computed on every read, never stored
rb schema finding
# what every command prints with --json
rb schema envelope
# read one record file and check it against the format
rb show .rb/experiments/int8-ptq/runs/<id>.yaml
```

## Rabbit Brain YAML

The files are YAML kept to a strict subset, so every YAML parser reads a record the same way: mappings, lists, text, numbers, `true`, `false`, `null` and comments, and nothing else. A plain value that YAML 1.1 and YAML 1.2 read differently is refused, with the spelling to write instead: `yes`, `no`, `on`, `off`, dates, numbers with leading zeros, and `1e-4` without a dot. `rb fmt` rewrites files in their canonical form, comments kept.

A comment or a reformat is not an edit: rb compares a record's content, never its text, so you can annotate the files.

## Format 1 and format 2

Format 1 is fixed: a later rb that adds a field writes that file in format 2, and an older rb says to upgrade (`E_STATE_NEWER`) rather than read part of it. `rb show` of such a file says a newer rb wrote it, as `rb status`, `rb doctor` and every write do.

Format 2 adds four kinds of rule to a criterion and nothing else: a count (`over: count`, with `count` and `of`), a best of N (`over: best`, with `of`), a relative change (`measure: relative`) and a cited difference of means (`measure: means`). [Criteria](https://rabbitbrain.ai/docs/concepts/criteria/#four-more-kinds-of-rule) says how each reads.

- rb 0.8.3 writes a criterion's file in format 2 only while it holds one of the four, and writes every other file in format 1, as rb 0.7 writes it. It reads both.
- A file that says `rb: 1` and holds one of the four is refused (`E_STATE_CORRUPT`): format 1 never held one.
- A file that says `rb: 2` and holds none of the four (its first line raised by hand) is no valid record either (`E_STATE_CORRUPT`). `rb doctor` names it with its fix (set its first line back to `rb: 1`, or `rb doctor --restore`), every gate fails closed, and rb writes nothing over it until it says `rb: 1` again. rb 0.8.0 said nothing.
- A retracted rule keeps the rule it was, so its file stays in format 2.
- `rb export --with-record` writes `record_format: 2` in the bundle's manifest when a criterion's file is in format 2, else 1.

rb 0.8.0, 0.8.1 and 0.8.2 read the files 0.8.3 writes as they read their own.

rb 0.7.0 and 0.7.1 read format 1 only. They read a record with no format-2 file, and write to it as before. Where a file is format 2, `rb doctor` names it ("is format 2; this rb reads format 1: it was written by a newer rb"), `rb status` counts such files, and both say to upgrade (`pip install --upgrade rabbit-brain`). They read every other file and fail every `--fail-on` gate. They write nothing to the record, and refuse to clone a bundle that holds such a file. Upgrade everyone who shares a record together.

## The vocabulary

`rb schema ir` prints every kind of record, what it means, the command that writes it, and the source rb requires:

```rb-session frame="terminal"
$ rb schema ir
rb's vocabulary: every record kind, what it means, the command that writes it, and the sources rb requires.

question     what the work is trying to find out
             a tree: a sub-question names its parent, and an experiment may be filed under several; where it stands is read from what is below it, never a child's negative as its own
             write: rb question add "<what are you trying to find out?>" --parent <q>
             provenance: wording as asked
experiment   what is tested
             its purpose or hypothesis, one baseline and variants with roles, varies naming the intended differences; planned until its first run; an undeclared difference between variants is a confound
             write: rb experiment add "<title>" --hypothesis "<h>" --baseline <b> --candidate <c> --varies <s>
             provenance: declared, then checked against its settings
variant      one arm of an experiment
             baseline, candidate, control or ablation; its own settings override the shared ones
             write: rb variant add <exp> <name> --role <r>
             provenance: role declared
setting      what the experiment changes or depends on
             a value with where it comes from; population.* declares the counting structure and never blocks
             write: rb spec set <exp> <name> <value> --source file#key
             provenance: REQUIRED: a source rb can read, or --unknown, or a vouch (rb decide <exp>/<name> accept)
criterion    what would count on one comparison
             a rule over one comparison (subject, reference, metric, estimand), with a role (primary, secondary, exploratory, gate), declared by someone at a time before the evidence it reads; explicit question links may keep the author's verbatim interpretation when met, never an answer inferred by rb
             write: rb criterion add "<c>" -e <exp> -q <question> --metric <m> --subject <a> --reference <b> --better
             provenance: declared by whoever wrote it, with the user's words when an agent gives them; question links snapshot the question and retain author, time, reason, session and revisions; REQUIRED for cited numbers: --source --quote --locator, checked against the file
run          one execution, and what it produced
             results with a receipt: actor, chat session, the agent (its name, version, provider and model), time, repository state, command, environment, the resources read and made, when its output existed; the same output recorded twice is one run; a repeat checks a run and adds no unit; a wrong run is retracted (whole, or --scope for named measurements), never deleted
             write: rb run add <exp> --from <file> --set seed=N --input <resource> --config <resolved>
             provenance: REQUIRED: the receipt is taken when it is recorded; --from and --config are read and hashed; typed numbers are marked typed
analysis     a computation over saved outputs
             the comparisons it produced, each with its estimate and interval, the unit it resampled, and where it came from; a reanalysis is an analysis, never a run; a later one supersedes it with a reason, and its readings stay as history
             write: rb analysis add <exp> --compare "a/b:m=est[lo,hi]" --method "<m>" --unit "<u>" --units N --source <file>
             provenance: REQUIRED: computed from an output file rb hashed, or reported from a document rb checked, or marked typed
comparison   subject against reference on a metric
             the unit of state: an estimate, its interval when there is one, what it resampled; changing the metric, the reference or the estimand is a different comparison
             write: read by a criterion; produced by an analysis or by rb from the runs
             provenance: an analysis, or rb's reading of the runs
result       a number a run reports
             named <variant>.<metric>, change.<metric> (candidate minus baseline) or plain; compared across variants in rb compare
             write: NAME=VALUE on rb run add, or --from <file>
             provenance: the run's receipt
resource     a dataset, checkpoint, code, environment or artifact
             identity (name, version, hash), locations (any provider), and how to get it again; availability is computed: present, changed, remote, reconstructible or lost; a run records what it found when it read or made one
             write: rb resource add <id> --type dataset --name "<n>" --url <u>
             provenance: declared; a local file is hashed when recorded
metric       what is measured, by name
             unit, direction, aliases; a result naming an alias is read through it
             write: rb metric add <m> --unit <u> --minimize
             provenance: declared
finding      a criterion read on axes, computed
             result, rule, timing, units, integrity, review and history, computed from the runs and analyses on every read, never stored; no single word stands for it
             write: read it: rb show <criterion>
             provenance: derived
decision     a call, with what it was made on
             accept, reject or investigate a finding, an experiment, a question or a setting; continue, park, stop, pivot or ship a line of work; read against the readings it was made on, with when to revisit; one recorded in an agent session is proposed until the user confirms it (rb confirm), and a past one (--past) is Retrospective
             write: rb decide <subject> <action> -m "..." --revisit "..."
             provenance: REQUIRED: who made it and why, with the chat session when known and the user's words verbatim when an agent acts on them; for a past call, when it was made and the document that states it
note         words about an experiment or the project
             an assumption, a caveat, context; kept verbatim with who wrote it; changes no finding
             write: rb note add "<text>" -e <exp>
             provenance: text as written
relation     how one experiment stands to another
             reproduces, extends, follows_from or compares_with an experiment here, a uid, or a published manifest; structure, not evidence
             write: rb experiment relate <exp> reproduces <target>
             provenance: declared
```

---

# llms.txt

https://rabbitbrain.ai/docs/agents/llms-txt/

This page tells a coding agent where rb's documentation is, in the forms an agent reads best.

If you are an agent working in a repository with a `.rb/` folder, start with `rb context --model <your model id>`, not with these pages: it is the state of the work, and it replaces anyone's summary of it.

## These docs, as text

| Address | What it is |
| --- | --- |
| [/docs/llms.txt](https://rabbitbrain.ai/docs/llms.txt) | An index of every page of these docs, with a one-line description each. |
| [/docs/llms-full.txt](https://rabbitbrain.ai/docs/llms-full.txt) | Every page of these docs but the Changelog, in one plain-text Markdown file. |

[rabbitbrain.ai/llms.txt](https://rabbitbrain.ai/llms.txt) is rb's own summary for agents: what the record is, the rules an agent follows, and what it must pass on to the user.

## Built into rb

The manual ships inside every rb, so it always matches the installed version:

```sh
rb docs            # the whole manual: every rule, every command, the JSON envelope, every error and its fix
rb docs --start    # the protocol for starting the record in a project already under way
rb docs --errors   # the error table
rb skill           # the skill rb installs for coding agents
```

Over MCP (`rb mcp`), the same manual is the resource `rb://docs`, beside `rb://start`, `rb://context`, `rb://status`, `rb://schema` and `rb://format`.

## What an agent must not do

- Compute a finding, an error, a ranking or a regression itself: record the numbers, and read the finding with `rb show`.
- Edit `.rb/` by hand.
- Record a call the user did not ask for, or run `rb confirm`, which only the user runs, in their own terminal.
- Set or unset `RB_ACTOR` or `RB_HUB_TOKEN`, or read the credentials file `rb login` writes. See [Auth](https://rabbitbrain.ai/docs/agents/auth/).

---

# Auth

https://rabbitbrain.ai/docs/agents/auth/

This page says which commands use an account, how rb signs a machine in, where the credential lives, and what an agent must leave alone.

Only three commands use an account: `rb login`, `rb sync`, and `rb logout`, which undoes the first. Everything else in rb, recording, `rb status`, `rb show`, `rb page` and `rb export`, needs no account and no network.

## rabbitbrain.ai

Anyone can make an account at [rabbitbrain.ai/signin](https://rabbitbrain.ai/signin): an email address, a code sent to it in place of a password, and a handle. On a machine not signed in yet, `rb sync` says so and sends nothing:

```rb-session frame="terminal"
$ rb sync
rb sync: No credential for rabbitbrain.ai on this machine.
  → Run rb login once per machine, or set RB_HUB_TOKEN for CI.
```

## How rb calls a hub

rb calls rabbitbrain.ai, or another hub named with `--hub <address>` or `RB_HUB_URL`, as follows.

`rb login` signs this machine in with a device flow:

1. The terminal shows a short code, and where to approve it:

   ```rb-session frame="terminal"
   $ rb login

     First, this terminal gets a code:

         PJWM-24RF

     Open https://rabbitbrain.ai/link?code=PJWM-24RF on any device; it signs you in first if needed, and approve the code.
   ```

2. The person approves it at the hub's `/link` page, on any device where they are signed in. Since the person approves it, the credential is theirs, whoever started `rb login`. While rb waits for the approval, it asks again when a check's connection drops or times out, three in a row at most; after that it says the hub could not be reached and stores nothing.
3. The token arrives once. rb stores it with mode 0600 in `~/.config/rabbit-brain/credentials.json`, or under `$RB_CONFIG_DIR`: outside the repository and outside the environment.

`rb logout` revokes the credential on the hub and removes it from this machine. For CI, `RB_HUB_TOKEN` overrides the file. With no credential stored, `rb logout` changes nothing:

```rb-session frame="terminal"
$ rb logout
Nothing stored for <https://rabbitbrain.ai>; nothing to do.
```

`rb sync` exports the record's page as `rb export` writes it, and uploads it as a private copy in the account. `rb sync --with-record` sends the record too, which keeps who made each write and the words the user gave, and rb says so. `rb sync --watch` looks every 30 seconds and syncs again when the record or its page changed, at most once a minute, until Ctrl-C.

A hub's refusal comes back as `E_HUB_REFUSED`, with its reason. A hub rb cannot reach is `E_HUB_UNREACHABLE`. Either way, nothing on this machine changed.

## For an agent

An agent can run `rb login` for the user: the user approves the code in their own browser. Beyond that:

- Never read the credentials file, and never print, copy or commit a token.
- Never set or unset `RB_HUB_TOKEN` or `RB_ACTOR`.
- Keep tokens out of chat, logs, commits and files.
- Plain `rb sync` sends the record's page; `rb sync --with-record` sends the record too. Send the record only when the user asks for it.

---