rb.INT8 ResNet-20 on CIFAR-10Rerun this research2026-10-29 23:49 UTC

INT8 ResNet-20 on CIFAR-10

23 recorded runs · changed

Open question

Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?

No answer yet: the fp32 baseline doesn't match its cited number, and 8 of 13 rules are not settled.

Rules: 4 met · 1 not reproduced · 5 can't tell yet · 3 not tested

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

How far to trust it

9 of 13 rules were written after their results came in.

None of the 23 runs has a command from when it ran.

Treat them as a record of what happened, not as a test planned in advance.

Needs judgmentNeeds your judgmentStill open

1 call · 2 designs waiting for a confirmation1 call · 2 designs waiting for your confirmation

Open one to see why it waits and decide it.

Not decided yet. Open one to see why it waits.

Also open: 15 items that need no decision

See the open calls →

See your calls →

Waiting for a confirmationWaiting for your confirmation

Experiment designs not confirmed (2)

The design of 2 experiments is not confirmed: rb confirm lists them.

The design of 2 experiments is not confirmed.

Research questions

Kept with rb since 20 Oct 2026. All 23 runs and 1 call were recorded afterwards, not as they happened.

How rb reads this repository

rb looks in out/, outputs/, results/, eval-out/ and metrics/, and at the files in the project's top folder. It has not looked yet. It reads what code, configs, docs and database rows say only where the record points to them, and it sees only the commands run through rb journal run.

Recent changes

Latest changes: 2026-10-25, 01:08 UTC · 1 change · no calls made

Recent calls and changes

2026-10-25, 01:08 UTC · 1 change · no calls made

  1. 01:08
    Recorded a run on e-ptq: run fp32 and int8 · seed 1 · scheme per_channel · calib images 128 · calib seed 0

Every change, newest first →

Notes on the project (1)
  • note on the project (2026-10-20): Recipe: python sweep.py configs/sweep.yaml --jobs 4 --threads 2, then python summarize.py (README.md · lines 42-43). FP32 baselines 90.22 to 90.64 (results/r014, r016, r017 metrics.json) sit below the 91.25 reference of He et al. 2016; notes.md · line 5 keeps 91.25 a reference, not a target. Older runs left out of the record: r001 and r013 smoke tests, r002/r003 10-epoch pilot. · not confirmed

Reproduce this project

Everything on this page was computed from the record this bundle carries. Rerun that computation, and the runs behind it, on your machine:

Rerun this research

Research

Research

Each question has its own answer, evidence and open questions.

Open question

Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?

No answer yet: the fp32 baseline doesn't match its cited number, and 8 of 13 rules are not settled.

Rules: 4 met · 1 not reproduced · 5 can't tell yet · 3 not tested

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

Open question

How many calibration images does INT8 need?

Measured differences; uncertainty remains

Rules: 4 met · 1 can't tell yet · 3 not tested

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

Open question

Do per-channel weight scales matter, against one scale per tensor?

Evidence includes point estimates only

Rules: 5 can't tell yet · 3 not tested

Research›Does INT8 post-training quantization… q-int8

Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?

Open question

No answer yet: the fp32 baseline doesn't match its cited number, and 8 of 13 rules are not settled.

Rules: 4 met · 1 not reproduced · 5 can't tell yet · 3 not tested

Its rules (13)

9 of 13 rules were written after their results came in

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07), 3 seeds. c-pt-512

    details of c-pt-512

    INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07) · unresolved

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-512 · primary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2264 inside 1top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor]: -0.07667 [-0.2264, +0.07307] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: 1024 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07), 3 seeds. c-pt-1024

    details of c-pt-1024

    INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07) · loss

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-1024 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2737 inside 1top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor]: -0.1733 [-0.2737, -0.07294] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: 128 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04), 3 seeds. c-pt-128

    details of c-pt-128

    INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04) · loss

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-128 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2681 inside 1top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor]: -0.1533 [-0.2681, -0.0386] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 and 8 more)

A note, not confirmed: 2048 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09), 3 seeds. c-pt-2048

    details of c-pt-2048

    INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09) · unresolved

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-2048 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.3141 inside 1top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor]: -0.1133 [-0.3141, +0.08746] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Rb can't tell yet whether its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

Rules no note speaks to

Open callsYour call (1)

Not reproduced · estimate 90.48 outside 91.25 ± 0.005FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10 c-ref

Calls are made with rb in a terminal, never on this page.

Your calls are made with rb in a terminal; this page gives you the command.

Calls are made with rb in a terminal, not on this page; it shows where they stand.

Experiments behind this question

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs
Experiment · open · 15 runs
Rerun recipe: 0 of 4 parts ready
Per-channel against per-tensor weights, seed 0, [0.5] normalization
Experiment · open · 8 runs
Rerun recipe: 0 of 4 parts ready

Next step

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

Sub-question

Open question

How many calibration images does INT8 need?

Measured differences; uncertainty remains

Rules: 4 met · 1 can't tell yet · 3 not tested

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

Open question

Do per-channel weight scales matter, against one scale per tensor?

Evidence includes point estimates only

Rules: 5 can't tell yet · 3 not tested

Which comparisons carry these qualifications?

no interval

one estimate with no interval, so how much it could vary is unknown. rb gives one from 3 units, and none across fixed design cells.

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rule written after the results

the rule or its question link was declared after the evidence existed or after the data was seen: never read as a test planned in advance.

  • c-pt-1024 Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-128 Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-ref FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10
  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

Uncertainty remains

unresolved: the interval includes 0.

  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
By reading

not tested · 3

  • c-pc-1024 Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-2048 Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-512 Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

can't tell yet · 5

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rule met · 4

  • c-pt-1024 Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-128 Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

not reproduced · 1

  • c-ref FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10

Research›Does INT8 post-training quantization… q-int8›How many calibration images does INT8 need? q-calib

How many calibration images does INT8 need?

Open question

Measured differences; uncertainty remains

Rules: 4 met · 1 can't tell yet · 3 not tested

Its rules (8)

4 of 8 rules were written after their results came in

A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07), 3 seeds. c-pt-512

    details of c-pt-512

    INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07) · unresolved

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-512 · primary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2264 inside 1top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor]: -0.07667 [-0.2264, +0.07307] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: 1024 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07), 3 seeds. c-pt-1024

    details of c-pt-1024

    INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07) · loss

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-1024 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2737 inside 1top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor]: -0.1733 [-0.2737, -0.07294] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: 128 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04), 3 seeds. c-pt-128

    details of c-pt-128

    INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04) · loss

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-128 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.2681 inside 1top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor]: -0.1533 [-0.2681, -0.0386] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 and 8 more)

A note, not confirmed: 2048 calibration images (per-tensor) are enough to stay within 1 point of FP32. Its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

  • Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

    INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09), 3 seeds. c-pt-2048

    details of c-pt-2048

    INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09) · unresolved

    3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

    rule
    c-pt-2048 · secondary · on e-ptq
    as declared
    rule met: the interval's worse bound -0.3141 inside 1top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
    reading
    top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor]: -0.1133 [-0.3141, +0.08746] % · paired difference, t interval, 3 seeds, 95% (rb)
    units
    3 seeds · declared: unit = seed
    timing
    declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

A note, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Rb can't tell yet whether its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

Calls made

  • accepted 2026-10-07 (a past call from notes.md:5, "Decision: from now on every FP32 baseline trains for 30 epochs, not the paper's 64k iterations (about 182 epochs).", Retrospective) · note: notes.md · line 5 e-ptq/train.epochs

Experiments behind this question

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs
Experiment · open · 15 runs
Rerun recipe: 0 of 4 parts ready
Per-channel against per-tensor weights, seed 0, [0.5] normalization
Experiment · open · 8 runs
Rerun recipe: 0 of 4 parts ready

Next step

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

Which comparisons carry these qualifications?

no interval

one estimate with no interval, so how much it could vary is unknown. rb gives one from 3 units, and none across fixed design cells.

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rule written after the results

the rule or its question link was declared after the evidence existed or after the data was seen: never read as a test planned in advance.

  • c-pt-1024 Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-128 Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

Uncertainty remains

unresolved: the interval includes 0.

  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
By reading

not tested · 3

  • c-pc-1024 Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-2048 Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-512 Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

can't tell yet · 1

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rule met · 4

  • c-pt-1024 Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-128 Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-2048 Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pt-512 Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

Research›Does INT8 post-training quantization… q-int8›Do per-channel weight scales matter… q-scheme

Do per-channel weight scales matter, against one scale per tensor?

Open question

Evidence includes point estimates only

Rules: 5 can't tell yet · 3 not tested

Its rules (8)

4 of 8 rules were written after their results came in

A note, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Rb can't tell yet whether its rule is met.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

A note, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines. Its rule has no results yet.

when and why

declared 2026-10-20 01:04 UTC · note: README.md:3 · not confirmed

Rules no note speaks to

Experiments behind this question

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs
Experiment · open · 15 runs
Rerun recipe: 0 of 4 parts ready
Per-channel against per-tensor weights, seed 0, [0.5] normalization
Experiment · open · 8 runs
Rerun recipe: 0 of 4 parts ready

Next step

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

Which comparisons carry these qualifications?

no interval

one estimate with no interval, so how much it could vary is unknown. rb gives one from 3 units, and none across fixed design cells.

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rule written after the results

the rule or its question link was declared after the evidence existed or after the data was seen: never read as a test planned in advance.

  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
By reading

not tested · 3

  • c-pc-1024 Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-2048 Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-pc-512 Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

can't tell yet · 5

  • c-pc-128 Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-1024 Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-128 Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-2048 Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32
  • c-s0-pc-512 Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

Research›Rules

Rules (13)

Rules: 4 met · 1 not reproduced · 5 can't tell yet · 3 not tested

9 of 13 rules were written after their results came in

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs

e-ptq9 rules
Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule metINT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04), 3 seeds. c-pt-128

INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04) · loss

rule
c-pt-128 · secondary · on e-ptq
as declared
rule met: the interval's worse bound -0.2681 inside 1top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor]: -0.1533 [-0.2681, -0.0386] % · paired difference, t interval, 3 seeds, 95% (rb)
units
3 seeds · declared: unit = seed
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 and 8 more)

3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32can't tell yetINT8 is 0.18 points lower than the FP32 baseline, 1 seed, no interval yet (rb gives one from 3). written before its results c-pc-128

INT8 is 0.18 points lower than the FP32 baseline · no interval

rule
c-pc-128 · secondary · on e-ptq
as declared
can't tell yet: a noninferior rule reads the interval: 1 unit: rb gives an interval from 3top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_channel]: -0.18 % · paired difference, 1 seed (rb)
units
1 seed · declared: unit = seed
timing
declared before its runs (by production times; its baseline's runs came first)

1 run · 14 runs not counted · its run recorded afterwards · a noninferior rule reads the interval: 1 unit: rb gives an interval from 3

Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule metINT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07), 3 seeds. c-pt-512

INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07) · unresolved

rule
c-pt-512 · primary · on e-ptq
as declared
rule met: the interval's worse bound -0.2264 inside 1top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor]: -0.07667 [-0.2264, +0.07307] % · paired difference, t interval, 3 seeds, 95% (rb)
units
3 seeds · declared: unit = seed
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not testedNo results yet. c-pc-512

no reading yet · not tested

rule
c-pc-512 · secondary · on e-ptq
as declared
criterion untestedtop1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_channel]: no reading yet
units
declared: unit = seed
timing
declared; no evidence recorded yet

all 15 of its runs recorded afterwards

Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule metINT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07), 3 seeds. c-pt-1024

INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07) · loss

rule
c-pt-1024 · secondary · on e-ptq
as declared
rule met: the interval's worse bound -0.2737 inside 1top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor]: -0.1733 [-0.2737, -0.07294] % · paired difference, t interval, 3 seeds, 95% (rb)
units
3 seeds · declared: unit = seed
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not testedNo results yet. c-pc-1024

no reading yet · not tested

rule
c-pc-1024 · secondary · on e-ptq
as declared
criterion untestedtop1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_channel]: no reading yet
units
declared: unit = seed
timing
declared; no evidence recorded yet

all 15 of its runs recorded afterwards

Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule metINT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09), 3 seeds. c-pt-2048

INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09) · unresolved

rule
c-pt-2048 · secondary · on e-ptq
as declared
rule met: the interval's worse bound -0.3141 inside 1top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor]: -0.1133 [-0.3141, +0.08746] % · paired difference, t interval, 3 seeds, 95% (rb)
units
3 seeds · declared: unit = seed
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)

3 runs · 12 runs not counted · all 3 of its runs recorded afterwards

Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not testedNo results yet. c-pc-2048

no reading yet · not tested

rule
c-pc-2048 · secondary · on e-ptq
as declared
criterion untestedtop1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_channel]: no reading yet
units
declared: unit = seed
timing
declared; no evidence recorded yet

all 15 of its runs recorded afterwards

FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10not reproducedFP32 baseline is 90.48% (89.92% to 91.04%), 3 seeds; the cited number is 91.25%. c-ref

FP32 baseline is 90.48% (89.92% to 91.04%); the cited number is 91.25% · measured

rule
c-ref · secondary · on e-ptq · cited
as declared
not reproduced: estimate 90.48 outside 91.25 ± 0.005fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor] = 91.25 ± 0.005rule = 91.25% ± 0.005 points · margin -0.76 points
reading
fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor]: 90.48 [89.92, 91.04] % · mean, t interval, 3 seeds, 95% (rb)
units
3 seeds · declared: unit = seed
timing
declared after the evidence existed: all 3 units were produced before it was declared
review
Open callsYour call

3 runs · all 3 of its runs recorded afterwards · estimate 90.48 outside 91.25 ± 0.005 Open calls

Per-channel against per-tensor weights, seed 0, [0.5] normalization

e-scheme-s04 rules
Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yetINT8 is 0.12 points lower than the FP32 baseline, 1 seed, no interval yet (rb gives one from 3). c-s0-pc-128

INT8 is 0.12 points lower than the FP32 baseline · no interval

rule
c-s0-pc-128 · secondary · on e-scheme-s0
as declared
can't tell yet: a noninferior rule reads the interval: 1 unit: rb gives an interval from 3top1: int8 vs fp32 [subject where calib_images=128, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=128, scheme=per_channel]: -0.12 % · paired difference, 1 seed (rb)
units
1 seed · declared: unit = seed
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)

1 run · 7 runs not counted · its run recorded afterwards · a noninferior rule reads the interval: 1 unit: rb gives an interval from 3

Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yetINT8 is 0.09 points lower than the FP32 baseline, 1 seed, no interval yet (rb gives one from 3). c-s0-pc-512

INT8 is 0.09 points lower than the FP32 baseline · no interval

rule
c-s0-pc-512 · secondary · on e-scheme-s0
as declared
can't tell yet: a noninferior rule reads the interval: 1 unit: rb gives an interval from 3top1: int8 vs fp32 [subject where calib_images=512, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=512, scheme=per_channel]: -0.09 % · paired difference, 1 seed (rb)
units
1 seed · declared: unit = seed
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)

1 run · 7 runs not counted · its run recorded afterwards · a noninferior rule reads the interval: 1 unit: rb gives an interval from 3

Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yetINT8 is 0.09 points lower than the FP32 baseline, 1 seed, no interval yet (rb gives one from 3). c-s0-pc-1024

INT8 is 0.09 points lower than the FP32 baseline · no interval

rule
c-s0-pc-1024 · secondary · on e-scheme-s0
as declared
can't tell yet: a noninferior rule reads the interval: 1 unit: rb gives an interval from 3top1: int8 vs fp32 [subject where calib_images=1024, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=1024, scheme=per_channel]: -0.09 % · paired difference, 1 seed (rb)
units
1 seed · declared: unit = seed
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)

1 run · 7 runs not counted · its run recorded afterwards · a noninferior rule reads the interval: 1 unit: rb gives an interval from 3

Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yetINT8 is 0.05 points lower than the FP32 baseline, 1 seed, no interval yet (rb gives one from 3). c-s0-pc-2048

INT8 is 0.05 points lower than the FP32 baseline · no interval

rule
c-s0-pc-2048 · secondary · on e-scheme-s0
as declared
can't tell yet: a noninferior rule reads the interval: 1 unit: rb gives an interval from 3top1: int8 vs fp32 [subject where calib_images=2048, scheme=per_channel] no worse than 1, its interval's worse bound inside 1rule no worse than 1 point
reading
top1: int8 vs fp32 [subject where calib_images=2048, scheme=per_channel]: -0.05 % · paired difference, 1 seed (rb)
units
1 seed · declared: unit = seed
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)

1 run · 7 runs not counted · its run recorded afterwards · a noninferior rule reads the interval: 1 unit: rb gives an interval from 3

Experiments

Experiments

Open an experiment for its comparison, settings and record.

Experiments›INT8 PTQ against FP32, CIFAR-10… e-ptq

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs

Experiment · open · 15 runs

INT8 against the FP32 baseline · varies precision

design not confirmed · list it to confirm: rb confirm e-ptq

The design is not confirmed yet.

hypothesis
INT8 post-training static quantization (fbgemm) keeps ResNet-20 within 1 point of its own FP32 top-1 on CIFAR-10
note
README.md · line 3 states the question with its 1-point margin · not confirmed

Comparison

Top-1 (%)

FP32 baselineINT8Difference
Per channel · Calib images: 128 · Calib seed: 090.22 · 1 run90.04 · 1 run0.18 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 128 · Calib seed: 090.48 · 3 seeds90.33 · 3 seeds0.15 points lower than FP32 (-0.27 to -0.04)
Per tensor · Calib images: 128 · Calib seed: 190.64 · 1 run90.49 · 1 run0.15 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 128 · Calib seed: 290.64 · 1 run90.61 · 1 run0.03 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 512 · Calib seed: 090.48 · 3 seeds90.40 · 3 seeds0.08 points lower than FP32 (-0.23 to +0.07)
Per tensor · Calib images: 1024 · Calib seed: 090.48 · 3 seeds90.31 · 3 seeds0.17 points lower than FP32 (-0.27 to -0.07)
Per tensor · Calib images: 2048 · Calib seed: 090.48 · 3 seeds90.37 · 3 seeds0.11 points lower than FP32 (-0.31 to +0.09)

Size (KB)

FP32 baselineINT8Difference
Per channel · Calib images: 128 · Calib seed: 01104.50 · 1 run327.00 · 1 run777.50 lower than FP32 · 1 seed, so no interval

Top-5

FP32 baselineINT8Difference
Per channel · Calib images: 128 · Calib seed: 099.75 · 1 run99.75 · 1 runlevel with FP32 · 1 seed, so no interval

Each row is measured under its own settings. Compare within a row, not across.

From 15 runs.

Runs (15)

SeedSchemeCalib imagesCalib seedTop-1 (%)Wall sTop-5Size (KB)ProducedRecordedrun
FP32INT8DifferenceFP32INT8DifferenceFP32INT8Difference
0per_tensor128090.6490.460.18 lower2026-10-142026-10-20recorded afterwards
1per_tensor128090.2290.040.18 lower2026-10-142026-10-20recorded afterwards
2per_tensor128090.5890.480.10 lower2026-10-142026-10-20recorded afterwards
0per_tensor512090.6490.570.07 lower2026-10-142026-10-20recorded afterwards
1per_tensor512090.2290.080.14 lower2026-10-142026-10-20recorded afterwards
2per_tensor512090.5890.560.02 lower2026-10-142026-10-20recorded afterwards
0per_tensor1024090.6490.430.21 lower2026-10-142026-10-20recorded afterwards
1per_tensor1024090.2290.040.18 lower2026-10-142026-10-20recorded afterwards
2per_tensor1024090.5890.450.13 lower2026-10-142026-10-20recorded afterwards
0per_tensor2048090.6490.480.16 lower2026-10-142026-10-20recorded afterwards
1per_tensor2048090.2290.200.02 lower2026-10-142026-10-20recorded afterwards
2per_tensor2048090.5890.420.16 lower2026-10-142026-10-20recorded afterwards
0per_tensor128190.6490.490.15 lower2026-10-152026-10-20recorded afterwards
0per_tensor128290.6490.610.03 lower2026-10-152026-10-20recorded afterwards
1per_channel128090.2290.040.18 lower11.8099.7599.75no difference1104.50327.00777.50 lower2026-10-212026-10-25recorded afterwards

Its rules (9)

5 of 9 rules were written after their results came in

Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

c-pt-128

Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32can't tell yet

written before its results c-pc-128

Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

c-pt-512

Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not tested

c-pc-512

Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

c-pt-1024

Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not tested

c-pc-1024

Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32rule met

c-pt-2048

Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32not tested

c-pc-2048

FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10not reproduced

c-ref

Reproduce

Rerun recipe: 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

Rerun this research with Rabbit Brain (rb). Its record: https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/

1. Install rb: pip install rabbit-brain
2. Clone the record into a new folder: rb clone https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/bundle.zip int8-resnet-20-on-cifar-10. This bundle holds the record alone: the code is its owner's to give, and the runs need it. Tell me so and stop: once I give you an address of the code, rb clone https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/bundle.zip <another new folder> --code <that address> gets it at the commit the record names and puts the record in it.
3. In that folder, run rb context and read it before anything else. Its "Rerun the parent" part names the code and whether it is here, each resource, such as a model or a dataset, with how to get it, and the first command for each experiment.
4. Get those resources. Run experiment e-ptq, “INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs”: its commands in the order rb context names them, each as rb journal run -- <command>; where rb context names a command that puts a run's code here, run that one first. After each, run rb reconcile and record the run it proposes with the rb run add line it prints.
5. Then run rb status, and tell me which of its rules (c-pc-1024, c-pc-128, c-pc-2048, c-pc-512, c-pt-1024, c-pt-128, c-pt-2048, c-pt-512, c-ref) reproduced and which did not, with their numbers. Do not change a rule. If something cannot run here, say what and why.
The whole recipe as one script
# Reproduce e-ptq: INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9, b008869efc · not in this repository: 2982f0fae9, b008869efc (get the code repository to rerun) · 1 of 15 runs was made before it was recorded and names no commit
# environment: unknown · the runs were made before they were recorded, in environments rb did not see
# checkpoint: partial · fp32-r014, fp32-r017, fp32-r016: reconstructible
# files: missing · none of the 29 files the runs named is here unchanged; not here: results/r018_int8_pt_c128_s0/quantize.log, results/r018_int8_pt_c128_s0/metrics.json, results/r019_int8_pt_c128_s1/quantize.log and 26 more

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# checkpoint · fp32-r017
python train.py --seed 1 --out results/r017_fp32_s1

# checkpoint · fp32-r016
python train.py --seed 2 --out results/r016_fp32_s2

# per run (15): seed · scheme · calib images · calib seed
#   fp32: 0 per_tensor 128 0 · 1 per_tensor 128 0 · 2 per_tensor 128 0 · 0 per_tensor 512 0 · 1 per_tensor 512 0 · 2 per_tensor 512 0 · 0 per_tensor 1024 0 · 1 per_tensor 1024 0 · 2 per_tensor 1024 0 · 0 per_tensor 2048 0 · 1 per_tensor 2048 0 · 2 per_tensor 2048 0 · 0 per_tensor 128 1 · 0 per_tensor 128 2 · 1 per_channel 128 0
#   int8: 0 per_tensor 128 0 · 1 per_tensor 128 0 · 2 per_tensor 128 0 · 0 per_tensor 512 0 · 1 per_tensor 512 0 · 2 per_tensor 512 0 · 0 per_tensor 1024 0 · 1 per_tensor 1024 0 · 2 per_tensor 1024 0 · 0 per_tensor 2048 0 · 1 per_tensor 2048 0 · 2 per_tensor 2048 0 · 0 per_tensor 128 1 · 0 per_tensor 128 2 · 1 per_channel 128 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command (as given) · recorded for 14 of 15 runs · 14 given later, not observed
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r018_int8_pt_c128_s0
#   fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0: command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r019_int8_pt_c128_s1
#   fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0: command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r020_int8_pt_c128_s2
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r021_int8_pt_c512_s0
#   fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0: command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r022_int8_pt_c512_s1
#   fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0: command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r023_int8_pt_c512_s2
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:38 UTC: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r024_int8_pt_c1024_s0
#   fp32 and int8 · seed 1 · scheme per_tensor · calib images 1024 · calib seed 0: command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r025_int8_pt_c1024_s1
#   fp32 and int8 · seed 2 · scheme per_tensor · calib images 1024 · calib seed 0: command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r026_int8_pt_c1024_s2
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:39 UTC: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r027_int8_pt_c2048_s0
#   fp32 and int8 · seed 1 · scheme per_tensor · calib images 2048 · calib seed 0: command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r028_int8_pt_c2048_s1
#   fp32 and int8 · seed 2 · scheme per_tensor · calib images 2048 · calib seed 0: command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r029_int8_pt_c2048_s2
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 1: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 1 --out results/r030_int8_pt_c128_s0_d1
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 2: command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 2 --out results/r031_int8_pt_c128_s0_d2

What it takes to run it again

  • codemissing · commit 2982f0fae9, b008869efc · not in this repository: 2982f0fae9, b008869efc (get the code repository to rerun) · 1 of 15 runs was made before it was recorded and names no commit
  • environmentunknown · the runs were made before they were recorded, in environments rb did not see
  • checkpointpartial · fp32-r014, fp32-r017, fp32-r016: reconstructible
  • filesmissing · none of the 29 files the runs named is here unchanged; not here: results/r018_int8_pt_c128_s0/quantize.log, results/r018_int8_pt_c128_s0/metrics.json, results/r019_int8_pt_c128_s1/quantize.log and 26 more

Recipe, from what the record holds

  1. code · missing: commit 2982f0fae9, b008869efc · not in this repository: 2982f0fae9, b008869efc (get the code repository to rerun) · 1 of 15 runs was made before it was recorded and names no commit
  2. checkpoint · fp32-r014
    python train.py --seed 0 --out results/r014_fp32_s0
  3. checkpoint · fp32-r017
    python train.py --seed 1 --out results/r017_fp32_s1
  4. checkpoint · fp32-r016
    python train.py --seed 2 --out results/r016_fp32_s2
  5. command (as given) · recorded for 14 of 15 runs · 14 given later, not observed
  6. per run · seed · scheme · calib_images · calib_seed
    fp320 per_tensor 128 0 · 1 per_tensor 128 0 · 2 per_tensor 128 0 · 0 per_tensor 512 0 · 1 per_tensor 512 0 · 2 per_tensor 512 0 · 0 per_tensor 1024 0 · 1 per_tensor 1024 0 · 2 per_tensor 1024 0 · 0 per_tensor 2048 0 · 1 per_tensor 2048 0 · 2 per_tensor 2048 0 · 0 per_tensor 128 1 · 0 per_tensor 128 2 · 1 per_channel 128 0int80 per_tensor 128 0 · 1 per_tensor 128 0 · 2 per_tensor 128 0 · 0 per_tensor 512 0 · 1 per_tensor 512 0 · 2 per_tensor 512 0 · 0 per_tensor 1024 0 · 1 per_tensor 1024 0 · 2 per_tensor 1024 0 · 0 per_tensor 2048 0 · 1 per_tensor 2048 0 · 2 per_tensor 2048 0 · 0 per_tensor 128 1 · 0 per_tensor 128 2 · 1 per_channel 128 0
  7. settings
    shareddata.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512fp32precision=fp32int8precision=int8
Commands as recorded (history, not the recipe)

Next step

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

"run one more top1 measurement on INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs, int8 at scheme=per_channel, int8 at calib_images=128, int8 at calib_seed=0, and record it in rb"

Source details · settings: 4 per-run · 3 provisional · 7 verified
shareddata.normalize.mean[0.4914, 0.4822, 0.4465]verifiedconfigs/data.yaml#normalize.mean
shareddata.normalize.std[0.247, 0.2435, 0.2616]verifiedconfigs/data.yaml#normalize.std
sharedtrain.epochs30verifiedconfigs/train.yaml#epochs
sharedquant.backendfbgemmverifiedconfigs/quant.yaml#backend
sharedquant.activation_observerhistogramverifiedconfigs/quant.yaml#activation_observer
sharedquant.calibration.batch_size32verifiedconfigs/quant.yaml#calibration.batch_size
sharedeval.batch_size512verifiedconfigs/eval.yaml#batch_size
fp32precisionfp32provisionalnote
int8precisionint8provisionalnote
notes (6)
  • note on e-ptq: Each quantize run evaluates the FP32 model it starts from and its INT8 model on the same 10,000 test images (quantize.py, README.md Quantize section), so fp32.top1 and int8.top1 in one metrics.json are paired by seed. · not confirmed
  • note on e-ptq: results/r015_fp32_s1 (seed 1, first try) has no metrics.json: runs.db gives status crashed, exit code -9. results/r017_fp32_s1 is its rerun (runs.db rerun_of r015_fp32_s1), and seed 1 quantize runs start from it. · not confirmed
  • note on e-ptq: Calibration draws 1 and 2 at 128 images, seed 0 (results/r030_int8_pt_c128_s0_d1, results/r031_int8_pt_c128_s0_d2) are recorded with calib_seed 1 and 2. Every rule here reads draw 0 only, so the spread across calibration draws is not read by any rule yet. · not confirmed
  • note on e-ptq: Every quantize run's config.yaml records git dirty: true at its commit, so the code that ran may differ from the commit named. · not confirmed
  • note on e-ptq: No file read states a threshold or a direction for comparing calibration sizes with each other (README.md, notes.md, logs/agent.log, configs/), so rb computes no comparison between sizes; the rules only say whether each size stays within 1 point of FP32. · not confirmed
  • note on e-ptq: Per-channel on these baselines has no runs yet. configs/sweep.yaml (uncommitted since 1e2147e) holds per_channel at 128, 512, 1024 and 2048 images for seeds 1 and 2 only, from results/r017_fp32_s1 and results/r016_fp32_s2 (planned at logs/agent.log · line 2793 (Not quoted on a shared page), config written at logs/agent.log · line 2811 (Not quoted on a shared page), not run). Seed 0 per-channel exists only from results/r004_fp32_s0, with [0.5] normalization (e-scheme-s0), so c-pc-* would read 2 seeds unless per-channel from results/r014_fp32_s0 is added. · not confirmed

Experiments›Per-channel against per-tensor weights… e-scheme-s0

Per-channel against per-tensor weights, seed 0, [0.5] normalization

Experiment · open · 8 runs

INT8 against the FP32 baseline · varies precision

design not confirmed · list it to confirm: rb confirm e-scheme-s0

The design is not confirmed yet.

why: not recorded

purpose
Earlier stage: calibration size sweep with per-tensor and per-channel weight scales, both quantized from results/r004_fp32_s0 (normalize mean and std 0.5, eval batch 256)
if it holds
add seeds for per-channel weights · not confirmed

Comparison

Top-1 (%)

FP32 baselineINT8Difference
Per channel · Calib images: 12890.39 · 1 run90.27 · 1 run0.12 points lower than FP32 · 1 seed, so no interval
Per channel · Calib images: 51290.39 · 1 run90.30 · 1 run0.09 points lower than FP32 · 1 seed, so no interval
Per channel · Calib images: 102490.39 · 1 run90.30 · 1 run0.09 points lower than FP32 · 1 seed, so no interval
Per channel · Calib images: 204890.39 · 1 run90.34 · 1 run0.05 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 12890.39 · 1 run90.30 · 1 run0.09 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 51290.39 · 1 run90.28 · 1 run0.11 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 102490.39 · 1 run90.30 · 1 run0.09 points lower than FP32 · 1 seed, so no interval
Per tensor · Calib images: 204890.39 · 1 run90.29 · 1 run0.10 points lower than FP32 · 1 seed, so no interval

Each row is measured under its own settings. Compare within a row, not across.

From 8 runs.

Runs (8)

Every run: seed 0, calib seed 0. Recorded 2026-10-20

SchemeCalib imagesTop-1 (%)Producedrun
FP32INT8Difference
per_tensor12890.3990.300.09 lower2026-10-08recorded afterwards
per_tensor51290.3990.280.11 lower2026-10-08recorded afterwards
per_tensor102490.3990.300.09 lower2026-10-08recorded afterwards
per_tensor204890.3990.290.10 lower2026-10-08recorded afterwards
per_channel12890.3990.270.12 lower2026-10-09recorded afterwards
per_channel51290.3990.300.09 lower2026-10-09recorded afterwards
per_channel102490.3990.300.09 lower2026-10-09recorded afterwards
per_channel204890.3990.340.05 lower2026-10-09recorded afterwards

Its rules (4)

All 4 rules were written after their results came in

Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yet

c-s0-pc-128

Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yet

c-s0-pc-512

Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yet

c-s0-pc-1024

Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32can't tell yet

c-s0-pc-2048

Reproduce

Rerun recipe: 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

Rerun this research with Rabbit Brain (rb). Its record: https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/

1. Install rb: pip install rabbit-brain
2. Clone the record into a new folder: rb clone https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/bundle.zip int8-resnet-20-on-cifar-10. This bundle holds the record alone: the code is its owner's to give, and the runs need it. Tell me so and stop: once I give you an address of the code, rb clone https://rabbitbrain.ai/u/sample/int8-resnet-20-on-cifar-10/bundle.zip <another new folder> --code <that address> gets it at the commit the record names and puts the record in it.
3. In that folder, run rb context and read it before anything else. Its "Rerun the parent" part names the code and whether it is here, each resource, such as a model or a dataset, with how to get it, and the first command for each experiment.
4. Get those resources. Run experiment e-scheme-s0, “Per-channel against per-tensor weights, seed 0, [0.5] normalization”: its commands in the order rb context names them, each as rb journal run -- <command>; where rb context names a command that puts a run's code here, run that one first. After each, run rb reconcile and record the run it proposes with the rb run add line it prints.
5. Then run rb status, and tell me which of its rules (c-s0-pc-1024, c-s0-pc-128, c-s0-pc-2048, c-s0-pc-512) reproduced and which did not, with their numbers. Do not change a rule. If something cannot run here, say what and why.
The whole recipe as one script
# Reproduce e-scheme-s0: Per-channel against per-tensor weights, seed 0, [0.5] normalization
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 1d79e5c4ec, defcbf0a63 · not in this repository: 1d79e5c4ec, defcbf0a63 (get the code repository to rerun)
# environment: unknown · the runs were made before they were recorded, in environments rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · none of the 16 files the runs named is here unchanged; not here: results/r005_int8_pt_c128_s0/quantize.log, results/r005_int8_pt_c128_s0/metrics.json, results/r006_int8_pt_c512_s0/quantize.log and 13 more

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run (8): seed · scheme · calib images · calib seed
#   fp32: 0 per_tensor 128 0 · 0 per_tensor 512 0 · 0 per_tensor 1024 0 · 0 per_tensor 2048 0 · 0 per_channel 128 0 · 0 per_channel 512 0 · 0 per_channel 1024 0 · 0 per_channel 2048 0
#   int8: 0 per_tensor 128 0 · 0 per_tensor 512 0 · 0 per_tensor 1024 0 · 0 per_tensor 2048 0 · 0 per_channel 128 0 · 0 per_channel 512 0 · 0 per_channel 1024 0 · 0 per_channel 2048 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command (as given) · recorded for all 8 runs · 8 given later, not observed
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r005_int8_pt_c128_s0
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r006_int8_pt_c512_s0
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r007_int8_pt_c1024_s0
#   fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:33 UTC: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r008_int8_pt_c2048_s0
#   fp32 and int8 · seed 0 · scheme per_channel · calib images 128 · calib seed 0: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 128 --calib-seed 0 --out results/r009_int8_pc_c128_s0
#   fp32 and int8 · seed 0 · scheme per_channel · calib images 512 · calib seed 0: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 512 --calib-seed 0 --out results/r010_int8_pc_c512_s0
#   fp32 and int8 · seed 0 · scheme per_channel · calib images 1024 · calib seed 0: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 1024 --calib-seed 0 --out results/r011_int8_pc_c1024_s0
#   fp32 and int8 · seed 0 · scheme per_channel · calib images 2048 · calib seed 0: command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 2048 --calib-seed 0 --out results/r012_int8_pc_c2048_s0

What it takes to run it again

  • codemissing · commit 1d79e5c4ec, defcbf0a63 · not in this repository: 1d79e5c4ec, defcbf0a63 (get the code repository to rerun)
  • environmentunknown · the runs were made before they were recorded, in environments rb did not see
  • checkpointpartial · fp32-r004: reconstructible
  • filesmissing · none of the 16 files the runs named is here unchanged; not here: results/r005_int8_pt_c128_s0/quantize.log, results/r005_int8_pt_c128_s0/metrics.json, results/r006_int8_pt_c512_s0/quantize.log and 13 more

Recipe, from what the record holds

  1. code · missing: commit 1d79e5c4ec, defcbf0a63 · not in this repository: 1d79e5c4ec, defcbf0a63 (get the code repository to rerun)
  2. checkpoint · fp32-r004
    python train.py see results/r004_fp32_s0/config.yaml
  3. command (as given) · recorded for all 8 runs · 8 given later, not observed
  4. per run · seed · scheme · calib_images · calib_seed
    fp320 per_tensor 128 0 · 0 per_tensor 512 0 · 0 per_tensor 1024 0 · 0 per_tensor 2048 0 · 0 per_channel 128 0 · 0 per_channel 512 0 · 0 per_channel 1024 0 · 0 per_channel 2048 0int80 per_tensor 128 0 · 0 per_tensor 512 0 · 0 per_tensor 1024 0 · 0 per_tensor 2048 0 · 0 per_channel 128 0 · 0 per_channel 512 0 · 0 per_channel 1024 0 · 0 per_channel 2048 0
  5. settings
    shareddata.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256fp32precision=fp32int8precision=int8
Commands as recorded (history, not the recipe)

Next step

"run one more top1 measurement on Per-channel against per-tensor weights, seed 0, [0.5] normalization, int8 vs fp32, int8 at scheme=per_channel, int8 at calib_images=128, and record it in rb"

"run one more top1 measurement on Per-channel against per-tensor weights, seed 0, [0.5] normalization, int8 vs fp32, int8 at scheme=per_channel, int8 at calib_images=128, and record it in rb"

Source details · settings: 4 per-run · 3 provisional · 7 verified
shareddata.normalize.mean[0.5, 0.5, 0.5]verifiedconfigs/data.yaml#normalize.mean
shareddata.normalize.std[0.5, 0.5, 0.5]verifiedconfigs/data.yaml#normalize.std
sharedtrain.epochs30verifiedconfigs/train.yaml#epochs
sharedquant.backendfbgemmverifiedconfigs/quant.yaml#backend
sharedquant.activation_observerhistogramverifiedconfigs/quant.yaml#activation_observer
sharedquant.calibration.batch_size32verifiedconfigs/quant.yaml#calibration.batch_size
sharedeval.batch_size256verifiedconfigs/eval.yaml#batch_size
fp32precisionfp32provisionalnote
int8precisionint8provisionalnote
notes (1)
  • note on e-scheme-s0: No file read states a threshold or direction for per-channel against per-tensor (README.md, notes.md, logs/agent.log); notes.md · line 7 says only 'If it looks different from per-tensor, we add seeds', so rb computes no per-channel against per-tensor comparison yet. FP32 here is results/r004_fp32_s0 (90.39), not the CIFAR-10-normalized baselines of e-ptq. · not confirmed

Record

Runs

Each run keeps when its output was produced and when it was recorded, the repository's state when it was recorded, and what it read and made.

23 runs

INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs

e-ptq15 runs
SeedSchemeCalib imagesCalib seedTop-1 (%)Wall sTop-5Size (KB)ProducedRecordedrun
FP32INT8DifferenceFP32INT8DifferenceFP32INT8Difference
0per_tensor128090.6490.460.18 lower2026-10-142026-10-20recorded afterwards
1per_tensor128090.2290.040.18 lower2026-10-142026-10-20recorded afterwards
2per_tensor128090.5890.480.10 lower2026-10-142026-10-20recorded afterwards
0per_tensor512090.6490.570.07 lower2026-10-142026-10-20recorded afterwards
1per_tensor512090.2290.080.14 lower2026-10-142026-10-20recorded afterwards
2per_tensor512090.5890.560.02 lower2026-10-142026-10-20recorded afterwards
0per_tensor1024090.6490.430.21 lower2026-10-142026-10-20recorded afterwards
1per_tensor1024090.2290.040.18 lower2026-10-142026-10-20recorded afterwards
2per_tensor1024090.5890.450.13 lower2026-10-142026-10-20recorded afterwards
0per_tensor2048090.6490.480.16 lower2026-10-142026-10-20recorded afterwards
1per_tensor2048090.2290.200.02 lower2026-10-142026-10-20recorded afterwards
2per_tensor2048090.5890.420.16 lower2026-10-142026-10-20recorded afterwards
0per_tensor128190.6490.490.15 lower2026-10-152026-10-20recorded afterwards
0per_tensor128290.6490.610.03 lower2026-10-152026-10-20recorded afterwards
1per_channel128090.2290.040.18 lower11.8099.7599.75no difference1104.50327.00777.50 lower2026-10-212026-10-25recorded afterwards

Per-channel against per-tensor weights, seed 0, [0.5] normalization

e-scheme-s08 runs

Every run: seed 0, calib seed 0. Recorded 2026-10-20

SchemeCalib imagesTop-1 (%)Producedrun
FP32INT8Difference
per_tensor12890.3990.300.09 lower2026-10-08recorded afterwards
per_tensor51290.3990.280.11 lower2026-10-08recorded afterwards
per_tensor102490.3990.300.09 lower2026-10-08recorded afterwards
per_tensor204890.3990.290.10 lower2026-10-08recorded afterwards
per_channel12890.3990.270.12 lower2026-10-09recorded afterwards
per_channel51290.3990.300.09 lower2026-10-09recorded afterwards
per_channel102490.3990.300.09 lower2026-10-09recorded afterwards
per_channel204890.3990.340.05 lower2026-10-09recorded afterwards

Resources

What the runs read and made, how each is identified, how to get it again, and what depends on it.

fp32-r004FP32 ResNet-20, results/r004_fp32_s0reconstructiblecheckpoint · checkpoint · read by 8 runs · 4 rules depend on ithow to get it again · python train.py see results/r004_fp32_s0/config.yaml
identity
sha256 41c34ba09dcc
where
local:results/r004_fp32_s0/model.pt · gone
read by
fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0 fp32 and int8 · seed 0 · scheme per_channel · calib images 128 · calib seed 0 fp32 and int8 · seed 0 · scheme per_channel · calib images 512 · calib seed 0 fp32 and int8 · seed 0 · scheme per_channel · calib images 1024 · calib seed 0 fp32 and int8 · seed 0 · scheme per_channel · calib images 2048 · calib seed 0
fp32-r014FP32 ResNet-20, results/r014_fp32_s0reconstructiblecheckpoint · checkpoint · read by 6 runs · 9 rules depend on ithow to get it again · python train.py --seed 0 --out results/r014_fp32_s0
identity
sha256 61a697a6b408
where
local:results/r014_fp32_s0/model.pt · gone
read by
fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0 fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 1 fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 2
fp32-r016FP32 ResNet-20, results/r016_fp32_s2reconstructiblecheckpoint · checkpoint · read by 4 runs · 9 rules depend on ithow to get it again · python train.py --seed 2 --out results/r016_fp32_s2
identity
sha256 49a88e9c8fe3
where
local:results/r016_fp32_s2/model.pt · gone
read by
fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 fp32 and int8 · seed 2 · scheme per_tensor · calib images 1024 · calib seed 0 fp32 and int8 · seed 2 · scheme per_tensor · calib images 2048 · calib seed 0
fp32-r017FP32 ResNet-20, results/r017_fp32_s1reconstructiblecheckpoint · checkpoint · read by 4 runs · 9 rules depend on ithow to get it again · python train.py --seed 1 --out results/r017_fp32_s1
identity
sha256 118b55ae518f
where
local:results/r017_fp32_s1/model.pt · gone
read by
fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0 fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0 fp32 and int8 · seed 1 · scheme per_tensor · calib images 1024 · calib seed 0 fp32 and int8 · seed 1 · scheme per_tensor · calib images 2048 · calib seed 0

History

Each call keeps its reason and its date. A call does not change a finding.

106 changes recorded

  1. 2026-10-25, 01:08 UTC · 1 change · no calls made

    Raw log

    1. 01:08
      Recorded a run on e-ptq: run fp32 and int8 · seed 1 · scheme per_channel · calib images 128 · calib seed 0

  2. 2026-10-20, 01:02 to 01:09 UTC · 105 changes · 1 call made

    Raw log

    1. 01:09
      Confirmed a call: accepted train.epochs of e-ptq · note: notes.md:5
    2. 01:08
      Added rule c-ref to e-ptq
    3. 01:06
      Added a note on the project
    4. 01:06
      Retracted a note on the project · note: README lines are 42-43, not 44-45 · not confirmed
    5. 01:06
      Added a note on e-ptq
    6. 01:06
      Retracted a note on e-ptq · note: cited logs/agent.log:2809; the plan is at logs/agent.log:2793 and 2811 · not confirmed
    7. 01:06
      Added a note on the project
    8. 01:06
      Added a note on e-scheme-s0
    9. 01:06
      Added 5 notes on e-ptq
    10. 01:05
      Mentioned a past call to accept train.epochs of e-ptq, dated 2026-10-07, from notes.md:5, not confirmed · note: notes.md:5 · not confirmed
    11. 01:05
      Recorded 22 runs: 8 on e-scheme-s0, 14 on e-ptq
    12. 01:05
      Added 4 resources: fp32-r004, fp32-r014, fp32-r016, fp32-r017
    13. 01:05
      Set the gate of e-scheme-s0 · note: notes.md:7 · not confirmed
    14. 01:05
      Added rule c-s0-pc-2048 to e-scheme-s0 · note: README.md:3 · not confirmed
    15. 01:04
      Added 2 rules to e-ptq: c-pt-2048, c-pc-2048 · note: README.md:3 · not confirmed
    16. 01:04
      Added rule c-s0-pc-1024 to e-scheme-s0 · note: README.md:3 · not confirmed
    17. 01:04
      Added 2 rules to e-ptq: c-pt-1024, c-pc-1024 · note: README.md:3 · not confirmed
    18. 01:04
      Added rule c-s0-pc-512 to e-scheme-s0 · note: README.md:3 · not confirmed
    19. 01:04
      Added 2 rules to e-ptq: c-pt-512, c-pc-512 · note: README.md:3 · not confirmed
    20. 01:04
      Added rule c-s0-pc-128 to e-scheme-s0 · note: README.md:3 · not confirmed
    21. 01:04
      Added 2 rules to e-ptq: c-pt-128, c-pc-128 · note: README.md:3 · not confirmed
    22. 01:04
      Set 5 settings on e-scheme-s0: population.unit, seed, scheme, calib_images, calib_seed
    23. 01:04
      Set 5 settings on e-ptq: population.unit, seed, scheme, calib_images, calib_seed
    24. 01:04
      Set 7 settings on e-scheme-s0, each stated by its source: data.normalize.mean, data.normalize.std, train.epochs, quant.backend, quant.activation_observer and 2 more
    25. 01:04
      Set 7 settings on e-ptq, each stated by its source: data.normalize.mean, data.normalize.std, train.epochs, quant.backend, quant.activation_observer and 2 more
    26. 01:04
      Changed int8.precision on e-scheme-s0: a new source
    27. 01:04
      Changed int8.precision on e-ptq: a new source
    28. 01:04
      Set 2 settings on e-scheme-s0: fp32.precision, int8.precision
    29. 01:04
      Set 2 settings on e-ptq: fp32.precision, int8.precision
    30. 01:04
      Added an experiment: Per-channel against per-tensor weights, seed 0, [0.5] normalization (e-scheme-s0)
    31. 01:04
      Added a note on e-ptq
    32. 01:04
      Added an experiment: INT8 PTQ against FP32, CIFAR-10 normalization, 30 epochs (e-ptq)
    33. 01:04
      Added metric top1
    34. 01:04
      Added 2 questions under q-int8: q-calib, q-scheme
    35. 01:04
      Added question q-int8
    36. 01:02
      Started the record: INT8 ResNet-20 on CIFAR-10

Raw log

  1. 2026-10-25 01:08 UTC record run fp32 and int8 · seed 1 · scheme per_channel · calib images 128 · calib seed 0 · e-ptq
  2. 2026-10-20 01:09 UTC decide a call to accept train.epochs of e-ptq
  3. 2026-10-20 01:08 UTC add criterion c-ref: fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor] = 91.25 ± 0.005 · e-ptq
  4. 2026-10-20 01:06 UTC note project int8-resnet-20-on-cifar-10
  5. 2026-10-20 01:06 UTC retract a note on the project: note: README lines are 42-43, not 44-45 · not confirmed
  6. 2026-10-20 01:06 UTC note experiment e-ptq
  7. 2026-10-20 01:06 UTC retract a note on e-ptq: note: cited logs/agent.log:2809; the plan is at logs/agent.log:2793 and 2811 · not confirmed
  8. 2026-10-20 01:06 UTC note project int8-resnet-20-on-cifar-10
  9. 2026-10-20 01:06 UTC note experiment e-scheme-s0
  10. 2026-10-20 01:06 UTC note experiment e-ptq
  11. 2026-10-20 01:06 UTC note experiment e-ptq
  12. 2026-10-20 01:06 UTC note experiment e-ptq
  13. 2026-10-20 01:06 UTC note experiment e-ptq
  14. 2026-10-20 01:06 UTC note experiment e-ptq
  15. 2026-10-20 01:05 UTC decide a past call to accept train.epochs of e-ptq mentioned, not confirmed
  16. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 2 · e-ptq
  17. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 1 · e-ptq
  18. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 2 · scheme per_tensor · calib images 2048 · calib seed 0 · e-ptq
  19. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 1 · scheme per_tensor · calib images 2048 · calib seed 0 · e-ptq
  20. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:39 UTC · e-ptq
  21. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 2 · scheme per_tensor · calib images 1024 · calib seed 0 · e-ptq
  22. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 1 · scheme per_tensor · calib images 1024 · calib seed 0 · e-ptq
  23. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:38 UTC · e-ptq
  24. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 · e-ptq
  25. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0 · e-ptq
  26. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC · e-ptq
  27. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 · e-ptq
  28. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0 · e-ptq
  29. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC · e-ptq
  30. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_channel · calib images 2048 · calib seed 0 · e-scheme-s0
  31. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_channel · calib images 1024 · calib seed 0 · e-scheme-s0
  32. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_channel · calib images 512 · calib seed 0 · e-scheme-s0
  33. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_channel · calib images 128 · calib seed 0 · e-scheme-s0
  34. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:33 UTC · e-scheme-s0
  35. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC · e-scheme-s0
  36. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC · e-scheme-s0
  37. 2026-10-20 01:05 UTC record run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC · e-scheme-s0
  38. 2026-10-20 01:05 UTC add resource fp32-r017
  39. 2026-10-20 01:05 UTC add resource fp32-r016
  40. 2026-10-20 01:05 UTC add resource fp32-r014
  41. 2026-10-20 01:05 UTC add resource fp32-r004
  42. 2026-10-20 01:05 UTC gate experiment e-scheme-s0: note: notes.md:7 · not confirmed
  43. 2026-10-20 01:05 UTC add criterion c-s0-pc-2048: note: README.md:3 · not confirmed · e-scheme-s0
  44. 2026-10-20 01:04 UTC add criterion c-pc-2048: note: README.md:3 · not confirmed · e-ptq
  45. 2026-10-20 01:04 UTC add criterion c-pt-2048: note: README.md:3 · not confirmed · e-ptq
  46. 2026-10-20 01:04 UTC add criterion c-s0-pc-1024: note: README.md:3 · not confirmed · e-scheme-s0
  47. 2026-10-20 01:04 UTC add criterion c-pc-1024: note: README.md:3 · not confirmed · e-ptq
  48. 2026-10-20 01:04 UTC add criterion c-pt-1024: note: README.md:3 · not confirmed · e-ptq
  49. 2026-10-20 01:04 UTC add criterion c-s0-pc-512: note: README.md:3 · not confirmed · e-scheme-s0
  50. 2026-10-20 01:04 UTC add criterion c-pc-512: note: README.md:3 · not confirmed · e-ptq
  51. 2026-10-20 01:04 UTC add criterion c-pt-512: note: README.md:3 · not confirmed · e-ptq
  52. 2026-10-20 01:04 UTC add criterion c-s0-pc-128: note: README.md:3 · not confirmed · e-scheme-s0
  53. 2026-10-20 01:04 UTC add criterion c-pc-128: note: README.md:3 · not confirmed · e-ptq
  54. 2026-10-20 01:04 UTC add criterion c-pt-128: note: README.md:3 · not confirmed · e-ptq
  55. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting calib_seed: new, unknown (per-run)
  56. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting calib_images: new, unknown (per-run)
  57. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting scheme: new, unknown (per-run)
  58. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting seed: new, unknown (per-run)
  59. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting population.unit: new, seed (optional)
  60. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting calib_seed: new, unknown (per-run)
  61. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting calib_images: new, unknown (per-run)
  62. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting scheme: new, unknown (per-run)
  63. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting seed: new, unknown (per-run)
  64. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting population.unit: new, seed (optional)
  65. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: eval.batch_size: checked as stated
  66. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting eval.batch_size: new, 256
  67. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: quant.calibration.batch_size: checked as stated
  68. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting quant.calibration.batch_size: new, 32
  69. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: quant.activation_observer: checked as stated
  70. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting quant.activation_observer: new, histogram
  71. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: quant.backend: checked as stated
  72. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting quant.backend: new, fbgemm
  73. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: train.epochs: checked as stated
  74. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting train.epochs: new, 30
  75. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: data.normalize.std: checked as stated
  76. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting data.normalize.std: new, [0.5, 0.5, 0.5]
  77. 2026-10-20 01:04 UTC spec check experiment e-scheme-s0: data.normalize.mean: checked as stated
  78. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting data.normalize.mean: new, [0.5, 0.5, 0.5]
  79. 2026-10-20 01:04 UTC spec check experiment e-ptq: eval.batch_size: checked as stated
  80. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting eval.batch_size: new, 512
  81. 2026-10-20 01:04 UTC spec check experiment e-ptq: quant.calibration.batch_size: checked as stated
  82. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting quant.calibration.batch_size: new, 32
  83. 2026-10-20 01:04 UTC spec check experiment e-ptq: quant.activation_observer: checked as stated
  84. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting quant.activation_observer: new, histogram
  85. 2026-10-20 01:04 UTC spec check experiment e-ptq: quant.backend: checked as stated
  86. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting quant.backend: new, fbgemm
  87. 2026-10-20 01:04 UTC spec check experiment e-ptq: train.epochs: checked as stated
  88. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting train.epochs: new, 30
  89. 2026-10-20 01:04 UTC spec check experiment e-ptq: data.normalize.std: checked as stated
  90. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting data.normalize.std: new, [0.247, 0.2435, 0.2616]
  91. 2026-10-20 01:04 UTC spec check experiment e-ptq: data.normalize.mean: checked as stated
  92. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting data.normalize.mean: new, [0.4914, 0.4822, 0.4465]
  93. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting int8.precision: source configs/quant.yaml#backend -> note: INT8 model from quantize.py; int8.top1 in each quantize run's metrics.json
  94. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting int8.precision: source configs/quant.yaml#backend -> note: INT8 model from quantize.py; int8.top1 in each quantize run's metrics.json
  95. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting int8.precision: new, int8
  96. 2026-10-20 01:04 UTC spec set experiment e-scheme-s0: setting fp32.precision: new, fp32
  97. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting int8.precision: new, int8
  98. 2026-10-20 01:04 UTC spec set experiment e-ptq: setting fp32.precision: new, fp32
  99. 2026-10-20 01:04 UTC add experiment e-scheme-s0
  100. 2026-10-20 01:04 UTC note experiment e-ptq
  101. 2026-10-20 01:04 UTC add experiment e-ptq
  102. 2026-10-20 01:04 UTC add metric top1
  103. 2026-10-20 01:04 UTC add question q-scheme · q-int8
  104. 2026-10-20 01:04 UTC add question q-calib · q-int8
  105. 2026-10-20 01:04 UTC add question q-int8
  106. 2026-10-20 01:02 UTC init project int8-resnet-20-on-cifar-10

Open callsYour call (1)

Calls are made with rb in a terminal, never on this page.

Your calls are made with rb in a terminal; this page gives you the command.

Calls are made with rb in a terminal, not on this page; it shows where they stand.

Not reproduced · estimate 90.48 outside 91.25 ± 0.005FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10 c-ref

Review this result

  • Accept
    rb decide c-ref accept --why '‹why›'
    The record will say: accepted ‹day›: ‹why›
  • Reject
    rb decide c-ref reject --why '‹why›'
    The record will say: rejected ‹day›: ‹why›
  • Investigate
    rb decide c-ref investigate --why '‹why›'
    The record will say: investigating since ‹day›: ‹why›

Waiting for a confirmation

2 designs wait for your confirmation, listed apart

Also open in the record (15) →

Research › q-int8 › e-ptq › c-pt-128

Per-tensor INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pt-128

Met: INT8 is 0.15 points lower than the FP32 baseline (-0.27 to -0.04); its worse end is 0.73 points past the -1 point it asks for.

What rb measured

-0.15 points (-0.27 to -0.04)

3 runs · 12 runs not counted

the rule: -1 point or more-10-0.15

shaded: what the rule accepts · dot: estimate · bar: interval

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
3 seeds · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 3 runs · 12 runs not counted

12 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceseed 0, per-tensor, calib images 128, calib seed 0-0.18 points
differenceseed 1, per-tensor, calib images 128, calib seed 0-0.18 points
differenceseed 2, per-tensor, calib images 128, calib seed 0-0.10 points
FP32seed 0, per-tensor, calib images 512, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 512, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 512, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 1024, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 1024, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 1024, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 2048, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 2048, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 2048, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 128, calib seed 1not counted90.64%
FP32seed 0, per-tensor, calib images 128, calib seed 2not counted90.64%
FP32seed 1, per-channel, calib images 128, calib seed 0not counted90.22%

What depends on this

  • A note, not confirmed: 128 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-int8
  • A note, not confirmed: 128 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-calib
Source details
rule
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1
rb reads
the interval's worse bound -0.2681 inside 1
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0 and 8 more)
comparison
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor]: -0.1533 [-0.2681, -0.0386] % · paired difference, t interval, 3 seeds, 95% (rb)
evidence
all 3 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_tensor]: int8 against fp32
method
paired difference, t interval · 95% interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pc-128

Per-channel INT8 at 128 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pc-128

Can't tell yet: INT8 is 0.18 points lower than the FP32 baseline (no interval yet); the rule reads the interval, which rb gives from 3 seeds.

What rb measured

-0.18 points

1 run · 14 runs not counted

the rule: -1 point or more-10-0.18

shaded: what the rule accepts · dot: estimate (no interval)

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written before its results.
units
1 seed · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 1 run · 14 runs not counted

10 runs are not counted: its unit has no int8 run, so the paired reading leaves it out.

4 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceseed 1, per-channel, calib images 128, calib seed 0-0.18 points
FP32seed 0, per-tensor, calib images 128, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 128, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 128, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 512, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 512, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 512, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 1024, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 1024, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 1024, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 2048, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 2048, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 2048, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 128, calib seed 1not counted90.64%
FP32seed 0, per-tensor, calib images 128, calib seed 2not counted90.64%

What depends on this

  • A note for when this rule is met, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-int8
  • A note for when this rule is met, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-calib
  • A note for when this rule is met, not confirmed: per-channel INT8 at 128 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-scheme
Source details
rule
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
rb reads
a noninferior rule reads the interval: 1 unit: rb gives an interval from 3
timing
declared before its runs (by production times; its baseline's runs came first)
comparison
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_channel]: -0.18 % · paired difference, 1 seed (rb)
evidence
its run recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=128, calib_seed=0, scheme=per_channel]: int8 against fp32
method
paired difference · no interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pt-512

Per-tensor INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

ruleprimary in e-ptqc-pt-512

Met: INT8 is 0.08 points lower than the FP32 baseline (-0.23 to +0.07); its worse end is 0.77 points past the -1 point it asks for.

What rb measured

-0.08 points (-0.23 to +0.07)

3 runs · 12 runs not counted

the rule: -1 point or more-10-0.08

shaded: what the rule accepts · dot: estimate · bar: interval

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
3 seeds · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 3 runs · 12 runs not counted

12 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceseed 0, per-tensor, calib images 512, calib seed 0-0.07 points
differenceseed 1, per-tensor, calib images 512, calib seed 0-0.14 points
differenceseed 2, per-tensor, calib images 512, calib seed 0-0.02 points
FP32seed 0, per-tensor, calib images 128, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 128, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 128, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 1024, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 1024, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 1024, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 2048, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 2048, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 2048, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 128, calib seed 1not counted90.64%
FP32seed 0, per-tensor, calib images 128, calib seed 2not counted90.64%
FP32seed 1, per-channel, calib images 128, calib seed 0not counted90.22%

What depends on this

  • A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-int8
  • A note, not confirmed: 512 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-calib
Source details
rule
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1
rb reads
the interval's worse bound -0.2264 inside 1
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)
comparison
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor]: -0.07667 [-0.2264, +0.07307] % · paired difference, t interval, 3 seeds, 95% (rb)
evidence
all 3 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · primary
metric
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_tensor]: int8 against fp32
method
paired difference, t interval · 95% interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pc-512

Per-channel INT8 at 512 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pc-512

Not tested: nothing is read for this rule yet.

What rb measured

no reading yet

The rule as written

passes when
The interval's worse end is at least -1 (INT8 against the FP32 baseline).
timing
No results yet.
units
declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on

Every run here is FP32.

armrunvalue
seed 0, per-tensor, calib images 128, calib seed 090.64
seed 1, per-tensor, calib images 128, calib seed 090.22
seed 2, per-tensor, calib images 128, calib seed 090.58
seed 0, per-tensor, calib images 512, calib seed 090.64
seed 1, per-tensor, calib images 512, calib seed 090.22
seed 2, per-tensor, calib images 512, calib seed 090.58
seed 0, per-tensor, calib images 1024, calib seed 090.64
seed 1, per-tensor, calib images 1024, calib seed 090.22
seed 2, per-tensor, calib images 1024, calib seed 090.58
seed 0, per-tensor, calib images 2048, calib seed 090.64
seed 1, per-tensor, calib images 2048, calib seed 090.22
seed 2, per-tensor, calib images 2048, calib seed 090.58
seed 0, per-tensor, calib images 128, calib seed 190.64
seed 0, per-tensor, calib images 128, calib seed 290.64
seed 1, per-channel, calib images 128, calib seed 090.22

What depends on this

  • A note for when this rule is met, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-int8
  • A note for when this rule is met, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-calib
  • A note for when this rule is met, not confirmed: per-channel INT8 at 512 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-scheme
Source details
rule
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
timing
declared; no evidence recorded yet
comparison
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_channel]: no reading yet
evidence
all 15 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=512, calib_seed=0, scheme=per_channel]: int8 against fp32
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pt-1024

Per-tensor INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pt-1024

Met: INT8 is 0.17 points lower than the FP32 baseline (-0.27 to -0.07); its worse end is 0.73 points past the -1 point it asks for.

What rb measured

-0.17 points (-0.27 to -0.07)

3 runs · 12 runs not counted

the rule: -1 point or more-10-0.17

shaded: what the rule accepts · dot: estimate · bar: interval

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
3 seeds · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 3 runs · 12 runs not counted

12 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceseed 0, per-tensor, calib images 1024, calib seed 0-0.21 points
differenceseed 1, per-tensor, calib images 1024, calib seed 0-0.18 points
differenceseed 2, per-tensor, calib images 1024, calib seed 0-0.13 points
FP32seed 0, per-tensor, calib images 128, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 128, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 128, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 512, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 512, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 512, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 2048, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 2048, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 2048, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 128, calib seed 1not counted90.64%
FP32seed 0, per-tensor, calib images 128, calib seed 2not counted90.64%
FP32seed 1, per-channel, calib images 128, calib seed 0not counted90.22%

What depends on this

  • A note, not confirmed: 1024 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-int8
  • A note, not confirmed: 1024 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-calib
Source details
rule
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1
rb reads
the interval's worse bound -0.2737 inside 1
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)
comparison
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor]: -0.1733 [-0.2737, -0.07294] % · paired difference, t interval, 3 seeds, 95% (rb)
evidence
all 3 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_tensor]: int8 against fp32
method
paired difference, t interval · 95% interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pc-1024

Per-channel INT8 at 1024 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pc-1024

Not tested: nothing is read for this rule yet.

What rb measured

no reading yet

The rule as written

passes when
The interval's worse end is at least -1 (INT8 against the FP32 baseline).
timing
No results yet.
units
declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on

Every run here is FP32.

armrunvalue
seed 0, per-tensor, calib images 128, calib seed 090.64
seed 1, per-tensor, calib images 128, calib seed 090.22
seed 2, per-tensor, calib images 128, calib seed 090.58
seed 0, per-tensor, calib images 512, calib seed 090.64
seed 1, per-tensor, calib images 512, calib seed 090.22
seed 2, per-tensor, calib images 512, calib seed 090.58
seed 0, per-tensor, calib images 1024, calib seed 090.64
seed 1, per-tensor, calib images 1024, calib seed 090.22
seed 2, per-tensor, calib images 1024, calib seed 090.58
seed 0, per-tensor, calib images 2048, calib seed 090.64
seed 1, per-tensor, calib images 2048, calib seed 090.22
seed 2, per-tensor, calib images 2048, calib seed 090.58
seed 0, per-tensor, calib images 128, calib seed 190.64
seed 0, per-tensor, calib images 128, calib seed 290.64
seed 1, per-channel, calib images 128, calib seed 090.22

What depends on this

  • A note for when this rule is met, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-int8
  • A note for when this rule is met, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-calib
  • A note for when this rule is met, not confirmed: per-channel INT8 at 1024 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-scheme
Source details
rule
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
timing
declared; no evidence recorded yet
comparison
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_channel]: no reading yet
evidence
all 15 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=1024, calib_seed=0, scheme=per_channel]: int8 against fp32
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pt-2048

Per-tensor INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pt-2048

Met: INT8 is 0.11 points lower than the FP32 baseline (-0.31 to +0.09); its worse end is 0.69 points past the -1 point it asks for.

What rb measured

-0.11 points (-0.31 to +0.09)

3 runs · 12 runs not counted

the rule: -1 point or more-10-0.11

shaded: what the rule accepts · dot: estimate · bar: interval

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
3 seeds · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 3 runs · 12 runs not counted

12 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceseed 0, per-tensor, calib images 2048, calib seed 0-0.16 points
differenceseed 1, per-tensor, calib images 2048, calib seed 0-0.02 points
differenceseed 2, per-tensor, calib images 2048, calib seed 0-0.16 points
FP32seed 0, per-tensor, calib images 128, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 128, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 128, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 512, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 512, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 512, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 1024, calib seed 0not counted90.64%
FP32seed 1, per-tensor, calib images 1024, calib seed 0not counted90.22%
FP32seed 2, per-tensor, calib images 1024, calib seed 0not counted90.58%
FP32seed 0, per-tensor, calib images 128, calib seed 1not counted90.64%
FP32seed 0, per-tensor, calib images 128, calib seed 2not counted90.64%
FP32seed 1, per-channel, calib images 128, calib seed 0not counted90.22%

What depends on this

  • A note, not confirmed: 2048 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-int8
  • A note, not confirmed: 2048 calibration images (per-tensor) are enough to stay within 1 point of FP32
    on question q-calib
Source details
rule
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor] no worse than 1, its interval's worse bound inside 1
rb reads
the interval's worse bound -0.3141 inside 1
timing
declared after the evidence existed: all 3 units were produced before it was declared, and so were 11 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC, run fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0, run fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0 and 8 more)
comparison
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor]: -0.1133 [-0.3141, +0.08746] % · paired difference, t interval, 3 seeds, 95% (rb)
evidence
all 3 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_tensor]: int8 against fp32
method
paired difference, t interval · 95% interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-pc-2048

Per-channel INT8 at 2048 calibration images (draw 0) is no more than 1 point of top-1 below its own FP32

rulesecondary in e-ptqc-pc-2048

Not tested: nothing is read for this rule yet.

What rb measured

no reading yet

The rule as written

passes when
The interval's worse end is at least -1 (INT8 against the FP32 baseline).
timing
No results yet.
units
declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on

Every run here is FP32.

armrunvalue
seed 0, per-tensor, calib images 128, calib seed 090.64
seed 1, per-tensor, calib images 128, calib seed 090.22
seed 2, per-tensor, calib images 128, calib seed 090.58
seed 0, per-tensor, calib images 512, calib seed 090.64
seed 1, per-tensor, calib images 512, calib seed 090.22
seed 2, per-tensor, calib images 512, calib seed 090.58
seed 0, per-tensor, calib images 1024, calib seed 090.64
seed 1, per-tensor, calib images 1024, calib seed 090.22
seed 2, per-tensor, calib images 1024, calib seed 090.58
seed 0, per-tensor, calib images 2048, calib seed 090.64
seed 1, per-tensor, calib images 2048, calib seed 090.22
seed 2, per-tensor, calib images 2048, calib seed 090.58
seed 0, per-tensor, calib images 128, calib seed 190.64
seed 0, per-tensor, calib images 128, calib seed 290.64
seed 1, per-channel, calib images 128, calib seed 090.22

What depends on this

  • A note for when this rule is met, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-int8
  • A note for when this rule is met, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-calib
  • A note for when this rule is met, not confirmed: per-channel INT8 at 2048 calibration images stays within 1 point of FP32 on the CIFAR-10-normalized baselines
    on question q-scheme
Source details
rule
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
timing
declared; no evidence recorded yet
comparison
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_channel]: no reading yet
evidence
all 15 of its runs recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-ptq · secondary
metric
top1: int8 vs fp32 [subject where calib_images=2048, calib_seed=0, scheme=per_channel]: int8 against fp32
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-calib · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-ptq › c-ref

FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10

rulesecondary in e-ptqcitedc-ref

Not reproduced: FP32 baseline: 90.48% (89.92% to 91.04%), not within 0.005% of the cited 91.25%.

What rb measured

90.48% (89.92% to 91.04%)

3 runs

the rule: 91.25% ± 0.005%91.2590.48

shaded: what the rule accepts · dot: estimate · bar: interval

The rule as written

passes when
The estimate is within 0.005% of 91.25%.
margin
-0.76 points
timing
Written after its results came in.
units
3 seeds · declared: unit = seed
checks
ok
review
No call made on it. · Open callsYour call

Runs it rests on 3 runs

Every run here: per-tensor, calib images 128, calib seed 0.

Every run here is FP32 baseline.

armrunvalue
seed 090.64%
seed 190.22%
seed 290.58%

What depends on this

  • question link · q-int8
    Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
Source details
rule
fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor] = 91.25 ± 0.005
rb reads
estimate 90.48 outside 91.25 ± 0.005
timing
declared after the evidence existed: all 3 units were produced before it was declared
comparison
fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor]: 90.48 [89.92, 91.04] % · mean, t interval, 3 seeds, 95% (rb)
evidence
all 3 of its runs recorded afterwards
declared
2026-10-20 01:08 UTC
experiment
e-ptq · secondary · cited
metric
fp32.top1 [where calib_images=128, calib_seed=0, scheme=per_tensor]
reference
91.25 ± 0.005
method
mean, t interval · 95% interval
source
README.md:10 · read at commit 1e2147e · sha256 54fe5e466749
question link
q-int8 · 2026-10-20 01:08 UTC
cited
README.md:10 · README.md:10ResNet-20 reaches 91.25% top-1 on CIFAR-10read at commit 1e2147ef3da5c08b635a5323db589ef98e71ddef · sha256 54fe5e466749f4dfb72fb0114d5b3ed3d5043294f890513d643ddd5e9a8226ce

Record a call

Still open

Nothing is recorded on it yet.

Its actions and their lines are on its call card.

Research › q-int8 › e-scheme-s0 › c-s0-pc-128

Per-channel INT8 at 128 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rulesecondary in e-scheme-s0c-s0-pc-128

Can't tell yet: INT8 is 0.12 points lower than the FP32 baseline (no interval yet); the rule reads the interval, which rb gives from 3 seeds.

What rb measured

-0.12 points

1 run · 7 runs not counted

the rule: -1 point or more-10-0.12

shaded: what the rule accepts · dot: estimate (no interval)

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
1 seed · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 1 run · 7 runs not counted

Every run here: seed 0, calib seed 0.

7 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceper-channel, calib images 128-0.12 points
FP32per-tensor, calib images 128not counted90.39%
FP32per-tensor, calib images 512not counted90.39%
FP32per-tensor, calib images 1024not counted90.39%
FP32per-tensor, calib images 2048not counted90.39%
FP32per-channel, calib images 512not counted90.39%
FP32per-channel, calib images 1024not counted90.39%
FP32per-channel, calib images 2048not counted90.39%

What depends on this

  • question link · q-int8
    Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
  • question link · q-scheme
    Do per-channel weight scales matter, against one scale per tensor?
Source details
rule
top1: int8 vs fp32 [subject where calib_images=128, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
rb reads
a noninferior rule reads the interval: 1 unit: rb gives an interval from 3
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)
comparison
top1: int8 vs fp32 [subject where calib_images=128, scheme=per_channel]: -0.12 % · paired difference, 1 seed (rb)
evidence
its run recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-scheme-s0 · secondary
metric
top1: int8 vs fp32 [subject where calib_images=128, scheme=per_channel]: int8 against fp32
method
paired difference · no interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-scheme-s0 › c-s0-pc-512

Per-channel INT8 at 512 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rulesecondary in e-scheme-s0c-s0-pc-512

Can't tell yet: INT8 is 0.09 points lower than the FP32 baseline (no interval yet); the rule reads the interval, which rb gives from 3 seeds.

What rb measured

-0.09 points

1 run · 7 runs not counted

the rule: -1 point or more-10-0.09

shaded: what the rule accepts · dot: estimate (no interval)

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
1 seed · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 1 run · 7 runs not counted

Every run here: seed 0, calib seed 0.

7 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceper-channel, calib images 512-0.09 points
FP32per-tensor, calib images 128not counted90.39%
FP32per-tensor, calib images 512not counted90.39%
FP32per-tensor, calib images 1024not counted90.39%
FP32per-tensor, calib images 2048not counted90.39%
FP32per-channel, calib images 128not counted90.39%
FP32per-channel, calib images 1024not counted90.39%
FP32per-channel, calib images 2048not counted90.39%

What depends on this

  • question link · q-int8
    Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
  • question link · q-scheme
    Do per-channel weight scales matter, against one scale per tensor?
Source details
rule
top1: int8 vs fp32 [subject where calib_images=512, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
rb reads
a noninferior rule reads the interval: 1 unit: rb gives an interval from 3
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)
comparison
top1: int8 vs fp32 [subject where calib_images=512, scheme=per_channel]: -0.09 % · paired difference, 1 seed (rb)
evidence
its run recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-scheme-s0 · secondary
metric
top1: int8 vs fp32 [subject where calib_images=512, scheme=per_channel]: int8 against fp32
method
paired difference · no interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-scheme-s0 › c-s0-pc-1024

Per-channel INT8 at 1024 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rulesecondary in e-scheme-s0c-s0-pc-1024

Can't tell yet: INT8 is 0.09 points lower than the FP32 baseline (no interval yet); the rule reads the interval, which rb gives from 3 seeds.

What rb measured

-0.09 points

1 run · 7 runs not counted

the rule: -1 point or more-10-0.09

shaded: what the rule accepts · dot: estimate (no interval)

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
1 seed · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 1 run · 7 runs not counted

Every run here: seed 0, calib seed 0.

7 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceper-channel, calib images 1024-0.09 points
FP32per-tensor, calib images 128not counted90.39%
FP32per-tensor, calib images 512not counted90.39%
FP32per-tensor, calib images 1024not counted90.39%
FP32per-tensor, calib images 2048not counted90.39%
FP32per-channel, calib images 128not counted90.39%
FP32per-channel, calib images 512not counted90.39%
FP32per-channel, calib images 2048not counted90.39%

What depends on this

  • question link · q-int8
    Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
  • question link · q-scheme
    Do per-channel weight scales matter, against one scale per tensor?
Source details
rule
top1: int8 vs fp32 [subject where calib_images=1024, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
rb reads
a noninferior rule reads the interval: 1 unit: rb gives an interval from 3
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)
comparison
top1: int8 vs fp32 [subject where calib_images=1024, scheme=per_channel]: -0.09 % · paired difference, 1 seed (rb)
evidence
its run recorded afterwards
declared
2026-10-20 01:04 UTC
experiment
e-scheme-s0 · secondary
metric
top1: int8 vs fp32 [subject where calib_images=1024, scheme=per_channel]: int8 against fp32
method
paired difference · no interval
question link
q-int8 · 2026-10-20 01:04 UTC
question link
q-scheme · 2026-10-20 01:04 UTC

Research › q-int8 › e-scheme-s0 › c-s0-pc-2048

Per-channel INT8 at 2048 calibration images, [0.5] normalization, seed 0, is no more than 1 point of top-1 below its own FP32

rulesecondary in e-scheme-s0c-s0-pc-2048

Can't tell yet: INT8 is 0.05 points lower than the FP32 baseline (no interval yet); the rule reads the interval, which rb gives from 3 seeds.

What rb measured

-0.05 points

1 run · 7 runs not counted

the rule: -1 point or more-10-0.05

shaded: what the rule accepts · dot: estimate (no interval)

The rule as written

passes when
The interval's worse end is at least -1 point (INT8 against the FP32 baseline).
timing
Written after its results came in.
units
1 seed · declared: unit = seed
checks
ok
review
No call made on it.

Runs it rests on 1 run · 7 runs not counted

Every run here: seed 0, calib seed 0.

7 runs are not counted: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

armrunvalue
differenceper-channel, calib images 2048-0.05 points
FP32per-tensor, calib images 128not counted90.39%
FP32per-tensor, calib images 512not counted90.39%
FP32per-tensor, calib images 1024not counted90.39%
FP32per-tensor, calib images 2048not counted90.39%
FP32per-channel, calib images 128not counted90.39%
FP32per-channel, calib images 512not counted90.39%
FP32per-channel, calib images 1024not counted90.39%

What depends on this

  • question link · q-int8
    Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
  • question link · q-scheme
    Do per-channel weight scales matter, against one scale per tensor?
Source details
rule
top1: int8 vs fp32 [subject where calib_images=2048, scheme=per_channel] no worse than 1, its interval's worse bound inside 1
rb reads
a noninferior rule reads the interval: 1 unit: rb gives an interval from 3
timing
declared after the evidence existed: the only unit was produced before it was declared, and so were 7 unpaired runs (run fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC, run fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC and 4 more)
comparison
top1: int8 vs fp32 [subject where calib_images=2048, scheme=per_channel]: -0.05 % · paired difference, 1 seed (rb)
evidence
its run recorded afterwards
declared
2026-10-20 01:05 UTC
experiment
e-scheme-s0 · secondary
metric
top1: int8 vs fp32 [subject where calib_images=2048, scheme=per_channel]: int8 against fp32
method
paired difference · no interval
question link
q-int8 · 2026-10-20 01:05 UTC
question link
q-scheme · 2026-10-20 01:05 UTC

FP32 and INT8, seed 0, per-tensor, calib images 128, calib seed 0, recorded 2026-10-20 01:05:32 UTC

runin e-scheme-s0 · produced 2026-10-08 · recorded 2026-10-20

Counts toward 0 rules · not counted in 4

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.30
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r005_int8_pt_c128_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-08 01:48 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r005_int8_pt_c128_s0/quantize.log · sha256 431c3df408df · 503 bytes
results
results/r005_int8_pt_c128_s0/metrics.json · sha256 4534fefc65ec · 284 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.3
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r005_int8_pt_c128_s0/quantize.log, results/r005_int8_pt_c128_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:32 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r005_int8_pt_c128_s0/quantize.log, results/r005_int8_pt_c128_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_tensor · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r005_int8_pt_c128_s0

FP32 and INT8, seed 0, per-tensor, calib images 512, calib seed 0, recorded 2026-10-20 01:05:33 UTC

runin e-scheme-s0 · produced 2026-10-08 · recorded 2026-10-20

Counts toward 0 rules · not counted in 4

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.28
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r006_int8_pt_c512_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-08 01:48 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r006_int8_pt_c512_s0/quantize.log · sha256 1b645185b62e · 503 bytes
results
results/r006_int8_pt_c512_s0/metrics.json · sha256 bd16400d35ee · 284 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.28
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r006_int8_pt_c512_s0/quantize.log, results/r006_int8_pt_c512_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:33 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r006_int8_pt_c512_s0/quantize.log, results/r006_int8_pt_c512_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_tensor · calib images 512 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r006_int8_pt_c512_s0

FP32 and INT8, seed 0, per-tensor, calib images 1024, calib seed 0, recorded 2026-10-20 01:05:33 UTC

runin e-scheme-s0 · produced 2026-10-08 · recorded 2026-10-20

Counts toward 0 rules · not counted in 4

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.30
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r007_int8_pt_c1024_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-08 01:48 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r007_int8_pt_c1024_s0/quantize.log · sha256 d43b4d30dede · 507 bytes
results
results/r007_int8_pt_c1024_s0/metrics.json · sha256 12f729f62228 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.3
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r007_int8_pt_c1024_s0/quantize.log, results/r007_int8_pt_c1024_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:33 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r007_int8_pt_c1024_s0/quantize.log, results/r007_int8_pt_c1024_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_tensor · calib images 1024 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r007_int8_pt_c1024_s0

FP32 and INT8, seed 0, per-tensor, calib images 2048, calib seed 0, recorded 2026-10-20 01:05:33 UTC

runin e-scheme-s0 · produced 2026-10-08 · recorded 2026-10-20

Counts toward 0 rules · not counted in 4

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.29
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r008_int8_pt_c2048_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-08 01:48 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r008_int8_pt_c2048_s0/quantize.log · sha256 2dda33ca62ca · 507 bytes
results
results/r008_int8_pt_c2048_s0/metrics.json · sha256 d1b7670b9e71 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.29
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r008_int8_pt_c2048_s0/quantize.log, results/r008_int8_pt_c2048_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:33 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit defcbf0a63 · not in this repository: defcbf0a63 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r008_int8_pt_c2048_s0/quantize.log, results/r008_int8_pt_c2048_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_tensor · calib images 2048 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r008_int8_pt_c2048_s0

FP32 and INT8, seed 0, per-channel, calib images 128, calib seed 0

runin e-scheme-s0 · produced 2026-10-09 · recorded 2026-10-20

Counts toward 1 rule · not counted in 3

Counted in 1 rule.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.27
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 128 --calib-seed 0 --out results/r009_int8_pc_c128_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-09 02:23 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r009_int8_pc_c128_s0/quantize.log · sha256 f82f510f0ea8 · 504 bytes
results
results/r009_int8_pc_c128_s0/metrics.json · sha256 e235eb77dd8b · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.27
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r009_int8_pc_c128_s0/quantize.log, results/r009_int8_pc_c128_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_channel · calib images 128 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r009_int8_pc_c128_s0/quantize.log, results/r009_int8_pc_c128_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_channel · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 128 --calib-seed 0 --out results/r009_int8_pc_c128_s0

FP32 and INT8, seed 0, per-channel, calib images 512, calib seed 0

runin e-scheme-s0 · produced 2026-10-09 · recorded 2026-10-20

Counts toward 1 rule · not counted in 3

Counted in 1 rule.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.30
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 512 --calib-seed 0 --out results/r010_int8_pc_c512_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-09 02:23 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r010_int8_pc_c512_s0/quantize.log · sha256 d1d3b2c7ba60 · 504 bytes
results
results/r010_int8_pc_c512_s0/metrics.json · sha256 f2f9b317d200 · 283 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.3
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r010_int8_pc_c512_s0/quantize.log, results/r010_int8_pc_c512_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_channel · calib images 512 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r010_int8_pc_c512_s0/quantize.log, results/r010_int8_pc_c512_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_channel · calib images 512 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 512 --calib-seed 0 --out results/r010_int8_pc_c512_s0

FP32 and INT8, seed 0, per-channel, calib images 1024, calib seed 0

runin e-scheme-s0 · produced 2026-10-09 · recorded 2026-10-20

Counts toward 1 rule · not counted in 3

Counted in 1 rule.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.30
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 1024 --calib-seed 0 --out results/r011_int8_pc_c1024_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-09 02:23 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r011_int8_pc_c1024_s0/quantize.log · sha256 617f10890b11 · 508 bytes
results
results/r011_int8_pc_c1024_s0/metrics.json · sha256 020f86c2b442 · 284 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.3
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r011_int8_pc_c1024_s0/quantize.log, results/r011_int8_pc_c1024_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_channel · calib images 1024 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r011_int8_pc_c1024_s0/quantize.log, results/r011_int8_pc_c1024_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_channel · calib images 1024 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 1024 --calib-seed 0 --out results/r011_int8_pc_c1024_s0

FP32 and INT8, seed 0, per-channel, calib images 2048, calib seed 0

runin e-scheme-s0 · produced 2026-10-09 · recorded 2026-10-20

Counts toward 1 rule · not counted in 3

Counted in 1 rule.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.39
Top-1, INT8
90.34
Source details
command given later, not observedpython quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 2048 --calib-seed 0 --out results/r012_int8_pc_c2048_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-09 02:23 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r004 (checkpoint) · present when recorded · sha256 41c34ba09dcc
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r012_int8_pc_c2048_s0/quantize.log · sha256 5baf4cd3e8cf · 508 bytes
results
results/r012_int8_pc_c2048_s0/metrics.json · sha256 eaca933cc645 · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.39 · int8: top1=90.34
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-scheme-s0

missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r004: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r012_int8_pc_c2048_s0/quantize.log, results/r012_int8_pc_c2048_s0/metrics.json

# Reproduce a run of e-scheme-s0 · fp32 and int8 · seed 0 · scheme per_channel · calib images 2048 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 1d79e5c4ec · not in this repository: 1d79e5c4ec (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r004: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r012_int8_pc_c2048_s0/quantize.log, results/r012_int8_pc_c2048_s0/metrics.json

# checkpoint · fp32-r004
python train.py see results/r004_fp32_s0/config.yaml

# per run: seed 0 · scheme per_channel · calib images 2048 · calib seed 0
# settings: data.normalize.mean=[0.5, 0.5, 0.5] data.normalize.std=[0.5, 0.5, 0.5] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=256
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r004_fp32_s0 --scheme per_channel --calib-images 2048 --calib-seed 0 --out results/r012_int8_pc_c2048_s0

FP32 and INT8, seed 0, per-tensor, calib images 128, calib seed 0, recorded 2026-10-20 01:05:35 UTC

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 5 rules · not counted in 4

Counted in 5 rules.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.46
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r018_int8_pt_c128_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:53 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r018_int8_pt_c128_s0/quantize.log · sha256 2597406602c1 · 520 bytes
results
results/r018_int8_pt_c128_s0/metrics.json · sha256 650585aee7da · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.46
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r018_int8_pt_c128_s0/quantize.log, results/r018_int8_pt_c128_s0/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 0, recorded 2026-10-20 01:05:35 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r018_int8_pt_c128_s0/quantize.log, results/r018_int8_pt_c128_s0/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r018_int8_pt_c128_s0

FP32 and INT8, seed 1, per-tensor, calib images 128, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 5 rules · not counted in 4

Counted in 5 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.22
Top-1, INT8
90.04
Source details
command given later, not observedpython quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r019_int8_pt_c128_s1how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:53 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r017 (checkpoint) · present when recorded · sha256 118b55ae518f
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r019_int8_pt_c128_s1/quantize.log · sha256 e02d34e8a63b · 520 bytes
results
results/r019_int8_pt_c128_s1/metrics.json · sha256 f6e109d9bc57 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.22 · int8: top1=90.04
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r017: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r019_int8_pt_c128_s1/quantize.log, results/r019_int8_pt_c128_s1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 1 · scheme per_tensor · calib images 128 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r017: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r019_int8_pt_c128_s1/quantize.log, results/r019_int8_pt_c128_s1/metrics.json

# checkpoint · fp32-r017
python train.py --seed 1 --out results/r017_fp32_s1

# per run: seed 1 · scheme per_tensor · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r019_int8_pt_c128_s1

FP32 and INT8, seed 2, per-tensor, calib images 128, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 5 rules · not counted in 4

Counted in 5 rules.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.58
Top-1, INT8
90.48
Source details
command given later, not observedpython quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r020_int8_pt_c128_s2how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:53 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r016 (checkpoint) · present when recorded · sha256 49a88e9c8fe3
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r020_int8_pt_c128_s2/quantize.log · sha256 44c5e1a95cdb · 520 bytes
results
results/r020_int8_pt_c128_s2/metrics.json · sha256 da7d227e81a4 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.58 · int8: top1=90.48
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r016: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r020_int8_pt_c128_s2/quantize.log, results/r020_int8_pt_c128_s2/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 2 · scheme per_tensor · calib images 128 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r016: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r020_int8_pt_c128_s2/quantize.log, results/r020_int8_pt_c128_s2/metrics.json

# checkpoint · fp32-r016
python train.py --seed 2 --out results/r016_fp32_s2

# per run: seed 2 · scheme per_tensor · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 128 --calib-seed 0 --out results/r020_int8_pt_c128_s2

FP32 and INT8, seed 0, per-tensor, calib images 512, calib seed 0, recorded 2026-10-20 01:05:36 UTC

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.57
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r021_int8_pt_c512_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:53 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r021_int8_pt_c512_s0/quantize.log · sha256 634aecb046e4 · 520 bytes
results
results/r021_int8_pt_c512_s0/metrics.json · sha256 d4fd8529ac24 · 284 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.57
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r021_int8_pt_c512_s0/quantize.log, results/r021_int8_pt_c512_s0/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 512 · calib seed 0, recorded 2026-10-20 01:05:36 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r021_int8_pt_c512_s0/quantize.log, results/r021_int8_pt_c512_s0/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 512 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r021_int8_pt_c512_s0

FP32 and INT8, seed 1, per-tensor, calib images 512, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.22
Top-1, INT8
90.08
Source details
command given later, not observedpython quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r022_int8_pt_c512_s1how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r017 (checkpoint) · present when recorded · sha256 118b55ae518f
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r022_int8_pt_c512_s1/quantize.log · sha256 6ab6da5d2e0c · 520 bytes
results
results/r022_int8_pt_c512_s1/metrics.json · sha256 601011b6fbd9 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.22 · int8: top1=90.08
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r017: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r022_int8_pt_c512_s1/quantize.log, results/r022_int8_pt_c512_s1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 1 · scheme per_tensor · calib images 512 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r017: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r022_int8_pt_c512_s1/quantize.log, results/r022_int8_pt_c512_s1/metrics.json

# checkpoint · fp32-r017
python train.py --seed 1 --out results/r017_fp32_s1

# per run: seed 1 · scheme per_tensor · calib images 512 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r022_int8_pt_c512_s1

FP32 and INT8, seed 2, per-tensor, calib images 512, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.58
Top-1, INT8
90.56
Source details
command given later, not observedpython quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r023_int8_pt_c512_s2how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r016 (checkpoint) · present when recorded · sha256 49a88e9c8fe3
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r023_int8_pt_c512_s2/quantize.log · sha256 ba84336b9436 · 520 bytes
results
results/r023_int8_pt_c512_s2/metrics.json · sha256 a4f947c7ceb2 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.58 · int8: top1=90.56
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r016: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r023_int8_pt_c512_s2/quantize.log, results/r023_int8_pt_c512_s2/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 2 · scheme per_tensor · calib images 512 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r016: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r023_int8_pt_c512_s2/quantize.log, results/r023_int8_pt_c512_s2/metrics.json

# checkpoint · fp32-r016
python train.py --seed 2 --out results/r016_fp32_s2

# per run: seed 2 · scheme per_tensor · calib images 512 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 512 --calib-seed 0 --out results/r023_int8_pt_c512_s2

FP32 and INT8, seed 0, per-tensor, calib images 1024, calib seed 0, recorded 2026-10-20 01:05:38 UTC

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.43
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r024_int8_pt_c1024_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r024_int8_pt_c1024_s0/quantize.log · sha256 83e24700a36d · 524 bytes
results
results/r024_int8_pt_c1024_s0/metrics.json · sha256 e306ea156123 · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.43
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r024_int8_pt_c1024_s0/quantize.log, results/r024_int8_pt_c1024_s0/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 1024 · calib seed 0, recorded 2026-10-20 01:05:38 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r024_int8_pt_c1024_s0/quantize.log, results/r024_int8_pt_c1024_s0/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 1024 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r024_int8_pt_c1024_s0

FP32 and INT8, seed 1, per-tensor, calib images 1024, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.22
Top-1, INT8
90.04
Source details
command given later, not observedpython quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r025_int8_pt_c1024_s1how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r017 (checkpoint) · present when recorded · sha256 118b55ae518f
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r025_int8_pt_c1024_s1/quantize.log · sha256 ed2ab274d5f8 · 524 bytes
results
results/r025_int8_pt_c1024_s1/metrics.json · sha256 8094d17eac96 · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.22 · int8: top1=90.04
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r017: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r025_int8_pt_c1024_s1/quantize.log, results/r025_int8_pt_c1024_s1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 1 · scheme per_tensor · calib images 1024 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r017: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r025_int8_pt_c1024_s1/quantize.log, results/r025_int8_pt_c1024_s1/metrics.json

# checkpoint · fp32-r017
python train.py --seed 1 --out results/r017_fp32_s1

# per run: seed 1 · scheme per_tensor · calib images 1024 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r025_int8_pt_c1024_s1

FP32 and INT8, seed 2, per-tensor, calib images 1024, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.58
Top-1, INT8
90.45
Source details
command given later, not observedpython quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r026_int8_pt_c1024_s2how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r016 (checkpoint) · present when recorded · sha256 49a88e9c8fe3
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r026_int8_pt_c1024_s2/quantize.log · sha256 99d38a16dc56 · 524 bytes
results
results/r026_int8_pt_c1024_s2/metrics.json · sha256 277143ae0cca · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.58 · int8: top1=90.45
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r016: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r026_int8_pt_c1024_s2/quantize.log, results/r026_int8_pt_c1024_s2/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 2 · scheme per_tensor · calib images 1024 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r016: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r026_int8_pt_c1024_s2/quantize.log, results/r026_int8_pt_c1024_s2/metrics.json

# checkpoint · fp32-r016
python train.py --seed 2 --out results/r016_fp32_s2

# per run: seed 2 · scheme per_tensor · calib images 1024 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 1024 --calib-seed 0 --out results/r026_int8_pt_c1024_s2

FP32 and INT8, seed 0, per-tensor, calib images 2048, calib seed 0, recorded 2026-10-20 01:05:39 UTC

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.48
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r027_int8_pt_c2048_s0how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r027_int8_pt_c2048_s0/quantize.log · sha256 7e16d08a29c6 · 524 bytes
results
results/r027_int8_pt_c2048_s0/metrics.json · sha256 315737b6461e · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.48
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r027_int8_pt_c2048_s0/quantize.log, results/r027_int8_pt_c2048_s0/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 2048 · calib seed 0, recorded 2026-10-20 01:05:39 UTC
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r027_int8_pt_c2048_s0/quantize.log, results/r027_int8_pt_c2048_s0/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 2048 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r027_int8_pt_c2048_s0

FP32 and INT8, seed 1, per-tensor, calib images 2048, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.22
Top-1, INT8
90.20
Source details
command given later, not observedpython quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r028_int8_pt_c2048_s1how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r017 (checkpoint) · present when recorded · sha256 118b55ae518f
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r028_int8_pt_c2048_s1/quantize.log · sha256 2d7431b9e40e · 524 bytes
results
results/r028_int8_pt_c2048_s1/metrics.json · sha256 c7ecc81f9447 · 285 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.22 · int8: top1=90.2
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r017: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r028_int8_pt_c2048_s1/quantize.log, results/r028_int8_pt_c2048_s1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 1 · scheme per_tensor · calib images 2048 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r017: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r028_int8_pt_c2048_s1/quantize.log, results/r028_int8_pt_c2048_s1/metrics.json

# checkpoint · fp32-r017
python train.py --seed 1 --out results/r017_fp32_s1

# per run: seed 1 · scheme per_tensor · calib images 2048 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r017_fp32_s1 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r028_int8_pt_c2048_s1

FP32 and INT8, seed 2, per-tensor, calib images 2048, calib seed 0

runin e-ptq · produced 2026-10-14 · recorded 2026-10-20

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 3 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.58
Top-1, INT8
90.42
Source details
command given later, not observedpython quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r029_int8_pt_c2048_s2how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-14 01:54 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r016 (checkpoint) · present when recorded · sha256 49a88e9c8fe3
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r029_int8_pt_c2048_s2/quantize.log · sha256 1a896cf4517d · 524 bytes
results
results/r029_int8_pt_c2048_s2/metrics.json · sha256 82759cf66269 · 286 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.58 · int8: top1=90.42
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r016: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r029_int8_pt_c2048_s2/quantize.log, results/r029_int8_pt_c2048_s2/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 2 · scheme per_tensor · calib images 2048 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit 2982f0fae9 · not in this repository: 2982f0fae9 (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r016: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r029_int8_pt_c2048_s2/quantize.log, results/r029_int8_pt_c2048_s2/metrics.json

# checkpoint · fp32-r016
python train.py --seed 2 --out results/r016_fp32_s2

# per run: seed 2 · scheme per_tensor · calib images 2048 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r016_fp32_s2 --scheme per_tensor --calib-images 2048 --calib-seed 0 --out results/r029_int8_pt_c2048_s2

FP32 and INT8, seed 0, per-tensor, calib images 128, calib seed 1

runin e-ptq · produced 2026-10-15 · recorded 2026-10-20

Counts toward 3 rules · not counted in 5

Counted in 3 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.49
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 1 --out results/r030_int8_pt_c128_s0_d1how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-15 02:11 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r030_int8_pt_c128_s0_d1/quantize.log · sha256 5bf4001c9d51 · 526 bytes
results
results/r030_int8_pt_c128_s0_d1/metrics.json · sha256 a51d9065d064 · 288 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.49
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit b008869efc · not in this repository: b008869efc (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r030_int8_pt_c128_s0_d1/quantize.log, results/r030_int8_pt_c128_s0_d1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 1
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit b008869efc · not in this repository: b008869efc (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r030_int8_pt_c128_s0_d1/quantize.log, results/r030_int8_pt_c128_s0_d1/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 128 · calib seed 1
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 1 --out results/r030_int8_pt_c128_s0_d1

FP32 and INT8, seed 0, per-tensor, calib images 128, calib seed 2

runin e-ptq · produced 2026-10-15 · recorded 2026-10-20

Counts toward 3 rules · not counted in 5

Counted in 3 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Not counted in 1 rule: its unit has no int8 run, so the paired reading leaves it out.

Results

Top-1, FP32
90.64
Top-1, INT8
90.61
Source details
command given later, not observedpython quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 2 --out results/r031_int8_pt_c128_s0_d2how rb knowsit was made before it was recorded, and neither rb's journal nor a log line written then shows this command made it; logs/agent.log was written 2026-10-16, after the run
produced
2026-10-15 02:11 UTC (as reported) · recorded 2026-10-20 01:05 UTC
inputs
read: fp32-r014 (checkpoint) · present when recorded · sha256 61a697a6b408
repository
when recorded, not when run: commit 1e2147ef3d · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.12.15 · rb 0.6.0 · pydantic 2.14.1 (where it was recorded: the run was made before it was recorded)
results
results/r031_int8_pt_c128_s0_d2/quantize.log · sha256 2a1b879eaa54 · 526 bytes
results
results/r031_int8_pt_c128_s0_d2/metrics.json · sha256 b9324a31eb2e · 288 bytes
type
recorded afterwards
results as rb gives them
fp32: top1=90.64 · int8: top1=90.61
Reproduce this run · 0 of 4 parts ready

to run again: ◐ 1 partial · ✗ 2 missing · ○ 1 unknown

the whole recipe for e-ptq

missing · commit b008869efc · not in this repository: b008869efc (get the code repository to rerun)

unknown · the run was made before it was recorded, in an environment rb did not see

partial · fp32-r014: reconstructible

missing · neither of the 2 files the runs named is here unchanged; not here: results/r031_int8_pt_c128_s0_d2/quantize.log, results/r031_int8_pt_c128_s0_d2/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 0 · scheme per_tensor · calib images 128 · calib seed 2
# Recipe, from what the record holds. rb did not run these commands.
# code: missing · commit b008869efc · not in this repository: b008869efc (get the code repository to rerun)
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# checkpoint: partial · fp32-r014: reconstructible
# files: missing · neither of the 2 files the runs named is here unchanged; not here: results/r031_int8_pt_c128_s0_d2/quantize.log, results/r031_int8_pt_c128_s0_d2/metrics.json

# checkpoint · fp32-r014
python train.py --seed 0 --out results/r014_fp32_s0

# per run: seed 0 · scheme per_tensor · calib images 128 · calib seed 2
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command given later, not observed: python quantize.py --from results/r014_fp32_s0 --scheme per_tensor --calib-images 128 --calib-seed 2 --out results/r031_int8_pt_c128_s0_d2

FP32 and INT8, seed 1, per-channel, calib images 128, calib seed 0

runin e-ptq · produced 2026-10-21 · recorded 2026-10-25

Counts toward 4 rules · not counted in 4

Counted in 4 rules.

Not counted in 4 rules: its unit has a run that gives the difference by itself, so the paired reading leaves it out.

Results

Top-1, FP32
90.22
Top-5, FP32
99.75
Size kb, FP32
1104.50
Top-1, INT8
90.04
Top-5, INT8
99.75
Size kb, INT8
327.00
Source details
command (as given)not recorded
produced
2026-10-21 05:17 UTC (its file's time) · recorded 2026-10-25 01:08 UTC
repository
when recorded, not when run: commit 870726b59f · tree dirty
config
8 matched · absent: train.epochs
basis
file · spec not locked
environment
python 3.14.4 · rb 0.6.0 (where it was recorded: the run was made before it was recorded)
results
results/r032_int8_pc_c128_s1/metrics.json · sha256 9cbcadc61501 · 285 bytes
type
recorded afterwards
results as rb gives them
wall_s=11.8 · fp32: top1=90.22 top5=99.75 size_kb=1104.5 · int8: top1=90.04 top5=99.75 size_kb=327.0
Reproduce this run · 0 of 3 parts ready

to run again: ✗ 1 missing · ○ 2 unknown

the whole recipe for e-ptq

unknown · the run was made before it was recorded, and names no commit

unknown · the run was made before it was recorded, in an environment rb did not see

missing · no file the runs named is here unchanged; not here: results/r032_int8_pc_c128_s1/metrics.json

# Reproduce a run of e-ptq · fp32 and int8 · seed 1 · scheme per_channel · calib images 128 · calib seed 0
# Recipe, from what the record holds. rb did not run these commands.
# code: unknown · the run was made before it was recorded, and names no commit
# environment: unknown · the run was made before it was recorded, in an environment rb did not see
# files: missing · no file the runs named is here unchanged; not here: results/r032_int8_pc_c128_s1/metrics.json

# per run: seed 1 · scheme per_channel · calib images 128 · calib seed 0
# settings: data.normalize.mean=[0.4914, 0.4822, 0.4465] data.normalize.std=[0.247, 0.2435, 0.2616] train.epochs=30 quant.backend=fbgemm quant.activation_observer=histogram quant.calibration.batch_size=32 eval.batch_size=512
#   fp32: precision=fp32
#   int8: precision=int8
# command (as given) · not recorded

Open callsYour call › q-int8 › c-ref

Open callYour callStill open · c-ref

FP32 ResNet-20 reaches the published 91.25% top-1 on CIFAR-10

Not reproduced: FP32 baseline: 90.48% (89.92% to 91.04%), not within 0.005% of the cited 91.25%.

the rule: 91.25% ± 0.005%91.2590.48

From 3 runs · the whole finding

Nothing is recorded on it yet.

Source details

cited: README.md:10 ResNet-20 reaches 91.25% top-1 on CIFAR-10

Record a call

  • Accept
    rb decide c-ref accept --why '‹why›'
    The record will say: accepted ‹day›: ‹why›
  • Reject
    rb decide c-ref reject --why '‹why›'
    The record will say: rejected ‹day›: ‹why›
  • Investigate
    rb decide c-ref investigate --why '‹why›'
    The record will say: investigating since ‹day›: ‹why›

It is run with rb in a terminal; rb records the call and the reason.

Run it in your terminal; rb records the call and your reason.

FP32

variantbaselinee-ptq

fp32

cellTop-1Size kbTop-5
per-channel, calib images 128, calib seed 090.22% n=11104.50 n=199.75 n=1
per-tensor, calib images 128, calib seed 090.48% n=3
per-tensor, calib images 128, calib seed 190.64% n=1
per-tensor, calib images 128, calib seed 290.64% n=1
per-tensor, calib images 512, calib seed 090.48% n=3
per-tensor, calib images 1024, calib seed 090.48% n=3
per-tensor, calib images 2048, calib seed 090.48% n=3

the runs behind each mean · 15 runs

INT8

variantcandidatee-ptq

int8

cellTop-1Size kbTop-5
per-channel, calib images 128, calib seed 090.04% n=1327.00 n=199.75 n=1
per-tensor, calib images 128, calib seed 090.33% n=3
per-tensor, calib images 128, calib seed 190.49% n=1
per-tensor, calib images 128, calib seed 290.61% n=1
per-tensor, calib images 512, calib seed 090.40% n=3
per-tensor, calib images 1024, calib seed 090.31% n=3
per-tensor, calib images 2048, calib seed 090.37% n=3

the runs behind each mean · 15 runs

README.md · lines 42-43

cited file

The cited lines

41  ```
42▸ python sweep.py configs/sweep.yaml --jobs 4 --threads 2
43▸ python summarize.py
44  ```

read at commit 870726b

Cited by

Source details
file
README.md
commit
870726b59fbb098146ce5c529b5e377436dcc217
sha256
54fe5e466749f4dfb72fb0114d5b3ed3d5043294f890513d643ddd5e9a8226ce

notes.md · line 5

cited file

The cited lines

4  
5▸ 2026-10-07: Decision: from now on every FP32 baseline trains for 30 epochs, not the paper's 64k iterations (about 182 epochs). We compare INT8 with our own FP32, so a short schedule is enough, and the paper's 91.25 stays a reference, not a target.
6

read at commit 870726b

Cited by

Source details
file
notes.md
commit
870726b59fbb098146ce5c529b5e377436dcc217
sha256
1195765d3f04bbb1def2183a27fd7f469ddf55a71baf12037bd7a63f4e28a6c6

README.md · line 3

cited file

The cited lines

2  
3▸ Does INT8 post-training quantization keep a ResNet-20 within 1 point of FP32 top-1 on CIFAR-10?
4

read at commit 870726b

Cited by

Source details
file
README.md
commit
870726b59fbb098146ce5c529b5e377436dcc217
sha256
54fe5e466749f4dfb72fb0114d5b3ed3d5043294f890513d643ddd5e9a8226ce

README.md · line 10

cited file

The cited lines

9  
10▸ Reference: ResNet-20 reaches 91.25% top-1 on CIFAR-10 (He et al. 2016, "Deep Residual Learning for Image Recognition", table 6, 8.75% error).
11

read at commit 870726b

Cited by

Source details
file
README.md
commit
870726b59fbb098146ce5c529b5e377436dcc217
sha256
54fe5e466749f4dfb72fb0114d5b3ed3d5043294f890513d643ddd5e9a8226ce

logs/agent.log · line 2793

cited file

The citation

Not quoted on a shared page · rb open shows these lines on the record's own machine.

Cited by

Source details
file
logs/agent.log
commit
870726b59fbb098146ce5c529b5e377436dcc217

logs/agent.log · line 2811

cited file

The citation

Not quoted on a shared page · rb open shows these lines on the record's own machine.

Cited by

Source details
file
logs/agent.log
commit
870726b59fbb098146ce5c529b5e377436dcc217

notes.md · line 7

cited file

The cited lines

6  
7▸ 2026-10-09: Per-channel weights only on seed 0 for now. If it looks different from per-tensor, we add seeds.
8

read at commit 870726b

Cited by

Source details
file
notes.md
commit
870726b59fbb098146ce5c529b5e377436dcc217
sha256
1195765d3f04bbb1def2183a27fd7f469ddf55a71baf12037bd7a63f4e28a6c6

logs/agent.log · line 2809

cited file

The citation

Not quoted on a shared page · rb open shows these lines on the record's own machine.

Cited by

Only rb's own lines cite it here, such as the history or a recipe.

Source details
file
logs/agent.log
commit
870726b59fbb098146ce5c529b5e377436dcc217

Also open in the record (15)

Rules rb can't judge yet (5). Each needs results from 3 or more runs for an interval.

details

Rules rb can't judge yet: no interval yet

Rules with no runs yet (3).

details

Rules with no runs yet

Rules written after their results (1).

details

Rules written after their results or data were seen

Settings never checked against a source (4).

  • given as fp32, not checked against a source. Its note: FP32 model as trained; fp32.top1 in each quantize run's metrics.json. e-ptq/fp32.precision
  • given as int8, not checked against a source. Its note: INT8 model from quantize.py; int8.top1 in each quantize run's metrics.json. e-ptq/int8.precision
  • given as fp32, not checked against a source. Its note: FP32 model as trained; fp32.top1 in each quantize run's metrics.json. e-scheme-s0/fp32.precision
  • given as int8, not checked against a source. Its note: INT8 model from quantize.py; int8.top1 in each quantize run's metrics.json. e-scheme-s0/int8.precision
details

Settings no source confirms

Metrics the runs report that no rule reads (1)

  • Runs report fp32.size_kb, fp32.top5, int8.size_kb, int8.top5, wall_s, which no rule reads e-ptq

Runs whose command was given later, not seen (1).

details

Commands given after the runs

About this record

Research record int8-resnet-20-on-cifar-10 · rb 0.6.1

Kept with rb since 20 Oct 2026. All 23 runs and 1 call were recorded afterwards, not as they happened.

A decision records a call; it does not change a reading. rb computes; it never decides.

made with rb page · a snapshot of the record at 2026-10-29 23:49 UTC
How to read this record

each rule is read on axes: the comparison it reads with its interval, whether the rule as declared is met, when it was declared against its evidence, how many independent units it rests on, and anything wrong with what it rests on. No single word stands for all of that.

findings are computed, never stored, so regenerate this file after new runs. If this page and rb status ever disagree, rb status is right.

What the words on this page mean
open (question)
no call has closed it. A call on the question closes it: answered (accept), dropped (reject), parked, stopped, pivoted or shipped; investigate and continue keep it open. A finding or an interpretation does not make that call.
note (on a question)
recorded words on what meeting a rule would mean for the question. A note counts only while its rule is met; rb computes the finding and never asserts the inverse.
Computed comparison
rb's reading of a declared reversal between two rules: both numbers and intervals stay visible, and it makes no call on the question.
headline
rb's one line on a question's evidence, the first that applies: No rule tests this question yet (rules are filed here but none is linked to it), Evidence needs review (an integrity problem), No evidence recorded yet, Measured differences; uncertainty remains (an interval includes 0), Evidence includes point estimates only (a reading has no interval), else Evidence available for review.
rule written before the results
the rule was declared before the evidence it reads existed: a test planned in advance.
rule written after the results
the rule or its question link was declared after the evidence existed or after the data was seen: never read as a test planned in advance.
no interval
one estimate with no interval, so how much it could vary is unknown. rb gives one from 3 units, and none across fixed design cells.
Uncertainty remains
unresolved: the interval includes 0.
Fixed design cells
the reading spans fixed design choices: each cell is read on its own, with how many meet and their range, and no sampling interval across cells.
Reversal observed; repeatability uncertain
the two sides of a declared reversal measured opposite directions, but not both with intervals clear of 0.
rb's interval
95%, from 3 units: a Student t interval when the arms pair up by their unit, Welch (unpaired) when they never share one.
straddles
the interval crosses the rule's threshold: the rule word holds on the estimate, not across the whole interval.
rule
a threshold or comparison declared for a question; commands call it a criterion.
can't tell yet
there are results, but they don't settle the rule: it needs an interval, more units, or the metric's direction.
result
gain or loss (the interval clear of 0, read in the metric's direction), unresolved (the interval includes 0), equivalent (inside a declared margin), no interval (one estimate, no interval yet), measured (a level), higher or lower (the metric has no direction), not tested.
timing
declared before the evidence (before its runs, when only its baseline's runs came first: timing is read on the subject's runs); after the evidence existed; after the data was seen; order unknown.
units
the independent units a reading rests on. A repeat checks a run and adds no unit; a reanalysis adds none.
checks
problems with what a finding depends on (a stale source, an undeclared difference, a record edited outside rb). ok when there are none. Not the repository's own checks.
estimate meets the rule
met on the estimate, not across the whole interval; never drawn green.
verified
the named file, run or database row states this setting's value. It does not mean a run used it. provisional: given, not checked.
open (experiment)
planned: an experiment nothing tests yet. open: evidence and no call. accepted, rejected, investigating, parked, stopped, pivoted, shipped: what the latest call on it made it.
locked
the spec and rules are locked; a later change is recorded as an amendment.
not locked
rules and settings can still change; a freeze locks them.
not tested
no results for this rule yet: no recorded run reads this comparison
measured
one arm on its own, with its interval, so neither better nor worse
command (as given)
--command records what produced the numbers. rb did not run it.
not confirmed
it waits to be confirmed with rb in a terminal; until then it reads as a note, never as a call made or an answer.
reconstructible
present, changed, remote, reconstructible or lost: reconstructible means no copy is recorded, but how to get it again is.
gain
the interval is clear of 0 on the better side.
loss
the interval is clear of 0 on the worse side.
unresolved
the interval includes 0, which is not evidence of no difference.
equivalent
the interval lies inside the margin an equivalence rule declares.
reproduced
a rule from a cited source or from the record this one was cloned from, met by the runs recorded here.
not reproduced
a rule from a cited source or from the record this one was cloned from, not met by the runs recorded here.
paired difference
each unit's subject minus its reference, averaged over the units.
cited
a number someone else stated (a paper's table, say), checked against the source's own words.
margin
how far the value the rule judges (the estimate, an interval bound or the worst unit) sits from its threshold: positive inside it, negative outside. Not a declared equivalence or noninferiority margin.
question link
identity
how exactly a resource is named: exact (the hash of its bytes), version (a release) or name only.
amended
changed after the rules and settings were locked; the change, its reason and what it changed from are kept.