A notebook shows a strong result on screen — but anyone who reruns it gets a worse number, an error, or both. That includes the author: after a kernel restart, the celebrated score never comes back. The file is unchanged, the data is unchanged, and yet the result on the screen cannot be reproduced from them. Usually discovered by a teammate who cloned the project, or on Monday morning after a Friday demo.
End of an exploration sprint: dozens of cells, hours of editing, fitting and re-fitting. Somewhere along the way a cell was edited — a feature dropped from a list, a threshold changed — without re-running the cells below it, or cells were run in a hand-picked order. The screen kept showing results computed from the kernel's memory, not from the file's text. Common right before demos, when people edit fast and rerun selectively. Linked module: 4.1 (Notebook to repo).
Hidden state. A notebook kernel keeps every variable alive between runs, so the
result of a cell can depend on code that has since been edited away. There are two
forms. The loud form shows execution brackets out of order — In [12] sitting
above In [3] — and is easy to spot. The silent form is worse: a cell edited
after its last run keeps its old bracket number, so the screen looks clean while
the memory still holds the old definition. Confirm with one action: Kernel →
Restart & Run All. If any number changes, the screen result was memory-born and
the file cannot rebuild it. In the 4.1 churn repo this was 0.824 on screen against
0.816 in the file — two churners' difference, invisible until the restart.
Accept the file's number as the only truth, even when it is worse — the on-screen
number describes code that no longer exists. Run Restart & Run All and record what
survives. If the lost result mattered, reconstruct it deliberately: work out which
edit broke it (re-add the dropped feature, revert the threshold), re-run top to
bottom, and commit that version so the good number has a home in the file. Then
move the surviving code out of the notebook: functions into src/, knobs into
config.py, and a train.py entry point that runs top to bottom from a clean
terminal. A script cannot have hidden state; every run is a restart.
House rule: no notebook number enters a report, a ticket, a demo or a README until
Restart & Run All has reproduced it. Keep notebooks for exploration only; anything
that must last or be shared lives in the repo with entry points. Clear all outputs
before committing a notebook, so stale Out[] cells cannot pose as results. And
treat out-of-order brackets in a teammate's screenshot the way you treat a failing
test — say something.
Q: "Your notebook shows 0.82 but your teammate runs the same file and gets 0.79. Same data, same library versions. What happened?" Strong answer: name hidden state in one sentence — the 0.82 came from the kernel's memory, not the file — give Restart & Run All as the one-step confirmation, and finish with the fix: promote the code to scripts with a clean-terminal entry point, where every run starts from nothing. Mentioning that an edited cell keeps its old bracket number signals you have met the silent form, not only read about the loud one.