A score improves and nobody can explain it. One commit changed the learning rate, the batch size, the dropout probability, the optimizer and the number of hidden layers, and validation accuracy moved from 0.71 to 0.81. The team ships it. A week later the same recipe is applied to a second dataset and performs worse than the old one, a reviewer asks which change earned the gain, and there is no answer. Worse symptom: the next experiment moves the score down and nobody can tell which of the five knobs to turn back.
Any tuning session under time pressure — the evening before a demo, the last hours of a hackathon, the day a mentor asks for "better numbers". The bug is social as much as technical: five changes feel five times faster than five runs, and on a laptop where each run takes two seconds that reasoning is not even true. Common companions: no results file, several notebook cells edited in place, and screenshots of numbers instead of a table. The 2.4 measured demo shows what a one-at-a-time table looks like on 281 churn rows — baseline test 0.712; dropout 0.5 alone 0.808; weight decay 1e-2 alone 0.784; early stopping alone 0.824; and all three stacked together 0.792, below the best single knob. Linked modules: 2.4, 1.8 (tuning), 4.4 (tracking).
An experiment answers one question. Change five things and the run answers no question at all: it produces one number that could have come from any of thirty-one combinations of the five changes, including combinations where one change helped and another hurt by nearly the same amount. The measured table above is the proof that this is not pedantry — stacking three genuinely helpful tools scored 0.792 while one of them alone scored 0.824, so a five-knob commit can easily hide a change that made things worse. Confirm the problem by asking one question of any tuning log: can you name, for each knob, the two runs that differ only in that knob? If you cannot, the log records history, not evidence. A second confirming signal is that the gain does not survive a new seed or a new dataset, because part of it was random init luck (WR-129) rather than any knob.
Go back to the last run you can reproduce and rebuild forward, one variable per run, writing a row after each. The row needs six fields at least: the knob and its value, epochs run, final training loss, best validation loss with its epoch, the test-set score, and one sentence of reading. Keep every other setting and the seed identical across rows, so each row is a controlled comparison. When two knobs must move together — batch size and learning rate often do — say so explicitly in the row and treat the pair as one knob. Then, and only then, test a stacked combination as its own row, and accept it only if it beats the best single knob on the validation set. On this course's hardware a row costs seconds, so the discipline is free.
Start every tuning session by writing the baseline row before changing anything. Keep the results table in the repository next to the code, not in a chat message, so it survives the evening. Put the configuration in a file the script reads rather than inline numbers, which makes a run's identity a diff instead of a memory. Adopt the rule out loud in reviews: a pull request that changes more than one training knob must carry the table that separates them. Module 4.4 turns this same habit into MLflow runs and a registry; the discipline is the point, the tool is a convenience.
Q: "You changed the learning rate, the batch size and the model depth together and accuracy went up four points. Your lead asks what to write in the report. What do you say?" Strong answer: state plainly that the run does not support any claim about which change helped, and that reporting one is guessing; then propose the fix — rerun from the baseline with one knob per run, same seed, one row each, and report the table; then note that stacked settings must be tested as their own row because tools can cancel each other, citing an example where three good tools together scored below the best single one. Bonus: mention that some knobs are coupled (batch size and learning rate) and must be moved as a pair with that coupling stated.