A pretrained backbone is loaded, the head is replaced, the code runs without a single warning, and the accuracy is merely mediocre. Nothing looks broken. The model beats a coin toss, the loss falls, the curves look healthy, and yet the score sits far below every blog post and every teammate's number on the same task. A second signature often appears beside it: the frozen-feature run and the fine-tuned run land close together, because both are reading the same damaged input. Teams then blame the dataset, add more layers, or train for longer, and none of it helps.
Any transfer-learning job from module 2.6 onwards, and the most common quiet bug
in the whole module. The usual birth is a transform pipeline copied from 2.5,
where transforms.ToTensor() alone was correct because the model was trained from
scratch on those same 0-to-1 pixels. A pretrained backbone has no such freedom: it
was trained on inputs that had ImageNet's channel means subtracted and ImageNet's
channel standard deviations divided out. Other births are an OpenCV read that
hands back channels in BGR order, a pipeline that leaves pixels on the 0-to-255
scale, a Normalize copied from a different backbone's documentation, and an
evaluation pipeline that normalises while the training pipeline does not. Measured
example from 2.6, resnet18 frozen features on 600 training photos: the documented
mean and standard deviation score 0.953 on 300 test photos, raw 0-to-1 pixels
score 0.900, mean 0.5 with standard deviation 0.5 scores 0.940, BGR channel order
scores 0.927, and 0-to-255 pixels collapse to 0.657. Linked modules: 2.6, 2.5
(transform pipelines), 0.4 (array ranges), 2.10 (Hugging Face processors).
A trained network is a stack of thresholds tuned to one specific input
distribution. ImageNet training subtracted 0.485, 0.456 and 0.406 from the red,
green and blue channels and divided by 0.229, 0.224 and 0.225, so every filter in
the backbone expects numbers centred near zero with a spread near one. Hand it
0-to-1 pixels and every activation shifts; hand it 0-to-255 pixels and the first
layers saturate, so the 512 features arriving at your head carry far less
information than they should. The damage is proportional to the mismatch, which is
exactly why it hides: a small offset costs one or two accuracy points that look
like ordinary variance, while a scale error of 255 costs everything. Confirm it
with two prints rather than a guess. Print the mean and the standard deviation of
one training batch after your pipeline: correct input reads roughly 0.0 and 1.0,
a missing Normalize reads roughly 0.45 and 0.27, and a missing ToTensor scale
reads in the tens. Then run the backbone's own classifier on a handful of ordinary
photos with the head still attached and read the predicted ImageNet labels: with
the right pipeline they are sensible, and with the wrong one they are nonsense.
Stop hand-writing the transform. Every torchvision weight enum carries its own
documented pipeline, so ask the weights for it:
weights = ResNet18_Weights.IMAGENET1K_V1 and then
preprocess = weights.transforms(), which prints the resize, the crop, the
interpolation, the mean and the standard deviation it was trained with. Use that
object for training and evaluation alike. When compute forces a smaller input than
224 pixels, as this course does on CPU, change the resize and the crop but never
the mean and the standard deviation — those belong to the weights, not to the
image size. If the images arrive through OpenCV, convert BGR to RGB at the read,
before anything else touches them. Then make it enforceable: assert in one line
that a training batch has a mean inside plus or minus 0.5 and a standard deviation
between 0.5 and 2.0, and fail loudly when it does not. In the 2.6 run this single
change moved the score from 0.657 back to 0.953.
Treat the input pipeline as part of the checkpoint. Whenever you save a model,
save the transform beside it, and whenever you load somebody's weights, load their
documented transform with them. Keep the numbers out of your own code: a hardcoded
Normalize([0.485, 0.456, 0.406], ...) is correct only until the day you switch
backbones, and then it is a silent bug with your name on it. Print the batch mean
and standard deviation once at the start of every training run, the same way you
print shapes. In review, ask one question of any transfer-learning script: where
did these normalisation numbers come from? "From the weights object" is the only
safe answer.
Q: "You fine-tuned a pretrained ResNet and got 66 percent where the same recipe
gets 95 percent for everyone else. Nothing errors. Where do you look first?"
Strong answer: name the input pipeline before the model, and explain why — a
pretrained backbone encodes an expected input distribution, so a scale or offset
mismatch degrades every layer without raising anything. Give the confirming check
as a printed batch mean and standard deviation, and the fix as
weights.transforms() rather than hand-copied constants. Then rank the mismatches
by damage: a wrong mean costs a little, a wrong channel order costs a little more,
and a 0-to-255 scale costs almost everything. Bonus: note that this bug makes
frozen and fine-tuned runs converge to similarly poor numbers, which is itself a
clue that the fault is upstream of the training choice.