A convolutional network crashes on its very first batch, before a single epoch
finishes, with a message like RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x800 and 1568x128). Nothing in the traceback points at a conv
layer; the failing line is the first nn.Linear after nn.Flatten(). The same
script ran last week. A second, quieter version of this report exists: the model
does train, but only after someone inserted a pooling layer to make the numbers
agree, and the accuracy is a point or two below what the architecture should give.
Any CNN built by hand from module 2.5 onwards, and hardest the first time a
student changes an architecture rather than copying one. The usual births: a
padding=1 removed or added, a kernel size changed from 3 to 5, an extra pooling
layer inserted, or the same model pointed at a new image size — 32-pixel CIFAR
rows fed to a stack sized for 28-pixel FashionMNIST. Measured example from 2.5:
a two-block stack with padding=1 ends at (64, 32, 7, 7), so the flatten is
1568; drop the padding and the same stack ends at (64, 32, 5, 5), so the flatten
is 800, and the nn.Linear(1568, 128) written for the first version now receives
800 columns. Linked modules: 2.5, 2.3 (shape audits), 2.6 (pretrained stacks with
their own input sizes).
A conv layer does not care about the height and width it is handed, so it never complains. Its output simply changes size, and the change is silent all the way down the stack until it meets the first fully connected layer, which was built with one fixed input count. That layer is a matrix multiply, and a matrix multiply is where a shape disagreement finally becomes an error. Read the two pairs in the message in order: the first pair is what arrived — batch 64, flatten 800 — and the second is what the layer expects — 1568 in, 128 out. So the question is never "why is the Linear wrong"; it is "which layer above changed the height and width". Answer it with the output-size formula, applied once per layer: output equals input plus twice the padding, minus the kernel size, divided by the stride, plus one. A 3-kernel with padding 1 keeps the size, a 5-kernel with padding 2 keeps the size, and a 2 by 2 max pool with stride 2 halves it, rounding down.
Print the shape after every layer before you train anything. Ten lines do it:
push torch.zeros(2, C, H, W) through the model one layer at a time and print
tuple(x.shape) with the layer's class name. The number you need is the last
shape before nn.Flatten(), multiplied out: channels times height times width.
Write that product into the nn.Linear — or better, compute it in code with a
small flatten_size helper so it can never drift from the layers above it.
nn.LazyLinear will also infer it on the first forward pass, and it is fine for
experiments, but the helper is what you want in a repo other people read. Resist
the tempting wrong fix: inserting nn.AdaptiveAvgPool2d((7, 7)) makes any stack
agree with any Linear and hides the real bug. On 2.5's FashionMNIST subset that
patch scored 0.8505 while correcting the padding scored 0.8695 — the patch threw
away spatial detail nobody had chosen to throw away.
Treat a CNN's shape trace the way 2.3 taught you to treat dtypes at the tensor boundary: audit it once, in code, at the top of every project. Keep the formula where you can see it, and do the arithmetic on paper before you type the layer — it is a standard interview question precisely because it separates people who have built a CNN from people who have copied one. Never hard-code a flatten count in two places. And when you change the input image size, expect every downstream number to change with it; a model written for 28 by 28 is not a model for 32 by 32.
Q: "Your CNN throws mat1 and mat2 shapes cannot be multiplied (64x800 and 1568x128). Walk me through the fix." Strong answer: read the message aloud —
64 by 800 arrived, 1568 by 128 was expected — and say the mismatch is at the
flatten, not inside any conv layer. Then give the formula and apply it down the
stack, naming the three habits (3 with padding 1 keeps size, 5 with padding 2
keeps size, 2 by 2 pool halves it, rounding down). Then give the fix: compute the
flatten in code, and print shapes per layer before training. Bonus: name
AdaptiveAvgPool2d as the fix that silences the error without solving it, and say
what it costs.