A NumPy forward pass dies with
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0 ... (size 3 is different from 10) — or a similar pair of numbers. The traceback points
at one @ somewhere in the middle of the network, the numbers in the message match
nothing you remember typing, and the batch ran fine yesterday.
First hand-built networks, module 2.1 onwards: a chain of x @ W + b layers written
in a row, often after editing one layer's width or adding a new layer in the middle.
Also triggered by loading weights saved from a differently sized network, or by
passing a transposed matrix. The error surfaces at the first multiply that cannot
happen — which may be several layers after the real mistake. Linked module: 2.1.
The rule is module 0.4's: (a, b) @ (b, c) works only when the inner numbers match.
The error message hands you the mismatched pair — here an input arriving with 3
columns meeting a matrix expecting 10 rows, or the reverse. The layer that raised
the error is usually innocent; the guilty layer is upstream, producing the wrong
width, and every layer between them silently passed the bad shape along. Confirm by
printing shapes at every station: add print(name, out.shape) after each layer (or
loop over the layers and print inside the loop). The first shape that differs from
your written plan marks the real fault line. Common causes, in order: a layer's W
built as (out, in) instead of (in, out); an edited hidden width updated in one
matrix but not the next; weights loaded from an older network size.
Write the intended shape sequence as a comment first, e.g.
(32, 10) -> (32, 16) -> (32, 8) -> (32, 1). Then print the actual shape after
every layer and compare against the plan, top to bottom. Fix the first divergence
only — resist editing the layer the traceback named until upstream shapes are
clean. If a matrix is transposed, fix its construction rather than sprinkling .T
at the call site; a .T patch hides the mistake and breaks again on the next edit.
Rerun and confirm the whole printed sequence matches the comment.
Adopt the shape-print reflex from 0.4 at network scale: any function that builds a
layer asserts x.shape[1] == W.shape[0] before multiplying, with a message naming
the layer. Keep the planned shape sequence as a comment above the network code and
update it with every edit. When changing one hidden width, grep for every matrix
that touches that width — a hidden layer's size appears in two weight matrices,
never one.
Q: "Your five-layer NumPy network throws a matmul shape error at layer four. Walk
me through your debugging." Strong answer: state the inner-dimension rule in one
sentence, note that the raising layer is rarely the faulty one, describe printing
the shape after every layer against a written plan, and name the two usual culprits
— a (out, in) weight matrix and a half-edited hidden width. Bonus: mention
asserting shapes inside the layer function so the error names its layer next time.