Two faces, one border crossing. The loud face: the first forward pass crashes
with RuntimeError: mat1 and mat2 must have the same dtype, but got Double and Float, or a loss call raises ValueError: Target size (torch.Size([125])) must be the same as input size (torch.Size([125, 1])). The quiet face: nothing
crashes, but a metric is nonsense — accuracy prints a constant, impossible value
(2.3's measured demo: 0.58 where the true score is 0.79), or an MSE loss trains
toward the wrong target while printing only a one-line UserWarning that scrolled
past.
The boundary where NumPy and pandas data becomes PyTorch tensors — the first ten
lines of nearly every training script from module 2.3 onward. torch.from_numpy
faithfully keeps NumPy's default float64 (Double), while every nn.Linear weight
is born float32 (Float). Labels arrive from pandas as flat (N,) arrays, while
the model emits (N, 1) columns. Both mismatches are born in the same five
lines, and each surfaces at a different distance from its cause. Linked module:
2.3.
The dtype crash is honest: PyTorch refuses to multiply Double by Float, and the
traceback points at the model call, one step downstream of the from_numpy that
caused it. The shape failures are the dangerous ones because broadcasting — the
0.4 superpower — keeps working when you no longer want it. Compare a (125,)
prediction against a (125, 1) label and NumPy rules stretch both to
(125, 125): fifteen thousand pairwise comparisons whose mean prints as a
confident, meaningless accuracy. The same stretch inside MSE builds a loss over
every prediction-label pair instead of matched pairs — a warning prints once,
training continues, and the model optimises the wrong quantity.
BCEWithLogitsLoss refuses mismatched shapes outright, which is a gift: the
loud error at the loss line is the same bug that elsewhere stays silent. Confirm
any suspicion the same way: print .shape and .dtype for every tensor at the
boundary, and print the shape of any comparison before taking its mean.
Fix it where it was born, not where it surfaced. Convert once, explicitly, at the
boundary: torch.tensor(features, dtype=torch.float32) for inputs, and labels as
float32 reshaped to (-1, 1) so they match the model's output column. For the
silent accuracy, align shapes before comparing — reshape, or compare
pred.squeeze() against y.squeeze(), but pick one convention and keep it. If a
metric already printed, recompute it after the reshape; the 0.58 was not a worse
model, it was not a measurement at all. Where float64 is the deliberate choice
(2.3's gradient verification), convert the model instead with .double() — the
rule is one dtype per project, chosen on purpose.
Give every project one boundary function that turns raw arrays into
model-ready tensors, and let nothing else call torch.tensor or from_numpy.
Print shapes and dtypes once at that boundary — four lines that end the whole
class of bug. Treat any comparison or loss between tensors of different
dimensionality as a red flag in review, and treat a UserWarning about sizes as an
error. The habit is 0.4's shapes reflex with a new border post: NumPy's flexible
world ends where the model's strict world begins.
Q: "A teammate's binary classifier prints accuracy 0.58 every epoch, but the loss
is falling nicely. What is your first suspicion?" Strong answer: a shape
mismatch in the metric, not the model — a (N,) versus (N, 1) comparison
broadcasts to (N, N) and its mean is meaningless; confirm by printing the
comparison's shape; fix by aligning shapes at the tensor boundary and recomputing.
Bonus: name the dtype cousin (from_numpy float64 meeting float32 weights fails
loudly), note that BCEWithLogitsLoss errors on mismatched shapes while MSE only
warns, and propose the single boundary-conversion function as the lasting fix.