np.exp(1000) prints inf with RuntimeWarning: overflow encountered in exp.
A product of thousands of probabilities prints exactly 0.0, so every candidate
gets the same score and your ranking is meaningless. Downstream, divisions start
producing nan, and a dashboard that worked on small data breaks on the full
dataset with no exception anywhere.
Scoring pipelines that multiply many per-event chances (fraud, spam, churn), softmax or sigmoid code fed large inputs, and first likelihood computations. Typically appears the day the input grows from a 50-row sample to the real lakh-row table. Linked module: 0.7 (The maths you will actually use).
A float64 has hard limits: the largest value is about 1.8e308 and the smallest
positive value is about 5e-324. exp(1000) is around 1e434 — past the ceiling —
so NumPy stores inf and warns (overflow). Multiplying thousands of numbers
below 1 sinks the result under the floor, where it flattens to 0.0 with no
warning at all (underflow is silent). Once inf or a false 0.0 enters, the
damage spreads quietly: inf - inf is nan, and 0.0 divides into nonsense.
Confirm the value really existed: np.log of the collapsed product gives -inf,
but summing the logs of the factors gives a healthy finite number — proof the
true answer was real and only the container failed.
Work in log space. Replace p.prod() with np.log(p).sum() — because
log(a×b) = log(a) + log(b), the sum of logs carries the same information without
ever leaving float range. Since log preserves order, you can rank and compare
log-scores directly; only convert back with np.exp if a human needs the raw
number, and often nobody does. For exp overflow, subtract the maximum first:
np.exp(x - x.max()) — the standard log-sum-exp trick that softmax
implementations use. Prefer library versions (scipy.special.logsumexp) over
hand-rolling.
Treat any product of many probabilities as a log-sum by default — decide to
multiply only when you can bound the count. Never silence RuntimeWarnings; they
are the only loud symptom overflow gives you. Sanity-print arr.min() and
arr.max() of intermediate arrays when scaling up. Remember the asymmetry:
overflow warns, underflow does not — a suspicious exact 0.0 is your only clue.
Q: "Your likelihood score prints 0.0 for every single candidate — what happened
and what is the standard fix?" Strong answer: names underflow and the float64
floor, explains that multiplying thousands of sub-1 numbers escapes float range,
gives the log-space fix (np.log(p).sum()), and notes log preserves ordering so
ranking still works. Mentioning log-sum-exp for the matching overflow case shows
you know both edges of the number line.