An amount column refuses to behave like numbers. df["amount"].sum() prints a glued
monster like 12080250 instead of 450. mean() raises a TypeError, or
describe() shows count/unique/top/freq instead of mean and std. Sorting puts 1000
before 60. A comparison like df["amount"] > 100 either crashes or matches nonsense
rows. Everything "worked" right up until the first calculation.
Immediately after loading data — read_csv on a bank or UPI settlement file, an
Excel export, or a hand-built list. Indian data files make it worse: amounts arrive
as "₹1,200", "1,20,000" in lakh-style grouping, or with one stray "not paid"
or "NA" in the middle of a numeric column. Linked module: 0.5 (Pandas).
A column has exactly ONE dtype, and one non-numeric value drags the whole column to
text (pandas 3 shows str; pandas 2 shows object — same trap). From then on +
means concatenation, so sum() glues digits; ordering is alphabetical, so "1000"
sorts before "60"; and statistics are meaningless. The two-second diagnostic is
df.info() or df["amount"].dtype right after loading — if a column you believe is
numeric prints str/object, this is your bug. To find WHICH values caused it, run
pd.to_numeric(df["amount"], errors="coerce") and inspect the rows where the result
is NaN but the original is not — those are the culprits, listed for free.
Clean the text first, then convert, then decide about the failures. Strip currency
symbols and separators: df["amount"].str.replace("₹", "").str.replace(",", "").
Convert with pd.to_numeric(..., errors="coerce") so real numbers become float64 and
junk becomes NaN, then COUNT the NaN with .isna().sum() — a handful means a few bad
cells to fill or drop with a stated reason; thousands means the file itself is broken
and the fix belongs at the source. Never loop over rows casting one by one, and never
paper over the column with astype(str) tricks — recover the numbers or reject the
file.
Run df.info() immediately after every load — reading the dtype line is a two-second
habit that catches this before any maths runs. Where columns are known, pass
dtype= hints or converters to read_csv instead of trusting inference. Keep the
rule from module 0.4: money and quantities live in numeric dtypes; formatting with
symbols and commas happens only at display time. And treat any absurd sum — too huge,
glued, or negative — as a dtype alarm first and a data mystery second.
Q: "A fresher reports that the day's UPI total is 12080250 rupees from three small
shops. What happened and what do you do?" Strong answer: recognise digit-gluing as
string concatenation, prove it with df["amount"].dtype, clean symbols and commas,
convert with to_numeric(errors="coerce"), and audit the NaN it produces. Adding the
prevention line — dtype checks at load time, numbers stored as numbers — shows you
have been burned once and built the reflex.