ML system design / case round
Hemanth had a whiteboard and forty-five minutes in a Karimnagar office. The interviewer gave one line: build a system that spots bad seller listings. Hemanth started drawing boxes at once, a queue and a store and a large model. Twenty minutes later the interviewer asked how many listings arrive in a day. Hemanth had never asked. The answer was nine hundred. He had designed for four million. The panel wrote one word on the sheet: assumptions.
- This round has no single right answer, so it cannot be crammed the night before. It scores how you think while the question is still fuzzy.
- One framework covers every case they can hand you, from loan queues to support bots to fraud sweeps. You learn six steps instead of sixty diagrams.
- Every step here is a module you already finished. This is the round where Phase 1 to Phase 4 finally meet on one page.
A system design round — an open question where you design a whole system out loud — opens with one vague line. Build a system that flags risky loans. There is no dataset, no metric and no scale. The interviewer wants to watch you turn that line into a plan.
So the thing being scored is your reasoning trace — the visible path from the vague ask to a working design. Two candidates can draw different boxes and both pass. Two can draw the same boxes and only one passes. The marks come from the questions you asked and the reasons you gave.
Six steps cover every case you will meet. Clarify the ask, then define the ML problem. Plan the data, then pick a baseline and a model path. Design the serving, then close with evaluation, monitoring and failure. Each step is a module you have already done.
Step one is the step freshers skip. Always ask before you design. Four questions carry most of it: who uses this, how many items a day, how fast must the answer come, and what happens when it is wrong. A clarifying question — a short question that pins down one unknown before any design work — costs fifteen seconds and saves twenty minutes.
Step two turns the ask into an ML problem. Name the unit of prediction — the single thing one row of output describes, such as one loan or one ticket. Name the target with a threshold and a window, and name who produces that label. Then run the honest check from module 1.1: is this even ML? If a written rule does the job, say so out loud, because that earns marks rather than losing them.
Step three is data, and it is where designs quietly die. Say where the rows come from, how many arrive per day, and how many carry a label. Name the leakage risk yourself, the way module 1.7 taught, before anyone asks. Then give one privacy sentence: under the DPDP Act, personal data needs a stated purpose and a limit, which module 3.11 covers.
Step four is the baseline and the model path. Start with the dumbest thing that could work and say its likely score. Then give one upgrade, with the reason you would pay for it, using the ladder from module 1.4. A design that opens with a large model and no baseline has skipped the only cheap answer in the room.
Step five is serving. Ask one question first: must the answer arrive in milliseconds, or will a nightly run do? Batch scoring — running predictions on a schedule and storing the results — is cheaper and simpler, and it fits more problems than freshers expect. If it must be live, state the latency budget — the total milliseconds the whole answer is allowed to take — split across each part.
Step six separates a fresher from a hire. Tie the offline metric to a rupee cost, so precision and recall stop being letters. Then say what you will watch daily: input drift, output drift, latency and cost, as module 4.8 set out. Finally tell the failure story, which is what breaks, who sees it first, and what the system does meanwhile.
Here is the contrast this module exists for. The wrong way opens with a tool: I would use a transformer, with a queue and a feature store. It sounds fluent and answers a question nobody asked. The right way spends five minutes asking, writes the answers where the interviewer can see them, and then draws boxes that fit those numbers.
One habit lifts the whole round: name the trade-off before the interviewer does. Precision against recall, priced in rupees. Latency against accuracy, priced in users who leave. Build against buy, priced in months. Module 3.10 gave you the table for the last one, where prompting, retrieval and fine-tuning each carry their own bill.
Halfway through, a good interviewer will change the question on you. Traffic goes up a hundred times. Labels now arrive sixty days late. The model must run on a phone with no network. None of these has one right answer, and each one bends a different joint in your design.
Run the clock on purpose. Roughly five minutes to clarify, five to frame, six for data, seven for the model path, seven for serving, and eight for the last step. Draw while you talk, and keep every assumption written in one corner. When the interviewer corrects an assumption, thank them and change one box, not the whole picture.
There is no stored answer that your diagram is being matched against. The panel is asking one question about you. If we hand this person a fuzzy problem on Monday, will Friday look sane? Everything in this module is practice for that single judgement.
Six worked pieces, not code. Read the block, then read the notes under it. The first block is the checklist you carry into every case; the next three are full cases; the last two are the pressure and the clock.
Piece 1 — the six-step checklist you write on the board first. Learn this until you can draw it in ninety seconds:
THE SIX-STEP DESIGN CHECKLIST
STEP 1 CLARIFY THE ASK ~5 min
Who uses the output, and what do they DO with it?
How many items per day? How many users?
How fast must the answer come back?
What happens when it is wrong? Which mistake hurts more?
-> Write every answer in the top-left corner under ASSUMPTIONS.
The interviewer can now correct you cheaply, which is the point.
STEP 2 DEFINE THE ML PROBLEM ~5 min
Unit of prediction : one what? one loan, one listing, one ticket?
Target : the exact column, with a threshold and a window
Label source : who or what produces that column, and when
Honest check : is this even ML, or does a written rule do it?
-> If a rule does it, SAY SO, then say what would make it ML later.
STEP 3 DATA ~6 min
Where do the rows come from? Which system owns each field?
How many rows, and how many LABELLED rows?
Leakage: is any feature knowable only AFTER the outcome?
Privacy: personal data, stated purpose, retention, who can see it
-> Name one leak you would hunt for. Every real table has one.
STEP 4 BASELINE, THEN MODEL PATH ~7 min
Baseline : the dumbest thing that could work, and its likely score
V1 : the simplest real model, and why it beats the baseline
V2 : ONE upgrade, plus the trigger that would make you pay
-> Never open at V2. The baseline is what makes V1 measurable.
STEP 5 SERVING AND SCALE ~7 min
Batch or real-time? Justify it from step 1's speed answer.
Latency budget: total ms, split across each part of the path
Where the model lives, and what serves it
What happens when that box is down (the fallback)
-> Do the sum out loud: items/day -> requests/second at peak.
STEP 6 EVALUATION, MONITORING, FAILURE ~8 min
Offline metric, tied to a rupee cost per mistake type
Online: what you watch daily, and the alert threshold
Drift: inputs, outputs, and the label delay that hides it
Failure story: what breaks, who notices, what the system does
-> The step freshers never reach. Protect the time for it.
TOTAL 38 min of design + ~7 min for their questions = 45.
The checklist is deliberately made of questions, not answers, because the answers change with every case and the questions never do. Steps 1 and 2 are the cheap half and they decide the round: a wrong unit of prediction makes every later box wrong, however well you draw it. Step 3 is where honest candidates separate themselves, since naming your own leakage risk is a thing only somebody who has been burnt does. Steps 5 and 6 are the half that most freshers run out of time for, which is exactly why the minute budget exists. Write the six headings on the board before you say anything else, and the interviewer already knows this is not your first design.
Piece 2 — case (a), the classical one, worked end to end. The ask is one line: "design a system that decides which loan applications need a human reviewer."
CASE A - LOAN APPLICATIONS THAT NEED MANUAL REVIEW (an NBFC)
STEP 1 - CLARIFY (what I asked, and what I was told)
Q: Who acts on the output? A: A 12-person credit ops team.
Q: How many applications a day? A: About 4,000 on weekdays.
Q: How many can they review? A: About 600 a day. Hard limit.
Q: How fast must it answer? A: The applicant is in the app, so
under 2 seconds.
Q: Cost of each mistake? A: Missing a bad loan costs the
principal, near Rs 80,000. A
needless review costs about 15
minutes of an officer's time.
ASSUMPTIONS ON THE BOARD: 4,000/day - 600 review slots - 2 s budget
- a miss costs roughly 1,000x a false alarm in rupees.
>>> The 600-slot answer changed the problem. This is not "predict
default". It is "rank 4,000 and hand over the worst 600".
That re-framing came from ONE question, asked in minute two.
STEP 2 - DEFINE THE ML PROBLEM
Unit of prediction : one application
Target : defaulted = 1 if 90+ days past due within 12
months of disbursal (the definition the risk
team already uses, module 1.1)
Label source : the collections system, about 12 months later
Problem type : RANKING by predicted risk, not a yes/no
classifier, because capacity is fixed at 600
Is it even ML? : partly. Hard policy checks - age, missing KYC,
PAN mismatch - are rules and go BEFORE the
model. The model only orders what survives.
STEP 3 - DATA
Sources : application form, credit-bureau pull, bank-statement
parse, app telemetry, past repayment history
Volume : ~4,000/day, ~1.2 M applications a year, but ONLY
disbursed loans ever get a label
Leakage : (1) any field written after disbursal, such as
collections notes; (2) "number of reminder calls" is a
post-outcome field; (3) the bureau pull timestamp must
sit before the decision timestamp
Bias : we only see repayment for approved loans, so the model
learns from a filtered population. Say this out loud;
it is the honest limit of the whole design.
Privacy : PAN and bank statements are personal data. Stated
purpose, masked in logs, retention agreed with legal.
STEP 4 - BASELINE, THEN MODEL PATH
Baseline: sort by bureau score alone, take the worst 600. Cheap,
explainable, already catches something. Measure it: what
share of eventual defaults sits inside that 600?
V1 : logistic regression on ~25 features, with the policy
rules in front. Reason: an NBFC must be able to explain
a decision to a customer and to an auditor.
V2 : gradient boosting, IF V1's capture rate is clearly short
of the baseline plus a useful margin. Cost: harder to
explain, needs SHAP for the credit committee.
I would NOT start at V2. If V1 barely beats a bureau score, the
problem is the data, and a bigger model will not fix data.
STEP 5 - SERVING AND SCALE
Arithmetic: 4,000/day over ~10 busy hours = ~0.11 requests/second,
peak maybe 1/s. This is a tiny load. SAY THAT - it
stops you from designing a queue you do not need.
Choice : real-time scoring inside the application flow, because
the applicant is waiting. One small service is enough.
Latency : 2,000 ms budget = 300 ms bureau call (cached),
50 ms feature lookup, 20 ms model, the rest is slack.
Fallback : if the model service is down, fall back to the bureau-
score baseline and mark the batch for a later re-score.
Never block a loan application on a model being up.
STEP 6 - EVALUATION, MONITORING, FAILURE
Offline : recall of defaults inside the top 600 (capture at 600),
and its rupee twin - expected loss caught minus review
cost. NOT accuracy; defaults are a small share of rows.
Online : daily score distribution, share of applications hitting
each policy rule, review-queue length, decision latency.
Drift : labels are 12 months late, so default cannot be measured
weekly. Proxies: 30-day-past-due rate on recent cohorts,
plus input drift on income and bureau score.
Failure : (1) festival spike overflows the queue -> cap the queue
and let the overflow fall to the baseline rule;
(2) a new sourcing channel shifts the input mix -> input
drift alert, then re-score a held-out slice;
(3) a bureau field is renamed and arrives null -> per-
feature null-rate alert; fail loudly, never quietly.
Read step 1 again, because that is where this case was won. One question about review capacity turned a classification problem into a ranking problem with a fixed budget, and every later box follows from it. Step 3's bias line is worth copying into your own answers: the model only ever sees repayment for loans that were approved, so it is trained on a filtered slice of the world. Step 5's arithmetic is the cheapest mark in the round, since 4,000 a day is about one request every nine seconds, and saying so proves you size things before you build them. Step 6 does the work that separates candidates, because the label being twelve months late is a design constraint, not a footnote, and the proxy metrics are the answer to it.
Piece 3 — case (b), the GenAI one, worked end to end. The ask: "design an assistant that answers our support agents' questions from our own documents."
CASE B - A SUPPORT-TICKET ASSISTANT OVER COMPANY DOCUMENTS
(a telecom operator, 900 agents)
STEP 1 - CLARIFY
Q: Who uses it? A: Support agents on a live call, not
customers directly.
Q: How many queries? A: 900 agents, about 30 lookups each per
day = about 27,000 queries a day.
Q: How fast? A: A customer is on the line. Under 3 s to
the first useful line of text.
Q: What is "wrong"? A: A confident wrong policy answer. The
agent repeats it, and the company is
held to it.
Q: What documents? A: About 8,000 pages - plans, tariffs,
refund policy. Content changes weekly.
Q: Any past answers? A: Four years of resolved tickets with the
agent's final reply.
ASSUMPTIONS: 27k queries/day - 3 s - 8k docs - weekly content
churn - a wrong answer costs far more than "I could not find it".
STEP 2 - DEFINE THE ML PROBLEM
This is not a classifier. It is retrieval plus generation.
Unit of prediction : one agent question
Output : a short answer plus the exact source lines
Target? : there is no column to fit. Success here is
judged, so my "target" is an eval set, not a
label column. Say this; it surprises people.
Honest check : for the 50 most repeated questions, a curated
answer card is better, cheaper and safer.
Route those to cards. RAG earns its cost on
the long tail, not on the head.
STEP 3 - DATA
Corpus : 8,000 documents. Chunk by heading; keep the source URL,
the effective date and the product it applies to.
Freshness: content changes weekly, so a scheduled re-index and a
"valid from" filter at query time are part of the design.
Eval set : 200 real agent questions with a human-checked answer
and the correct source page, drawn from the archive.
Leakage : if the eval questions come from tickets that are also
indexed, the system finds the answer word for word and
I over-score myself. Hold the eval tickets OUT of the
index. This is 1.7 wearing GenAI clothes.
Privacy : tickets carry customer numbers and addresses. Mask
personal fields BEFORE indexing. Under DPDP the stated
purpose is agent support, not model training.
STEP 4 - BASELINE, THEN MODEL PATH
Baseline : keyword search (BM25) over the same chunks, top 5 shown
raw to the agent, no generation at all. Measure hit
rate at 5 on the 200-question set. Some teams stop here
and are right to.
V1 : embeddings + vector search, top 5 chunks, one grounded
answer with citations, and a refusal path when the best
score is below a threshold.
V2 : hybrid keyword + vector with a reranker, ONLY if the
eval shows retrieval misses are the main error type.
Fine-tune? No. The content changes weekly and fine-tuning bakes it
in, while retrieval keeps it swappable. That is the 3.10 decision
table applied, not recited.
STEP 5 - SERVING AND SCALE
Arithmetic: 27,000/day over an 8-hour shift = ~0.9 queries/second
average, peak maybe 5/s. Small for the vector store,
real money for the generator.
Latency : 3,000 ms budget = 40 ms embed the query, 60 ms vector
search, 120 ms rerank if used, ~1,200 ms generation,
rest is slack. Stream tokens so the agent reads while
the answer is still being written.
Cost : 27k generator calls a day is the number to price.
Cache repeated questions; the answer cards take the
head of the distribution for free.
Where : managed vector index, generator behind our own
gateway, so API keys never sit in an agent's browser.
STEP 6 - EVALUATION, MONITORING, FAILURE
Offline : on the 200-question set - retrieval hit rate at 5,
groundedness (is every claim inside a cited chunk), and
refusal correctness (did it refuse when it should have).
Rupee : one wrong policy answer costs a credit note plus a
call-back. A refusal costs about 90 seconds of the
agent's time. So I tune the threshold to prefer refusing.
Online : citation rate, refusal rate, thumbs-down per agent, p95
latency, cost per day, and the share of questions with
no chunk above threshold. That last number is the
content-gap list, and it is the most useful report this
system will ever produce.
Drift : new tariff plans appear and the index goes stale. Alert
on a rising "no good chunk" rate, and on citations that
point at documents past their valid-to date.
Failure : (1) generator API down -> show the top 5 chunks with no
generated answer; the agent can still read;
(2) prompt injection inside an indexed document -> strip
instruction-shaped text at index time and never let
retrieved text alter the system prompt;
(3) invented policy -> every answer carries citations and
the interface shows them first, so the agent can check.
The move that scores here is step 2's admission that there is no label column, because most candidates try to force RAG into a classifier shape and lose the thread. The answer-card routing is the same "is this even ML?" check from case A, applied to the head of the question distribution rather than to the whole problem. Step 3's eval leakage is the trap that catches good students: build your eval set from tickets you also indexed and your hit rate looks wonderful and means nothing. In step 4 the refusal to fine-tune is not a preference, it is an argument from the weekly content churn you were told about in step 1, which is what "grounded in the constraints" means. Step 6's content-gap report is worth memorising, since a system that tells the business what its documents do not cover is often worth more than the answers themselves.
Piece 4 — case (c), the ambiguous one, where rules beat ML. The ask: "detect fraudulent seller listings on our marketplace."
CASE C - FRAUDULENT SELLER LISTINGS. THE FIRST ANSWER IS NOT ML.
STEP 1 - CLARIFY (and the answers that change everything)
Q: New listings a day? A: About 900.
Q: How many are fraud? A: Nobody knows. Last month the trust
team removed 11.
Q: How do you find them? A: Buyer complaints, mostly after the
money is gone.
Q: What does fraud mean? A: Three different things - fake brand,
price bait, and sellers pushing
buyers to pay off-platform.
Q: Who acts on the output? A: A 3-person trust team.
Q: Cost of a mistake? A: Pulling a genuine listing costs a
seller and a support fight. Missing
one costs a refund and trust.
ASSUMPTIONS: 900/day - about 11 known frauds a month - 3 reviewers
- no agreed definition - no label pipeline at all.
STEP 2 - DEFINE THE ML PROBLEM, AND THE HONEST ANSWER
Try to write the target: fraud = 1 if... if what? There are three
behaviours here, not one, and about 11 positive examples a month.
So I say it plainly:
"I cannot train a supervised model on 11 labels a month, and I
should not average three behaviours into one column. Version 1
of this system is rules, not ML."
What version 1 actually is:
- three separate problems, handled separately
- a rule set per behaviour, written WITH the trust team
- a review queue and a decision log
The design goal of version 1 is to MANUFACTURE LABELS, not to
predict. In six months this same design has thousands of judged
rows, and then ML has something to learn from.
STEP 3 - DATA (and how version 1 creates the missing part)
Available now : listing text and images, price against the
category median, seller account age, KYC status,
payout history, on-platform message text, device
and network signals.
Missing now : a label. That is the real blocker, not the model.
The decision log gives, per reviewed listing: which rule fired,
the reviewer's verdict, the reason code, the date.
Leakage, later: "removed by trust team" is an outcome, and it will
be sitting in the same table as the features. Flag it now.
Privacy : seller KYC and buyer messages are personal data.
Reviewers see the minimum needed, and every access is logged.
STEP 4 - BASELINE, THEN MODEL PATH
V0 (week 1) : SQL rules, one per behaviour. Price under 40% of
the category median from a seller under 14 days
old. A phone number in the listing text from a new
seller. An image hash matching a brand catalogue.
Output: a scored queue for the 3 reviewers.
V1 (month 3) : rules stay, and their outputs become features. Once
the log holds a few thousand judged listings, a
gradient-boosted ranker orders the queue better
than a rule count does. Same queue, better order.
V2 (later) : one model per behaviour, plus text embeddings for
the off-platform push, which is a language problem.
The judgement to state out loud: the rules are not a stepping
stone I am embarrassed by. With 900 listings a day and only 3
reviewers, THE QUEUE ORDER IS THE PRODUCT. A model that reorders
a queue nobody can work through has changed nothing.
STEP 5 - SERVING AND SCALE
900 listings a day is one every 96 seconds. There is no scale
problem here, and saying so is worth a mark.
Choice: batch scoring every 15 minutes. Nobody is waiting - the
reviewer works a queue, the seller does not sit on a screen.
Name the trigger that would change this: if the platform later
wants to HOLD a listing before it goes live, the seller IS
waiting, and it becomes a real-time path with a latency budget.
Design the joint, do not just pick the side.
STEP 6 - EVALUATION, MONITORING, FAILURE
V0 has no ROC curve, and that is correct. Measure it as an
operations system:
- queue precision: what share of reviewed listings were acted
on? A 3-person team can live with roughly 1 in 3.
- coverage: of the frauds later found by complaints, how many
had been in the queue and were passed over?
- queue age: anything older than 24 hours is a design failure,
not a modelling one.
Monitoring: rule fire rates per day. A rule firing 400 times is
not a rule, it is a filter, and it buries the reviewers.
Failure : (1) sellers learn the price rule and sit just above the
line -> adversarial drift, so rotate thresholds and never
publish them; (2) a category-wide promotion drops prices
legitimately and floods the queue -> use a rolling per-category
median; (3) reviewers stop writing reason codes -> the label
pipeline dies silently and version 1 never arrives, so make the
reason field mandatory from day one.
Case C is in this module because the honest answer disappoints people, and saying it anyway is the whole test. Eleven labelled examples a month cannot train anything, and a candidate who proposes a deep model over that data has told the panel they do not check whether the data exists. The reframing in step 2 is the reusable idea: when labels are missing, version 1's job is to manufacture labels, and the review queue is the machine that makes them. Step 4's sentence about the queue order being the product is the kind of line that gets remembered, because it puts the three reviewers, not the algorithm, at the centre of the design. Step 6 then shows you can evaluate a rules system properly, which many candidates believe is impossible.
Piece 5 — the mid-round changes, applied to case A. The interviewer is not testing memory. She is testing whether your design has joints that bend:
THREE CURVEBALLS, APPLIED TO CASE A (the loan review queue)
CHANGE 1 - "Now it is 400,000 applications a day, not 4,000."
Say what does NOT change first: the target, the label source, the
metric, and the review-capacity framing. That shows the design was
built on the right thing.
What DOES change:
- the sum: 400k/day over 10 hours = ~11 requests/second, maybe
40/s at peak. Still not enormous. Do the arithmetic out loud
BEFORE reaching for a message queue.
- the bureau call at 300 ms x 400k is now a cost and a rate-limit
problem. Cache hard, batch the pulls, add a circuit breaker.
- feature lookups move from a database read to a key-value store
holding precomputed aggregates.
- review capacity does NOT scale with traffic. 600 slots against
400,000 applications means the ranking must be far sharper, and
one global slot count becomes a per-segment budget.
The honest closing line: "at this scale the model is not my
bottleneck - the bureau vendor and the review team are."
CHANGE 2 - "Labels now arrive 60 days late, not 12 months."
This one is a gift, and you should say so.
- retraining cadence moves from yearly to monthly.
- a real early metric on recent cohorts becomes possible, so the
proxy monitoring from step 6 gets a proper backstop.
- BUT a 60-day label is a DIFFERENT target: 60-day delinquency is
not 12-month default. I would keep both - the fast one for
monitoring and retraining signals, the slow one for the final
model - and check how well they correlate before trusting the
fast one.
The trap: quietly swapping the target and not saying so. Naming
the swap is the mark; making it silently is the failure.
CHANGE 3 - "It must run on the field officer's phone, offline."
Now the constraints invert.
- no bureau call, because there is no network. The strongest
feature is gone. Say that loss FIRST, before the engineering.
- the model must be small: a compact tree model or a logistic
regression on a handful of on-device fields, shipped with the
app.
- the model file is now a release artefact with a version, and
old app versions will keep running an old model for months. So
log the model version with every prediction.
- offline means no fresh features and no live logging. Predictions
sync when the phone reconnects, so drift is detected days late.
- privacy changes: personal data now sits on a device. Encrypt at
rest and keep the raw application form off the phone.
The judgement: on-device scoring with the bureau feature removed
may be WORSE than sending the application for a 5-minute batch
decision. Ask whether that path is truly unavailable before you
accept the constraint. Interviewers respect a pushed-back premise
far more than a design that silently gets worse.
Every change here follows the same three moves, and that pattern is what you practise: say what does not change, do the arithmetic, then move only the boxes the new constraint actually touches. Change 1 punishes the reflex to reach for heavy infrastructure, because eleven requests a second is still a small number and admitting that is stronger than naming five tools. Change 2 is really a framing question in disguise, since a shorter label delay quietly redefines the target, and the candidate who notices that is the candidate who read module 1.1 properly. Change 3 is the hardest because it removes your best feature, and the professional response is to price the loss out loud rather than to pretend the design survives untouched. Notice the last line: asking whether a constraint is real is allowed, and it usually scores.
Piece 6 — the 45-minute clock, with what to say at each stage. Rehearse this until the timings are automatic:
THE 45-MINUTE CLOCK - WHAT TO SAY, AND WHEN TO STOP SAYING IT
00:00-00:05 CLARIFY
Say: "Before I design anything, I want to check a few things."
Ask 4-6 questions. Write every answer in the top-left corner
under the heading ASSUMPTIONS. Draw no boxes yet.
If they say "you decide": state the number yourself - "I will
assume 5,000 a day" - and write it down. An assumed number that
is visible beats no number at all.
00:05-00:10 FRAME
Unit of prediction, target with its window, label source, and
whether this is ML at all. One sentence each, no more. Then name
the shape: "so this is a ranking problem with fixed capacity."
00:10-00:16 DATA
Sources, volume, labelled share. Then say the sentence that wins
marks: "the leakage I would go looking for is X." Then one
privacy sentence. Raising leakage before being asked is the
cheapest signal of experience available to you.
00:16-00:23 BASELINE AND MODEL PATH
Draw nothing complicated. Baseline and its expected score, then
V1 and the reason, then V2 and the trigger. The sentence frame:
"I would start with ... because ... and I would only move to ...
if ..."
00:23-00:30 SERVING
Now draw boxes. Do the requests-per-second sum out loud. State
batch or real-time and why. Give the latency budget as a split,
never as one number. Draw the fallback path too.
00:30-00:38 EVALUATION, MONITORING, FAILURE
Offline metric with a rupee attached. Three things you watch
daily. Two ways it breaks and what happens then. This is the
block most candidates never reach, so protect the time for it.
00:38-00:45 THEIR QUESTIONS, THEN YOUR WRAP
Close with 30 seconds: "the two risks I would worry about most
are the label delay and the review capacity, and the first thing
I would build is the queue, not the model."
TIME RULES THAT SAVE ROUNDS
- Still clarifying at minute 8? Stop and design.
- Still on the model at minute 25? Jump to serving.
- Never end with steps 5 and 6 unspoken. If the clock is short,
compress step 4 and say so out loud: "I will move to serving so
that we cover monitoring - happy to come back to the model."
- If you go quiet for more than ten seconds, narrate instead:
"I am deciding between batch and real-time here, and the thing
that decides it is whether the applicant is waiting."
The clock exists because the round has a predictable failure mode: candidates spend twenty-five minutes on the model, which is the part they enjoy, and never reach monitoring. Minutes 0 to 5 feel wasted while you are in them and pay for themselves every single time, which is why the instruction is to draw no boxes at all in that window. The "you decide" recovery is worth rehearsing on its own, since a stated assumption written on the board converts a stuck moment into a scoring one. The narration rule at the end handles the other common failure, silence, because an interviewer cannot give marks for thinking they cannot hear. Practise the wrap sentence until it is automatic, as the last thirty seconds are what the panel writes down.
Work in a folder called design-round with four files: frames.md for your six-step write-ups, designs/ for full case artefacts, rubric.md for the grading sheet, and curveballs.md for the changes you were thrown and how your design bent. Grade every attempt on the twelve-mark rubric below, including when you are grading yourself and would rather not.
The twelve-mark rubric, two marks per step: (1) clarify — asked before designing, wrote assumptions visibly; (2) frame — unit of prediction, target with window, label source, and the is-this-ML check; (3) data — sources, volume, labelled share, one named leak, one privacy sentence; (4) path — baseline with expected score, V1 with a reason, V2 with a trigger; (5) serving — the requests-per-second sum, batch-or-live with a reason, latency split, fallback; (6) close — metric tied to rupees, three daily monitors, two failure stories with responses.
The clarifying opener, drilled across five asks. Write in frames.md. Take these five one-line asks: (a) "predict which electricity connections will not pay this month" for a state discom; (b) "recommend kirana items to re-order" for a distributor app; (c) "read scanned KYC documents and pull out the fields" for a bank's onboarding team; (d) "tell us which delivery riders are about to quit" for a logistics firm; (e) "answer employee HR questions from our policy PDFs". For each ask, write the six clarifying questions you would ask in minute one, and then write a plausible set of answers as if the interviewer gave them, including one number for volume and one number for speed. Then, for each, write the single sentence that the answers change — the way the 600-slot answer turned case A into a ranking problem. Finish with steps 1 and 2 only for all five: unit of prediction, target with threshold and window, label source, and your is-this-ML verdict with a reason. Check: all five asks have six questions each, and at least four of the six differ across asks rather than being copied; every ask carries an invented but plausible volume number and speed number, written as assumptions; each ask has one "this changes the problem" sentence naming which step it changes; all five have a unit of prediction, a target with both a threshold and a time window, and a named label source; and at least one of the five is marked as not-ML-first with a two-sentence defence.
Re-run the three cases from memory, then diff against the lesson. Close this lesson. In designs/, write your own six-step design for all three cases — the loan review queue, the support-ticket assistant, and the seller listings — from memory, at about half the length of the worked versions. Give each one a real leakage risk, a real privacy sentence, an arithmetic line converting items per day into requests per second, and two failure stories. Only then reopen the lesson and mark, in a different colour, every point the worked version made that you missed and every point you made that it did not. For each miss, write which of the six steps it belonged to. Finish with a tally of your misses by step, since that tally is your revision plan. Check: three design files exist, each with all six steps and none of them copied from the lesson; every design has an explicit requests-per-second calculation with the arithmetic shown; every design names at least one leakage risk and one privacy constraint; your miss list is tagged by step number and totalled, and you state which step you lose most marks in; and you name at least two points you made that the lesson did not, with a sentence on whether each is right.
The timed solo design: 45 minutes, one fresh prompt, one artefact. Set a timer for 45 minutes and do not stop it. Your prompt: "an IRCTC-scale ticketing platform wants to detect and block bot accounts that grab tatkal tickets." You have no interviewer, so write the clarifying questions and then answer them yourself as assumptions, marked clearly as assumptions. Follow the clock from piece 6 exactly: five minutes clarify, five frame, six data, seven path, seven serving, eight close, and stop writing at minute 38. Produce one artefact in designs/bots.md with the six steps as headings, a drawn or described box diagram in the serving section, and the assumptions listed at the top. When the timer ends, grade yourself on the twelve-mark rubric and write one sentence of evidence for every mark you refuse yourself. Check: the artefact has all six headings and was finished inside 38 minutes, with the actual finish time recorded; the assumptions block sits at the top and holds at least one volume number, one speed number and one cost-of-mistake statement; step 4 names a non-ML baseline and states what it would catch; step 6 has a metric with a rupee cost, three daily monitors and two failure stories with responses; and your self-grade totals out of 12 with written evidence for every refused mark, plus the one step you would redo first.
The pair interview, with a curveball list, then swap. Pair with a classmate. One of you designs for 30 minutes on a prompt the other picks from these: a crop-advisory assistant for a seed company, a duplicate-claim detector for a health insurer, a delay predictor for an intercity bus operator, or a KYC document checker for a small bank. The interviewer's job is strict: give only the answers asked for, invent consistent numbers and write them down, stay silent otherwise, and at minute 18 throw one change from this list — 100x the traffic, labels now arrive 90 days late, it must run on-device with no network, the review team shrinks from ten people to two, or the business now wants an explanation for every decision. Score the design on the twelve marks as it happens, with one line of evidence per refused mark. Then swap roles, use a different prompt and a different change, and finish with 10 minutes comparing the two transcripts. Check: curveballs.md records both rounds with the prompt, the invented numbers, the change thrown and the minute it landed; both designs are scored out of 12 with evidence for every refused mark; for each round you record what the designer said did NOT change after the curveball, and whether that was correct; you name which chair taught you more and why, in three sentences; and each of you writes the one habit that cost the most marks, quoting your own words from the transcript.
Three design-round failures, all common and all fixable with the six steps. Each has a War Room entry with the full diagnosis path:
- WR-160 — the candidate who never asked a question. Forty minutes of confident design against assumptions the interviewer never made; the fix is the clarifying opener, with the exact questions to ask in minute one and where to write the answers.
- WR-161 — the design with no failure story. A tidy architecture with no metric, no monitoring and no answer to "what happens when it is wrong"; the fix is treating steps 5 and 6 as compulsory and protecting their minutes on the clock.
- WR-162 — the over-engineered fresher answer. A queue, a feature store and a fine-tuned model proposed for a problem one SQL rule would solve; the fix is the simplest thing that works first, then one upgrade with a named trigger.
The design round hands you a vague line and scores the path you take from there, not the diagram you end with. Six steps cover every case: clarify the ask, define the ML problem, plan the data, set a baseline and a model path, design the serving, then close with evaluation, monitoring and failure. Ask before you design, write your assumptions where the interviewer can see them, and check whether a written rule would do the job before reaching for a model. Do the arithmetic out loud, price each mistake in rupees, and always leave time for what breaks and what the system does then. When the interviewer changes the question, say what does not change first, then bend only the boxes the new constraint touches.
- Banks and NBFCs hiring risk and analytics freshers run this round on their own products, often a loan queue or a fraud sweep. The capacity question from case A is real work there, because a review team of twelve is a hard limit that no model removes.
- GCC and product teams building internal assistants use case B almost word for word, since most large Indian employers now have some document assistant in flight. Naming the eval set, the refusal path and the content-gap report separates a candidate who has built one from a candidate who has read about one.
- Marketplace, logistics and quick-commerce trust teams live in case C. A design that starts with rules, builds a review queue and treats the decision log as the label pipeline matches how these teams actually start.
Answer these aloud before opening the gate assessment. Each question names its kind.
type=interpretation— A candidate spends minutes 0 to 25 on model choice, reaches serving at minute 30, and never mentions monitoring. The panel's note says "strong on models, would not let him own a system". Interpret what the note is actually about, and name the two steps whose absence produced it.type=scenario-judgement— You ask how many rows arrive per day and the interviewer replies "you decide". Give your ruling on what you do next, the exact sentence you say, and justify it against what the round is measuring.type=trade-off— For the seller-listing case, compare batch scoring every 15 minutes against a real-time service that scores a listing before it goes live. Give one concrete gain and one concrete cost of each, then name the single business change that would flip your choice.type=debug— A candidate designs a RAG assistant and builds the eval set from the same support tickets that were indexed. Retrieval hit rate reads 0.94. Name the flaw, predict which direction the true number moves, and give the one change that fixes the measurement.type=output-prediction— A candidate proposes fine-tuning a model on company policy documents that change every week. Predict the interviewer's next two questions, what a weak candidate answers to each, and what the round note will say.
Explain to a friend from a non-technical degree what happens in a system design interview, and why the person with the neatest diagram is not always the one who gets the offer. Use an everyday comparison, such as a contractor who is asked to build a house and starts by asking how many people will live in it and what the budget is. Then tell your friend the first thing you always do before designing, and the last thing you always cover before the time runs out. Use at most 4 sentences and no word your friend would need to look up.