Serving with FastAPI
Sanjana works at an NBFC in Udaipur. The churn repo from module 4.1 sits on her laptop, clone-tested and clean. Then the app team calls. Their Android app must warn field agents about risky customers, and it speaks only HTTP. Send us a URL, they say. We POST a customer, you return a verdict. She offers the repo; they cannot run Python inside an app. The model works, yet nothing else can use it. A model becomes a product on the day other software can call it.
- Every model you have built so far can answer only you. At work a model earns its keep when apps, dashboards and other teams can call it over HTTP — this module builds that door.
- Interview panels test the server side: "what does your API return on bad input" separates engineers from tutorial followers. After today you answer with measured numbers, not guesses.
- Serving appears in most Indian ML and GenAI job posts, and FastAPI is the tool they name. The /predict service you build today is the same shape bank and GCC teams run.
Module 3.2 put you on the client side of an HTTP call: your script sent a request and read a reply. Today you cross the desk and become the server. A web framework — a library that handles the HTTP plumbing so your code only supplies the answers — makes this practical. Ours is FastAPI, and uvicorn is the small program that runs it and listens on a port. Everything Aisha sent from Lucknow in 3.2, some other program will now send to you.
A served model has few moving parts. A route — one URL path plus a method that your app answers — is the unit of service, like POST /predict. The request body carries a customer as JSON, and the reply carries the verdict as JSON. A status code — the three-digit number that names the result of a request — rides on every reply. The caller reads 200 as success, 422 as rejected input, 400 as a broken rule, and 500 as your fault.
The door of the service is a contract. A schema — the declared shape of the input: field names, types, and allowed ranges — is written once as a pydantic class. FastAPI checks every request against it before your function runs. Bad input never reaches the model, and the caller gets a 422 that names the failing field. This is the validate-then-execute idea from 3.8 on the receiving side, and the cure for WR-060's schema drift.
Here is the contrast that decides whether your service can be trusted. The wrong way reads the request as a raw dict and passes it straight to the model. Columns in a new order, income sent in thousands, a word where a number belongs — the model still answers, and the number is nonsense. The right way puts the schema at the door: wrong shapes die there with a 422, loudly and at once. A fast, loud no protects the caller better than a quiet wrong yes.
Loading the model is the second trap. The wrong way calls joblib.load inside the endpoint, so every request pays the load time again. The right way loads the model once at startup and keeps it in memory, so each request pays only prediction time. On the small churn model that is 138 milliseconds against 9; on a large model it is 30 seconds against 50 milliseconds. A cold start — the slow first stretch after a process boots, before models and caches sit in memory — should be paid once, never per request.
Failures deserve design, because callers meet them more often than you expect. Wrong shape earns a 422 from the schema, with no code of yours involved. Right shape but wrong meaning — an order date after the snapshot — earns a 400 with a short reason from your own check. Your own bug becomes a 500, and the caller must see a generic message while the full story goes to your logs. A stack trace names file paths, library versions and column names — a map of your insides handed to a stranger.
Deployers need two more doors. A health check — a tiny endpoint that reports whether the service is alive and ready — lets a machine ask "are you fine" every few seconds. Load balancers and Docker, which arrives in 4.3, restart or bypass a service whose health check fails. FastAPI also reads your routes and schema and writes a live reference page at /docs on its own. That is the same kind of page you read as an outsider in 3.2 — now other people read yours.
The loop closes with measurement. When the service runs, you call it with the requests library, the way you called a server in 3.2 — the student who called APIs now answers them. Time many requests, report the middle and the worst, and keep the cold first call separate from the warm rest. The 4.1 clone test also gains a serving twin. The module is done when a teammate can POST a customer and read a verdict without ever opening your code.
Work inside churn-repo from module 4.1, venv active. We build the hello world first, then grow the real service file by file.
Step 1 — the smallest possible server. One file, one route:
# hello.py — your first server-side program
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def home():
return {"message": "Namaste from Udaipur", "service": "hello"}
There is no loop, no socket code, no JSON encoding — the web framework does all of it. You declare a route with a decorator, return a dict, and FastAPI turns it into a JSON reply with a 200 status code.
Step 2 — install, run, call. New pins go into requirements.txt first, the 4.1 way, then uvicorn brings the file to life:
(.venv) $ pip install fastapi uvicorn pydantic requests
Successfully installed fastapi-0.115.12 uvicorn-0.34.2 pydantic-2.11.4 ...
(.venv) $ uvicorn hello:app --reload
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Application startup complete.
# a second terminal — the server owns the first one now:
$ curl http://127.0.0.1:8000/
{"message":"Namaste from Udaipur","service":"hello"}
hello:app means "the object named app inside hello.py", and --reload restarts the server on every file save — development only, never production. The server keeps running until you stop it, which is the point: a service waits. Now open http://127.0.0.1:8000/docs in a browser — FastAPI has already written an interactive reference page by reading your routes. Every endpoint you add below appears there without any extra work.
Step 3 — the churn service. Create app/__init__.py (empty) and app/main.py inside churn-repo. This one file carries the whole module — read it top to bottom:
# app/main.py — the churn repo grows a serving layer
import logging
from contextlib import asynccontextmanager
from datetime import date
from typing import Literal
import joblib
import pandas as pd
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
import config
from src.features import add_features
log = logging.getLogger("churn-service")
state = {}
@asynccontextmanager
async def lifespan(app: FastAPI):
state["model"] = joblib.load(config.MODEL_PATH) # ONCE, at process start
log.info("model loaded from %s", config.MODEL_PATH)
yield
state.clear()
app = FastAPI(title="Churn service", version="1.0", lifespan=lifespan)
class CustomerIn(BaseModel):
model_config = {"extra": "forbid"} # unknown fields -> 422
city: Literal["Ludhiana", "Jalandhar", "Amritsar", "Patiala", "Bathinda"]
monthly_income: float = Field(gt=0, le=1_000_000) # rupees per month
orders_last_90d: int = Field(ge=0, le=270)
avg_order_value: float = Field(gt=0, le=100_000)
last_order_date: date
class VerdictOut(BaseModel):
prediction: int
churn_prob: float
@app.get("/health")
def health():
return {"status": "ok", "model_loaded": "model" in state}
@app.post("/predict", response_model=VerdictOut)
def predict(customer: CustomerIn):
if customer.last_order_date > config.SNAPSHOT_DATE.date():
raise HTTPException(400, "last_order_date is after the snapshot date")
row = pd.DataFrame([customer.model_dump()])
row["last_order_date"] = pd.to_datetime(row["last_order_date"])
prob = state["model"].predict_proba(add_features(row))[0, 1]
return {"prediction": int(prob >= 0.5), "churn_prob": round(float(prob), 2)}
@app.exception_handler(Exception)
async def unhandled(request: Request, exc: Exception):
log.exception("unhandled error on %s", request.url.path)
return JSONResponse(status_code=500,
content={"detail": "internal error — the team has the log"})
Run it with uvicorn app.main:app from the repo root. Walk the layers. The lifespan block runs once when the process starts: the joblib file from 4.1's train.py loads into memory and stays there — the log prints one "model loaded" line per process, never per request. CustomerIn is the contract: types, ranges with real-world meaning, the five 4.1 cities as the only legal values, and extra: "forbid" so a misspelled field name fails loudly instead of being ignored. The endpoint itself reuses add_features from src/ — the same function train.py used, so serving and training cannot drift apart. The 400 check is a business rule the schema cannot express, and the exception handler at the bottom is the 500 policy: the caller gets one generic line, your logs get the full stack trace.
Step 4 — one good caller, one bad caller. The same endpoint, two very different mornings:
# caller A — the shape the contract expects
POST /predict
{"city": "Jalandhar", "monthly_income": 24000, "orders_last_90d": 1,
"avg_order_value": 650, "last_order_date": "2026-03-02"}
HTTP 200
{"prediction": 1, "churn_prob": 0.7}
# caller B — income negative, orders spelled as a word
POST /predict
{"city": "Jalandhar", "monthly_income": -24000, "orders_last_90d": "one",
"avg_order_value": 650, "last_order_date": "2026-03-02"}
HTTP 422
{"detail": [
{"type": "greater_than", "loc": ["body", "monthly_income"],
"msg": "Input should be greater than 0", "input": -24000},
{"type": "int_parsing", "loc": ["body", "orders_last_90d"],
"msg": "Input should be a valid integer, unable to parse string as an integer",
"input": "one"}
]}
Caller A gets the exact verdict predict.py printed in 4.1 — prediction 1, churn prob 0.70 — the same model, now reachable over HTTP. Caller B's request never touched the model: pydantic stopped it at the door and wrote the rejection itself. Read the 422 body the way a caller will: loc names the exact field, msg says what rule broke, input echoes what arrived. A caller can fix their bug from this reply alone, without messaging you — that is what a good contract buys. Send a last_order_date of 2026-08-01 instead and you get the third kind of reply: HTTP 400 with {"detail": "last_order_date is after the snapshot date"} — right shape, wrong meaning, your own rule speaking.
Step 5 — call your own API, and time it. The requests skills from 3.2, pointed at your own port:
# call_api.py — 3.2 comes full circle: you are your own caller now
import time
import requests
URL = "http://127.0.0.1:8000"
jalandhar = {"city": "Jalandhar", "monthly_income": 24000, "orders_last_90d": 1,
"avg_order_value": 650, "last_order_date": "2026-03-02"}
print(requests.get(f"{URL}/health", timeout=5).json())
r = requests.post(f"{URL}/predict", json=jalandhar, timeout=5)
r.raise_for_status()
print(r.json())
times = []
for _ in range(30):
t0 = time.perf_counter()
requests.post(f"{URL}/predict", json=jalandhar, timeout=5)
times.append((time.perf_counter() - t0) * 1000)
times.sort()
print(f"median: {times[14]:.0f} ms p95: {times[27]:.0f} ms worst: {times[-1]:.0f} ms")
Output on our laptop: {'status': 'ok', 'model_loaded': True}, then {'prediction': 1, 'churn_prob': 0.7}, then median: 9 ms p95: 14 ms worst: 21 ms. Note the habits carried over from 3.2: a timeout on every call, raise_for_status() before trusting a body. The timing loop is the honest way to talk about speed: thirty requests, sorted, then read the median (the typical case) and the p95 — the time 95% of requests beat, which is what the slowest twentieth of your callers actually feels. One number from one run is an anecdote; a sorted list of thirty is a measurement.
Step 6 — the anti-pattern, measured. Copy app/main.py to app/slow_main.py and move the load inside the endpoint — wrong on purpose:
# app/slow_main.py — the only change:
@app.post("/predict", response_model=VerdictOut)
def predict(customer: CustomerIn):
model = joblib.load(config.MODEL_PATH) # WRONG: every request pays this
...
30 requests against slow_main: median: 138 ms p95: 171 ms worst: 240 ms
30 requests against app.main: median: 9 ms p95: 14 ms worst: 21 ms
first request after a restart: 231 ms (cold — imports and caches warm once)
Same model, same laptop, fifteen times slower — and our churn joblib is only about one megabyte. A 400 MB model turns this mistake into a 20-to-30-second predict, which is WR-094's story. The cold number matters too: the first request after any restart is slow while Python's imports and caches warm up, so report it separately instead of letting it pollute your median. From today, "is it fast" is never an opinion — it is this table.
Work inside churn-repo with the 4.1 venv active and models/churn_pipeline.joblib trained. Each exercise ends with a check — you are done only when the check passes.
Hello, server. Build hello.py with two GET routes: / returning a greeting dict, and /about returning your name, city, and the service version. Run it with uvicorn, call both routes with curl or the browser, then open /docs and fire a request from inside the page. Finally stop the server and run a requests.get against it from another terminal. Check: both routes return JSON you wrote; the /docs page lists both routes and executes one; with the server stopped, your requests call raises ConnectionError — the same failure you diagnosed from the caller's side in 3.2, now seen from the owner's chair.
The full churn service. Build app/main.py in churn-repo exactly as the watch section grew it: lifespan loading of the 4.1 joblib, CustomerIn with types, realistic ranges, the five-city Literal and extra: "forbid", VerdictOut, /health, /predict with the snapshot-date 400 rule, and the generic 500 handler. Add the three new pinned lines to requirements.txt and commit. Check: the Jalandhar customer from 4.1 returns {"prediction": 1, "churn_prob": 0.7} — the same verdict predict.py printed; /health returns model_loaded: true; /docs shows CustomerIn with its ranges visible; the server log prints exactly one "model loaded" line per start and none per request; git ls-files shows app/main.py but still no joblib and no csv.
Attack your own contract. Write attack.py that sends six malformed payloads to /predict: a string where a number belongs, a negative income, a missing required field, a city outside the Literal list, a misspelled field name, and a last_order_date after the snapshot. Capture every status code and response body into attacks.md as a table. If any attack gets a 200 — or a confusing error — tighten the schema and attack again until all six fail cleanly with the right code and a loc or detail that names the problem. Finish with one legal boundary payload that must still pass. Check: attacks.md shows six attacks, each with its status code (five 422s, one 400) and the field the response names; at least one attack slipped through your first schema and forced a tightening, and a sentence records which; the boundary payload still returns 200.
The batch endpoint. Add POST /predict-batch taking a list of CustomerIn (use a pydantic list with max_length=100) and returning a list of verdicts in the same order. Reject a 101-customer batch with a clean 422 naming the size cap. Then measure honestly with a timing script: 100 customers sent one-by-one through /predict versus the same 100 in one /predict-batch call; record cold, median and worst numbers for both in latency.md. Check: a 100-customer batch returns 100 verdicts in order and a 101-customer batch is refused with the cap named; your table shows the batch call beating 100 single calls by a wide margin (ours: about 60 ms against about 1.2 s); two sentences explain where the one-by-one time went — 100 rounds of HTTP overhead and 100 tiny predicts against one round and one vectorised predict.
Three serving failures cover most first-month API pain. Each has a War Room entry with the full diagnosis path:
- WR-094 — the predict that took 30 seconds. A model was loaded inside the endpoint function, so every request paid the full load time; moving the load to startup and measuring before and after turned a 30-second predict into 60 milliseconds.
- WR-095 — the API that answered nonsense. A raw-dict service accepted a caller's renamed, re-scaled columns without complaint and returned confident wrong scores for weeks; a pydantic contract with ranges, forbidden extras and a schema version makes that silence impossible.
- WR-096 — the stack trace that leaked. A crashing endpoint returned the full traceback — file paths, library versions, column names — to an external partner; a generic 500 handler for callers plus full logging for the team closed the leak.
A model becomes a product when other software can call it, and FastAPI with uvicorn turns your churn repo into that callable service. Load the model once at startup, because a load inside the endpoint makes every caller pay the load time again. A pydantic schema at the door rejects wrong-shaped input with a 422 that names the field, before the model can answer nonsense. Business rules speak through a 400 with a short reason, and your own bugs return a generic 500 while the stack trace stays in your logs. Prove the service from outside: a health check that answers, auto docs at /docs, and latency reported as cold, median and worst.
- UPI fraud scoring. Every UPI payment is scored by a model behind an internal API with a budget of a few dozen milliseconds. The shape is today's /predict with a harsher clock — which is why the median-and-p95 habit is trained now.
- Bank and NBFC credit checks. Loan models in India sit behind internal HTTP services with strict schemas, so the branch system, the website and the field-agent app all call one consistent truth instead of three copies.
- Hosted model APIs. The endpoints you called and paid per token in 3.2 are this module at world scale: typed contracts on input, health checks, versioned schemas, and generic errors that never leak the insides.
Answer these before opening the gate assessment. Each question names its kind.
type=output-prediction— A caller POSTs{"city": "Mumbai", "monthly_income": 24000, "orders_last_90d": 3, "avg_order_value": 500, "last_order_date": "2026-05-10"}to the watch section's/predict. Predict the status code, what the response body'slocnames, and whether the model ran for this request.type=debug— A teammate's service answers/healthinstantly, but every/predicttakes about 140 ms even after fifty warm requests, and the server log prints "model loaded" on every call. Name the exact line that causes this, the fix, and the two measurements that prove the fix worked.type=interpretation— A latency report reads: first request after restart 231 ms; median 9 ms; p95 14 ms; 100 customers one-by-one 1.2 s total; the same 100 in one batch call 60 ms. Explain what each number tells you, and where the one-by-one time actually went.type=scenario-judgement— Before launch, a teammate proposes removingCustomerInand accepting a raw dict "so partner apps are never blocked by 422 errors". Judge the proposal, name the failure it invites, and say who a fast loud 422 actually protects.type=trade-off— A strict contract (Literal city list, tight ranges, extras forbidden) versus a loose one (free strings, wide ranges): give one concrete gain and one concrete cost of each, pick one for Sanjana's NBFC service, and name what you would log so the strict ranges can be loosened safely later.
Explain to a friend who runs a kirana shop why your model needed a shop counter, not an open storeroom. The storeroom is your repo — everything is in there, but customers cannot walk in and serve themselves. The counter is your API: one window, a fixed way to ask, and a polite refusal when a request makes no sense. Use at most 4 sentences and no technical word beyond "counter". If your friend can say why refusing a bad request at the counter protects both sides, you own this module.