Hugging Face ecosystem
Fathima checks quality at a knitwear exporter in Tiruppur. Her team wants a tool that reads buyer complaints and sorts them. She has three weeks, and one laptop with 8GB of memory. Her lead tells her something odd: do not build a model, find one. Read its label, prove it fits, then write code. That evening a model loads in two seconds and answers in less than one. Then she opens the label, and the real work starts.
- Nobody at work builds a model from an empty file. They start from a public checkpoint, and picking the right one is the skill that gets paid for. Tonight you pick three, price them, and write the paragraph a manager can act on.
- Two questions end careers early: what licence is this under, and what is inside that weights file? Both are answered in four minutes of reading, and almost no fresher reads them.
- Everything here runs on an 8GB laptop with no GPU. The biggest model in this module is 268 MB, answers in 0.08 seconds, and fine-tunes in 11 seconds, and every number below came off that machine.
Phase 2 has been about building. You wrote a neuron, a training loop, a convolution, an attention head, and a whole Transformer block. This module is about the other half of the job. Work does not start from an empty file. It starts from somebody else's finished model, and the craft is choosing well.
The Hugging Face Hub is the field's shared shelf. It holds three kinds of thing. Models are trained weights with a page attached. Datasets are rows you can load or read over the network. Spaces are small hosted demos, which is where your project can live as a link instead of a screenshot.
Each of those is reached by a library, and there are only four worth learning now. The transformers library loads a model and runs it. The datasets library brings rows in and reshapes them. The sentence-transformers library turns text into meaning vectors, which is module 3.4's engine with a real checkpoint behind it. The huggingface_hub library holds the plumbing and the paperwork: file sizes, licences, and the cache all of them share.
The page attached to a model is where the profession keeps its honesty. A model card — the page shipped with a model, holding its licence, its training data, its intended use and its known limits — is a label, not marketing. Read it the way you read the back of a food packet. Many cards are thin, and one of the three real models measured below has no licence field at all, with more than three lakh downloads a month. A thin card is not a safe card; it is an unanswered question.
There is a wrong way and a right way to pick a model. The wrong way searches the task, sorts by downloads, takes row one, and ships it. The right way reads four lines before any code: licence, training data, limits, and file format with size. That reading costs about four minutes per model, once. The wrong way costs a stopped launch, which is WR-145.
Start at the top layer, because the first result should arrive in seconds. A pipeline — a one-line wrapper joining a tokeniser, a model and the tidy-up code for one task — hides everything. Sentiment, entity finding, mask filling and label matching all become one call. Below, four of them ran on a CPU in 0.08, 0.02, 0.03 and 0.07 seconds. Speed is not the lesson here; the lesson is that an instant answer is still an unchecked answer.
One level down is where you become an engineer. AutoTokenizer turns text into ids, which is module 3.1's tokens made of real numbers. AutoModel turns ids into hidden states and logits, which is module 2.8's block stack wearing a checkpoint. Below, one sentence became 14 ids, six attention tensors of shape (1, 12, 14, 14), and two logits. Everything you built by hand in 2.8 is sitting inside a 66,955,010-parameter file you did not train.
The tokeniser is also where Indian text quietly breaks. In the run below, "kurta" became kurt plus ##a, and "Tiruppur" became three pieces. That is why the entity finder split one name into two people and one town into four places. The model is not broken and the code is not wrong. Its word list was built on English news, which is exactly what its card says.
Data has the same two layers. load_dataset fetches rows with their splits and column names, and map, filter and train_test_split reshape them in seconds. When the data is too big for your disk, streaming — reading rows over the network without saving them — changes the arithmetic. Below, a 72 GB dataset gave up three Hindi articles while the disk grew by 0.0 MB. That is the difference between "I cannot work on this" and "I can start tonight".
Fine-tuning has a wrapper too. The Trainer class runs module 2.4's loop for you, and it writes a folder every N steps without being asked. That folder holds the weights, the optimizer state, the scheduler and the random state — module 2.9's five items, already correct. Below, a session was killed at step 60 and resumed from checkpoint-40, and both runs ended at the same 0.738 accuracy. Module 3.10 does this for large language models with small adapters; this is the ordinary case, and it runs on your laptop.
Now the file format, which is a safety question wearing a file extension. The old .bin and .pt files are Python pickles, and loading a pickle runs code the file's author chose, before your next line. The safetensors format — a weights file holding names, shapes and bytes, with no place to hide code — removes that risk entirely. Below, a small pickle printed a line from inside the weights file itself. Prefer safetensors, keep weights_only=True, and never load weights from a link a stranger sent, which is WR-146.
Licence literacy is the last habit, and it is the one that reaches a payslip. A permissive licence — one that allows commercial use with credit, such as Apache-2.0 or MIT — lets a product ship. Research-only and custom community licences do not, and a blank licence field means blocked until somebody answers. What you fine-tune from, you inherit from, so an adapter over a research-only model is still research-only. Put licence in the first row of your comparison table, above accuracy, so an unusable model dies in minute one.
Two closing habits cost nothing. Pin a revision — the exact commit of a repository, named by a long hash — so tomorrow's load gets today's weights. And check size before you download: parameters times bytes each, against your free memory, not your total memory. A 7-billion-parameter model at two bytes each is about 14 GB, which an 8GB laptop cannot hold, and that is WR-147. This module closes Phase 2, and Phase 3 opens with module 3.1 on the same shelf, at a larger size.
Work inside ai-lab. One-time setup: pip install torch==2.9.* transformers==4.57.* datasets==4.2.* sentence-transformers==5.1.*, the CPU build only. Unlike every other module in this phase, this one downloads things, so the sizes and the first-run seconds are part of the lesson. Timings come from a plain CPU laptop with four threads and a warm cache; your seconds will differ, and the megabytes should not. Where a number below is a model output rather than a timing, yours should land within about 1% of it — seeded torch runs vary slightly across builds — while counts, shapes and vocabulary sizes should match exactly.
If your numbers are close but not identical, triage in a fixed order before touching your code. Everything off by a little — every value drifting in the last decimal or two — points at a library version: run pip show torch transformers, compare against the pinned install line above, and match the pins, because the printed numbers were produced on those versions. One number wildly off while the rest agree points at stale state: restart from a fresh kernel or a fresh python and run the file top to bottom, since seeded draws only reproduce when they happen in the same order from a clean start. Only when both checks pass and the numbers still disagree should you suspect your own logic — diff your script line by line against the printed code.
Step 1 — four tasks, four one-line answers, measured. Save as step1_pipelines.py:
import time
import torch
from pathlib import Path
from transformers import pipeline
from transformers.pipelines import SUPPORTED_TASKS
torch.set_num_threads(4)
HUB = Path.home() / ".cache/huggingface/hub"
def cache_mb(repo):
"""What this checkpoint really costs on disk. Count blobs, not the symlinks."""
blobs = HUB / ("models--" + repo.replace("/", "--")) / "blobs"
return sum(f.stat().st_size for f in blobs.glob("*")) / 1e6 if blobs.exists() else 0.0
JOBS = [
("text-classification", "distilbert-base-uncased-finetuned-sst-2-english",
"The kurta arrived torn and the seller stopped replying.", {}),
("token-classification", "dslim/distilbert-NER",
"Fathima Rasheed joined Zerodha in Tiruppur last March.",
{"aggregation_strategy": "simple"}),
("fill-mask", "distilbert-base-uncased",
"The parcel was [MASK] by the courier.", {"top_k": 3}),
("zero-shot-classification", "typeform/distilbert-base-uncased-mnli",
"Money was debited from my UPI app but the recharge did not happen.",
{"candidate_labels": ["payment failure", "delivery delay",
"login problem", "refund status"]}),
]
print(f"{'pipeline task':<24s} {'checkpoint':<46s} {'MB':>6s} {'load s':>7s} {'run s':>6s}")
for task, repo, text, kw in JOBS:
t0 = time.perf_counter(); pipe = pipeline(task, model=repo)
load = time.perf_counter() - t0 # first run also downloads
t0 = time.perf_counter(); out = pipe(text, **kw); run = time.perf_counter() - t0
print(f"{task:<24s} {repo:<46s} {cache_mb(repo):6.0f} {load:7.1f} {run:6.2f}")
print(f" text: {text}")
print(f" out : {out}\n")
print(f"pipeline tasks on this transformers version: {len(SUPPORTED_TASKS)}")
print("print this list yourself; it changes between versions, and blogs do not.")
Output (second run, so the load column is cache only — the first run added the MB column over the network):
pipeline task checkpoint MB load s run s
text-classification distilbert-base-uncased-finetuned-sst-2-english 268 2.5 0.08
text: The kurta arrived torn and the seller stopped replying.
out : [{'label': 'NEGATIVE', 'score': 0.9996920824050903}]
token-classification dslim/distilbert-NER 262 1.8 0.02
text: Fathima Rasheed joined Zerodha in Tiruppur last March.
out : [{'entity_group': 'PER', 'score': 0.9968401, 'word': 'Fat', 'start': 0, 'end': 3},
{'entity_group': 'PER', 'score': 0.9590288, 'word': '##hima Rasheed', ...},
{'entity_group': 'ORG', 'score': 0.9905977, 'word': 'Zero', ...},
{'entity_group': 'ORG', 'score': 0.9966531, 'word': '##dha', ...},
{'entity_group': 'LOC', 'score': 0.9923548, 'word': 'T', ...},
{'entity_group': 'LOC', 'score': 0.9826891, 'word': '##ir', ...},
{'entity_group': 'LOC', 'score': 0.9910622, 'word': '##up', ...},
{'entity_group': 'LOC', 'score': 0.9940937, 'word': '##pur', ...}]
fill-mask distilbert-base-uncased 269 2.3 0.03
text: The parcel was [MASK] by the courier.
out : [{'score': 0.0822, 'token_str': 'purchased', ...},
{'score': 0.0559, 'token_str': 'delivered', ...},
{'score': 0.0333, 'token_str': 'distributed', ...}]
zero-shot-classification typeform/distilbert-base-uncased-mnli 268 2.0 0.07
text: Money was debited from my UPI app but the recharge did not happen.
out : {'labels': ['refund status', 'payment failure', 'delivery delay', 'login problem'],
'scores': [0.9508491, 0.0286570, 0.0108129, 0.0096810]}
pipeline tasks on this transformers version: 24
print this list yourself; it changes between versions, and blogs do not.
Four tasks, four working answers, and three of them are wrong in ways worth more than the right one. Start with the honest win: the sentiment row read a torn-kurta complaint as NEGATIVE at 0.9997, from a 268 MB file, in 0.08 seconds on a CPU with no GPU anywhere. Now the entity row. It found the right kinds of thing, then broke Fathima into Fat and ##hima Rasheed, Zerodha into Zero and ##dha, and Tiruppur into T, ##ir, ##up and ##pur — one person became two people and one town became four places. Nothing failed; that model's word list was built on English news, and Step 2 shows the exact mechanism. The mask row is the same story from a different angle: the top guess for what a courier does to a parcel is purchased at 0.082, ahead of delivered at 0.056, because a base model is trained to complete text and not to know courier work. The zero-shot row is the one to be frightened of, because it is confident and wrong: a failed UPI recharge was labelled refund status at 0.951, with payment failure down at 0.029. Read that pair of numbers twice. A pipeline gives you a result in one line, and the result carries a score that looks like certainty and is not evidence. The last two lines matter too — the task list is a property of your installed version, and on this one summarization is not in it, so print SUPPORTED_TASKS rather than trusting a tutorial.
Step 2 — the same model, one level down. Save as step2_internals.py:
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
torch.set_num_threads(4)
REPO = "distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForSequenceClassification.from_pretrained(
REPO, attn_implementation="eager") # eager, or attention maps come back empty
model.eval()
text = "The kurta arrived torn and the seller stopped replying."
enc = tok(text, return_tensors="pt")
ids = enc["input_ids"][0]
print("1. TEXT -> IDS (3.1's tokens, as real numbers)")
print(f" ids shape {tuple(enc['input_ids'].shape)}, {len(ids)} tokens for {len(text.split())} words")
print(f" ids {ids.tolist()}")
print(f" pieces {tok.convert_ids_to_tokens(ids)}")
with torch.no_grad():
out = model(**enc, output_hidden_states=True, output_attentions=True)
print("\n2. IDS -> MODEL (2.8's block stack, wearing a checkpoint)")
print(f" layers {model.config.num_hidden_layers}, heads {model.config.num_attention_heads},"
f" width {model.config.dim}")
print(f" parameters {sum(p.numel() for p in model.parameters()):,d}")
print(f" hidden_states {len(out.hidden_states)} tensors of {tuple(out.hidden_states[0].shape)}")
print(f" attentions {len(out.attentions)} tensors of {tuple(out.attentions[0].shape)}")
print(f" last layer head 0 attention rows sum to {float(out.attentions[-1][0, 0].sum(-1).mean()):.6f}")
print("\n3. MODEL -> LOGITS -> LABEL")
probs = torch.softmax(out.logits, dim=-1)[0]
print(f" logits {tuple(out.logits.shape)} {[round(v, 4) for v in out.logits[0].tolist()]}")
print(f" softmax {[round(v, 6) for v in probs.tolist()]}")
print(f" id2label {model.config.id2label}")
best = int(probs.argmax())
print(f" argmax {best} -> {model.config.id2label[best]} score {float(probs[best]):.6f}")
print("\n4. WHERE THE TOKENISER BITES")
for s in ["Fathima", "Tiruppur", "kurta", "delivery"]:
print(f" {s:<10s} -> {tok.tokenize(s)}")
Output:
1. TEXT -> IDS (3.1's tokens, as real numbers)
ids shape (1, 14), 14 tokens for 9 words
ids [101, 1996, 9679, 2050, 3369, 7950, 1998, 1996, 14939, 3030, 7514, 2075, 1012, 102]
pieces ['[CLS]', 'the', 'kurt', '##a', 'arrived', 'torn', 'and', 'the', 'seller',
'stopped', 'reply', '##ing', '.', '[SEP]']
2. IDS -> MODEL (2.8's block stack, wearing a checkpoint)
layers 6, heads 12, width 768
parameters 66,955,010
hidden_states 7 tensors of (1, 14, 768)
attentions 6 tensors of (1, 12, 14, 14)
last layer head 0 attention rows sum to 1.000000
3. MODEL -> LOGITS -> LABEL
logits (1, 2) [4.4587, -3.6264]
softmax [0.999692, 0.000308]
id2label {0: 'NEGATIVE', 1: 'POSITIVE'}
argmax 0 -> NEGATIVE score 0.999692
4. WHERE THE TOKENISER BITES
Fathima -> ['fat', '##hima']
Tiruppur -> ['ti', '##rup', '##pur']
kurta -> ['kurt', '##a']
delivery -> ['delivery']
This output is the whole of module 2.8 printed from a file somebody else trained, and it deserves a slow read. Nine words became 14 ids, because the tokeniser adds [CLS] and [SEP] and splits anything its word list does not hold — that is 3.1's token, with the actual integers on screen. The model reports 6 layers, 12 heads and a width of 768, which is 66,955,010 numbers, and the two shapes underneath are the ones you built by hand: hidden_states is 7 tensors of (1, 14, 768), one per layer plus the embedding output, and attentions is 6 tensors of (1, 12, 14, 14) — one square of 14 by 14 scores per head per layer. The check on the next line is worth keeping as a habit: attention rows sum to 1.000000, which is the softmax from 2.7 still doing exactly what you proved it does. Then the head: two logits, 4.4587 and -3.6264, turned by softmax into 0.999692 and 0.000308, and only the id2label map makes index 0 mean NEGATIVE — read that map rather than assuming, because a different checkpoint will order its labels differently and your code will be silently inverted. Section 4 explains Step 1's broken entities in four lines. delivery survives whole because it is common in English news; kurta becomes kurt plus ##a, and Tiruppur becomes three pieces, so an entity tagger that labels pieces has no way to hand you one clean name. Fathima's own name is split in half by the model she was about to ship.
Step 3 — read the card and the file format as a checklist. Save as step3_card_audit.py. The first half asks the Hub about three real models; the second half proves why the format matters, using a file you write yourself:
import os, pickle, re, torch
from huggingface_hub import HfApi, ModelCard
from safetensors.torch import save_file, load_file
api = HfApi()
REPOS = ["distilbert-base-uncased-finetuned-sst-2-english",
"dslim/distilbert-NER",
"typeform/distilbert-base-uncased-mnli"]
WANT = {"licence": r"licen[cs]e",
"training data": r"training data|dataset|trained on",
"intended use": r"intended use|how to use|uses",
"limitations": r"limitation|bias|risk|out-of-scope"}
for repo in REPOS:
info = api.model_info(repo, files_metadata=True)
weights = {f.rfilename: (f.size or 0) for f in info.siblings
if f.rfilename.endswith((".safetensors", ".bin", ".h5", ".onnx"))}
body = ModelCard.load(repo).text
print(f"== {repo}")
print(f" licence tag {info.card_data.get('license') if info.card_data else None}")
print(f" downloads/month {info.downloads:,d} last change {info.last_modified.date()}")
print(f" sha (pin this) {info.sha}")
print(f" card length {len(body):,d} characters")
for label, pat in WANT.items():
hits = len(re.findall(pat, body, re.I))
print(f" {label:<15s} {'mentioned ' + str(hits) + 'x' if hits else 'MISSING'}")
print(f" safetensors? {'yes' if any(k.endswith('.safetensors') for k in weights) else 'NO'}")
print(f" weight files {', '.join(f'{k} {v/1e6:.0f} MB' for k, v in sorted(weights.items()))}\n")
class Sneaky:
"""A pickle is a program. This is what a hostile publisher writes."""
def __reduce__(self):
return (os.system, ('echo " >>> this line came from the WEIGHTS FILE, not your code"',))
with open("weights.pt", "wb") as f:
pickle.dump({"layer1": [0.1, 0.2], "note": Sneaky()}, f)
print("loading weights.pt the old way (pickle):", flush=True)
with open("weights.pt", "rb") as f:
pickle.load(f)
print("\nthe same file through torch.load(weights_only=True):")
try:
torch.load("weights.pt", weights_only=True)
except Exception as e:
print(f" refused: {type(e).__name__}")
save_file({"layer1": torch.tensor([0.1, 0.2])}, "weights.safetensors")
print("\nloading weights.safetensors:")
print(" ", load_file("weights.safetensors"))
print(" safetensors holds names, shapes and bytes. There is no code slot to abuse.")
Output:
== distilbert-base-uncased-finetuned-sst-2-english
licence tag apache-2.0
downloads/month 3,346,574 last change 2023-12-19
sha (pin this) 714eb0fa89d2f80546fda750413ed43d93601a13
card length 3,990 characters
licence mentioned 1x
training data mentioned 6x
intended use mentioned 3x
limitations mentioned 17x
safetensors? yes
weight files model.safetensors 268 MB, onnx/model.onnx 268 MB,
pytorch_model.bin 268 MB, tf_model.h5 268 MB
== dslim/distilbert-NER
licence tag apache-2.0
downloads/month 215,423 last change 2024-10-08
sha (pin this) dfa2838a127384aabb82ed7719e16dab84c42a2a
card length 5,934 characters
licence MISSING
training data mentioned 14x
intended use mentioned 2x
limitations mentioned 4x
safetensors? yes
weight files model.safetensors 261 MB, onnx/model.onnx 261 MB
== typeform/distilbert-base-uncased-mnli
licence tag None
downloads/month 338,223 last change 2023-03-22
sha (pin this) cfa538a0fddbbd978fefe8966c1aeff7ad409c90
card length 3,761 characters
licence mentioned 1x
training data mentioned 5x
intended use mentioned 3x
limitations mentioned 10x
safetensors? yes
weight files model.safetensors 268 MB, pytorch_model.bin 268 MB, tf_model.h5 268 MB
loading weights.pt the old way (pickle):
>>> this line came from the WEIGHTS FILE, not your code
the same file through torch.load(weights_only=True):
refused: UnpicklingError
loading weights.safetensors:
{'layer1': tensor([0.1000, 0.2000])}
safetensors holds names, shapes and bytes. There is no code slot to abuse.
Two audits in one script, and both change how you shop. Take the third model first, because it is the uncomfortable one: typeform/distilbert-base-uncased-mnli has a licence tag of None and 338,223 downloads a month, so a third of a million pulls happened without an answer to the question legal will ask. Popularity is not permission. The second model is subtler — it carries an apache-2.0 tag while its card body never uses the word licence, so the machine-readable field and the human-readable page disagree, and you would quote the tag in a review and be right by luck. The first model is the good case and still teaches something: 17 mentions of limitations and bias, a card of 3,990 characters, and a sha you can pass as revision= so tomorrow's load gets today's weights. Look at its file list too, because a repository can publish the same 268 MB of weights four times over — safetensors, ONNX, a pickle and a TensorFlow file — and cloning the whole repository costs four times what the loader needs. Then the second half, which takes ten seconds and settles the format argument for good. The line >>> this line came from the WEIGHTS FILE, not your code was printed by the file, not the script. That is pickle.load doing its documented job: rebuilding objects, which means running code, before any check of yours can run. torch.load(..., weights_only=True) refuses the same file with an UnpicklingError, which is a defence you should never switch off, and safetensors hands back the tensor with nowhere for a payload to hide. This is WR-146, and it is module 3.11's supply-chain thinking arriving early.
Step 4 — rows in, and rows too big to keep. Save as step4_datasets.py:
import time
from pathlib import Path
from datasets import load_dataset
from huggingface_hub import HfApi
CACHE = Path.home() / ".cache/huggingface/datasets"
def cache_mb():
return sum(f.stat().st_size for f in CACHE.rglob("*") if f.is_file()) / 1e6 if CACHE.exists() else 0.0
t0 = time.perf_counter()
ds = load_dataset("cornell-movie-review-data/rotten_tomatoes") # full id, namespace included
print(f"downloaded and opened in {time.perf_counter() - t0:.1f} s")
print(ds)
print("one row:", ds["train"][0]["text"][:60], "| label", ds["train"][0]["label"])
short = ds["train"].filter(lambda r: len(r["text"]) < 120)
print(f"\nfilter: {len(ds['train']):,d} -> {len(short):,d} rows")
mapped = short.map(lambda b: {"n_words": [len(t.split()) for t in b["text"]]},
batched=True, batch_size=256) # batched=True, or it crawls
print(f"map: new column -> {mapped.column_names}")
split = mapped.train_test_split(test_size=0.2, seed=7) # 1.7's split rule, one call
print(f"train_test_split -> train {len(split['train']):,d} test {len(split['test']):,d}")
print("label balance in train:", {l: split["train"]["label"].count(l) for l in (0, 1)})
REPO = "wikimedia/wikipedia"
info = HfApi().dataset_info(REPO, files_metadata=True)
total = sum(s.size or 0 for s in info.siblings)
hindi = sum(s.size or 0 for s in info.siblings if "20231101.hi/" in s.rfilename)
print(f"\n{REPO}: {len(info.siblings):,d} files, {total / 1e9:.0f} GB in the whole repo")
print(f"the Hindi part alone is {hindi / 1e6:,.0f} MB")
before = cache_mb()
t0 = time.perf_counter()
stream = load_dataset(REPO, "20231101.hi", split="train", streaming=True)
print(f"opened as a stream in {time.perf_counter() - t0:.1f} s (nothing downloaded yet)")
for r in stream.take(3):
print(f" {r['title'][:24]:<26s} {len(r['text']):,d} characters")
big = stream.filter(lambda r: len(r["text"]) > 20000).map(
lambda r: {"title": r["title"], "kb": len(r["text"]) // 1024})
for r in big.take(3):
print(f" filtered+mapped: {r['title'][:24]:<26s} {r['kb']} KB")
print(f"dataset cache grew by {cache_mb() - before:.1f} MB")
Output:
downloaded and opened in 10.9 s
DatasetDict({
train: Dataset({features: ['text', 'label'], num_rows: 8530})
validation: Dataset({features: ['text', 'label'], num_rows: 1066})
test: Dataset({features: ['text', 'label'], num_rows: 1066})
})
one row: the rock is destined to be the 21st century's new " cona | label 1
filter: 8,530 -> 4,794 rows
map: new column -> ['text', 'label', 'n_words']
train_test_split -> train 3,835 test 959
label balance in train: {0: 1955, 1: 1880}
wikimedia/wikipedia: 557 files, 72 GB in the whole repo
the Hindi part alone is 238 MB
opened as a stream in 4.7 s (nothing downloaded yet)
हम होंगे कामयाब 420 characters
दैनिक पूजा 3,827 characters
हिन्दी 22,266 characters
filtered+mapped: हिन्दी 21 KB
filtered+mapped: देवनागरी 26 KB
filtered+mapped: भारत 46 KB
dataset cache grew by 0.1 MB
The first half is ordinary and should feel like Pandas with the splits already made. One call returned a DatasetDict with train, validation and test already separated, and the features named, so you never have to guess which column is the label. filter cut 8,530 rows to 4,794, map added a column, and train_test_split(seed=7) produced 3,835 and 959 rows with a balance of 1,955 against 1,880 — module 1.7's split discipline as a single seeded call, and the seed is what makes it repeatable next week. Two details save hours later: pass the full namespace/name id, because bare dataset names stopped resolving in current versions, and pass batched=True to map or you pay per row instead of per 256 rows. The second half is the part that changes what you believe is possible on your laptop. wikimedia/wikipedia is 557 files and 72 GB, which will not fit on most student machines, and the Hindi part alone is 238 MB. With streaming=True the dataset opened in 4.7 seconds, handed over three Hindi articles, then accepted a filter and a map over the stream — and the cache grew by 0.1 MB, which is metadata. Nothing was stored, because streaming reads rows over the network as you ask for them. That is the honest answer to "the dataset is bigger than my disk": you do not need the dataset, you need the rows you are about to use.
Step 5 — fine-tune on a CPU, kill it, and resume. Save as step5_trainer.py. Each child process is one free session, exactly as in module 2.9:
import json, os, shutil, subprocess, sys, time
import torch
from datasets import load_dataset
from transformers import (AutoTokenizer, AutoModelForSequenceClassification, Trainer,
TrainingArguments, TrainerCallback, set_seed)
torch.set_num_threads(4)
MODEL = "google/bert_uncased_L-4_H-256_A-4" # 11.2M parameters, 44 MB of weights
TOKENISER = "distilbert-base-uncased" # same uncased word list, already on disk
KILL_AT = 60
def data(tok):
ds = load_dataset("cornell-movie-review-data/rotten_tomatoes")
enc = lambda b: tok(b["text"], truncation=True, max_length=64, padding="max_length")
return (ds["train"].shuffle(seed=7).select(range(2000)).map(enc, batched=True),
ds["validation"].shuffle(seed=7).select(range(500)).map(enc, batched=True))
def accuracy(p):
return {"accuracy": float((p.predictions.argmax(-1) == p.label_ids).mean())}
class SessionDeath(TrainerCallback):
"""A free session does not ask permission before it ends."""
def on_step_end(self, args, state, control, **kw):
if state.global_step == KILL_AT:
print(f" *** session ended at step {state.global_step}, no warning ***", flush=True)
os._exit(9) # no cleanup: an honest death
def session(out, resume=False, kill=False):
set_seed(7)
tok = AutoTokenizer.from_pretrained(TOKENISER)
train, val = data(tok)
model = AutoModelForSequenceClassification.from_pretrained(MODEL, num_labels=2)
args = TrainingArguments(output_dir=out, num_train_epochs=2, per_device_train_batch_size=32,
learning_rate=5e-4, save_steps=20, save_total_limit=2,
logging_steps=32, eval_strategy="epoch", seed=7,
report_to=[], use_cpu=True, disable_tqdm=True, log_level="error")
tr = Trainer(model=model, args=args, train_dataset=train, eval_dataset=val,
compute_metrics=accuracy, callbacks=[SessionDeath()] if kill else [])
if resume:
last = max((d for d in os.listdir(out) if d.startswith("checkpoint-")),
key=lambda d: int(d.split("-")[1]))
st = json.load(open(os.path.join(out, last, "trainer_state.json")))
print(f" resuming from {last}: step {st['global_step']} of {int(st['max_steps'])}", flush=True)
t0 = time.perf_counter()
tr.train(resume_from_checkpoint=resume) # True = pick up the newest checkpoint
r = tr.evaluate()
print(f" {time.perf_counter() - t0:5.1f} s in this session | "
f"eval_loss {r['eval_loss']:.4f} | accuracy {r['eval_accuracy']:.3f}", flush=True)
if __name__ == "__main__":
if len(sys.argv) > 1:
session(sys.argv[1], resume=sys.argv[2] == "resume", kill=sys.argv[2] == "kill")
sys.exit(0)
for d in ("run-A", "run-B"):
shutil.rmtree(d, ignore_errors=True)
go = lambda *a: subprocess.run([sys.executable, __file__, *a], check=False)
m = AutoModelForSequenceClassification.from_pretrained(MODEL, num_labels=2)
print(f"model {MODEL}: {sum(p.numel() for p in m.parameters()):,d} parameters\n")
print("A - one session, never interrupted", flush=True)
go("run-A", "plain")
print("\nB - the session dies at step 60", flush=True)
go("run-B", "kill")
ck = sorted(os.listdir("run-B"))[-1]
print(f" on disk: {sorted(os.listdir('run-B'))}")
print(f" inside {ck}:")
for f in sorted(os.listdir(os.path.join("run-B", ck))):
p = os.path.join("run-B", ck, f)
print(f" {f:<24s} {os.path.getsize(p) / 1e6:7.2f} MB")
print("\nB - a new session, same script, nothing edited", flush=True)
go("run-B", "resume")
Output (training log lines trimmed to the ones that matter):
model google/bert_uncased_L-4_H-256_A-4: 11,171,074 parameters
A - one session, never interrupted
{'loss': 0.7007, 'epoch': 0.51}
{'eval_loss': 0.6173, 'eval_accuracy': 0.672, 'epoch': 1}
{'loss': 0.4062, 'epoch': 1.52}
{'eval_loss': 0.5479, 'eval_accuracy': 0.738, 'epoch': 2}
11.3 s in this session | eval_loss 0.5479 | accuracy 0.738
B - the session dies at step 60
{'loss': 0.7007, 'epoch': 0.51}
*** session ended at step 60, no warning ***
on disk: ['checkpoint-20', 'checkpoint-40']
inside checkpoint-40:
config.json 0.00 MB
model.safetensors 44.69 MB
optimizer.pt 89.43 MB
rng_state.pth 0.01 MB
scheduler.pt 0.00 MB
trainer_state.json 0.00 MB
training_args.bin 0.01 MB
B - a new session, same script, nothing edited
resuming from checkpoint-40: step 40 of 126
{'eval_loss': 0.6173, 'eval_accuracy': 0.672, 'epoch': 1}
{'loss': 0.4062, 'epoch': 1.52}
{'eval_loss': 0.5479, 'eval_accuracy': 0.738, 'epoch': 2}
8.2 s in this session | eval_loss 0.5479 | accuracy 0.738
Read the last line of run A against the last line of run B, because that pair is module 2.9's whole doctrine in a library you did not write. Run A trained 126 steps in one go and ended at eval_loss 0.5479 and accuracy 0.738. Run B was killed at step 60 by os._exit(9), which skips every cleanup handler and is therefore an honest imitation of a free session ending on somebody else's schedule. Its next session found checkpoint-40, printed step 40 of 126, and finished at eval_loss 0.5479 and accuracy 0.738 — the same numbers, to the printed digits, including the intermediate 0.6173 and 0.672 at epoch 1. Your own pair may land within about 1% of these printed values, because seeded torch runs vary slightly across builds — but on your machine run A and run B must agree with each other to the digit, and that agreement is the point. Twenty steps of work were lost because save_steps=20 and the death came at 60, and that gap is a cadence you chose. Now look inside the checkpoint folder, because it is module 2.9's list of five items with library names on it: model.safetensors is the weights at 44.69 MB, optimizer.pt is Adam's two buffers per weight at 89.43 MB — twice the model, which surprises everyone once — scheduler.pt holds the learning-rate position, rng_state.pth holds the random state, and trainer_state.json holds the step number and the data order. That is why the resume was exact rather than close, and it is why save_total_limit=2 matters on a small disk. One honest note on scale: an 11.2M-parameter model on 2,000 rows fine-tunes in 11 seconds on a CPU, and a 66M model on 50,000 rows is a Kaggle GPU job by 2.9's channel rule. Module 3.10 does the large-language-model version of this with adapters of about 100 MB, and the resume habit is the same.
Step 6 — embeddings, and the model card question made visible. Save as step6_embeddings.py:
import time
import numpy as np, torch
from pathlib import Path
from sentence_transformers import SentenceTransformer
torch.set_num_threads(4)
HUB = Path.home() / ".cache/huggingface/hub"
def mb(repo):
b = HUB / ("models--" + repo.replace("/", "--")) / "blobs"
return sum(f.stat().st_size for f in b.glob("*")) / 1e6 if b.exists() else 0.0
SENTS = [
"I want my money back for this torn kurta.", # 0 English refund
"Mera paisa wapas chahiye, kurta phata hua aaya.", # 1 Hindi in Roman letters
"मुझे अपना पैसा वापस चाहिए, कुर्ता फटा हुआ आया।", # 2 the same, in Devanagari
"Where is my parcel? Ten days and no update.", # 3 English delivery
"Parcel kahan hai? Das din ho gaye, koi update nahi.", # 4 Hindi in Roman letters
"How do I change my shop's pickup address?", # 5 unrelated
]
LABEL = ["EN refund", "Hinglish refund", "Hindi refund",
"EN parcel", "Hinglish parcel", "address"]
for repo in ["sentence-transformers/all-MiniLM-L6-v2",
"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"]:
t0 = time.perf_counter(); m = SentenceTransformer(repo); load = time.perf_counter() - t0
t0 = time.perf_counter()
v = m.encode(SENTS, normalize_embeddings=True) # normalised, so v @ v.T is cosine
enc = time.perf_counter() - t0
sim = np.round(v @ v.T, 2)
print(f"\n{repo}")
print(f" {sum(p.numel() for p in m.parameters()):,d} params | {mb(repo):.0f} MB |"
f" load {load:.1f} s | encode 6 sentences {enc:.2f} s | vector {v.shape[1]} numbers")
print(f" {'':<17s}" + "".join(f"{l[:9]:>10s}" for l in LABEL))
for i, row in enumerate(sim):
print(f" {LABEL[i]:<17s}" + "".join(f"{x:10.2f}" for x in row))
print(f" EN refund vs Hinglish {sim[0, 1]:.2f} | vs Devanagari {sim[0, 2]:.2f}"
f" | vs address {sim[0, 5]:.2f}")
Output:
sentence-transformers/all-MiniLM-L6-v2
22,713,216 params | 92 MB | load 5.7 s | encode 6 sentences 0.09 s | vector 384 numbers
EN refund Hinglish Hindi ref EN parcel Hinglish address
EN refund 1.00 0.27 0.05 0.05 0.04 0.07
Hinglish refund 0.27 1.00 0.48 -0.02 0.44 -0.06
Hindi refund 0.05 0.48 1.00 0.01 0.50 0.02
EN parcel 0.05 -0.02 0.01 1.00 0.25 0.32
Hinglish parcel 0.04 0.44 0.50 0.25 1.00 0.05
address 0.07 -0.06 0.02 0.32 0.05 1.00
EN refund vs Hinglish 0.27 | vs Devanagari 0.05 | vs address 0.07
sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
117,653,760 params | 480 MB | load 7.0 s | encode 6 sentences 0.04 s | vector 384 numbers
EN refund Hinglish Hindi ref EN parcel Hinglish address
EN refund 1.00 0.27 0.74 0.29 0.24 0.15
Hinglish refund 0.27 1.00 0.20 0.11 0.55 0.02
Hindi refund 0.74 0.20 1.00 0.37 0.15 0.23
EN parcel 0.29 0.11 0.37 1.00 0.29 0.34
Hinglish parcel 0.24 0.55 0.15 0.29 1.00 0.13
address 0.15 0.02 0.23 0.34 0.13 1.00
EN refund vs Hinglish 0.27 | vs Devanagari 0.74 | vs address 0.15
These two matrices are the same six sentences, and choosing between them is the entire model-selection lesson with a score attached. Both models return 384 numbers per sentence and encode all six in under a tenth of a second, so speed is not the deciding factor. Look at one cell: the English refund against the same refund in Devanagari scores 0.05 under the first model and 0.74 under the second. Module 3.4's promise — meaning matches across different words — is kept by one of these and broken by the other, and no error is raised either way. Read the first matrix a little further and it gets worse rather than better. Under the English-only model the Devanagari refund scores 0.48 against the Hinglish refund and 0.50 against the Hinglish parcel complaint, which are near-identical scores for a match and a mismatch: the vectors are being placed by strangeness rather than by meaning. A cutoff of 0.45 copied from a blog would have accepted both, which is WR-033 arriving from a new direction. The right answer is in the model card, where the first model states it was trained on English text — the failure was documented before it happened, and four minutes of reading would have found it. The price of the fix is honest and worth saying out loud: 92 MB and 22.7M parameters against 480 MB and 117.7M, roughly five times the disk for a model that handles your users' actual language. That trade-off, priced with your own numbers, is the paragraph a manager wants.
Work inside ai-lab with transformers, datasets and sentence-transformers installed. Everything here runs on a CPU. Downloads are real, so keep an eye on du -sh ~/.cache/huggingface, and prefer the small checkpoints named in the lesson over anything bigger you find while browsing. Where a shape or a count is given, your output should match the printed digits; where a score is given, land within about 1% of it — seeded torch runs vary slightly across builds; megabytes and seconds will differ with your machine and your connection, and reporting your own is the point. If your connection cannot pull a model at all, do the reading half of each exercise from the model pages on the Hub in a browser and say so in your notes.
Pick a model the way a professional picks one. The task: label incoming complaints for a Tiruppur knitwear exporter as one of four kinds — payment, delivery, product quality, or account. Create model_choice.md and bench.py. Part one: find three candidate checkpoints on the Hub for this task. At least one must be an English-only encoder of 66M parameters or fewer, and at least one must be multilingual, because roughly half your complaints arrive in Hindi or in Hindi written with Roman letters. Part two: before downloading anything, fill a comparison table with one row per model and these columns in this order — licence tag, licence in the card body, training data described, limitations section present, safetensors present, weight file size in MB, parameter count, and the sha you would pin. Use HfApi().model_info(...) and ModelCard.load(...) from Step 3 so the table is generated, not typed. Part three: write twelve test complaints of your own, three per category, with at least four in Hindi or Hinglish, and run all three models over them with bench.py, recording accuracy against your own labels plus seconds per complaint and megabytes on disk. Part four: write one paragraph of at most 150 words recommending a model to a manager who has not read any of this, naming the licence, the measured accuracy, the size, and the one risk you would watch. Check: your table has three complete rows and at least one cell reads MISSING or None rather than being quietly filled in; any model whose licence tag is blank is marked blocked in your recommendation rather than merely noted; your twelve test complaints and their true labels are in the file, and the accuracy column comes from running them rather than from the model card; the English-only model scores visibly worse on your Hindi complaints than on your English ones, and you name the training-data line that predicted it; and your paragraph gives one number for accuracy, one for size and one for speed, with the licence stated in the first sentence.
Fine-tune something small, then kill it twice. Create finetune_drill.py, starting from Step 5 but typed rather than pasted, because TrainingArguments and the resume line are the parts you must own. Part one: fine-tune google/bert_uncased_L-4_H-256_A-4 on 2,000 rows of any small labelled dataset you load with load_dataset, using save_steps=20, eval_strategy="epoch" and a fixed seed, and record the uninterrupted end-of-run eval_loss and accuracy as your reference. Part two: run it again and kill it twice at points you did not choose in advance — press Ctrl-C, or close the terminal — noting the step number each time, and restart with resume_from_checkpoint=True and no edits to the script. Part three: put the reference numbers and the interrupted numbers side by side, including the epoch-1 evaluation, and state the largest difference. Part four: open the newest checkpoint folder, list every file with its size in MB, and write one line per file saying what it restores and what would break without it. Part five: delete optimizer.pt from a checkpoint, resume from it, and record what happens to the loss at the next evaluation. Check: your interrupted run's final eval_loss and accuracy match your reference to the printed digits, across both kills, as run A and run B both reached 0.5479 and 0.738 here; your resume line prints the step it restarted from and it is a multiple of your save_steps; your file list names at least the weights, the optimizer state, the scheduler, the random state and the trainer state, with optimizer.pt roughly twice the size of the weights and a comment saying why; the run resumed without optimizer.pt gives a different number from the reference and you name Adam's buffers as the cause; and one closing line states how many steps of work your cadence can lose in the worst case.
Rows, vectors, and the format check. Create data_and_vectors.py. Part one: load a small dataset by its full namespace/name id, print its splits and features, then use filter, a batched map that adds one computed column, and a seeded train_test_split, reporting row counts and the label balance at every stage. Part two: open a dataset far larger than your free disk with streaming=True, pull five rows, apply one filter and one map over the stream, and print how much your dataset cache grew — measure it, do not assume it. Part three: embed ten sentences of your own with sentence-transformers/all-MiniLM-L6-v2 — five pairs that mean the same thing, at least two of those pairs written in an Indian language or in that language typed with Roman letters — and print the rounded 10 by 10 cosine matrix with labelled rows. Part four: repeat part three with paraphrase-multilingual-MiniLM-L12-v2, put the two matrices side by side, and write three lines naming which pairs each model got right, the disk cost of the better one, and the sentence in the weaker model's card that predicted its failure. Part five: run torch.load(..., weights_only=True) on any checkpoint you have produced in this module and record that it loads clean, then write one line on what that flag protects you from. Check: your stream section shows the remote dataset's total size in GB next to a cache growth under 1 MB; every intended pair beats every unrelated pair under at least one of the two models, and you say which; your two matrices disagree on at least one pair by 0.3 or more, as 0.05 against 0.74 here, and your explanation quotes the card rather than guessing; your seeded split gives the same counts on a second run and you show both runs; and your closing line on weights_only=True names code execution, not file corruption, as the risk.
Three ecosystem failures cost teams more than any modelling mistake in this module. Each has a War Room entry with the full diagnosis path:
- WR-145 — the licence that blocked the launch. A model picked from a search result went into a commercial product, and days before release the licence turned out to be research-only, with a blank field on a second candidate that had lakhs of downloads; the fix is a licence check at selection time, above accuracy in the table, plus a pinned revision and an approved-models list a fresher can use without asking.
- WR-146 — the pickle that ran code on load. An untrusted
.bincheckpoint executed its author's code the moment it was loaded, before any check in the script could run, and the damage showed up as an outbound connection rather than an error; the fix is preferring safetensors, keepingweights_only=Trueon every load, and treating publisher identity like a package name. - WR-147 — the model that did not fit the machine. A 7-billion-parameter checkpoint was downloaded onto an 8GB laptop, filled the disk, thrashed swap and killed the session; the fix is parameters times bytes each against free memory before the download, a smaller checkpoint that passes the same evaluation, and module 2.9's channel decision for anything genuinely large.
The Hub holds models, datasets and Spaces, and the model card is the label you read before writing code. A pipeline answered in 0.08 seconds on a CPU, and one level down the same 268 MB checkpoint showed 66,955,010 parameters and six attention tensors of shape (1, 12, 14, 14). The datasets library reshaped 8,530 rows in seconds and streamed a 72 GB repository while the disk grew by 0.1 MB. A Trainer fine-tune of an 11.2M model took 11 seconds, and a session killed at step 60 resumed from checkpoint-40 to the same 0.738 accuracy. Licence, training data, limits, file format and size decide whether a model may ship, and one English-only encoder scored 0.05 where a multilingual one scored 0.74.
- Indian support and complaint routing runs on small public encoders. Banking and e-commerce teams label UPI and delivery tickets with checkpoints of this size, because the volume is large and the budget is not. The multilingual choice from Step 6 is the difference between reaching Hindi customers and dropping them.
- Services firms and GCCs keep an approved-models list. A client contract decides what may ship, so licence, revision and file format are recorded per model, the same way library versions are. A fresher who arrives with that habit is trusted with real work sooner.
- Your own portfolio lives on this shelf. A Space with a working demo and a card that states the data, the limits and the licence is a stronger interview artifact than a slide. Module 6.8 turns that link into a resume line.
Answer these before opening the gate assessment. Each question names its type.
type=interpretation— An audit printslicence tag: None,downloads/month: 338,223andcard length: 3,761 charactersfor a model already wired into a product demo. State what each of the three numbers does and does not tell you, say which one decides whether the demo can ship, and give the one action you would take before the next standup.type=debug— A teammate's entity tagger returns eight entities for a five-word sentence containing two Indian names, splitting each name across several results with high scores. They want to raise a bug against the model. Diagnose the actual cause, name the one command that proves it in a line, and give the fix that does not involve retraining anything.type=output-prediction— A Trainer run hassave_steps=20, 126 total steps, and dies without warning at step 97, then restarts withresume_from_checkpoint=Trueand no edits. Predict the step printed by the resume line, the number of steps repeated, and what the finaleval_lossshould equal if the checkpoint folder was complete.type=trade-off— For a complaint router where about half the messages are Hindi or Hinglish, compare a 92 MB English-only embedding model against a 480 MB multilingual one, on an 8GB laptop and then on a small cloud server. Give one concrete gain and one concrete cost of each in each setting, name the two scores from this module you would quote, and say what you would measure before committing.type=scenario-judgement— A teammate found a checkpoint on a forum, downloadedpytorch_model.binfrom an account with no other uploads and no card, loaded it successfully, and wants to demo it tomorrow because "it works and the results look good". Give your judgement, name the two separate risks in that sentence, and state what you would require before the file is opened on any machine holding credentials.
Phase 2 is finished, so explain the whole of it to a friend who is starting it next month. Walk the arc in order — one neuron, then a network, then the craft of making training work, then CNNs and borrowed backbones, then sequences, then the Transformer block, then finding free compute, and finally the shared shelf where the field publishes its work. Use at most 4 sentences, name one thing you measured yourself along the way, and use no word your friend would have to look up. If they can then say what changed between "I build the model" and "I choose the model", you own this phase.