BXUP Track progress & test yourself — free
War Room · Field diagnosis · ~2 min read
WAR ROOM · FIELD DIAGNOSIS

WR-050 · The free-tier session died at 90% and took everything with it

Symptom

A fine-tuning run on a free GPU notebook (Kaggle or Colab) terminates without warning — browser tab dead, "session expired", or a silent disconnect. On reconnecting, the working directory is empty or gone, the model state is lost, and hours of training produced nothing. Often the run was minutes from finishing. The GPU hours are also gone from the weekly quota, so the retry starts poorer than the first attempt.

Context

Free GPU tiers are the training budget for most students: Kaggle publishes around 30 GPU hours per week, and Colab's free GPU is best-effort with no promised runtime. Both can end a session at any time — idle timeouts, runtime caps, capacity reclaims. The failure hits anyone who wrote a training script as one long uninterrupted run, which is how almost every tutorial writes it. Linked module: 3.10 (Fine-tuning + decision framework); the doctrine comes from the programme's compute ruling.

Diagnosis

The run kept all progress in GPU memory and never wrote a checkpoint — a mid-run copy of the adapter weights — to disk. When the session died, the process and its memory died with it; there was nothing on disk to resume from, so the only option was starting over. Confirm in one look at the training arguments: if there is no save strategy (or it saves only at the end) and no resume-from-checkpoint logic at startup, every session death is a total loss. The deeper error is the plan, not the crash: a six-hour unbroken run assumes a reliability the free tier never promised.

The fix

Make the script safe to kill, then rerun it. Set save_strategy="epoch" so the LoRA adapter (roughly 100 MB) is checkpointed every epoch, with save_total_limit=2 to cap disk use. At startup, glob the output directory for the newest checkpoint-* and pass it to trainer.train(resume_from_checkpoint=...) — the script then behaves the same on session one and session five. Before each session ends (or right after a death), save the checkpoint folder as a Kaggle dataset and attach it to the next session. The interrupted run resumes at its last epoch instead of epoch zero.

Prevention

Treat session-splitting as the normal case, not the emergency: plan every run as "N epochs, resumable anywhere", and rehearse the kill-and-resume path on a toy loop locally before spending quota (module 3.10, exercise 3). Budget runs against the published Kaggle quota — never against Colab, which is bonus when it appears. First action in any new training notebook: verify a checkpoint appears on disk after epoch one, before letting the run continue.

In an interview

Q: "You have no GPU and a weekly free-tier quota. How do you fine-tune a 3B model reliably?" Strong answer: QLoRA so the model fits free hardware, adapter checkpoints every epoch, resume-from-checkpoint logic so sessions can die harmlessly, checkpoints persisted between sessions, and runs planned inside the published Kaggle quota with Colab treated as bonus. The phrase that lands: "I plan the run to die, so it cannot fail."

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in