The churn service starts returning noticeably different verdicts — field agents
report customers flagged who never were before, and recall on the weekly sample
drops. Yet nothing was deployed: no code change, no new image, no release ticket.
The API logs show a normal restart two days ago and nothing else. The service code
still reads models:/churn-model@production, same as always.
A team one step past module 4.4: tracking in place, registry in use, the service
loading by alias. A teammate finished training version 3, saw good numbers "in the
terminal", and pointed the production alias at it so the demo next morning would
show the new model. No evaluation against the current production version, no
checklist, no announcement. The service restarted that night and loaded version 3.
Linked modules: 4.4, 4.2.
Loading by alias means the alias IS the deployment. Moving it is a production change
even though no code moved — that is the exact power that makes registries useful,
and the exact power that needs a gate. Confirm in minutes: ask the registry which
version production points at, then open that version's linked run. Here version
3's run told the story — recall below the version 2 run, and a different data hash,
meaning its scores were measured on a different table and were never comparable in
the first place. The "no deploy happened" belief was the trap: the registry recorded
the alias move with a timestamp and user, but nobody thought to look, because
nobody thought of an alias move as a release.
Roll back first, investigate second: point the production alias back at version 2
— one registry call, service restored on next model load. Then judge version 3
honestly using its run record: retrain or re-evaluate it against the same data hash
as version 2, and only then decide if it deserves promotion. Finally, close the gap
that let this happen: write the promotion checklist and make it the only path to an
alias move.
Treat every alias move as a deployment, with a deployment's ceremony. A minimum
checklist: the candidate's metrics beat or match current production on the same data
hash; its run record is complete (params, metrics, hash, commit); a second person
confirms; the move is announced where the team will see it. Keep a candidate alias
so new versions have a place to live while being judged. In 4.6 this checklist
becomes a CI gate, which is the grown-up form: humans propose, the pipeline checks,
then the alias moves.
Q: "Your registry lets anyone promote a model with one command. Is that a feature or a risk?" Strong answer: both, and name the mechanism — alias moves are deployments, so they need a metrics gate on the same data fingerprint, a complete run record, and a second pair of eyes; rollback being one move backwards is the reward for the discipline. Candidates who mention comparing on the same data hash — not raw metric against raw metric — show they have actually judged models, not read about them.