MLOps
The engineering around a model that makes it a system rather than a notebook — and the model is a few percent of it.
When not to use it
- (It's a discipline. The question is how much.)*
- Buying a platform before you have a problem. A container, an endpoint and a rollback covers two years for most teams.
- Testing code and calling it tested. Most ML failures are data failures, and unit tests don't touch data.
- Assuming abstraction will save you. CACE is a property of the artefact, not a discipline failure.
- Believing LLMs removed it. They removed the training pipeline. The entanglement and config debt moved into the prompt.
Reach for something else instead
- A container and an endpoint — genuinely enough for most teams, for a long time.
- Managed inference — someone else's serving problem.
- The ML Test Score — a rubric instead of a platform. Free.
- Not deploying ML — if a rule or a query solves it, the whole category disappears.
Sources & further reading
- Sculley et al. (2015), Hidden Technical Debt in Machine Learning Systems — the small box, CACE, and why ML debt is structural. Read this one.
- Breck et al. (2017), The ML Test Score: A Rubric for ML Production Readiness — score yourself; you'll do badly.
- Paleyes, Urma & Lawrence (2022), Challenges in Deploying Machine Learning: A Survey of Case Studies — what actually goes wrong, from people it went wrong to.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Focusing on the small box. The model is a few percent of the system and most of the attention.
- Reasoning locally about an ML system. CACE: changing anything changes everything.
- Not versioning data alongside code. You can't rebuild the model you're serving.
- Thinking LLMs made this go away. Configuration debt in English is still configuration debt.
At a glance
FieldTools & Ecosystem
The famous diagram"ML code" is the small box
The core claimML accrues technical debt through mechanisms ordinary software lacks
CACEchanging anything changes everything; there are no abstraction boundaries
What LLMs changedthe nouns
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
ML systems vs. ordinary software — one has contracts and boundaries you can reason within; the other is an entangled function of data you don't control, and it degrades rather than fails.