Episode 1 — Why an AI System Is Not Ordinary Software

This is where the course starts. Before learning any MLOps tool, you need to understand precisely what breaks when you put a statistical model inside a software system — otherwise every practice that follows looks like bureaucracy.

Learning objectives

By the end of this episode you will be able to:

State the three axes of change that distinguish an AI system from a classical application, and explain why they multiply rather than add.
Use the technical-debt taxonomy (Sculley et al., 2015) to name the failure modes of a real ML system.
Explain CACE and non-monotonic error behaviour, and why they defeat classical modular design.
Position the nine-stage ML workflow and identify where its feedback loops sit.
Say what the Twelve-Factor App still buys you — and precisely where it stops being sufficient.

Why it matters

An architect who treats an ML system as "a web service with a .pkl file inside" will design something that passes every test on day one and degrades silently for eighteen months. The whole discipline of MLOps is a set of countermeasures to a list of specific, documented, repeatable failures. Learn the failures first; the practices then become obvious rather than ceremonial.

1. The three axes of change

A classical application changes along one axis: code. Version control, code review, CI/CD, and rollback all assume that if the code is identical, the behaviour is identical.

An ML application changes along three axes: code, data, and model.

Read the right-hand box carefully. Two properties have no equivalent on the left:

The model artifact is derived, not written. Nobody can review it line by line. Its quality is only observable statistically, on data.
The world is an input at runtime. Behaviour changes even when nothing you own changes. This is the single most important sentence in this course.

Consequence: a build can be triggered by a code change, a data change, or a model change — and each of the three needs its own versioning, its own tests, and its own promotion rules. The rest of the course is largely the elaboration of that sentence.

Axis	Artefacts to version	Typical size	Reviewable by a human?
Code	Application code, pipeline code, configuration	KB–MB	Yes, by diff
Data	Datasets, features, labels, schemas	GB–TB	No — only by statistics
Model	Weights, hyperparameters, training runs, metrics	MB–GB	No — only by evaluation

Mnemonic — D-M-C. Data, Model, Code. Every MLOps table you will meet in this course (principles, tests, monitoring, maturity) is organised in those three columns. If you can reconstruct the three columns, you can reconstruct most of the discipline.

2. The canonical diagnosis: hidden technical debt

The founding text is "Hidden Technical Debt in Machine Learning Systems" (Sculley et al., Google, NIPS 2015). Its thesis in one line: ML gives you a fast win, and the win is not free — it is a loan.

Its central empirical observation is that "only a tiny fraction of the code in many ML systems is actually doing ML." Everything around it is the debt surface.

2.1 Boundary erosion

Entanglement — the CACE principle: "Changing Anything Changes Everything." No input to a model is truly independent of the others. Change one feature, one hyperparameter, or the data distribution, and the learned meaning of all the other features shifts. This is why ML components resist classical modular decomposition: there is no stable interface between "feature 7" and the rest.
Correction cascades. You build model m′ as a patch on top of model m, then m″ on m′. Now improving the base model can degrade the system, and the whole chain has to be maintained as one unit.
Undeclared consumers (visibility debt). Your predictions are being read by a downstream system nobody told you about. A model change breaks a consumer you did not know existed — and you find out from a customer.

2.2 Data dependencies cost more than code dependencies

Compilers give us static analysis for code dependencies. We have no compiler for data.

Unstable data dependencies — an input signal that is itself the output of another model, or a table that quietly changes.
Underutilised data dependencies — legacy, bundled, epsilon-valued or correlated features that add fragility for almost no accuracy. Find them with leave-one-feature-out evaluation and delete them.

2.3 Feedback loops

Direct feedback loop — the model influences its own future training data. A ranking model decides what gets shown, what gets shown determines what gets clicked, and clicks are the labels.
Hidden feedback loop — two systems influence each other through the world, with no code path between them (two independent recommenders on the same site).

2.4 System-level anti-patterns

Anti-pattern	What it looks like
Glue code	The ~95 % of supporting code written to get data in and out of a general-purpose ML package; it freezes the system to that package's idiosyncrasies.
Pipeline jungles	Data preparation grown organically — scrapes, joins, sampling — testable only end to end.
Dead experimental codepaths	Conditional branches left behind by experiments. The paper cites Knight Capital's $465 M loss in 45 minutes as the price of dead flags.
Abstraction debt	There is no agreed abstraction for "an ML system" comparable to the relational model for data.
Configuration debt	The config (which features, which date range, which pre-processing, which learning settings) is large, changes more often than the code, and is treated as an afterthought.

The paper's remedy for configuration is worth memorising as a checklist: a good configuration system is easy to diff against the previous config, hard to make manual errors in, visually inspectable, automatically assertable, and versioned and reviewed like code.

2.5 The world moves

Fixed thresholds in dynamic systems — a decision threshold chosen once becomes invalid after the next retraining.
Monitoring is the central operational recommendation. Three things to watch: prediction bias (the distribution of predicted labels should track the distribution of observed labels; slice it to localise the drift), action limits (cap the real-world actions a system may take and alert on the cap rather than acting on an anomaly), and up-stream producers (monitor the data producers, and propagate their SLAs into your own).

2.6 How debt is measured

Not by velocity — by the ability to change. The paper's diagnostic questions are excellent architecture-review material:

How easily can an entirely new algorithmic approach be tested at full scale?
What is the transitive closure of all data dependencies?
How precisely can the impact of a new change be measured?
Does improving one model or signal degrade others?
How quickly can a new team member be brought up to speed?

3. The empirical confirmation: a case study inside Microsoft

"Software Engineering for Machine Learning: A Case Study" (Amershi et al., ICSE 2019, Best Paper) studied how Microsoft teams actually build AI features. It found teams grafting ML onto existing Agile workflows, and it isolated three differences from traditional software engineering:

Discovering, managing and versioning data is much more complex than any dependency management in classical software.
Model customisation and model reuse require different skills from those typically present in a software team.
AI components are harder to isolate as modules, because of model entanglement and non-monotonic error behaviour — improving one component can degrade overall system quality.

Point 3 deserves a pause. In classical engineering, local improvement is monotonic: a faster sort makes the program faster. In ML, local improvement is not monotonic — a better sub-model can make the product worse. Every promotion decision must therefore be evaluated at system level, never at component level. That single fact justifies most of the gated-promotion machinery you will meet in Episode 4.

The nine-stage workflow

Two things to retain: the workflow is highly iterative — feedback loops exist between most stages, with large loops back to data collection and small loops between training and evaluation — and this nine-stage decomposition is the reference used by nearly every later maturity model. When Episode 3 shows you Google's levels 0/1/2 and Azure's levels 0–4, you will recognise these stages being progressively automated.

4. What classical software engineering still gives you

Nothing above invalidates ordinary engineering discipline. The Twelve-Factor App remains the baseline for the serving side of any AI system: one codebase, many deploys; explicitly declared dependencies; config in the environment; backing services as attached resources; strict separation of build / release / run; stateless processes; port binding; scale out via the process model; disposability; dev/prod parity; logs as event streams; admin tasks as one-off processes.

An AI platform that violates dev/prod parity or hard-codes config will fail for entirely classical reasons before it ever fails for ML reasons.

But note what the twelve factors do not have a factor for: data, models, and — from Episode 10 onward — prompts. The methodology assumes the behaviour of a deploy is fully determined by code + config. That assumption is exactly the one ML breaks.

That progression — code → +data/model → +prompt/context — is the spine of this whole course.

5. Putting it together: the mapping you should be able to recreate

Almost every practice taught later is a countermeasure to one item in the 2015 list.

Debt / failure mode	Countermeasure taught later	Episode
Reproducibility debt	Data + model versioning, deterministic pipelines	3, 7
Pipeline jungles	Declared, orchestrated ML pipelines	2, 7
Training/serving skew, underutilised features	Feature store	4, 7
Undeclared consumers, no lineage	Model registry + lineage + aliases	6, 7
Changes in the external world	Drift detection and continuous monitoring	8
Configuration debt	Configuration as code, environments as code	3, 9
Non-monotonic errors	Gated promotion, system-level evaluation, canary	4, 6
Fixed thresholds	Thresholds re-derived at each retraining	8

Numbers & names to memorize

3 axes of change: code, data, model — the D-M-C columns.
CACE — Changing Anything Changes Everything (entanglement).
9 stages in the ICSE 2019 ML workflow; 3 differences from classical SE.
"Only a tiny fraction of the code is actually doing ML" — Sculley et al., NIPS 2015; glue code ≈ 95 % of the codebase.
Knight Capital: $465 M in 45 minutes — the canonical cost of dead codepaths.
12 factors — the classical baseline; zero of them cover data or models.
Debt is measured by ability to change, not velocity.

Key takeaways

An AI system changes along three axes, and the world is one of its runtime inputs.
CACE and non-monotonic error behaviour mean ML components cannot be validated in isolation — only at system level.
The 2015 debt taxonomy is not history; it is the requirements document for MLOps.
The nine-stage workflow with its feedback loops is the decomposition every maturity model automates.
Twelve-Factor is necessary and insufficient: keep it, then add the data, model, and prompt axes.

Exercises

Exercise 1 — Diagnose the system (scenario)

A retailer runs a demand-forecasting service. Facts gathered during an architecture review:

The training set is assembled by a notebook that reads a sales_daily table maintained by the BI team; nobody in the ML team knows its refresh schedule.
A second model, promo_uplift, takes the forecast as one of its inputs.
The ordering system multiplies the forecast by a safety factor of 1.15 hard-coded in 2023.
A finance dashboard reads the forecast table directly. The ML team learned this last month, from a complaint.
Retraining is a manual Friday job; the last four runs used slightly different feature code, kept in four notebook copies.

Name five distinct debt items from the taxonomy, and give one countermeasure for each.

Exercise 2 — The monotonicity trap (scenario)

Your team improves the recall of a fraud-detection model from 0.71 to 0.79 on the offline test set, at constant precision. The product owner wants it shipped on Monday. Give two mechanisms by which this genuine improvement could make the product worse, and state the minimum evidence you would require before promoting.

Exercise 3 — MCQ

Which statement best characterises the CACE principle?

A. Any change to production code must be reviewed by two engineers.
B. Changing any input, hyperparameter or data distribution alters the learned meaning of all the others, so ML components resist modular decomposition.
C. Every model change must trigger a full retraining of every downstream model.
D. Model accuracy always degrades over time in production.

Exercise 4 — MCQ

An ML system's behaviour changed noticeably last week. No commit was merged, no model was redeployed, and no configuration was edited. Which axis should you investigate first?

A. The code axis — there must be an unmerged hotfix.
B. The model axis — the artifact must have been corrupted.
C. The data axis and the input distribution — the world moved.
D. None; with no change on any axis, behaviour cannot change.

Answers

Exercise 1. Any five of the following, with countermeasures:

Debt item	Evidence in the scenario	Countermeasure
Unstable data dependency	`sales_daily` owned elsewhere, refresh schedule unknown	Contract + SLA with the producer, freshness assertion in the pipeline, monitor up-stream producers
Correction cascade	`promo_uplift` built on top of the forecast	Evaluate the pair as one system; forbid new corrective layers
Fixed threshold in a dynamic system	Safety factor 1.15 frozen in 2023	Re-derive the factor at each retraining; treat it as a versioned parameter, not a constant
Undeclared consumer / visibility debt	Finance dashboard reading the table directly	Published interface with an explicit consumer list; access via a governed endpoint
Reproducibility + configuration debt	Four divergent notebook copies, manual Friday job	One versioned pipeline, config as code, automated trigger
Direct feedback loop (bonus)	Orders driven by the forecast shape future sales data	Log the decision alongside the prediction, so future training data can be de-biased

Exercise 2. Two plausible mechanisms: (a) non-monotonic error behaviour — the additional true positives may be concentrated in a segment where blocking a transaction is far more costly than the fraud avoided, so a metric gain becomes a business loss; (b) direct feedback loop — a stricter model blocks more transactions, so future training data contains fewer fraud examples, biasing the next model toward "good" traffic and degrading it over time. Minimum evidence before promotion: system-level, not component-level evaluation — sliced metrics on the segments that matter, a shadow or canary run on live traffic, an explicit action limit, and a re-derived decision threshold (the old one is invalid for the new model).

Exercise 3. B. CACE is about entanglement of inputs and the loss of modularity, not about review policy or scheduled retraining. D describes model decay, a different phenomenon.

Exercise 4. C. The distinguishing property of an ML system is that the input distribution is a runtime input. With code, model and config unchanged, the data axis — upstream producers, feature coverage, distribution shift — is the first place to look. D is the classical-software intuition this episode exists to destroy.