Supervised Learning
Teaching an AI by showing it labelled examples — inputs paired with the correct answers — so it can predict answers for new inputs.
When not to use it
- When you have no labels and can't afford to make them. Labelling is the real cost of supervised learning, and it's usually underestimated by an order of magnitude.
- When the thing you're predicting changes faster than you can relabel. A model trained on last year's fraud catches last year's fraud.
- When the rule is known. If a human can write the condition down in a sentence, write the condition down. Don't learn what you already know.
Reach for something else instead
- Rules — cheaper, instant, auditable, and correct for anything with a known decision boundary.
- Unsupervised methods (clustering, anomaly detection) when you want structure found rather than categories assigned.
- Foundation models with few-shot prompting now solve many small classification tasks with no training set at all — worth testing before you commission labels.
Sources & further reading
- Hastie, Tibshirani & Friedman, The Elements of Statistical Learning — still the reference, and free from the authors.
- Kaufman et al. (2012), Leakage in Data Mining — the failure that explains most implausibly good results.
- Sculley et al. (2015), Hidden Technical Debt in Machine Learning Systems — why the model is the small part.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Leaking the answer into the features. If a column is only populated after the outcome, your 99% accuracy is measuring the future, not predicting it.
- Optimising accuracy on imbalanced data. Predict "not fraud" every time and you're 99.9% accurate and completely useless.
- Testing on data that resembles training data more than reality does. The model looks great until launch day.
At a glance
FieldMachine Learning
Core idealearn from labelled examples
Two tasksclassification & regression
Main costgetting labels
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Supervised vs. Unsupervised — learning from labelled answers vs. finding structure in unlabelled data.