Machine learning is a way of programming computers with examples instead of instructions. Rather than writing down the rules for a task, you collect examples of the task done correctly, and a training process finds a function that maps inputs to outputs — one that keeps working on cases it has never seen. That last part, called generalization, is the entire point.
That's the short answer. The longer answer starts with a question the short one skips: why would anyone program with examples? Instructions are precise, auditable, and debuggable. Examples are none of those things. The reason is uncomfortable and mostly absent from the textbooks: we tried instructions first, for thirty years, and they failed — not for lack of computing power, but for a reason that says something real about human knowledge.
The rules we couldn't write
For most of AI's history, the obvious plan was to write the rules down. Intelligence was assumed to be reasoning, reasoning was manipulating symbols, so the job was to encode what experts know and let a program apply it. This was symbolic AI, it dominated the field into the 1980s, and it produced genuinely useful systems — as long as the domain was small, formal, and fully describable.
Then it hit a wall with a name. Ask a doctor how they recognise a particular rash and you get a partial answer, some contradictions, and eventually "I just know it when I see it." They're not being difficult. The philosopher Michael Polanyi put it in one line: we know more than we can tell. You can recognise your friend's face in a crowd and cannot write down how. You can ride a bicycle and cannot state the control law you're executing. The expertise is real; the rules-shaped version of it does not exist to be extracted.
Symbolic AI's engineers met this as a practical problem — they called it the knowledge acquisition bottleneck — and it broke the paradigm. The knowledge wasn't hidden or expensive. It was never available in the form the approach required. The failures compounded into funding collapse and an AI winter, and the field needed a way around the bottleneck rather than through it.
Machine learning is that way around. If the doctor can't state the rule but can label a thousand photos — rash, not rash — then the labelling is enough. Show a learning algorithm the photos and let it find the regularities the doctor couldn't articulate. Machine learning is the engineering workaround for Polanyi's paradox: it extracts what people know from what people do, skipping the step where they explain it. That's why it took over, and it's also — hold the thought — why its failures look the way they do. A program built from examples inherits everything in the examples, including the parts nobody meant to put there.
What "learning" actually means here
The word invites mysticism, so it's worth being blunt about the mechanics. The standard definition comes from Tom Mitchell's 1997 textbook: a program learns if its performance at a task, as measured somehow, improves with experience. Strip the abstraction and the modern practice looks like this: a model is a mathematical function with adjustable numbers in it — parameters. Training feeds it examples, measures how wrong its outputs are with a loss function, and nudges the parameters to be slightly less wrong, millions of times. For neural networks the nudging is gradient descent steered by backpropagation; for a decision tree it's greedy splitting; the family changes, the shape of the idea doesn't.
Statisticians look at this and say: that's curve fitting. They're right, and the field's occasional defensiveness about it is misplaced — Leo Breiman's "Two Cultures" essay drew exactly this line in 2001, between modelling how the data came to be and predicting what comes next, and machine learning simply is the second culture, industrialised. What makes it more than a curve through known points is the demand that the curve keep working on new points. A model that's merely memorised its training set is worthless; you can get perfect recall of the past from a database. The gap between performing on seen data and performing on unseen data is where the whole discipline lives — it's why data is split into train and test sets, why overfitting is the field's cardinal sin, and why every honest evaluation guards the test set like a exam paper before exam day.
There's one more piece the mechanics hide. Learning from finitely many examples is, strictly, impossible without assumptions — infinitely many different functions pass through any finite set of points. Every method therefore smuggles in a preference for some kinds of pattern over others, called its inductive bias: trees prefer axis-aligned splits, linear models prefer straight lines, convolutional networks prefer patterns that look the same wherever they appear in an image. "Learning from data alone" is a phrase with no referent. The choice of method is a choice of assumptions, made before the first example arrives.
The three kinds of learning
Nearly everything in practice is one of three setups, distinguished by what you provide and what the algorithm must figure out for itself.
| Setup | What you provide | What it learns | Typical use | The catch |
|---|---|---|---|---|
| Supervised | Inputs with correct answers | To predict the answer for new inputs | Spam filters, medical imaging, price prediction | Someone must label everything — and the labels are less clean than anyone assumes |
| Unsupervised | Inputs, no answers | Structure: clusters, patterns, compressions | Customer segments, anomaly detection | No answer key means no objective way to say the structure it found is the right one |
| Reinforcement | An environment and a reward signal | A strategy that maximises reward over time | Game playing, robotics, tuning language models | It optimises the reward you wrote, not the outcome you meant — and the gap bites |
Supervised learning dominates commercial practice because it's the setup with the clearest contract: here are the answers, learn to produce them. It's also, not coincidentally, the direct descendant of the Polanyi workaround — the labels are the expert demonstrations. The frontier has since blurred the categories: large language models are trained with a trick called self-supervision, where the labels are manufactured from the data itself by hiding the next word and asking the model to predict it. No human labels anything, yet the mechanics are supervised prediction. It's the reason the label bottleneck stopped limiting how big models could get, and it's covered properly in the deep learning entry.
The data is the program
Here's the mental shift that separates people who reason clearly about machine learning from people who don't: in classical software, behaviour comes from code; in machine learning, behaviour comes from data. The training set isn't an input to the program. It effectively is the program, with the algorithm acting as a compiler. Nobody would run code they'd never reviewed, yet teams routinely train on data they've never looked at.
What's in the data that nobody meant to put there? Start with the labels. The canonical benchmarks that progress is measured against were assumed clean for a decade, until a 2021 study led by Curtis Northcutt went and checked: across ten of the most-used test sets, an average of 3.3% of labels were simply wrong — in ImageNet's validation set, the one that ranked a decade of computer-vision breakthroughs, about 6%. Mislabeled test data doesn't just add noise; the study found it could flip which of two models ranked higher. The field spent years optimising against an answer key with wrong answers in it, and the errors were found not by a new technique but by looking.
Then there's what the data represents. A model learns the world as sampled by its training set, and samples have opinions: a hiring model trained on past decisions learns past decision-makers, a class-imbalanced fraud dataset teaches the model that predicting "not fraud" every time is 99.9% accurate, and labels made by people disagree with each other far more than anyone budgets for — measuring inter-annotator agreement before trusting a dataset is the cheap habit that catches this. None of these are exotic failures. They're the default, found in most real datasets by anyone who looks, which is the practical answer to "why did the model do that?": with high probability, because the data did that first.
Where it goes wrong — and how you'd know
Machine learning's failures have a signature: the number says fine while the behaviour is broken. Code fails loudly, with exceptions and stack traces. Models fail silently, with a respectable accuracy score attached. A short field guide:
| Failure | What it looks like | The check that catches it |
|---|---|---|
| Overfitting | Excellent on training data, mediocre on anything new — memorised, not learned | Held-out test data, cross-validation, regularization |
| Data leakage | Test performance too good to be true, because information from the test set (or the future) seeped into training | Split by time and entity, then audit every feature: could the model know this at prediction time? |
| Wrong metric | 99% accuracy on a dataset where 99% of examples are one class | Precision and recall, a confusion matrix — anything that looks at the errors, not the average |
| Miscalibration | The model says “90% confident” and is right 70% of the time | Measure calibration directly; modern networks are confidently wrong by default |
| Contamination | A model “aces” a benchmark whose questions leaked into its training data | Fresh, post-training-cutoff test material; deep suspicion of round-number triumphs |
| Drift | A model that was fine at launch quietly decays as the world changes under it | Monitor live performance, not launch performance; retrain on schedule |
Notice what column three has in common: every check is a form of measuring more carefully. That's the corpus-wide pattern this site keeps returning to — in machine learning, the mistakes overwhelmingly live in the measurement, not the mathematics. The famous results that dissolved under scrutiny (mislabeled benchmarks, contaminated test sets, leaderboard gains inside label-error bars) were not defeated by cleverer models. They were defeated by someone checking the ruler.
When you shouldn't use it
An honest account has to include the cases where the thirty-year-old failed paradigm is still the right call. If the rules can be written down — tax brackets, chess legality, eligibility criteria — write them down: a rule executes perfectly, explains itself, and never hallucinates, and using a learned model where a lookup table belongs is trading correctness for fashion. If you have dozens of examples rather than thousands, most methods will memorise rather than generalise, and regression with three features will beat anything deep. If a wrong answer is catastrophic and must be justified afterwards — sentencing, aviation, medical dosing — the honest framing is that you're deploying a system whose individual decisions nobody can fully explain, and sometimes the right engineering decision is the boring one. The full entry carries a longer version of this list.
The quiet truth of industrial practice follows the same logic: for tabular business data — the rows-and-columns world of churn, credit, and inventory — the consistently winning tools are not neural networks but gradient-boosted trees, and a well-tuned random forest remains one of the strongest baselines available. Deep learning's dominance is real but domain-shaped: it owns perception and language, not spreadsheets.
Where deep learning fits
Deep learning is machine learning — one family within it, built on neural networks with many layers. Its distinguishing move is that it absorbed a step the rest of the field did by hand: classical practice spent most of its effort on feature engineering, where humans decided which measurements of the raw input the model should see. Deep networks learn the features too, straight from pixels or audio or text. That — plus the discovery that performance keeps improving with scale in a way regular enough to plot as scaling laws — is why the loudest results of the past decade, including the language models this site is read alongside, all come from this one family. It's also why "machine learning" and "deep learning" get used interchangeably in the news, and why they shouldn't be: the perceptron-to-transformer story has its own entry, and its own pillar-length treatment of AI as a whole.
What to watch
Three checkable things, no vibes.
Whether data quality becomes a first-class discipline. The label-error results reframed a decade of progress; the interesting question is whether dataset auditing becomes as standard as code review, or stays a thing that wins best-paper awards precisely because nobody does it.
Whether the tabular exception holds. Every year brings a paper claiming neural networks finally beat trees on tabular data, and every year practitioners keep shipping gradient boosting. If that flips for real, it says something about whether deep learning's advantage is fundamental or domain-shaped.
Whether evaluation catches up with capability. Models are increasingly judged on benchmarks they may have partially memorised, with contamination checks that remain optional. Watch for evaluation on genuinely fresh material becoming the norm — until then, treat leaderboard movements inside a few percentage points as weather, not climate.
Everything linked above is explained at five depths — read as far as you need and stop. Start with Machine Learning itself, browse the machine learning field, look terms up in the glossary, or open the map and follow the edges.
Frequently asked questions
What is machine learning in simple terms?
It's teaching a computer by example instead of by instruction. Rather than writing rules for recognising spam, you show the computer thousands of emails marked spam or not spam, and a training process finds the patterns that separate them — patterns it can then apply to email it has never seen.
What's the difference between AI and machine learning?
AI is the goal — machines doing things that seem to require intelligence. Machine learning is a method for getting there: learning behaviour from data instead of hand-writing it. It's currently the dominant method by a wide margin, which is why the terms blur together, but AI also includes older rule-based approaches like symbolic AI. The full AI pillar untangles the terms properly.
Is machine learning just statistics?
They're deeply related, and the honest answer is "substantially yes, with a different goal." Statistics traditionally asks what process generated this data; machine learning asks what will the next data point be. Same mathematics, different contract — and the prediction-first culture turned out to scale to problems, like vision and language, that model-the-process statistics never cracked.
Do you need a lot of data for machine learning?
For classical methods on simple problems, hundreds to thousands of examples can work, especially with careful feature engineering. Deep learning is far hungrier, which is why the biggest models train on internet-scale text. With only dozens of examples, machine learning is usually the wrong tool — a hand-written rule or a simple regression will beat it.
Why do machine learning models make weird mistakes?
Because they learn whatever regularities are actually in the training data — including shortcuts, biases, and label errors nobody noticed. A model has no idea which patterns are "the real ones"; it has only the data. Roughly 3% of the labels in the field's most trusted test sets turned out to be wrong, and models trained and ranked on that data inherited every error. When a model behaves strangely, the first place to look is what it was shown.
Can machine learning models explain their decisions?
Mostly no, and be suspicious of confident claims otherwise. Simple models — decision trees, linear regression — are genuinely inspectable, which is a real reason to prefer them when stakes are high. Deep networks are not: explanation tools exist, but they produce plausible stories rather than guaranteed accounts, and the gap between the two matters exactly when explanation matters most.