Deep Learning
Machine learning using neural networks with many layers — the approach behind nearly every recent AI breakthrough.
When not to use it
- On small or tabular data. Deep learning's advantage shows up with scale and unstructured input. Below that, it's a slower, hungrier way to do worse.
- When compute or latency is tight. A model that needs a GPU per request is an architecture decision with a bill attached.
- When you need to explain the decision. Depth and interpretability trade against each other, and no amount of saliency mapping fully closes that gap.
Reach for something else instead
- Gradient boosting for tabular problems — still the state of the art, still faster.
- Classical CV/NLP for constrained tasks in controlled conditions.
- A pretrained model via API instead of training your own. Most teams don't need to train anything.
Sources & further reading
- LeCun, Bengio & Hinton (2015), Deep Learning (Nature) — the field's own account of itself.
- Goodfellow, Bengio & Courville, Deep Learning — the textbook, free online.
- Grinsztajn, Oyallon & Varoquaux (2022), Why do tree-based models still outperform deep learning on tabular data? — the honest limit.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reaching for deep learning because it's the interesting option, not the fitting one.
- Ignoring inference cost during model selection, then discovering the economics after the demo.
- Assuming more layers means more understanding. It means more capacity to memorise.
At a glance
FieldFoundations
Core ideamany-layered neural networks
Key powerlearns features from raw data
Enabled bydata + GPUs + architecture advances
DifficultyBeginner → Intermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4