Neural Network
A system of simple connected units that learns patterns from examples — the foundation underneath deep learning and modern AI.
Reviewed July 10, 2026Stable
When not to use it
- On tabular data. Gradient-boosted trees still beat neural networks on most spreadsheet-shaped problems, train in seconds, and explain themselves.
- With small datasets. A few hundred rows and a neural network is a recipe for memorising noise. Simpler models generalise better when data is scarce.
- When you must justify each decision. "The weights say so" doesn't survive a regulator, a clinician, or a loan applicant.
Reach for something else instead
- Gradient boosting (XGBoost, LightGBM) — the honest default for tabular prediction.
- Linear and logistic regression when interpretability is the requirement, not an afterthought.
- Classical algorithms — sometimes the task is a sort, a join, or a rule, and no learning is needed at all.
Read more on the blog
- How neural networks work: the idea under all of itUnder every transformer, every image generator, every language model, sits one idea: the neural network. Here's how it actually works, neurons, weights, layers, and the simple trick by which it learns from its own mistakes, explained so it finally makes sense.
- What is deep learning? The complete guideAlmost every AI system that impresses today, from chatbots to image generators to voice assistants, runs on deep learning. Its core idea is a single powerful shift: instead of humans hand-crafting the features a model uses, deep networks learn their own layered representations of data, from simple edges to whole objects. This guide explains what deep learning is, how it works, its architectures, and its limits.
- What actually caused the AI wintersThe book that supposedly killed neural networks proved a true theorem and attached a false conjecture. The field remembered the conjecture. Three myths about the AI winters, and what the record shows instead.
- How quantization shrinks AI models without breaking themA 70-billion-parameter model needs about 140 GB of memory at full precision. Your laptop has 16. Quantization is how the model fits anyway, by storing each weight in far fewer bits, and the surprising part is that you can throw away most of that precision and the model barely notices. Here is why, and where it finally breaks.
A neural network is layers of nodes, each connection a learned weight.
Input values enter on the left, flow through one or more hidden layers where each node combines all its inputs, weights them, and applies a non-linear activation, and exit as an output. Learning means adjusting the weights on every connection until the outputs are right. Depth — more hidden layers — is what makes a network deep.
Further reading
- Rumelhart, Hinton & Williams (1986), Learning representations by back-propagating errors — the algorithm everything still runs on.
- Grinsztajn, Oyallon & Varoquaux (2022), Why do tree-based models still outperform deep learning on tabular data? — the paper to cite when someone reaches for a neural net on a spreadsheet.
- LeCun, Bengio & Hinton (2015), Deep Learning (Nature) — the field's own summary of why depth mattered.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Adding layers to fix a data problem. More capacity memorises faster; it doesn't understand better.
- Skipping the simple baseline, so nobody knows whether the network is actually earning its complexity.
- Confusing training loss going down with the model getting good. That's the definition of overfitting, watched in real time.
At a glance
FieldDeep Learning
Core idealearn patterns from examples
Learns viabackpropagation + gradient descent
Needsdata and compute
DifficultyBeginner → Intermediate
Where this sits
3 concepts come first. Understanding it opens up 156 more.
3Levelsteps in
3Needs firstconcepts
156Opens up50% of 310
11Areascrosses fields
Learn these firstPerceptron
Computed from the prerequisite graph, not assigned. How this works