Geoffrey Hinton

b. 1947 · Quiet progress · 8 works · cited on 6 entries

His name is on the paper that made deep networks trainable, the one that made them win, and several of the parts they are still built from.

Contribution

Hinton's citations run through the whole of this encyclopedia rather than clustering on one idea. Backpropagation (1986) made deep networks trainable; AlexNet (2012) made them unignorable; t-SNE (2008) is how the field looks at its own representations; ReLU (2010), layer normalisation (2016) and distillation (2015) are components in almost every modern system. The through-line is not a single result but a bet held across two winters — that gradient descent on layered representations would work if the compute arrived. It did.

Common misreading

Backpropagation is often described as his invention. The 1986 paper says otherwise in its own citations — the algorithm had been derived several times before. What Rumelhart, Hinton and Williams showed was that it learned useful internal representations, which is a different and larger claim.

Influence

Entries shaped by this work

Deep LearningBackpropagation1986 — the paper that made depth trainable Computer VisionImage Classification2012 — AlexNet Deep LearningActivation Function2010 — ReLU Deep LearningDistillation2015 — distillation Machine LearningDimensionality Reduction2008 — t-SNE Deep LearningBatch Normalization2016 — layer normalisation

Selected works

Every reference below links to a search, not a stored URL — so it cannot rot or point at the wrong paper.

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. cited in the encyclopedia
Hinton, G. E. & Salakhutdinov, R. R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313(5786), 504–507. cited in the encyclopedia
van der Maaten, L. & Hinton, G. E. (2008). Visualizing Data using t-SNE. JMLR, 9, 2579–2605. cited in the encyclopedia
Nair, V. & Hinton, G. E. (2010). Rectified Linear Units Improve Restricted Boltzmann Machines. ICML. cited in the encyclopedia
Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS. cited in the encyclopedia
Hinton, G. E., Vinyals, O. & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv. cited in the encyclopedia
LeCun, Y., Bengio, Y. & Hinton, G. E. (2015). Deep Learning. Nature, 521, 436–444. cited in the encyclopedia
Ba, J. L., Kiros, J. R. & Hinton, G. E. (2016). Layer Normalization. arXiv. cited in the encyclopedia