A timeline of AI.

McCulloch and Pitts describe a mathematical model of a neuron — inputs, a threshold, an output. It is a drastic simplification of biology and it turns out to be enough. Every neural network since is built from descendants of this unit. Worth noting how early this is: the idea that thinking might be computation predates the computers to test it on.

SourceMcCulloch & Pitts (1943), A Logical Calculus of the Ideas Immanent in Nervous Activity — Bulletin of Mathematical Biophysics 5(4). The artificial neuron, twelve years before the field had a name.

Computing Machinery and Intelligence opens by asking whether machines can think and immediately abandons the question as too vague to answer. Turing proposes judging behaviour instead — if it converses indistinguishably from a person, that's the evidence we get. It's the most influential dodge in computer science, and it set the field's habit of measuring what systems do rather than settling what intelligence is.

SourceTuring (1950), Computing Machinery and Intelligence — where the field chose to sidestep the definition and ask about behaviour instead. Still the most influential dodge in computer science.

The Dartmouth workshop proposal coins the term artificial intelligence and suggests that a summer's work by ten people might make significant progress on language, abstraction and self-improvement. It's worth reading now for the confidence. That gap — between what looked close and what proved hard — is the field's oldest and most repeated pattern.

SourceMcCarthy, Minsky, Rochester & Shannon (1955), A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence — the document that coined the term, written a year before the workshop it proposed.

Rosenblatt builds a machine that learns to classify by adjusting weights from examples — the first working neural network, and the direct ancestor of everything modern. The press coverage promised machines that would walk, talk and reproduce. The actual device could separate simple patterns. Both facts matter: the technique was real, and the claims made about it were not.

SourceRosenblatt (1958), The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain — Psychological Review 65(6).

Minsky and Papert prove that a single-layer perceptron cannot learn XOR — a limitation real and narrow, since layered networks can. But the result landed as a verdict on the whole approach, funding moved elsewhere, and neural network research went quiet for over a decade. A cautionary tale about how a specific technical finding becomes a general mood.

SourceMinsky & Papert (1969), Perceptrons: An Introduction to Computational Geometry — MIT Press. Remembered as the book that killed neural networks; it proved a limit of single-layer ones.

A review commissioned by the UK government concludes that AI had failed to deliver on its promises, and British funding is largely withdrawn. Similar cuts follow elsewhere. The first AI winter isn't caused by the technology failing — it's caused by the technology failing to match what had been claimed for it. That distinction is the whole lesson.

SourceLighthill (1973), Artificial Intelligence: A General Survey — Science Research Council. The report behind the first AI winter.

Rumelhart, Hinton and Williams show how to assign blame across the layers of a network — trace the error backwards, work out how much each weight contributed, adjust accordingly. Backpropagation makes deep networks trainable, and every model running today still uses it. The algorithm was in place decades before the data and hardware that would make it matter.

SourceRumelhart, Hinton & Williams (1986), Learning representations by back-propagating errors — the paper that made neural networks trainable. :: https://doi.org/10.1038/323533a0

LeCun and colleagues apply convolutional networks to handwritten digits, and the system goes into production reading cheques. It's the first commercially significant deep learning system, and it works because the architecture encodes a real fact about images: nearby pixels are related, and a pattern means the same thing wherever it appears.

SourceLeCun et al. (1989), Backpropagation Applied to Handwritten Zip Code Recognition — Neural Computation 1(4). Convolutional networks, working, on real post.

Chess falls to a machine, and the reaction establishes a pattern the field has repeated ever since. Before: chess was the benchmark of machine intelligence. After: it was 'just' search and evaluation, and the real test moved elsewhere. The goalposts didn't move dishonestly — people genuinely learned that chess needed less than they'd assumed.

SourceHochreiter & Schmidhuber (1997), Long Short-Term Memory — Neural Computation 9(8). For Deep Blue: Campbell, Hoane & Hsu (2002), Deep Blue — Artificial Intelligence 134(1–2), published five years after the match.

Hochreiter and Schmidhuber design a recurrent unit that can hold information across long sequences, working around the vanishing gradient problem that made earlier recurrent networks forget. LSTMs dominate sequence modelling for nearly two decades — until attention makes recurrence itself optional.

SourceHochreiter & Schmidhuber (1997), Long Short-Term Memory — Neural Computation 9(8). For Deep Blue: Campbell, Hoane & Hsu (2002), Deep Blue — Artificial Intelligence 134(1–2), published five years after the match.

Fei-Fei Li's team publishes a labelled image dataset of unprecedented scale. It isn't an algorithm, and it changes more than most algorithms did. The lesson the field took years to absorb: the bottleneck had been data, not ideas. Nearly everything in the next decade runs on this insight.

SourceDeng et al. (2009), ImageNet: A Large-Scale Hierarchical Image Database — CVPR. The dataset, not the model, was the unlock.

A convolutional network wins the ImageNet competition by a margin wide enough to end the argument, trained on two consumer gaming GPUs. This is the result that starts the deep learning era — not because the technique was new (it wasn't), but because the data and the hardware had finally arrived to make a thirty-year-old idea work.

SourceKrizhevsky, Sutskever & Hinton (2012), ImageNet Classification with Deep Convolutional Neural Networks — AlexNet, the result that started the deep learning era.

Mikolov's team shows that words can be mapped to vectors where distance means similarity and directions carry meaning — the origin of king − man + woman ≈ queen, and of embeddings generally. Everything that searches by meaning rather than by keyword descends from this.

SourceMikolov et al. (2013), Efficient Estimation of Word Representations in Vector Space — word2vec, and the origin of the king−man+woman analogy. :: https://arxiv.org/abs/1301.3781

Goodfellow's setup pits a generator against a discriminator — one faking, one detecting, both improving by losing to each other. GANs make machine-generated images convincing for the first time and dominate for years, before diffusion largely displaces them. The training instability that plagued them is the reason.

SourceGoodfellow et al. (2014), Generative Adversarial Nets — the original, and unusually readable.

He and colleagues add shortcut connections so gradients have a path back that isn't multiplied down through every layer. Suddenly networks can be hundreds of layers deep instead of dozens. It's a small architectural idea that removes the ceiling on everything else.

SourceBaydin et al. (2015), Automatic Differentiation in Machine Learning: a Survey — what your framework is actually doing.

Go was supposed to be a decade away — too many positions to search, too much dependent on intuition. Reinforcement learning combined with search closes it, and move 37 of game two is still discussed because it was a move no human would play and it was correct. Then the goalposts moved again.

SourceClark & Amodei (2016), Faulty Reward Functions in the Wild — reward hacking demonstrated on a boat race, and the clearest illustration you will find.

Vaswani and colleagues drop recurrence entirely. Instead of reading a sentence word by word, the transformer weighs every word against every other word at once — and because that's parallel, it trains on hardware that recurrence couldn't exploit. Nearly every significant model since is built on this architecture. It's the single most consequential paper of the era.

SourceVaswani et al. (2017), Attention Is All You Need — the original, and still readable. :: https://arxiv.org/abs/1706.03762

BERT and the first GPT establish the recipe that still holds: train on an enormous general corpus, then adapt to your task afterwards. Transfer learning stops being a technique and becomes the assumption. This is why small teams can build things — you inherit the expensive part.

SourceHoward & Ruder (2018), Universal Language Model Fine-tuning for Text Classification — discriminative learning rates and gradual unfreezing.

Brown and colleagues show that making models bigger doesn't just improve them — it changes what they can do. Examples in the prompt start substituting for training, which nobody designed for. Kaplan's scaling laws arrive the same year and turn 'make it bigger' from a hunch into an equation. Large language models become a category.

SourceBrown et al. (2020), Language Models are Few-Shot Learners — GPT-3, and the demonstration that scale alone changes what models can do. :: https://arxiv.org/abs/2005.14165

Lewis and colleagues formalise fetching documents at query time and letting the model answer from them, rather than baking knowledge into weights. RAG becomes the standard answer to 'the model doesn't know our data' — and remains the most commonly misdiagnosed system in production.

SourceLewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — the paper that named it. Worth knowing it describes a different system to today's: DPR and a BART generator fine-tuned jointly, not a frozen model with text in the prompt. :: https://arxiv.org/abs/2005.11401

Radford's team trains on image–caption pairs until pictures and words share one embedding space. That alignment is the groundwork under multimodal systems — and under text-to-image generation, which needs a way to connect a prompt to a picture.

SourceJumper et al. (2021), Highly accurate protein structure prediction with AlphaFold — Nature 596.

Protein structure prediction — a fifty-year-old problem in biology — is substantially solved by a deep learning system. It's the clearest evidence to date that these methods do real scientific work rather than parlour tricks, and it's the result most likely to still matter in fifty years.

SourceJumper et al. (2021), Highly accurate protein structure prediction with AlphaFold — Nature 596.

Latent diffusion moves the denoising process into a compressed space, and suddenly image generation runs on consumer hardware. Within months it's in public hands. The copyright and consent arguments that follow are still unresolved.

SourceRombach et al. (2022), High-Resolution Image Synthesis with Latent Diffusion Models — Stable Diffusion, and the move to latent space that made it runnable on consumer hardware.

Ouyang and colleagues train on human preferences instead of correct answers — people compare two outputs and say which is better, and the model learns to produce the preferred kind. RLHF is the step that turns a text predictor into something that answers your question. Underrated, and the reason the next entry landed the way it did.

SourceOuyang et al. (2022), Training language models to follow instructions with human feedback — InstructGPT, the paper that made assistants work. :: https://arxiv.org/abs/2203.02155

The underlying model was not dramatically new. The interface was — a text box, no instructions needed, free. It reaches a hundred million users faster than anything before it, and AI stops being a topic and becomes a product category. The lesson most of the industry drew was about models. The lesson available was about interfaces.

SourceKaplan et al. (2020), Scaling Laws for Neural Language Models, and Hoffmann et al. (2022), Training Compute-Optimal Large Language Models — the second corrected the first on how to spend a compute budget. :: https://arxiv.org/abs/2001.08361

Llama 2 makes capable models downloadable under a licence that permits most commercial use, and an ecosystem forms around running models yourself. The argument about what open means — weights without training data isn't open source — starts here and hasn't finished.

SourceTouvron et al. (2023), Llama 2: Open Foundation and Fine-Tuned Chat Models — the release that made this mainstream, licence and all.

ReAct and tool use turn models into systems that act rather than answer. The demos are extraordinary and the production record is not, and the gap between them becomes the field's most expensive lesson: a system that's 90% reliable per step is a coin flip after seven steps.

SourceSchick et al. (2023), Toolformer — models learning when to call a tool, rather than being told.

Models that generate reasoning before answering shift spending from training time to serving time. It's a genuine change in the economics — capability becomes something you buy per request rather than something the model has — and it quietly retires the assumption that inference is the cheap part.

SourceSnell et al. (2024), Scaling LLM Test-Time Compute Optimally — the argument that inference-time compute can substitute for training-time compute.

As a named field, 1956 — the Dartmouth workshop coined the term. But the underlying ideas are older: McCulloch and Pitts modelled an artificial neuron in 1943, and Turing's paper asking whether machines can think was published in 1950.

No single person. The term was coined at the 1956 Dartmouth workshop organised by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon. The intellectual groundwork was laid earlier by Turing, McCulloch and Pitts among others.

Periods when funding and interest collapsed — roughly 1974–1980 and 1987–1993. The trigger wasn't the technology failing outright. It was the technology failing to match what had been promised for it, most visibly after the 1973 Lighthill report led the UK to withdraw funding.

2012, with AlexNet winning the ImageNet competition by a wide margin. The techniques were decades old — backpropagation dates to 1986, convolutional networks to 1989. What changed was the arrival of large labelled datasets and GPUs fast enough to train on them.

November 2022. Its underlying model was not a dramatic advance on what existed; the significant change was the interface and free public access, which took it to roughly a hundred million users faster than any consumer product before it.

Attention Is All You Need (Vaswani et al., 2017), which introduced the transformer. It removed recurrence from sequence modelling, which made training parallel and therefore scalable. Nearly every significant model since is built on that architecture.

Because intelligence has never had an agreed definition, so the frontier defines it. Chess was the benchmark until Deep Blue won in 1997, at which point it became 'just search'. The same happened with Go in 2016. Each time, people learned the task needed less than they'd assumed.