I spent a month writing down what every AI concept depends on

I run a free AI encyclopedia. It had 256 concepts, each written at five depths, each linked to the ones it relates to. What it did not have was any sense of order. A reader could see that attention relates to transformers, but nothing told them which to read first, or what either of them assumed you already knew.

So I spent a month writing that down. One line per concept: what must you understand before this makes sense. The result is 350 directed links across 256 concepts, and sorting it produced several things I did not expect, including two I had explicitly predicted wrong.

This is not research. It is one person's judgement about how a subject fits together, written down carefully enough that the consequences can be computed. The consequences turned out to be more interesting than I thought they would be.

The one rule

Direct prerequisites only. If A requires B, and B requires C, then C does not go on A.

That sounds pedantic and it is the entire method. The temptation is to list everything a concept assumes, which for something like a transformer is twenty other things. Do that and the graph becomes a hairball where every node points at every earlier node, and the depth numbers stop meaning anything.

Listing only the direct dependency keeps each line short and each judgement small. Instead of asking "what does a transformer need," which is overwhelming, you ask "what is the last thing you need before a transformer makes sense." That has an answer: self-attention, and positional encoding. Everything further back is reached through those, and the machine can work it out.

The transitive closure is computed, never authored. That distinction is what makes a month of work possible rather than a year of it.

I wrote down what I expected first

Before authoring the machine learning field, I committed a file to the repository with five predictions in it. Not because anyone asked, but because I could see the obvious failure mode: I was about to author a graph and then look for interesting patterns in it, which is a recipe for finding exactly the patterns I already believed.

If the results matched my predictions, that would be weak evidence the graph encoded my expectations rather than any structure. If they contradicted them, that would be stronger evidence something real was there.

Here is what I predicted for the most load-bearing concepts, in order:

1. Training Data 2. Supervised Learning 3. Generalization 4. Overfitting 5. Regression

I got one, two and five right. Three and four were badly wrong, and finding out why was the most useful thing that came out of the whole exercise.

What the graph says

Across all 256 concepts, ranked by how many others become reachable once you understand them:

Two concepts gate more than seven tenths of the field. Training Data sits at the root, and almost nothing in modern AI is reachable without it. That is unsurprising when stated, and I had not appreciated the scale of it until the number came out: 211 of 255 other concepts sit downstream.

There are eleven true starting points, concepts that require nothing: Artificial Intelligence, Training Data, GPU, Markov Decision Process, Bayesian Inference, Knowledge Graph, PyTorch, Spectrogram, Information Theory, Bayes' Theorem, and Natural Language Processing. Those are the genuine entry points to the subject. Everything else is downstream of at least one of them.

The first thing I got wrong

Generalization ranks 35th. Overfitting ranks 54th.

I predicted them third and fourth. Generalization unlocks 11 concepts. Overfitting unlocks 4.

This felt like a bug when I first saw it. These are among the most-discussed ideas in machine learning. Every course covers them, every practitioner has opinions about them, and the entire bias-variance apparatus exists to talk about them.

But look at what actually depends on them. You can learn decision trees without generalization. You can learn clustering, k-nearest neighbours, gradient descent, backpropagation, embeddings, and attention without either. They are not components of those things. They are commentary on them: the vocabulary we use to describe whether learning worked, rather than machinery that learning is built from.

That is a real distinction and I had not seen it before doing this. Prominence in how a field talks about itself and position in how that field's ideas depend on each other are different properties, and they come apart more than I would have guessed.

The practical version: if you are learning AI and you keep hearing about overfitting, that does not mean you need it early. It means people talk about it a lot.

The second thing I got wrong

I predicted Double Descent would be the deepest concept in machine learning, since it seemed to require generalization, overfitting, bias-variance and regularization, each with their own prerequisites.

It sits at level 6. The deepest concepts in the whole corpus are at level 15: Red-teaming, Prompt Injection, and Task Decomposition.

The theory concepts sit off to one side of the build chain rather than at the end of it. Depth follows what must be constructed before something exists, and the long chains run through engineering rather than through theory. Reasoning Model, at level 12, requires 23 prior concepts: perceptron, network, loss function, gradient descent, attention, transformer, large language model, prompting, chain-of-thought, RLHF, RLVR, and so on. That is a construction sequence. Double descent is an observation about such systems, and observations do not need to be built.

Fields have different shapes

This is the finding I did not predict at all, because it had not occurred to me to look for it.

Machine Learning is the flattest field and the most load-bearing. Its deepest concept is level 6, its median is 3, and it contains the two things that gate most of the corpus. It is wide rather than tall: a plateau of sibling methods, trees and SVMs and nearest neighbours and Bayes, that do not build on each other. You can learn them in almost any order.

AI Agents is the opposite. Median depth 12, meaning a typical agent concept sits twelve layers into the subject. Nothing in that field is a starting point, because agents are built on language models, which are built on transformers, which are built on the rest.

Safety & Ethics is bimodal, deepest 15 but median 4. Some of it, bias and privacy and regulation, is reachable almost immediately. The rest, alignment and deceptive alignment and prompt injection, requires most of the technical stack first.

That has a consequence for anyone writing a curriculum. A course that treats these fields uniformly is fighting the structure. Machine learning can be taught in fragments; agents cannot.

What I am not claiming

These are authored judgements. A different person would produce a different graph, and therefore different numbers. I have tried to make that checkable rather than just admitting it: the predictions were committed before the analysis, the whole relation is published as open data, and every concept page shows its own prerequisites so you can disagree with any specific call.

It is also not complete in an interesting sense. The corpus is 256 concepts because that is what I have written, not because AI has 256 concepts. Adding more would change the numbers. The two-fields-only version of this graph told me the transformer unlocked nothing, which was an artifact of scope and disappeared once I added a third field.

And "level 12" is not a difficulty rating. It says twelve layers of dependency sit underneath a concept, not that it is hard. Some deep concepts are easy once you arrive. Some shallow ones are hard.

What it is now used for

The graph turned out to be more useful as infrastructure than as a result.

Every concept page now shows where it sits: how many concepts come first, how many it opens up, whether it is a starting point, a bridge between fields, or a destination. There is a tool where you name something you want to understand and it returns everything you need first, in order, with anything you have already read removed. And there are eight bounded tracks, each the full prerequisite closure of a destination: "how language models work" is 20 concepts, "computer vision" is 9. A corpus that has no end acquires routes that do.

None of that is authored. It is all the same 350 lines, sorted differently.

If you want to do this to your own subject

The method is not specific to AI. Anything with a body of interrelated concepts has a dependency structure that nobody has written down, and writing it down is tedious rather than difficult.

Three things I would tell you before starting. Direct prerequisites only, or the graph becomes unreadable and the numbers become meaningless. Write your predictions down first, because you will otherwise find whatever you expected to find. And expect the boring concepts to matter more than you think. The load-bearing things in my corpus were training data and supervised learning, not the ones with the interesting names.

The whole relation is open. If you think one of my prerequisite calls is wrong, it probably is, and I would like to know which.

Written by Azmath A., who built and edits Artifipedia.

Common questions

What is a prerequisite graph? A record of what must be understood before each concept in a subject makes sense, stored as directed links rather than the associative "related to" links most reference sites use. The distinction matters because association has no direction, so it cannot be sorted into an order, while a prerequisite relation can. Sorting it produces a learning order, a depth for every concept, and a measure of how much each one unlocks. In this case it covers 256 AI concepts joined by 350 direct prerequisite links.

Why only direct prerequisites? Because listing everything a concept assumes makes the graph unreadable and the derived numbers meaningless. A transformer transitively depends on twenty other things, but only two of them are the last step before it: self-attention and positional encoding. Recording just those keeps each judgement small and answerable, and the full chain is computed rather than written. That distinction is what makes the exercise a month of work instead of a year.

What does it mean that Generalization ranks 35th? It means only 11 of 255 other concepts require it before they can be understood, despite it being among the most-discussed ideas in machine learning. You can learn decision trees, clustering, gradient descent, backpropagation and attention without it. Generalization is vocabulary for describing whether learning worked rather than machinery learning is built from. The wider point is that prominence in how a field discusses itself and position in how its ideas depend on each other are different properties, and they come apart more than expected.

Which AI concepts unlock the most? Training Data, which gates 211 of 255 other concepts, or 83 percent of the corpus. Supervised Learning follows at 185, then Perceptron at 144 and Neural Network at 143. There are eleven concepts that require nothing at all and function as genuine entry points to the subject: Artificial Intelligence, Training Data, GPU, Markov Decision Process, Bayesian Inference, Knowledge Graph, PyTorch, Spectrogram, Information Theory, Bayes' Theorem, and Natural Language Processing.

Why pre-register predictions before building a graph? Because authoring a graph and then looking for interesting patterns in it is a reliable way to find the patterns you already believed. Committing predictions first makes confirmation bias falsifiable: if the results match, that is weak evidence the graph encodes expectations, and if they contradict, that is stronger evidence something structural is there. In this case five predictions produced three hits and two clear misses, and the misses were more informative than the hits.

Do different AI fields have different structures? Yes, and measurably so. Machine Learning is flat and wide, with a maximum depth of 6 and a median of 3, because its methods are siblings rather than a chain. AI Agents is the opposite, with a median depth of 12, since agents are built on language models which are built on transformers. Safety and Ethics is bimodal, reaching depth 15 while its median sits at 4, because bias and privacy are reachable early while alignment requires most of the technical stack. Any curriculum treating these fields uniformly is working against the structure.

What are the limits of this method? These are authored judgements, so a different person would produce a different graph and different numbers. The corpus contains 256 concepts because that is what has been written, not because the subject has 256 concepts, and adding more would shift the results. Depth is also not difficulty: a level 12 concept has twelve layers of dependency beneath it, which says nothing about how hard it is to grasp once you arrive. The whole relation is published as open data so any individual call can be checked and disputed.

Can this method be applied to other subjects? Yes. Nothing about it is specific to AI. Any body of interrelated concepts has a dependency structure nobody has written down, and writing it down is tedious rather than difficult. Three things matter: record direct prerequisites only, commit your predictions before running any analysis, and expect the load-bearing concepts to be less glamorous than you assume. In this corpus the two most structurally important concepts were training data and supervised learning, not the ones with the interesting names.