Field
Foundations
The words underneath everything else.
This field defines the terms the rest of the site assumes. It's also where the honest answers are most uncomfortable.
Intelligence has no agreed definition — over seventy published ones and counting — which is why arguments about whether AI is "really" intelligent never resolve. Artificial intelligence is defined by its frontier, so the definition moves every time the machines succeed: chess was the benchmark until a computer won, then it was "just search". AGI is a hypothetical nobody can measure, and seventy years of predictions about it have been wrong in the same direction.
The practical entries are here too: the difference between training and inference — which sets your entire cost structure — and the three ways machines learn (machine learning, deep learning, reinforcement learning).
Start with Artificial Intelligence, then Intelligence if you want to know why nobody can define it.
59 concepts in this field
Machine Learning
Getting computers to learn patterns from data and improve at a task, instead of being explicitly programmed with rules.
Deep Learning
Machine learning using neural networks with many layers — the approach behind nearly every recent AI breakthrough.
Reinforcement Learning
Learning by trial and error through rewards — the way you'd train a pet, applied to software.
Artificial Intelligence
The field of making machines do things that seem to require intelligence — a definition that has moved every time the machines succeed.
AGI (Artificial General Intelligence)
A hypothetical system with broad human-level capability across domains — undefined enough that people can argue about whether it's arrived.
Training vs Inference
Building the model versus using it — two completely different activities with different costs, hardware, and constraints.
Intelligence
The word underneath "artificial intelligence" — used constantly, defined by nobody, and the reason the field's biggest arguments never resolve.
Benchmark
A standard test used to compare AI systems — indispensable for progress, and routinely mistaken for a measure of the thing it approximates.
Scaling Laws
The finding that model performance improves predictably with size, data and compute — the empirical result that justified spending billions, and it isn't a law.
Benchmark Contamination
When the test is in the training data — the problem that makes most published model scores impossible to fully trust.
Markov Decision Process
The formal frame underneath all of reinforcement learning — built on an assumption that's almost always false, and it works anyway.
Reward Function
The number that tells an agent what you want — and the hardest thing to write correctly in all of AI.
Q-Learning
Learning the value of every action in every state, by bootstrapping off your own estimates — which converges beautifully in theory and diverges in practice.
Policy Gradient
Learning the behaviour directly instead of learning values — the method behind RLHF, and its entire difficulty is variance.
PPO
The policy gradient method that trains language models — and a careful study found its gains came from the implementation details, not the idea in the paper.
Exploration vs Exploitation
Take the best thing you know, or look for something better — the trade-off underneath every learning system, with a known optimal answer that almost nobody uses.
Turing Test
The 1950 proposal that a machine should count as thinking if it can pass for human in conversation — a test of deception, which Turing said plainly and everyone forgot.
Symbolic AI
The idea that intelligence is symbol manipulation, and you build it by writing down what you know — the paradigm that ruled AI for thirty years and lost.
Expert System
Encoding a specialist's knowledge as rules — AI's first commercial success, and its collapse taught the field something it's currently relearning.
AI Winter
The periods when AI's promises outran its results and the money left — twice, and the question of whether the pattern is over is genuinely open.
Search Algorithm
Systematically exploring possibilities to find a good one — AI's oldest technique, its most complete success, and nobody calls it AI anymore.
Emergence
Abilities that appear suddenly at scale rather than improving gradually — the most cited claim about large models, and a NeurIPS best paper says it's a measurement artefact.
Test-Time Compute
Spending more compute when the model answers rather than when it trains — the scaling axis the field found after the first one got expensive.
Foundation Model
A large model trained broadly once and adapted to many tasks — a term coined to name a shift in how AI gets built, and contested from the day it was proposed.
Frontier Model
The most capable models in existence at any moment — a term invented mainly so that regulation could point at something.
World Model
An internal model of how things change, learned well enough to imagine what happens next — and the leading candidate for what current AI is missing.
Superintelligence
An intellect that greatly exceeds humans in every domain — a concept whose most important argument is a 1965 speculation about what such a thing would build next.
Singularity
The hypothetical point past which technological change becomes unpredictable to humans — a term borrowed from physics, with a long record of confidently wrong dates.
Information Theory
Shannon's mathematics of surprise — a way to measure information in bits, which turns out to be the measure almost everything in machine learning is quietly optimising.
Entropy
The average surprise of a distribution — a single number saying how uncertain an outcome is, and the quantity most of machine learning is built to reduce.
Cross-Entropy
The cost of being wrong about a distribution — the loss function training most classifiers and every language model, and information theory's most-used export to ML.
KL Divergence
A measure of how far one distribution is from another — not a distance, but the workhorse behind variational inference, RLHF, distillation, and diffusion.
Mutual Information
How much knowing one thing tells you about another — the general measure of dependence that captures relationships correlation misses.
Bayes' Theorem
The rule for updating a belief when new evidence arrives — the mathematical backbone of learning from data, and a genuine fix for how badly human intuition handles probability.
Actor-Critic
The reinforcement-learning architecture that pairs a decision-maker with a judge — one network chooses actions, another scores them — and underpins most modern deep RL.
Moravec's Paradox
The observation that tasks humans find hard, like formal reasoning, are comparatively easy for machines, while tasks we do without thinking, like perception and movement, are extremely difficult.
Compositionality
The meaning of a whole is determined by its parts and how they combine — the principle that lets a finite vocabulary express unbounded thoughts, and the standard AI systems are measured against.
Symbol Grounding
The problem of how symbols get meaning if they are defined only by other symbols — the sharpest form of the objection that a text-trained system cannot understand anything.
Construct Validity
Whether a measurement actually captures the thing it claims to, rather than something correlated with it that is easier to count.
Pre-registration
Declaring what you will measure and how before you look at the data, which is what separates a test of a hypothesis from a search for one.
Task Redefinition
Changing what a system is asked to do so that it becomes tractable, which is how most successful automation actually happened.
Citation Decay
A claim losing its source through repetition, until a number everyone cites has no traceable origin.
Load-Bearing Assumption
A judgement that determines a reported result while presenting as a fact, so the result inherits an uncertainty nobody sees.
Scope Boundary
What a measurement counts and what it leaves out, which is usually the difference between two figures that appear to contradict each other.
Correlated Exposure
Several risks that look independent resolving to the same underlying variable, so they move together at exactly the moment separation would have helped.
Binding Constraint
The input that actually limits output, which is usually not the one being discussed and often not the one anyone is spending on.
Survivorship Bias
Drawing conclusions from what remains visible, when the thing you need to know is contained in what disappeared.
Error Asymmetry
When being wrong in one direction costs far more than being wrong in the other, which makes a single accuracy figure close to meaningless.
Rate Against Level
Where the current size of something is reassuring and its growth rate is not, so the two framings support opposite conclusions from the same data.
Proxy Decay
A measurement that genuinely worked, because of a correlation nobody wrote down, and stopped working when that correlation broke.
Refutation Cost
What it costs to check a claim relative to what it cost to make it, which breaks systems when only one of those numbers falls.
Selective Transmission
The qualifying material sitting next to a quoted figure, in the same document, that does not travel with it.
Commissioned Framing
Which question gets measured at all, determined by who was willing to pay for an answer, so a subject's evidence base takes the shape of its buyers rather than its importance.
Self-Report Gap
Where asking the operator and measuring the artefact give different answers, systematically in the operator's favour, because effort saved is felt and cost deferred is not.
Undecided Commitment
A dependency incurred through a choice made for local reasons, which binds for years without ever having been treated as a commitment.
Aggregate Evidence Gap
Where every individual study is rigorously produced and the field-level number is unreliable, because quality control attaches to the artefact and nobody owns the sum.
Comparator Choice
What a result was measured against, which is frequently unstated and often nothing, and which determines what the result can support.
Unrecorded Stratifier
A variable that plausibly changes a result, known to be unevenly sampled, cheap to record, and absent from the record, which makes an aggregate uninterpretable rather than merely imprecise.
Measurement Concentration
Research effort settling at the stage of a causal chain that is cheapest to instrument, which is rarely the stage that decides the outcome.
Read more on the blog
- The EU delayed the part with no standardsThree AI Act obligations take effect today and the widely reported headline says the opposite. What was deferred, what was not, and why the split falls exactly where it does.
- 72 seconds, or 30 minutes, and both are trialsTerritory 11 opens on the first subject this corpus has examined where the evidence is genuinely good. Registered trials, CONSORT-AI reporting, peer review, and effect sizes that still differ by a factor of twenty-five.
- The 12% was non-inferior, and P was 0.41MASAI is the best-evidenced AI deployment in medicine and the headline everyone quoted describes a result the trial did not claim.
- Nine subjects, and the model was almost never the blockerTerritory 10 closes. Across nine enterprise deployment subjects the binding constraint was organisational in every one, the evidence was commissioned in almost all of them, and the reconciling study exists nowhere.