The glossary.

Find the shortest path using a heuristic estimate of what's left. Provably optimal if the heuristic never overestimates — why your maps app works.

A protocol for AI agents to discover and talk to each other directly. The interoperability layer the agentic web is betting on. ::

Removing one part of a system to see how much it mattered. The standard way papers show a component earns its place. ::

Giving each token a fixed position signal. The original transformer recipe, largely displaced by relative schemes like RoPE.

The invisible, constant human fixing that made an unreliable data estate workable. An analyst receiving two conflicting customer counts notices and reconciles; an agent acts on what it retrieves. The requirement did not rise, the absorption stopped.

Letting a model answer "I don't know." The cheapest large win against hallucination, because much fabrication happens simply because refusing was never an option.

How often a system declines to answer. A rate of zero means the permission to say "I don't know" is not real, and every question is being answered from whatever was retrieved.

Showing two versions to two random groups and measuring. The only method that tells you whether your model actually helped anyone.

Any chip built to run neural networks faster than a CPU. GPUs by history, TPUs and NPUs by design. ::

The share of speculatively drafted tokens the target model accepts. Governs the entire speedup, is high on boilerplate and low on novel content, which makes the benefit content-dependent.

Whether each generation of training data is added to its predecessors or substituted for them. Under replacement, model collapse is proved; under accumulation, test error has a finite upper bound independent of iteration count. The condition is the result.

The share of predictions that were right. Misleading whenever classes are imbalanced: predicting "healthy" for everyone scores 99% on a rare disease.

The time between specifying a purchase and receiving it. A multi-year cycle applied to a technology redefined annually means the system bought at the end is not the system specified at the start.

Recomputing intermediate values during the backward pass instead of storing them. Trades compute for memory, and makes big-model training fit. ::

The small non-linear step applied after each layer. Without it, stacking layers would collapse into a single linear operation and depth would buy nothing.

Replacing an internal activation from one run with the value from another and observing whether the output changes. A causal test of whether a component matters, as distinct from correlational evidence that it varies with the answer.

A comparison against an existing option rather than against nothing, which answers the question a prospective user actually has. More informative, harder to justify ethically in a first trial, and the missing design in most AI intervention research.

A neuroscience-derived framework where acting and perceiving both minimise prediction error. Influential as theory; niche as engineering.

Letting the model choose which examples to have labelled next. Pays off exactly when labels are expensive and uncertainty is measurable. ::

How many of a model's parameters actually run for each token, as opposed to how many it stores. The number that sets the cost; the gap from the total is what makes MoE worth it.

RL with two learners: one picks actions, one estimates how good states are. The template behind most modern policy-gradient methods.

The default optimiser. Adapts the step size per parameter, which is why it works without much tuning.

Adam with weight decay applied correctly, decoupled from the gradient. The quiet default for training transformers.

Small trainable modules inserted into a frozen model. The family of tricks that made fine-tuning affordable.

A determination that another jurisdiction's data protection is equivalent enough to permit transfers without additional safeguards. One of the EU's transfer mechanisms.

The square matrix recording which nodes in a graph are connected. The standard dense representation, and impractical at scale, where sparse edge lists are used instead.

One that never overestimates the remaining cost. The condition that makes A* provably optimal.

Recorded AI usage falling in a sector, observed in utilities and construction during 2026. More informative than low adoption, since it means firms tried the technology, evaluated it and reduced usage rather than never starting.

Assembling multiple dies and memory stacks into one accelerator, distinct from fabricating the logic itself. Was the binding constraint on AI hardware supply, expanded repeatedly, and stopped binding while coverage continued to track it.

A deliberately crafted input — often imperceptibly altered — that makes a model confidently wrong. Prompt injection is its modern, language form.

An input nudged, often imperceptibly, to make a model fail. Evidence that models rely on patterns humans don't see. ::

Checking performed by a party whose interest is served by finding a fault. Structurally stronger than self-audit, and the reason legal AI failures are quantified while comparable failures elsewhere are not.

A decision to decline credit or similar, which carries a legal obligation to give the affected person a reason. It predates machine learning by decades and does not bend for it, which is why explainability in finance is a deployment gate rather than a preference.

A selection process producing substantially different outcomes across protected groups, whether or not any protected characteristic was used as an input. Proxies such as postcode, school, employment gaps and language patterns carry the signal without naming it.

A system that decides what to do next rather than answering once — usually a model in a loop with tools.

Measuring whether an agent actually works. Much harder than evaluating a model, and the reason demos and products are different things.

The controls around a system that acts rather than answers. Access control with a non-deterministic principal, not AI safety.

Transferring a task between agents or to a human. The transition where context is most often lost, and the step most orchestration frameworks specify least precisely.

The layer that runs agents: tool execution, state and memory across steps, round and spend limits, retries, and deciding what context each agent sees. Distinct from the framework, and it matters more.

Retrieval where an agent decides what to search, when, and whether the results suffice. RAG with a loop instead of a pipeline.

The cycle an agent runs: observe, decide, act, observe again. The loop is what separates an agent from a single model call.

How an agent remembers across sessions, since the model itself forgets everything. Retrieval and storage, not learning.

Many simple agents working loosely in parallel rather than under one orchestrator. More resilient, harder to predict — and currently more demo than deployment.

Marketing assistants, chatbots or scripted automation as agentic products without the underlying capability. Gartner estimates roughly 130 of the thousands of vendors claiming agentic AI sell one.

A global share hiding the local concentration that actually binds. Data centres at just under 3% of world electricity by 2030 is manageable; Ireland at around 21% of national electricity is a grid problem now, and the first figure conceals the second.

No detectable effect at the economy level, routinely reported as evidence that nobody was affected. It bounds the size of an effect and says nothing about individuals or cohorts: a 20% fall in one age band of one occupation is invisible in a national occupational mix.

Hypothetical AI with broad human-level capability. Undefined enough that people argue about whether it has arrived.

Predictions of when general AI arrives. The forecasts disagree by decades, and always have.

The point where a metric matches human annotators as closely as they match each other. Past it, further gains could be real discrimination or overfitting to one annotation protocol, and nothing in the measurement separates them.

The field of making machines do things that seem to require intelligence. The definition moves every time machines succeed.

A chip built for the matrix math of neural networks — GPUs, TPUs, and their rivals. The hardware layer the whole AI boom runs on.

A browser with an agent built in — reading pages, filling forms, completing tasks. The 2025–2026 wave of products racing to own the agentic web's front door.

An AI built to be a relationship rather than a tool. The ELIZA effect as a product, optimised for engagement.

Informal label for problems believed as hard as building general intelligence itself — solve one fully and you've solved AI. Machine translation was the classic example.

An assistant embedded in a workflow, suggesting rather than acting. The dominant product shape before autonomous agents. ::

A tool claiming to spot AI-written text. They don't reliably work, and they flag non-native English writers as machines.

Once a technique works reliably, it stops being called AI and becomes just software. The reason the field's successes keep disappearing.

What AI costs in electricity. Real and rising, and the most misreported set of figures in the field.

The field asking not whether AI can do something but whether it should — fairness, accountability, transparency, privacy, and who bears the consequences.

A proxy in front of model APIs handling routing, keys, rate limits, caching, and logging in one place. The emerging control point of LLM infrastructure.

The policies, oversight, and accountability structures around how an organisation builds and deploys AI — broader than regulation, which is only the part governments write.

An event that could plausibly lead to an incident but did not cause harm. Counted separately by the OECD monitor, which is why a total quoted without saying whether hazards are included can be wrong by thousands.

An event where an AI system is implicated in harm. There is no standard definition: one register counts alleged harm including near-misses, another counts established harm and separates hazards, and the two report 1,460 and 14,530 for the same phenomenon.

Software designed around model capabilities from the start, rather than adding AI to an existing product. ::

Seeing what your AI system actually did — traces of prompts, retrievals, tool calls, and outputs — so failures can be diagnosed rather than guessed at.

Coding with a model suggesting and reviewing in real time. The first AI workflow developers actually adopted at scale. ::

Data an agent can act on without supervision, as distinct from a data platform, which delivers a place to put data. Reported at 5%, 7% and 19% of organisations across four instruments using four different bars, with the direction unanimous and the magnitude unsettled.

Governments deciding what AI systems may do. Fast by legislative standards, slow by technological ones, and genuinely contested.

Running entirely inside your own perimeter with no external connection. The only deployment tier where the transfer question disappears rather than being managed, and it limits you to open-weight models.

The field working on making AI systems behave as intended and remain controllable — spanning today's guardrails and alignment research to long-term risk.

Content generated because it's cheap and published because nobody's accountable. A publishing decision, not a property of the tool.

Periods when AI's promises outran its results and the money left. Twice — and a winter is a funding event, not a research event.

The state where a warning fires often enough on non-events that its recipients stop attending to it. A prediction nobody acts on has a utility of zero regardless of its measured accuracy.

The 2012 CNN that won ImageNet by a margin nobody could ignore. The result that started the deep-learning era.

Who answers for an automated decision, and by what mechanism a person subject to one can contest it. Distinct from governance, which is about process, and from explainability, which is about mechanism. This is about consequence.

Systematic unfairness in a system's outputs. Usually inherited from data, occasionally amplified by the objective.

The problem of getting AI systems to actually do what people intend.

The capability a model loses when alignment training makes it safer. Ouyang et al. named it — and showed most of it can be bought back by mixing pretraining data into RLHF.

Hiding group assignment from the person doing the assigning. Frequently poor in education research, where its absence lets assignment drift toward the expected result without anyone intending it.

The collective operation that sums a value across all devices and returns the result to each. The core primitive of data-parallel training, and the one whose cost determines how far training scales.

Skipping branches of a game tree that provably can't change the decision. The trick that made classical chess programs fast enough to matter.

DeepMind's protein-structure predictor. The clearest case of deep learning solving a real scientific problem. ::

The system that beat the world's best Go players in 2016. The moment RL plus deep learning stopped being a curiosity.

Systems that listen to a clinician-patient conversation and draft a structured note for review. The most widely deployed clinical AI application, used by over 200,000 clinicians, and safe because a licensed human signs every note before it enters the record.

A system that records a clinical encounter and drafts the visit note. Reduces documentation time modestly and burnout more than the time saved explains, with roughly 7% of notes containing fabricated content, mandatory physician review, and no vendor accepting clinical liability.

Doubt created by a capability's existence rather than its use. The liar's dividend needs no fabrication: once convincing fakes are known to be possible, authentic evidence becomes deniable, and the denial costs a sentence.

Amdahl (1967): speedup from parallelism is capped by the fraction of work that must run sequentially. A job that is 5% serial cannot go more than twenty times faster however many processors you add. The ceiling on distributed training, and why scaling a cluster stops helping.

DORA's framing that AI magnifies existing organisational strengths and weaknesses rather than adding a fixed benefit. Where review and refactoring discipline exist, output improves; where they do not, the same tool produces more code needing rewriting, and both report high adoption.

Drawing from a generative model by sampling each variable in turn according to its probability, rather than taking the most likely value. The reason a model gives different answers to the same prompt.

The measurable part of a problem becoming the part everyone debates, regardless of its size. Individual query energy is about 2% of AI data centre consumption on the IEA's own arithmetic, and it is close to the whole of the public conversation.

Finding close enough matches fast instead of exact matches slowly. What every vector database runs on.

Humans labelling data for training or evaluation. The invisible workforce behind most supervised systems.

Finding the unusual thing. The base rate makes precision nearly impossible and almost every deployment drowns in false alarms.

A model natively processing and producing multiple modalities through one architecture, rather than routing between separate encoders and decoders. The distinction from earlier multimodal systems that bolted a vision encoder onto a language model.

The permissive license most open-weight models ship under. Commercial use allowed, patents granted, few obligations.

You can see an edge move perpendicular to itself but not slide along it. A limit on information, not on algorithms.

The interface you send a request to and get a model's answer back from. How nearly everyone uses AI.

A checkpoint where an agent must stop and get sign-off before acting — spending money, sending email, deleting data. Governance made concrete.

Trading exactness for speed in vector search. The word approximate is load-bearing: production typically runs at 90–95% recall.

Abstract grid puzzles trivial for humans and historically brutal for models — designed by Chollet to measure skill-acquisition, not memorised skill.

Operations performed per byte of memory traffic. Low intensity means memory-bound, which describes token-by-token generation, and is why adding compute does not speed it up.

Any file a training run produces: weights, logs, checkpoints. What experiment tracking exists to keep straight. ::

A system competent at specific tasks rather than generally. Describes every AI system that currently exists, and is the term usually omitted from discussions contrasting AGI with superintelligence.

The classic pathfinding algorithm: explore the route that looks cheapest counting both distance travelled and an estimate of distance remaining. Still everywhere, from games to robotics.

Silicon designed for one task rather than general computation. TPUs are the AI example: faster and more efficient than a GPU on the workload they were built for, and useless outside it.

Turning audio into text. The task Whisper made feel solved for high-resource languages.

The prompt sent was not the prompt intended: context in the wrong place, history truncated badly, an empty variable. The most common AI failure category and invisible unless you captured the assembled prompt rather than the template.

Deciding what to ask, examine, order and disregard, as distinct from reasoning over the result. A vignette supplies it and a patient does not, which is why model advantages are largest on curated cases and narrow on real clinical material.

Changing what is assessed rather than detecting how work was produced. Vivas, portfolios, drafts, revision memos, invigilated exams and hand-verified calculations are resistant for one shared reason: the submitted artefact stops being the only evidence.

Performance rising with a tool present and falling when it is removed. One randomised trial of roughly 1,000 students found assisted performance up 48% and unassisted exam performance down 17% against control, which is one result rather than two contradictory ones.

A conversation billed as solved because the customer stopped replying. Per one vendor's own documentation, 24 hours of silence after the agent's last reply marks the conversation resolved at the same rate as a confirmed one, so a customer who gave up and a customer whose problem was solved produce the same billable event.

The mechanism letting a model weigh every token against every other. The transformer's core idea.

As input grows, a model's finite attention spreads thinner across more tokens, so a passage strongly attended at 1K tokens can be functionally ignored at 100K.

One of many parallel attention computations in a layer. Different heads learn different relations, some interpretable, most not.

The tendency of attention to pile onto the first tokens regardless of content. A quirk that streaming-inference tricks exploit.

A figure produced by an analyst or journalist becoming attributed to the subject it describes, through repetition rather than any claim by that subject. One of three recognisable forms of citation decay, alongside orphaned statistics and projection hardening.

Sourcing that is missing, misleading or wrong, including facts falsely credited to a named publication. The largest error category in AI news summaries at 31%, and a direct consequence of a system that will not decline to answer needing something to cite.

The area under the ROC curve, and the probability a random positive outranks a random negative. Pure ranking quality; it says nothing about calibration or class balance.

Area under the precision-recall curve. The metric to reach for when positives are rare and ROC flatters.

A measure of how well a classifier separates classes across all thresholds. Useful when you haven't picked a threshold yet. ::

Naming what a sound is. The field borrowed vision's entire playbook, including its label problems.

AI assisting a person who stays in the loop, as opposed to replacing them. The same exposed task becomes augmentation or automation depending on how work is organised, not on the technology.

A human adding nothing measurable to a system's performance. Two randomised trials found physician plus model indistinguishable from model alone on vignettes, which may say more about what a vignette can detect than about what a clinician contributes.

A system knowing who is asking and what state their account is in. The condition separating a support deployment from a demo, since without it the system can only answer general questions the help centre already answered.

A qualification issued by the researchers themselves, which does not automatically travel with their headline. One team reported that their finding only becomes significant under the broadest controls from 2024; the original percentage is quoted constantly and the narrowing almost never.

A network trained to copy its input through a bottleneck, which forces it to learn what matters. Ancestor of most representation learning.

A determination about a person produced without individual human assessment. The scope that catches every case in this record, unlike scopes defined around artificial intelligence, which exclude the ones built from arithmetic.

Any computational process issuing a score, classification or recommendation that substantially assists or replaces a discretionary hiring or promotion decision. The phrase "substantially assists" is doing the work, and the employer decides.

Computing a sequence of actions that turns the current state into a goal state. Classical AI's core problem, now living inside every agent framework.

Getting machines to find mathematical proofs — symbolic AI's oldest ambition, now hybridised with LLMs proposing steps and verifiers checking them.

The tendency to over-trust automated output and under-weight contradicting evidence, including your own. Documented across aviation, medicine and driving. It is why a human-in-the-loop is a weaker safeguard than it appears, since the human defers.

Attention decaying when supervising a system that is right almost all the time. Not a character flaw but a predictable response to a task offering no feedback, documented in aviation for decades and named by the NTSB as a risk the operator had not addressed.

Automating model selection and tuning. Works for tabular problems; never displaced experts the way it promised. ::

An agent that pursues a goal over many steps without human sign-off on each one. The capability everyone demos and few ship.

Generating one token at a time, each conditioned on everything before it. Why models write at a steady pace rather than all at once.

Producing output one element at a time, each conditioned on everything generated so far. The mechanism under every language model, and the reason generation is sequential and therefore memory-bound.

Post-training quantization that protects the small fraction of weights most affecting activations, rather than treating all weights equally. Retains accuracy at lower bit widths than uniform methods.

The pretrained network a system builds on. Swap the head, keep the backbone: the standard transfer-learning move.

The algorithm that works out which weights caused a mistake, by tracing error backwards through the network.

Reasoning from the goal to the facts: to prove X, find rules that conclude X, then prove their conditions. How classic expert systems answered "why".

Training many models on random resamples of the data and averaging them. Cuts variance; the mechanism behind random forests.

Representing text as word counts, order ignored. Crude, and a strong baseline for longer than anyone liked. ::

The simple thing you must beat before claiming anything. Often a rule, a linear model, or "predict the most common answer." ::

A model after pretraining but before instruction tuning. Completes text rather than answering questions.

How common the thing you are testing for actually is. When it is rare, even an accurate detector produces mostly false accusations, which is why screening scores are not verdicts.

Detector precision collapsing as genuine violations become rare, because false positives track the compliant population while true positives shrink. A detector performs worst in exactly the institutions where the underlying problem is least severe.

Stating what a figure measures: run rate or booked, full company or product line, gross or net of revenue share, point-in-time or period. It costs nothing and would resolve most apparent conflicts between sources, and it is almost never attached.

The group of examples processed together in one step. Bigger batches give smoother gradients and cost more memory.

Processing many requests together for throughput. The cheap way to run models when latency doesn't matter. ::

Processing many requests together to use hardware that's mostly idle. The largest cost lever in inference, and it costs you latency.

Renormalising activations at every layer. One of deep learning's most important techniques, and its original explanation turned out to be wrong.

How many examples per training step. Interacts with learning rate; copying one without the other means nothing.

Updating beliefs with evidence, by the only rule that's coherent. Mathematically settled, practically expensive, and the thing modern models are bad at.

Pearl (1988): a directed graph where nodes are variables and edges encode conditional dependence, so a large joint distribution factorises into small local ones. The dominant framework for reasoning under uncertainty before deep learning, and still the representation causal inference is built on.

How to update a probability when evidence arrives. The one equation under all of probabilistic ML.

Keeping several candidate continuations alive and picking the best complete one. Standard in translation, mostly abandoned for chat.

Generation owned by a facility and sited on it, bypassing the grid connection queue entirely. Converts a four-to-seven-year utility wait into a twelve-to-eighteen-month equipment schedule, and converts a decarbonising grid mix into the emissions of whatever burns on site.

The recursion that defines value: what a state is worth equals immediate reward plus the discounted worth of what follows. Nearly all of RL is solving it approximately.

A standard test set used to compare models. Increasingly unreliable, since the test may be inside the training data.

Weighting a comparison population to match the conditions a system actually operated in, rather than using a wider regional average. Correct methodology, and the same choice that bounds what the comparison can tell you: it establishes performance inside the domain and provides no evidence about outside it.

When the test is in the training data. The problem that makes most published model scores impossible to fully trust.

Reporting improvement on the exact test set that exposed a flaw, once that set is public and known to everyone building against it. Real progress and a weak demonstration of it, and the distinction is not visible from the number.

When the best models all cluster near a test's ceiling, so the scores stop telling them apart. Often mistaken for continued progress on a benchmark that no longer discriminates.

When an overparameterized model fits its training data perfectly, noise and all, without hurting generalisation, defying the classical rule that says it should.

An early influential language model that reads text in both directions. Still used for classification and search, not generation.

float32's exponent range with fewer mantissa bits. Neural networks need range far more than precision — Google's format, now in everyone's silicon.

The constant added before activation. Lets a neuron fire even when inputs are zero.

A system producing unfair outcomes across groups of people.

Systematic error — the model is consistently wrong in one direction, regardless of data.

The classical tension between underfitting and overfitting. Modern deep learning complicates it without abolishing it.

Embedding query and document separately and comparing vectors. Fast enough to search millions; blind to interactions a cross-encoder would catch.

Sutton's observation that methods leveraging computation beat methods encoding human knowledge, every time, for seventy years — and researchers keep resisting it.

A classic AI pattern: specialists share one common workspace, each contributing when it can. The ancestor of most multi-agent coordination schemes.

Overturning convictions as a class by statute rather than case by case. Used for the Horizon convictions in 2024 because individual appeal was too slow for the number wrongly convicted, which is itself a finding about remedies for automated harm at scale.

Everything an agent could reach and change if its intent were fully compromised. A bounded, writable fact rather than a score, and a deployment that has not established it is assuming the answer is acceptable.

The classic machine-translation metric: n-gram overlap with references. Correlates with quality weakly, used anyway for decades.

Keeping participants or assessors unaware of which condition they are in. Nearly impossible in education technology, since a student knows they are using an AI tutor, which is why expectancy effects contaminate every self-reported outcome in that literature.

The keyword-search ranking formula from the 1990s. Still the baseline every retrieval paper must beat, and often doesn't.

A stochastic network that learns a probability distribution over its inputs. Hinton's generative ancestor of deep learning — historically pivotal, rarely deployed.

Training models in sequence, each focused on the previous one's mistakes. The engine of XGBoost and most tabular-data winners.

Estimating uncertainty by resampling your own data with replacement, many times. How to get error bars when you can't rerun the experiment.

A pilot succeeding under a curated slice, pre-cleaned data, limited users and manual review, all four of which production removes. The pass measures a system that will not exist, and the four removed conditions are exactly what determines whether the deployment works.

Narrow, repeated thousands of times, numerically toleranced, on a known surface, and not requiring the machine to understand the wider job. Layout marking, rebar tying and solar piling qualify, which is why they are the construction workflows where robots actually earn their keep.

The compression trick used to split text into tokens. Started as a 1994 compression algorithm.

Automatically stretching arrays of different shapes to work together. A NumPy convention every ML framework inherited. ::

How much sentence length and structure vary through a text. AI-detection tools pair it with perplexity on the theory that models write more evenly than people — a heuristic, not a reliable test.

Information produced for reasons unrelated to training a model, and therefore free. The entire scaling story in language and vision rests on it, and robotics has almost none, which is the structural reason physical automation proceeds by accommodation rather than capability.

Tokenizing raw bytes so nothing is ever out of vocabulary. Robust, at the cost of longer sequences.

The dominant subword tokenization algorithm: start from bytes, repeatedly merge the most frequent adjacent pair, stop at the target vocabulary size. The merge list is the tokenizer.

How much memory each stored number takes: 4 at full precision, 2 at half, 1 at 8-bit, about 0.5 at 4-bit. Multiply by parameter count and you have your VRAM floor.

Whether a model's confidence means anything. Modern neural networks are worse at it than the ones they replaced.

The distance between what someone believes they know and what they can demonstrate. In AI tutoring trials, satisfaction and confidence moved around 0.9 while knowledge moved around 0.5, so students report improving roughly twice as much as they measurably do.

Releasing a change to a small slice of users first and watching before full rollout. Standard software practice, doubly necessary for models.

A unique token deliberately placed in a dataset so that its later reproduction proves the data was memorised. One of the few contamination tests that works without access to the training corpus.

A computational screen narrowing an intractable search space to a set worth attempting. A legitimate contribution where the binding constraint is deciding what to try next, and a misleading one when the output is described as discovery rather than as hypotheses.

The professional duty not to mislead a court. It is the obligation engaged when unverified AI output is filed, and courts have stated directly that candour after an error is discovered substantially reduces the sanction.

Getting a model to demonstrate what it can do, as distinct from measuring what it does by default. A poor elicitation produces an underestimate, which means a low benchmark score is ambiguous between inability and inadequate prompting.

Capital spending moved onto another entity's balance sheet, converting it into operating expense for the original party. The economic reality is unchanged and the per-company view no longer shows it, while the aggregate view still does.

Routing easy inputs to a cheap model and hard ones to an expensive one. The practical answer to cost at scale. ::

Solving new problems by retrieving the most similar past case and adapting its solution. An old idea that rhymes with what retrieval-augmented systems do now.

When training on something new destroys what a model already knew. The main risk in fine-tuning.

Working out what causes what, not just what correlates — interventions and counterfactuals, not curve-fitting. The discipline (Pearl, Rubin) that keeps prediction from being mistaken for understanding.

Systematically replacing activations according to a hypothesised explanation and checking that behaviour is preserved. A method for testing whether an interpretability claim actually holds rather than merely sounding plausible.

Automated scoring producing more medium and fewer extreme grades than human raters, reported across multiple studies. Compresses the distribution, which harms the strongest and weakest work at once, and unlike a systematic offset it cannot be corrected by subtraction.

A formally verified guarantee that a model's prediction is stable within a bounded perturbation. Notably variance-prone: models differing by 0.057% in accuracy showed 28.9% standard deviation here, because a derived metric amplifies small differences in a decision boundary.

The record of which agent called which tool, with what arguments, what returned, and what the caller did next. Requires properly nested spans, and without it a multi-agent failure has several candidate explanations and no way to narrow them.

Getting a model to reason step by step before answering. Improves results, and the stated reasoning isn't always the real one.

Drafting an answer, generating verification questions about it, answering those independently, then revising. A structured anti-hallucination recipe from Meta (2023).

A revision applied prospectively when new information arrives, as distinct from a restatement, which would imply the earlier figure was wrong when made. The distinction is why revising a depreciation schedule is not an admission of error.

Software you talk to. The first one was 200 lines of pattern matching in 1966, and people confided in it anyway.

The public evaluation where anonymous models answer side by side and users vote. The Elo leaderboard the industry actually watches — and optimises for.

The exact formatting — role markers, special tokens — a chat model was trained to expect. Getting it wrong silently degrades output, a common invisible bug.

A saved snapshot of a model's weights mid-training. Also what you pin to when you need reproducibility. ::

The finding that models were oversized and undertrained; tokens should scale with parameters. Reset how labs spend compute.

Searle's argument that manipulating symbols by rules never produces understanding. Forty-five years unresolved.

Building a processor from several smaller dies rather than one large one, then packaging them together. Improves manufacturing yield, since a defect ruins a small die rather than a large one, and shifts the difficulty from fabrication to packaging.

The narrowest layer in a dependency chain, which is often not the most discussed one. AI hardware concentration is usually framed as a country risk, while every accelerator below roughly 7nm passes through lithography machines made by a single firm regardless of where the fab stands.

Where a document is split before indexing. A lossy decision made once that silently shapes every retrieval result, and the cause of many failures blamed on the retriever.

Splitting documents into pieces before indexing them. The most common invisible cause of bad RAG.

Forecasting which customers will leave. The classic applied-ML problem: easy to model, hard to act on. ::

Identifying the subgraph of a network responsible for a specific behaviour. Produces mechanistic explanations rather than feature attributions, and scales poorly, which is why published circuits are for narrow tasks.

The same dollar counted more than once across an ecosystem, where a supplier holds equity in its customers or a cloud provider invests in a company that buys its compute. The magnitude is not publicly quantifiable, which is itself the finding.

A rule that every factual claim must reference a retrieved passage, and that the model must abstain when none supports it. Turns "be accurate" into something checkable.

Retellings growing stronger as they move further from the source, each layer dropping a qualifier until the claim exceeds any evidence while the citation chain still looks intact. It operates fastest on the most quotable cases, which are also the least documented.

Making a system attach sources to its claims — and checking the cited passage actually supports the sentence, which models frequently get wrong.

Output that names a real source which does not support what was said. A chatbot linked to the correct policy page while contradicting it, and the tribunal held the link did not cure the misleading answer.

Predicting which category something belongs to.

A tracked object's label changing repeatedly, discarding the predicted path each time. In the Tempe crash the system reclassified a pedestrian across roughly four seconds and never predicted her path correctly, so continuous detection produced no cumulative understanding.

Steering diffusion output toward the prompt by contrasting conditioned and unconditioned predictions. The dial behind every "guidance scale" slider.

When one class vastly outnumbers another. The standard advice to resample is mostly wrong — move the threshold instead.

Software that informs a licensed clinician's judgement rather than replacing it. Essentially every cleared AI medical device is regulated under this intended use, which is why claims of autonomous diagnosis should be read sceptically.

A model that puts images and text in the same embedding space, so you can compare them. Underpins most image search and generation.

Measuring image-text alignment with CLIP embeddings. The standard automatic check for text-to-image faithfulness.

Grouping similar things without labels. Always returns groups, whether or not any exist.

A network that slides small filters across an image to find local patterns. Made computer vision work.

Following one voice among many. Cherry named it in 1953 and it stood for sixty years as the measure of machine hearing.

Code rewritten or deleted within two weeks of being committed. Nearly doubled from 3.1% to 5.7% between 2020 and 2024 as AI assistance scaled, which is the cheapest available proxy for whether generated output holds up.

Changing the target and the machine together rather than solving for an unchanged target. Row crops were bred for uniform height and simultaneous ripening alongside the harvesters that cut them; the processing tomato was developed as a joint programme with its machine.

Knowledge that can be written down and handed over explicitly. Disproportionately what entry-level work consists of, and disproportionately what AI does well.

A blueprint for a whole mind — memory, learning, action selection as one integrated design (SOAR, ACT-R). The ambitious old program today's agent stacks echo.

Words that are siblings under a category: cat and dog, red and blue, Tuesday and Thursday. They collide in embedding space for the same reason antonyms do, since they share contexts and grammatical roles.

The retrieval model that introduced late interaction (Khattab & Zaharia, 2020): per-token embeddings, matched with a cheap MaxSim at query time.

Making recommendations with no history for a new user or item. The problem collaborative filtering can't solve alone.

Recommending from patterns across users: people like you liked this. The engine of classic recommenders.

Coordinated data exchange across many devices, such as summing gradients from every worker. Frequently the limiting factor in distributed training, since compute scales with devices and communication scales worse.

The bias introduced by controlling for a variable caused by both treatment and outcome. Why adjusting for everything available is a mistake rather than a precaution.

Search spaces growing exponentially, so a method that solves a toy problem cannot solve a real one. No hardware fixes an exponential.

Getting machines to assume what every person assumes — water is wet, dropped things fall. Famously resistant to hand-coding; LLMs absorb much of it, unreliably.

What a system was measured against. Better than nothing, better than a non-specialist, as good as a specialist, and better than a specialist using the tool are four different claims, and most disagreement about medical AI is two parties assuming different ones.

A jailbreak mechanism: framing a request so a model's trained helpfulness pulls against its trained caution, and engineering the unsafe one to win.

Where a tool changes what work is done rather than reducing how much. The best-documented case in AI for science: protein structure prediction won a Nobel Prize and left the rate of experimental structure determination almost unchanged, redirecting it toward proteins that had no structural information.

Handling novel combinations of familiar components. Measured by training on some combinations and testing on structurally different ones: transformers scored 96-99% in-distribution and 16-35% on recombination of the same vocabulary.

Frege's principle: the meaning of a whole is determined by the meanings of its parts and how they are combined. The property that lets a finite vocabulary express unbounded thoughts, and the standard against which claims that a model understands are usually tested.

Where a total can be measured and its breakdown cannot. Data centre electricity is projectable from utility data and construction pipelines; how much is training against inference, or text against video, is disclosed by nobody, which makes it the most valuable missing number in the field.

A small deviation moving a system into states its demonstrations never covered, where the next action is worse, and so on. Why behavioural cloning fails on long episodes, and why demonstrations should deliberately include recoveries from bad states.

Bitwise identical results from the same code, data, software environment and hardware. What containerisation targets and what GPU non-determinism defeats, since floating-point addition is not associative and thread scheduling varies.

Allocating a fixed compute budget between model size and data for lowest loss. What Chinchilla made everyone recalculate.

An agent driving software through the screen, mouse and keyboard. The automation of last resort, and the benchmarks are humbling.

The field of getting machines to interpret images and video, turning raw pixels into meaning rather than merely processing them.

A direction in activation space corresponding to a human-nameable concept, found by training a classifier to separate examples containing it. Allows testing whether a model's decisions are sensitive to a concept it was never explicitly taught.

The world changing under a deployed model. Why production accuracy decays and monitoring exists.

The position that a term's meaning is constituted by its inferential relations to other terms rather than by reference to something outside language. The strongest philosophical basis for claiming a text-trained system could understand.

Lafferty, McCallum & Pereira (2001): labels a whole sequence at once, so neighbouring decisions constrain each other rather than being made independently. The standard for named-entity recognition and part-of-speech tagging before neural models, and still layered on top of them.

Tying a legal obligation's start date to a readiness condition rather than a calendar. Removed from the AI Act's final Omnibus text in favour of fixed dates, which eliminates a dependency that could slip again and also eliminates the guarantee that a compliance route exists when the duty binds.

A result quoted without the assumption that produced it, which makes it a different claim rather than a simplified one. Model collapse is proved under data replacement; the popular version omits that and reads as an inevitability, while the accumulation case has a proof going the other way.

Telling a generative model what to make. The difference between a slot machine and a tool.

The preferred term of some researchers for model hallucination: fluent invention without intent to deceive.

An annotation recording how certain a causal claim about an incident is, given that most entries rest on incomplete third-party reporting. Established, alleged, disputed and unknown are different claims, and writing the second as the first is the common failure in incident writing.

Approval trained into a rubber stamp. An agent requesting human sign-off on every action teaches the user to approve without reading, which converts a security control into a formality and is automation bias arriving from the opposite direction.

A pattern with several sufficient causes and no design separating them. Entry-level hiring weakness is predicted by AI capability, by interest rates, by a post-2021 sector correction and by a low-flow labour market, and the strongest available comparison does not distinguish among them.

A table of what got classified as what. The least sophisticated tool in evaluation and the one that tells you the most.

A generative model trained so that any point on a diffusion trajectory maps directly to the final sample, enabling one- or few-step generation instead of many denoising steps.

The reporting standard for randomised trials of AI interventions, with SPIRIT-AI as its protocol counterpart. Their use is the clearest available signal that a deployment claim was tested rather than surveyed, and they remain rare outside clinical settings.

Training a model against written principles instead of human ratings. Scales, and moves the question to who wrote the principles.

Forcing generation to follow a grammar or schema by masking invalid next-tokens. The mechanism behind reliable structured output.

Finding the application where a system's limitations do not bind. Small payloads, imprecise drop zones and short range are severe constraints for retail delivery and no constraint at all for blood to a rural clinic, which is where the value concentrated.

Finding values that satisfy a set of constraints simultaneously — scheduling, sudoku, configuration. A classic AI formulation with industrial-strength solvers.

Relieving one bottleneck promotes the next, so sequential fixes deliver less than each appears to promise. AI infrastructure moved from chip allocation to packaging capacity to grid interconnection in about four years, and coverage tracked each long after it eased.

Whether a test actually measures the ability it claims to. A coding benchmark measures puzzle-solving; software engineering is something else.

The share of conversations that stayed in the automated channel without transfer. Better than deflection because it excludes obvious handoffs, and it still counts a customer who quietly gave up.

Establishing whether benchmark items appeared in training data. Hard because the training set is usually unavailable, which is why the strongest evidence comes from performance discontinuities around a cutoff date rather than from direct inspection.

Cryptographic provenance attached at capture or creation and carried through editing, standardised as ISO/IEC 22144 in 2025. Records what was declared at signing, which is provenance rather than detection: a credential faithfully attests the capture of a staged scene.

Whether a person can see why a decision about them was made and challenge it. A determination that cannot be examined cannot be contested, which removes the last check on every failure upstream of it.

Many papers on one question using different methods and assumptions and reaching different conclusions. A signal that the assumptions differ rather than that nothing is known: the conditional structure is usually established and which condition the world is in usually is not.

Treating the context window as a scarce resource to allocate deliberately — so much for instructions, so much for retrieval, so much for history.

Reusing computed attention states for a shared prompt prefix. The trick that makes long system prompts affordable.

Shrinking retrieved or accumulated context — summarising, pruning, deduplicating — so what remains fits the window and stays attended to.

Deciding what goes in the context window and what doesn't. Took over from prompt engineering once the prompt stopped being the hard part.

Establishing a model's credibility for a particular job rather than in general, which is the FDA draft guidance's central mechanism and the correct structure. It also presumes behaviour stable enough to characterise, which a predictive model has and a general-purpose generative model does not.

An orchestrator's context filling with material relevant to other agents' work, degrading its ability to act on any one of them. Measured steering accuracy falls from about 60% at three agents to 21% at ten.

The practitioner term for performance quietly degrading as the context window fills — broader than lost-in-the-middle. Community-coined; the fix is usually using less context, better chosen.

An embedding that changes with surrounding text, so "bank" gets different vectors by context. The transformer-era fix for static, one-vector-per-word methods.

A retrieval method from Anthropic (2024) that prepends an LLM-generated summary of the whole document to each chunk before embedding it, so a chunk carries context it would otherwise lose when split.

How much text a model can consider at once. Bigger isn't the same as better-used.

Updating a model over time without forgetting. Largely unsolved; retraining remains the industry answer. ::

Evicting finished sequences and admitting new ones every step, so the batch stays full. The biggest single win in LLM serving.

Training by pulling matching pairs together and pushing non-matching pairs apart. How CLIP and most embedding models learn.

An adapter that lets an image model follow structural guidance — edges, poses, depth maps — alongside the text prompt. The standard way to control image generation precisely.

When training stops improving. Not the same as being finished, and often a plateau rather than an answer.

Independent accounts of the same anomaly from people with no contact with each other. The strongest evidence available that a system is at fault, and typically held by the institution with the least interest in acting on it.

Sliding a small filter across an input to detect local patterns. The operation that gave CNNs their edge on images.

Grice's assumption that conversation participants are working toward a shared purpose. It is what licenses inferring meaning beyond what was said, and it is why an under-informative answer reads as significant rather than merely brief.

Whether training on work you didn't license is lawful. Unresolved, consequential, and the industry is shipping into the uncertainty at scale.

Working out which mentions refer to the same entity. Classic NLP; now mostly absorbed into LLMs. ::

The body of text a model was trained on, or that you search over. ::

Time between a problem entering a record and being removed from it. Of more than 32,000 identified fraudulent papers, 29% had been retracted, and the remainder stay citable while investigation, notice and institutional cooperation proceed.

How an institution finds out it was wrong. Courts have appeal, banks have examiners, laboratories have replication. Generative outputs block the loop three ways: the error is not recorded, not reproducible, and has no addressee.

The route by which a publication's errors are identified, fixed and attached to the record permanently. Generative systems have none, which is why an error rate is only meaningful alongside a correction mechanism: a 5% rate with corrections beats 2% without.

Repeated attempts at the same task failing together rather than independently, which is better news than it sounds. Independent attempts at 61% would give 2% across eight runs; the observed 25% means some tasks are reliably handled and some reliably are not, so the reliable subset can be identified and routed.

Decaying the learning rate along a cosine curve rather than in steps. Widely used because it spends more time at low rates near the end of training, where fine adjustments matter.

Decaying the learning rate along a cosine curve from its peak to near zero. The default shape for training transformers, usually after a short warmup.

Measuring how close two embeddings point in the same direction. The standard way to compare meanings numerically.

What training minimizes. Choose it badly and the model optimizes the wrong thing perfectly.

A party publishing something that damages its own position, which carries more weight than any favourable claim. A company withdrawing its own detector while disclosing 26% detection and 9% false flags says more than any vendor accuracy page.

Total spend divided by successful outcomes rather than by attempts. The only unit that maps onto what is being bought, since failed runs consume tokens and deliver nothing. Reframes reliability work as cost work.

What gets metered when the valuable outcome is counterfactual. A security incident that never happened and a bad decision avoided have no timestamp, so the billable event becomes documents processed or actions taken, chosen by the party being paid.

Post-processing an embedding space using an external lexical resource to push antonym pairs apart and pull synonyms together. Works, and requires a curated list of the pairs that matter.

Attention from one sequence into another. How decoders read encoders and diffusion models read prompts.

A policy trained on data from several robot bodies performing better than one trained on a single body. Demonstrated by the Open X-Embodiment collaboration, with roughly 50% higher mean success and about threefold improvement on skills absent from the evaluation robot's own training data.

Scoring a query and document together in one forward pass. Slower and more accurate than embeddings — hence reranking.

The standard loss for classification. A differentiable stand-in for accuracy, which has no useful gradient.

The effect where learning something in one language makes it partly available in others, via a shared internal representation. Real, and weaker the further you move from the training data's dominant languages.

Rotating which slice of data is held out, to get a more reliable estimate on small datasets.

A loss that aligns an audio sequence to text without needing the alignment labelled. What let speech recognition train on transcripts alone.

NVIDIA's software layer for running computation on GPUs. Its maturity, not the silicon, is why alternatives struggle.

Ordering training data easy to hard. Intuitive, occasionally helpful, rarely decisive. ::

The family of ways intuition breaks in high dimensions — volumes concentrate in shells, data gets sparse, and neighbours stop being near. Why more features isn't automatically better.

An open interconnect standard allowing memory to be pooled and shared across processors and accelerators. Aims at the problem that memory is stranded inside individual machines while workloads need it elsewhere.

The information hidden in a teacher model's full probability distribution, how the classes relate, that a plain correct label throws away. Why distillation works.

The 1956 summer meeting where the field — and the phrase "artificial intelligence" — got its start. The traditional birthdate of AI.

Some corpora exist already as a byproduct of human activity; others must be manufactured. Text and images were produced by people for their own reasons. A robot trajectory has never existed until a robot performs it, in real time, once, which is a physical constraint rather than a financial one.

Making more training examples by flipping, cropping, or perturbing existing ones. Often beats a fancier architecture.

When benchmark questions leak into training data, so a high score partly reflects memorisation rather than skill. Hard to detect after the fact; assume old public benchmarks are affected.

How richly a task is represented in existing written material. A better predictor of AI performance than difficulty: common but undocumented work is harder than rare but well-recorded work.

Your model didn't get worse — the world moved. The most common way a working system quietly stops working.

Product usage generating data that improves the model that improves the product. The compounding loop platform AI companies chase. ::

Humans deciding what each example is. The least visible and most determinative work in supervised learning, done by people the field rarely names.

When information unavailable at prediction time sneaks into training, inflating scores and causing real-world failure. The symptom is results that look too good.

Tracking where data came from and what touched it. The prerequisite for debugging models and answering regulators.

A statutory requirement that certain data be stored inside a territory. The strongest of the sovereignty claims, because the law names both the category and the boundary.

The 1990s name for finding patterns in large datasets — association rules, clusters, anomalies. The commercial ancestor of modern ML practice.

The proportions of different sources in a training corpus. One of the highest-leverage and least-disclosed decisions in model development, and the reason two models of identical architecture and size can differ substantially.

Copying the model across devices and splitting the batch. The first and simplest way to scale training. ::

The automated flow that moves and transforms data from source to model — ingestion, cleaning, feature computation, delivery.

Corrupting training data so a model learns a hidden flaw or backdoor. An attack on the supply chain rather than the running system.

Knowing where your data came from and what you may do with it. The licence field on the dataset you're using is probably wrong.

A usually contractual commitment about where data is kept. Weaker than localisation: the difference between a promise and a statute, and what most cloud sovereignty features actually deliver.

When the data in production stops resembling the data you trained on. Why models decay after launch.

Documentation of a training dataset: sources, collection method, who is represented, preprocessing, and permitted uses. Proposed separately from the model card and answering the provenance question a model card only summarises.

The rule that data is governed by the law of the place it physically sits. Distinct from privacy: it concerns where the data is, not what it reveals, and it is what decides whether you may send it to a hosted model at all.

The looming exhaustion of high-quality human text to train on, projected for the late 2020s to early 2030s. The constraint pushing labs toward synthetic data and test-time compute.

Miles driven with no passenger aboard. In one autonomous ride-hailing fleet it fell from 51.5% to 44.3% between January 2024 and September 2025, with roughly 54% of California miles carrying a rider. A large operational cost that safety statistics do not touch.

A ReLU unit stuck outputting zero. One reason leaky variants exist.

Multiple agents arguing toward a conclusion with a judge resolving. At least double single-model cost before the judge, and a judge favouring confident style over correct substance produces higher-confidence wrong answers.

A model that behaves well because it's being watched. Speculative as a risk, and there's now a real experiment showing safety training can fail to remove it.

The surface where a classifier changes its mind. Linear models draw flat ones; deep networks bend them into arbitrary shapes — which is the point of depth.

A flowchart learned from data. The most interpretable model there is and, on its own, one of the least accurate.

Generating output one token at a time, each conditioned on all previous. Memory-bandwidth-bound rather than compute-bound, because every token requires reading the full model weights, which is why batching improves throughput and not latency.

The half of a model that generates output. Modern LLMs are decoder-only.

The architecture of modern LLMs: one stack, predicting the next token. Encoder-decoder's simpler sibling that won.

A human placed in a supervisory role without alerting, attention monitoring or an engaging task. Not redundancy but an allocation of blame, and the test is simple: at what point does the person find out?

Removing repeated documents from training data. Boring, and one of the highest-leverage data cleanups known.

IBM's chess machine that beat world champion Kasparov in 1997 — the symbolic era's most famous victory, won by search and engineering rather than learning.

Synthetic media convincingly depicting a real person doing something they didn't. Descended from GANs.

Machine learning with many-layered neural networks.

Microsoft's training-optimization library: ZeRO sharding and friends. How labs fit models that don't fit. ::

Operations execute when you call them, so print and pdb work. The property that beat a better-resourced static-graph competitor.

Where a sector's edge is drawn, and the effect that has on whether it reads as transformation or stagnation. Counting factory automation for building products as construction robotics makes the sector large and fast-growing; excluding it puts on-site penetration below 0.03% of construction spend. Same machines, same buildings.

The share of support conversations no human agent touched. It counts a customer who received an unhelpful answer and abandoned the chat as a success, which is why a platform can report 90% deflection on a 40% resolution rate with both figures accurate.

An agent acting for a specific user with a scoped, short-lived credential, where logs preserve both the agent and the delegating person. Contrasted with standing identity, and confusing the two is the most common agent permission error.

A measure confirming an impression happened as described, as distinct from one assessing whether it was worth anything. Viewability, invalid traffic and measurability are all delivery metrics, and none asks whether the page deserved to exist.

A fairness criterion: the model's positive predictions occur at equal rates across groups. Simple to state, and provably in tension with other fairness definitions.

Dennard (1974): as transistors shrink, power density stays constant, so smaller chips run faster at the same power. It broke down around 2005, which is why clock speeds stopped rising and the industry turned to parallelism instead. That turn is what made GPUs the substrate for AI.

Learning to reverse added noise. The pretext task underneath diffusion models.

Training a model to reverse a gradual noising process, so that starting from noise and repeatedly denoising produces a sample. The formulation that made diffusion practical for images.

Which base a figure is expressed against, and therefore whether it reads as marginal or essential. Eight million drone deliveries is 0.004% of global parcels and close to all of the blood moving outside Kigali. Neither denominator is neutral.

The clearance route used when no suitable predicate device exists, carrying a higher evidentiary burden than 510(k) and used rarely in AI. The first for an AI pathology product was granted in 2021.

How many layers of prerequisites sit beneath a concept, computed as the longest path to a root. Not a difficulty rating: a deep concept may be easy once you arrive.

Recovering which words in a sentence govern which others, producing a tree of grammatical relations rather than a hierarchy of phrases. A core NLP task with substantial linguistic theory behind it, and a standard probe for whether a model has learned syntax.

Treating the number of institutions running a system as proof it works. Scale tells you about procurement, not performance, and the two are routinely conflated in vendor material and press coverage alike.

Removing an agent's access on retirement, which requires revoking tokens, delegated credentials, scheduled jobs, tool grants, sub-agent permissions, stored secrets and external integrations. Each created separately, with usually no single record listing them.

Working out how far things are from an image. From a single photo the absolute scale is mathematically unknowable — the model recalls, it doesn't measure.

A decidable fragment of first-order logic for defining concepts and their relationships. The formal backbone of ontologies and the Semantic Web, and what distinguishes a knowledge graph with defined semantics from one that is merely a labelled graph.

The cognitive science finding that effortful engagement with material produces more durable learning than passive processing. Predicts that a student who reaches the right answer with help has not necessarily built the knowledge that produces it without help.

The principle that error rates are only knowable where someone is paid to find errors. Litigation has an adversary who reads every filing, so its AI failures become public data. Most domains have none, so an absence of reported failures indicates nothing.

The share of speaking time attributed to the wrong speaker, missed, or falsely marked as speech. Only meaningful alongside the scoring conditions, since oracle speaker counts, oracle VAD and a forgiveness collar all reduce it without improving anything.

Adding calibrated noise so no single person's data is identifiable in the output. Strong math, real utility cost.

Damage spread thinly across many decisions with no identifiable victim, date or operator. Structurally invisible to every incident register, which requires a reportable event, and plausibly the largest category of real harm.

Generates images by starting from noise and removing it step by step. Replaced GANs for most image work.

Whether a policy targets what can be built or what gets used. The two come apart when a model trained once on constrained hardware is released openly, generating usage that appears in no production statistic.

The older framing of educational technology inequity, concerning devices and connectivity. Now behind the evidence in AI education, where both groups have the tools and the measured gap is in training, which is the variable determining whether the deployment helps or harms.

Squeezing many features into fewer while keeping the structure. PCA, UMAP, t-SNE.

Choosing which problems are worth working on, judging which failed experiment is informative, and knowing when an approach is exhausted. The capability that has not moved while execution has, and the narrowest observable question in the superintelligence debate.

Information that is fully public and that almost nobody assembles. Filings from several companies across several quarters can each be adequate while the picture they jointly describe is read by very few. Transparency and understanding are different properties.

A stake stated alongside a claim, which a reader should weight without treating it as refutation. An investor holding put positions while arguing that earnings are overstated has an interest; so does a manufacturer co-authoring a study of its own device.

What an organisation gains by reporting its own failure. Currently nothing: disclosure brings scrutiny and liability while silence brings neither, which is why public registers are built almost entirely from failures that became public against the operator's interest.

An agreed rule for labelling AI involvement in published work. Journalism has not converged on one, which matters more here than elsewhere because disclosure is the profession's existing answer to almost every other conflict of interest.

A model that learns the boundary between classes, not how the data was made. The counterpart to generative: cheaper, and all you need if you only want to label.

When a neutral-looking rule produces unequal outcomes across groups. The legal lens, and the four-fifths rule of thumb, for detecting algorithmic bias.

Where the operator and the reporting disagree about whether anyone was affected, and no evidence exists that would settle it. Recording the dispute rather than picking a side is the only honest treatment.

Training a small model to imitate a big one. Better quality at size than heavy quantization, at the cost of a training run.

Content that resembles the answer without being it. Semantically similar near-misses actively mislead a model, and they are why long, coherent documents degrade accuracy more than shuffled ones.

The shape of your data — what's typical, what's rare. Almost every model failure is a distribution mismatch. ::

Firth (1957): you shall know a word by the company it keeps. Meaning can be recovered from patterns of co-occurrence rather than from definitions, which is the justification for every word embedding and the reason a model trained only on text can represent meaning at all.

Diffusion with a transformer backbone instead of a U-Net. The architecture behind recent image and video generators.

A benchmark built to evaluate algorithm performance across skin types, and the only remedy that works retrospectively on models already trained. Evaluations against it found sensitivity falling from 0.69 to 0.23 and from 0.41 to 0.12 between lighter and darker skin.

How far a metric has travelled from the field it was constructed in, which determines how much it still says. A tracking rate of 99.98% is close to the objective itself in physics-based animation, necessary-but-insufficient for simulated robotics, and near-uninformative for physical deployment. The number is identical at all three.

When conditions move outside what a system was validated for. Detecting it is harder than operating inside the domain, because it requires recognising a situation the system has no model for, and silent exit is more dangerous than a narrow domain that is enforced.

Change lead time, deployment frequency, change failure rate and time to restore. Outputs rather than inputs, which is why they are the right place to measure AI impact: merged pull requests can rise 98% while these stay flat.

A fine-tuning method that splits each weight into a magnitude and a direction, then uses LoRA to update only the direction. Closes some of LoRA's accuracy gap to full fine-tuning with no extra inference cost. Liu et al., ICML 2024.

Test error rises to a peak as models grow, then falls again. Classical theory predicts the first half and gets the second exactly backwards.

How long a compounding quantity takes to double, which decides outcomes that level comparisons cannot. Suspected research fraud doubles every 1.5 years against 3.3 years for retractions and 15 years for legitimate output, and the gap widens by construction.

Aligning on preference pairs directly, no reward model or RL loop. The simplification that made alignment accessible.

Retrieval by embedding questions and passages into the same space and matching on meaning. Beat keyword search on open-domain QA — except where the query already shares the passage's words.

The small fast model that proposes tokens for a larger one to verify. Too weak and acceptance falls; too strong and it costs as much as the model it accelerates.

Randomly switching off units during training to prevent memorisation.

Grouping incoming requests on the fly to fill the accelerator. Table stakes for serving efficiency. ::

Solving big problems by combining solutions to overlapping subproblems, stored rather than recomputed. Bellman's method — the mathematical spine of classical RL.

Halting training when validation performance stops improving, so the model doesn't start memorising.

Running models on the device instead of a server. For privacy, latency and cost, against a memory wall that doesn't move.

Presenting opinion or framing as reported fact. One of five criteria journalists used to evaluate AI news responses, alongside accuracy, sourcing, separating opinion from fact, and context.

Review of a model by a party with the standing, incentive and competence to disagree with it. The central concept of US bank model risk guidance since 2011, and the clearest existing statement of what meaningful oversight requires rather than nominal sign-off.

The magnitude of a difference, as distinct from whether it is statistically detectable. The quantity that usually matters in machine learning, where the question is how much better rather than whether the null can be rejected.

Attributing understanding and intent to a system producing plausible text. Named in 1966; the fluency improved, the inference didn't get better.

Ranking models by pairwise human preference votes, chess-style. How Chatbot Arena orders the leaderboard.

A list of numbers representing meaning, so similar things sit close together.

The length of each vector a model produces. Bigger holds more nuance and costs more everywhere downstream.

Intelligence in a body — robots and agents that learn through acting in the physical world. The counterargument to intelligence as pure text prediction.

Abilities appearing suddenly at scale rather than improving gradually. The most cited claim about large models, and a NeurIPS best paper says it's a measurement artefact.

Skills that show up in large models but not smaller ones, seeming to switch on past some scale. Real, or an artifact of harsh all-or-nothing metrics? Still debated.

A capability appearing only above some scale. Contested: some emergence is real, some is an artifact of the metric.

Two independent mechanisms landing on the same population without either being designed against it. Under-resourced students are more likely to receive unsupervised AI use and more likely to be flagged by a detector, through a budget effect and a text-statistics effect that no policy connects.

The half of a model that reads and represents input.

One stack to read, one to write. The original transformer shape; still standard for translation.

Mapping audio directly to per-speaker activity as a multi-label prediction, rather than the pipeline of detection, segmentation, embedding and clustering. Handles overlap natively because nothing forces one answer per moment.

A model that scores configurations rather than outputting probabilities — learning what's plausible by assigning low energy to it. A recurring alternative framing, championed by LeCun.

Combining several models and averaging them. Usually more accurate, always more expensive. ::

Combining several models so their errors partially cancel. Reliably improves accuracy and calibration at the cost of compute, and is why competition-winning systems are almost never a single model.

Testing whether retrieved context actually supports each statement in an answer, flagging anything that goes beyond the evidence. Automated verification that catches unsupported claims.

Searching a company's own private documents, tickets, and wikis rather than the web. The unglamorous problem RAG quietly turned into a product category.

Mapping a mention to the specific real-world thing it names. What lets systems know which "Paris" you meant.

The average surprise in a distribution — how uncertain an outcome is, in bits. The quantity most of ML is built to reduce.

Modifying the workspace so a machine can succeed in it: fixed lighting, known part geometry, fixtures presenting objects identically. The strategy behind 4.66 million working industrial robots, and the opposite of the general-purpose claim.

An agent's record of specific past events — what happened in which session. Retrieved, not learned into weights.

One full pass through the training data.

A fairness criterion: equal true-positive and false-positive rates across groups. Hardt et al.'s alternative to demographic parity — and incompatible with it in general.

Whether false positive and false negative rates are equal across groups. The criterion ProPublica measured and found violated. Cannot hold alongside predictive parity when groups have different base rates, which is arithmetic rather than a modelling failure.

Whether an occasional confident mistake is affordable. Decides more deployments than raw capability: identical performance is transformative where errors are cheap and unusable where they are not.

What a system can do between confident-and-wrong and certain-and-too-late. Repeated reclassification of an object in your path is itself a signal, and a system with no intermediate state has no way to consume it.

The EU's risk-tiered AI law. Prohibits some uses outright, including emotion recognition in workplaces and schools.

The practice of systematically testing model behavior. The word that signals someone ships models rather than demos them. ::

How you find out whether it works. The step everyone skips and then argues about vibes.

The infrastructure that runs a model against benchmarks reproducibly — prompts, parsing, scoring. Small harness choices move scores enough to reorder leaderboards.

The study design stated alongside a finding. Level I randomised, Level II prospective or observational, Level III retrospective. In one subject the two randomised findings were the modest ones and the dramatic figure came from a pre-post survey with no control, and the rows are quoted interchangeably everywhere.

A procedure that converts a weak or algorithmic signal into a form courts treat as strong, without adding any information. The mechanism is procedural, so the remedy has to be procedural too.

Giving a model more permissions than the task needs. The security failure that least privilege exists to prevent, and the one that turns a prompt injection into an incident.

The argument that sufficiently capable misaligned AI could permanently curtail humanity's future — contested, influential, and the reason 'x-risk' appears in policy debates.

A burden becoming visible only when the party carrying it stops. A closed bug bounty or a departed maintainer measures the failure rather than the load, and by then the load has usually been severe for some time.

Dempster, Laird & Rubin (1977): alternate between guessing the hidden variables given current parameters and re-fitting parameters given those guesses, until it settles. The algorithm behind Gaussian mixture models and hidden Markov models. Converges to a local optimum, not the best one.

Logging every ML run's code, data, parameters, and metrics so results are reproducible and comparable — core MLOps hygiene.

The learned gate in a mixture-of-experts that decides which experts see each token. Get it wrong and experts collapse or starve — the hard part of MoE training.

Encoding a specialist's knowledge as rules. AI's first commercial success, and its collapse taught the field something it's currently relearning.

Getting a reason out of a model. Most methods explain the explanation, not the decision.

Gradients growing unboundedly through deep or recurrent networks. Clipping exists because of this.

When the training signal compounds until numbers overflow. Fixed by clipping.

Take the best thing you know, or look for something better. Underneath every learning system, with a known optimal answer almost nobody uses.

Time between a model's decision and the outcome that tests it. A prediction resolving in an hour and one resolving in six months carry entirely different risk from the same error rate, because the longer gap is more for the forecast to be right about.

Testing a model on data from institutions that did not build it. The gap it exposes can be large: one widely deployed sepsis model reported above 0.9 internally and 0.63 in external validation.

A state's power to compel a provider to disclose data it controls, wherever that data sits. Why hosting in a region changes where data rests but not who can be ordered to produce it.

Obligations attaching to where output lands rather than where a provider sits. A system whose output reaches EU users falls within the AI Act's transparency duties regardless of where it was built or hosted, which is the structure the GDPR established.

The regulatory pathway most AI medical devices reach the market through. It requires showing substantial equivalence to a legally marketed predicate device, not that patients do better. Of 691 cleared devices reviewed, 1.6% cited randomised trial data.

The balance of precision and recall in one number. What you usually care about when accuracy lies.

A reference to authority that does not exist, produced by a model asked to supply support for a proposition. The early and most visible AI failure in law, superseded by a subtler form: real cases quoted with sentences they do not contain.

Identifying a person from their face. Works well on average and fails on specific people — under 1% error for lighter-skinned men, up to 34.7% for darker-skinned women.

Identifying or verifying a person from their face via embeddings. Verification (1-to-1) is well-behaved; identification (1-to-many) is technically and ethically fraught.

Harm caused by a system not acting when it should have. It leaves no artefact naming the system, so incident registers, which count events somebody noticed, systematically under-represent it. This is the failure mode of most assistive AI now being deployed.

The flexible US doctrine weighing purpose, nature, amount, and market effect. Applied case by case to AI training because no purpose-built exception exists, which is why the answer is still unsettled.

Whether generated text is supported by its source — the property RAG is supposed to guarantee and often doesn't. Measured, imperfectly, by entailment checks and LLM judges.

Presenting unequal evidence as two comparable positions. Where one side rests on multiple agreeing analysts and the other on syndicated content with weak attribution, describing them as two cases overstates the second, and neutrality can produce this rather than prevent it.

What happens to someone a system wrongly flags. The same 5% error rate is a spam filter's nuisance or a family's bankruptcy depending entirely on the policy applied afterwards, and no model evaluation measures it. The question that should precede any accuracy target.

Splitting a task across parallel agents and merging the results. Latency is the slowest branch rather than the sum; the merge, where partial results conflict, is usually harder than the original task.

One input column or measurement the model sees.

Assigning credit for a prediction across input features. Widely used, and different attribution methods routinely disagree on the same input, which limits what any single map establishes.

Turning raw data into things a model can use. Still where most accuracy comes from outside deep learning.

Deriving informative inputs from raw data — hand-crafted in classic ML, learned automatically by deep networks.

Choosing which input variables a model gets to see — for accuracy, speed, or interpretability. The classic craft deep learning partly automated away.

Turning a discovered internal feature up or down to change a model's behaviour. Proof the feature is causal, and a possible control knob beyond prompting.

Infrastructure serving the same computed features to training and production. Exists to kill training-serving skew.

Training a shared model across devices without collecting the data. "Your data never leaves" is not the guarantee it sounds like — gradients leak.

Putting a handful of examples in the prompt. Free, instant, and skipping it is the most common expensive mistake.

Getting a task right from a handful of examples in the prompt. GPT-3's headline trick.

Showing the model a handful of worked examples in the prompt before the real input. Often the cheapest large accuracy gain available.

Comparing generated and real image distributions in feature space. The standard image-generation quality score, flaws included.

Continuing a model's training on your own examples to change its behaviour.

Uploading data to adapt a hosted model you never see. Fine-tuning as a service, weights not included.

The representational language of classical AI: objects, relations between them, and quantifiers for "all" and "some". Expressive enough for most knowledge and decidable only in fragments, which is the trade every knowledge-representation system negotiates.

A six-type classification developed to predict propensity to sunburn, widely repurposed as a skin tone taxonomy. Automated classification using it shows balanced accuracy from 17% to 65%, against 58% to 75% for the Monk scale, so the instrument measuring skin tone disparity is inconsistent where the disparity is.

What exactly was measured, at what unit of repetition, by whom, under whose definition, and paid for by which party. None requires expertise, all are answerable from a source document, and their absence explains most of the confusion in enterprise AI evidence.

Recomputing a historical series under a single detection method so the trend reflects the world rather than the effort spent looking. Without it, a rise in screening activity is indistinguishable from a rise in the thing being screened for.

What share of flagged cases are actually violations, as distinct from the false positive rate. At a 15% false positive rate and 15% true prevalence, more than half of all flags are wrong, which is the number an integrity office experiences.

An exact attention algorithm that avoids writing the huge attention matrix to memory, computing it in tiles on-chip instead. Same result, far less memory, much faster.

Floating-point operations: the currency of compute. What scaling laws count.

Training generators by learning a straight-line flow from noise to data. The cleaner successor to diffusion training.

Regulating how hard a manipulator pushes rather than only where it moves. Required for contact-rich work on deformable objects whose properties vary between individuals, and the specific difficulty separating robotic harvesting from robotic weeding.

Whether predictions constitute separate evidence. Five short timelines from five people whose companies benefit when short timelines are believed is one forecast repeated, not five, and should be weighted accordingly.

A window either side of each speaker change excluded from diarization scoring. Since transitions are where most errors occur, applying one removes a disproportionate share of them and flatters the reported figure.

Proving mathematically that a network satisfies a property for all inputs in a region, rather than testing samples. Gives guarantees that testing cannot, and scales to networks far smaller than deployed ones.

Reasoning from facts to conclusions: apply every rule whose conditions hold, add what follows, repeat. The engine of rule-based systems.

The period from ~2020 when broadly-pretrained models became the default starting point for everything. Named after "foundation model".

A large model pretrained on broad data, meant to be adapted to many tasks.

If one group's selection rate falls below four fifths of the highest group's rate, that is treated as evidence of adverse impact worth investigating. A screening heuristic from US employment practice rather than a legal standard, so an audit reporting 0.81 has failed to trip a rule of thumb rather than proved fairness.

Eight-bit floating point for training and inference. Half of half precision; the frontier of going smaller.

The classic puzzle of stating what doesn't change when an action happens — trivial for people, explosive for logic. A famous lesson in how hard common sense is to formalise.

Raising the cost of submitting rather than detecting what was submitted. A reproducible test case is free to someone who reproduced the bug and expensive to someone who did not, which is why it works where classification does not.

Whichever models are currently at the leading edge. A regulatory handle, defined by a compute threshold that capability has already routed around.

A model whose weights are held fixed while something around it adapts. What RAG and most adapters assume.

Sharding a model's parameters, gradients, and optimizer state across devices, gathering each layer only when needed. The mainstream way to train large models in PyTorch.

Using a model to stand in for an unknown function. What neural networks are, stated plainly.

Giving a model a described set of functions it can request. The mechanism under tool use.

Logic with degrees of truth — 'somewhat tall' as 0.7 rather than true/false. Zadeh's framework, quietly ubiquitous in control systems from cameras to rice cookers.

Two networks trained against each other, one faking and one detecting. Largely replaced by diffusion.

Modelling data as a weighted combination of several Gaussian distributions, each with its own mean and covariance. Gives soft cluster assignments rather than hard ones, and is fitted by expectation-maximization.

A distribution over functions rather than over parameters, giving predictions with calibrated uncertainty. Excellent on small data and expensive as data grows, since inference is cubic in the number of observations.

Millions of 3D Gaussians, rasterised. Real-time where NeRF took seconds — and it threw out the neural network entirely.

A smooth activation that weights inputs by how far they are from zero, rather than hard-cutting like ReLU. The default in most transformers.

Google DeepMind's flagship model family. Multimodal from the start, and the Search integration everyone watches. ::

The 2018 audit that measured commercial classifiers across skin tone and gender, and changed practice where a decade of fairness papers hadn't.

Working on data you've never seen. The entire point; everything else is bookkeeping.

Working on data you've never seen. The only thing that matters, and nobody can explain why deep learning does it.

The distance between training and test performance. What regularization exists to close.

A model of how the data itself is produced, able to sample new examples. The other half of the ML world from discriminative.

Optimization by evolution: keep a population of candidate solutions, breed and mutate the fittest, repeat. The best-known member of evolutionary computation.

Variation that reflects the world rather than a definitional difference. Documentation savings of 72.6 seconds per encounter and 30 minutes per day reconcile once units are converted and track a real difference between emergency and ambulatory settings. Resolved not by agreeing a definition but by asking which setting you are in.

The quantized-model file format of the local-LLM ecosystem. What you download to run a model on a laptop.

The pattern where a task stops counting as intelligence once a machine performs it. Usually read as bad faith and better read as a research result: each reclassification recorded a real discovery that the task needed less general capability than assumed.

Good Old-Fashioned AI. The symbolic paradigm, named affectionately by the people who watched it lose.

The trusted, human-verified answer. Everything supervised assumes you have them and they're right. ::

When a measure becomes a target, it stops being a good measure. Reward hacking is this, formalised and automated.

Policy absent entirely rather than inadequate. 63% of organisations have no AI governance policy, which makes any subsequent conversation about enforcement premature and explains why 97% of breach-affected organisations had no access controls.

Graduate-level science questions written to be Google-proof — experts get ~65%, skilled non-experts with the web get ~34%. A test of knowledge that can't be looked up.

Generative pretrained transformer: the lineage that proved next-token prediction scales. The letters that named an era.

A post-training quantization method that compresses weights layer by layer using second-order information, allowing four-bit inference with modest quality loss and no retraining.

The chip that made deep learning possible — thousands of cores doing the same maths at once.

The standard framework for rating how much confidence to place in a body of evidence. A 2026 review of 66 AI education trials returned low to very low across every outcome, which is the most important number in that literature and the least quoted.

The direction and steepness of the error surface at your current position. What tells the model which way to move.

Summing gradients over several small batches before stepping. How small GPUs pretend to be big ones.

Trees built in sequence, each fixing the last one's mistakes. The most accurate thing on tabular data and the easiest to overfit.

Recomputing intermediate activations during the backward pass instead of storing them. Trades compute for memory so bigger models fit.

Capping the size of updates so one bad batch can't destroy a training run.

Walking downhill on the error surface, one step at a time.

Systems placed on the market before a date escaping requirements that apply after it, until substantially modified. Rewards shipping before a deadline and creates an incentive to leave deployed systems unchanged, since improvement resets the position and inaction does not.

A message-passing variant that learns how much each neighbour should contribute rather than weighting by degree, applying the attention mechanism to graph structure.

Labelling an entire structure rather than a node or an edge. How molecular property prediction works, where the whole molecule is the graph and the label is a chemical property.

A message-passing variant that weights each neighbour by node degree, adapting the convolution idea from grids to irregular structures. The most common baseline architecture in graph learning.

Constructing the graph a model will learn from: deciding what counts as a node, what counts as an edge, and what each carries. Usually the expensive part of a graph learning project and the decision that determines whether the model works at all.

A network that learns from data whose structure is relationships rather than a grid or sequence. Each node repeatedly aggregates information from its neighbours, so after k rounds its representation reflects the structure within k hops.

Retrieval over a knowledge graph an LLM built from your documents. Answers questions about a whole corpus, and the indexing bill is real.

Always taking the most likely next token. Deterministic, and prone to dull repetition.

Grice's four principles cooperative speakers are presumed to follow: quality, quantity, relation and manner. Implicature arises when one is visibly broken and the listener infers what would explain the break.

Connecting a large load to the electricity network. The real bottleneck on AI data centres: you can buy accelerators far faster than you can get permission to plug them in.

A model memorises, plateaus at chance for ages, then abruptly generalises. Interpretability showed the circuit was forming all along; only the metric jumped.

Producing output constrained to a retrieved source rather than from parameters alone. It narrows fabrication without eliminating it: around a quarter of frontier-model legal answers still cite law that does not support the proposition offered.

How much of an answer is actually supported by the documents retrieved for it, as distinct from whether it happens to be true. Computable automatically, and it degrades before users complain.

Tying model output to verifiable sources or the real world. The property RAG buys and hallucination lacks.

The correct answer you're comparing against. Only as good as whoever labelled it.

Treating individuals as suspect because of a category they fall into rather than anything established about them. Applied automatically it removes the case-by-case assessment administrative law exists to require.

Sharing key and value projections across groups of attention heads. Shrinks the KV cache with little quality loss, which is why most recent large models use it.

Group relative policy optimization: RL from comparing sampled answers within a group, no value model. The recipe behind DeepSeek-R1.

Gated Recurrent Unit — a streamlined LSTM with two gates and no separate memory cell. Faster, lighter, usually comparable in performance.

Getting a model past its own restrictions. The adversarial game jailbreaking plays.

The checks around a model deciding what it can receive, say, and do.

Where supervisors expect a standard to be met without a framework specifying how. The current position for generative AI in banking, since the 2026 model risk guidance placed it outside formal scope while expectations of applying model-risk principles remained.

How strongly a generative model is pushed toward the conditioning signal. Higher values follow the prompt more closely and reduce diversity, and past a point produce saturated or distorted output.

When a model produces something fluent, confident, and untrue. Frequently misdiagnosed — often the retrieval failed.

Automatically flagging unsupported claims — via self-consistency, entailment models, or judges. Partial solutions; nothing catches everything.

How often a model states falsehoods, measured. Hard to define, harder to measure, essential to track.

A provision allowing an administrator to reduce, stage or waive a recovery. Its absence from the 2005 Dutch Childcare Act meant the tax authority could not have been merciful had it wished to be, which is why an error at selection became irreversible.

Technical specifications that let a provider demonstrate conformity without case-by-case regulatory interpretation. Without them a legal duty exists with no defined route to satisfying it, which is why the EU deferred exactly those AI Act obligations whose standards were not delivered on time.

The stacked memory feeding modern accelerators. The scarce component behind GPU shortages. ::

The task-specific final layers on a backbone. What you replace to reuse a pretrained model.

Hebb (1949): neurons that fire together wire together. Connections strengthen when the units they join activate at the same time. The biological principle behind unsupervised learning, and the reason early neural networks were argued for on grounds of plausibility rather than performance.

Data withheld from training and used to estimate performance on unseen input. Only meaningful if it stayed withheld: a set consulted during tuning has become development data, which is among the most common evaluation errors.

How much studies in a meta-analysis disagree with each other, summarised as I². Above roughly 75% is considerable, and at 86% the studies are measuring meaningfully different things, so the pooled average is a weak guide to what any one deployment should expect.

Any layer between input and output. Where "deep" in deep learning happens.

A Markov chain you can't observe directly, only through noisy emissions. Ruled speech recognition for thirty years before deep learning took the throne.

A network's internal representation at a point in a sequence. What RNNs carry and transformers compute in parallel.

Grouping data by building a tree (dendrogram) of nested clusters — no need to pick K in advance, but it doesn't scale to large data.

Agent memory organised in tiers, with recent detail kept verbatim and older material progressively summarised. Addresses the fact that a context window is fixed while a task history is not.

Memory stacked vertically beside the processor die to open thousands of parallel channels, reaching terabytes per second. The industry's answer to the memory wall, and not a cure.

The proposal that jobs made of many varied tasks are less exposed to AI than jobs made of one repeated task, offered to explain why healthcare and professional services adopted heavily and still added jobs. Exposure would then be a property of job narrowness rather than sector adoption.

The simplest optimizer: repeatedly take the neighbouring step that improves things most, and stop when none does. Fast, greedy, and easily stuck — the baseline everything else improves on.

The graph index most vector databases actually run. Fast approximate search over embeddings.

Dalal & Triggs (2005): represents a region by the distribution of edge directions within it. Paired with a linear classifier it was the standard pedestrian detector before deep learning.

Computing directly on encrypted data so the result decrypts to the correct answer, without the processor ever seeing the input. Fully homomorphic schemes have existed since Gentry (2009) and remain orders of magnitude slower than plaintext computation.

A network whose dynamics settle into stored patterns — memory as attractor states. Hopfield's 1982 model helped revive neural nets and earned the 2024 physics Nobel.

A pattern that may describe the observer rather than the world. Four territories finding the operative variable structural rather than technical is either a regularity or a habit, and the discriminating test is whether a subject selected by someone else, for other reasons, produces the same shape.

The requirement in several jurisdictions that a work involve meaningful human creative contribution to be copyrightable. Why purely machine-generated output is often unprotectable.

OpenAI's code benchmark: write the function, pass the tests. The pass@k standard, contamination caveats included.

Designing the system so a person approves, corrects, or overrides at defined points. The standard control for agents whose actions have consequences.

Combining keyword and semantic search. Close to free and reliably better than either alone.

Answering the query hypothetically, then searching with that answer's embedding. A retrieval trick that shouldn't work as well as it does.

A setting you choose rather than one the model learns — learning rate, batch size, layer count.

Designing actions so repeating them is safe. The property that makes agent retries survivable.

Whether an effect can be separated from its confounds by the available data. Where two shocks coincide, the honest position is uncertainty, and the thing that eventually separates them is usually time rather than a better specification.

Describing an image in words. Declared solved on benchmarks a decade ago, and the benchmarks were measuring the wrong thing.

Predicting what an image is. The task that started the deep learning era in 2012.

The million-image dataset that made deep learning's reputation. The benchmark an era was measured by.

When one class vastly outnumbers another (fraud, disease). Accuracy becomes a liar; use precision, recall, F1, and resampling or reweighting.

Turing's own name for his test, and the whole argument. He proposed a behavioural substitute for a question he thought unanswerable.

Learning a policy from demonstrations rather than rewards. How robots borrow human competence.

One group's selection rate divided by the highest group's rate, computed across sex, race and their intersections. It measures outcomes rather than mechanism, and it is a snapshot of the data it was run on, so it does not transfer to another role, pool or quarter.

Grice (1975): meaning conveyed beyond what was literally said, inferred from the assumption that a speaker is being cooperative. "Can you pass the salt" is a request, not a question about ability. Where models handle indirect language inconsistently, this is what they are failing at.

The quiet bias of gradient descent toward simple solutions among the many that fit the data. The leading explanation for why deep networks generalise.

A proof that several fairness criteria cannot be satisfied at once. Established independently in 2016 and 2017, and frequently over-read: it shows specific formal criteria conflict under specific conditions, not that fairness is unachievable.

The proof that when base rates differ between groups, no classifier can be calibrated and have equal error rates at once. Fairness becomes a values choice, not a bug to fix.

Whether the party producing a number loses anything when it is high. A vendor billing per resolution earns nothing on a suppressed handoff; one billing per containment does. A metric produced by someone who loses money when it rises beats one produced by someone who gains.

Dividing an annual total into equal periods and treating the result as what was earned in each. A standard method whose premise, roughly even distribution, is false by construction for a population defined by irregular work, which is what made the Australian scheme unlawful rather than merely inaccurate.

Numbers sharing a label while measuring different quantities. A range from $6.5 billion to $47 billion for one company in one quarter is not a disagreement about a fact; it is several quantities travelling under one word.

A model adapting its behaviour from examples in the prompt, with no training.

What a system adds beyond the process already running, measured on the cases that process misses rather than on all cases. A sepsis model with 33% sensitivity overall caught 7% of the cases clinicians had not identified, and only the second figure says whether it was worth having.

Separating a mixed signal into statistically independent sources, classically the problem of isolating individual voices from a recording of several. Differs from PCA by seeking independence rather than uncorrelatedness.

Choosing tractable diseases to demonstrate a platform, which inflates success rates without the method contributing anything. Named as a concern in AI drug discovery, never quantified, and testable against data that already exists.

Hiding instructions in content the model will later read — a web page, a document, an email — so they hijack it without the user typing them. The dominant agent security risk.

The assumptions a model makes before seeing any data. Without them learning is impossible — there's a theorem.

Learning logic programs from examples and background knowledge. The symbolic counterpart to statistical learning: produces rules a person can read and check, at the cost of scaling poorly and needing clean structure.

Using a trained model to get an answer. Where the money goes at scale.

The software that actually runs a model fast — vLLM, TensorRT-LLM, llama.cpp — handling batching, caching, and kernels. Where serving performance is won.

Trading extra compute at answer-time for accuracy, instead of extra training. The scaling axis that reasoning models exploit.

Spending more computation at answer time — longer reasoning, more samples — for better results. The scaling axis reasoning models opened.

A framing of learning as compressing the input while retaining what predicts the output. Offers a theoretical account of why representations generalise, and its application to deep networks remains contested.

Shannon's mathematics of surprise — measuring information in bits. The hidden language of ML: most loss functions are information-theoretic quantities.

A test built for human education and reused for machines, which isolates the part of a task humans find hard. Reasoning is hard for people and gathering is routine; for a model the difficulty runs the other way, so the test flatters the opposite difficulty curve.

Filling in a masked region so it matches the rest. Commercially the most useful generative feature, and it quietly ended photographic evidence.

Labelling every pixel and separating individual objects, so you can count them.

Branding lending weight a piece of work has not independently earned. A project originating at an institution and not administered by it acquires that institution's authority through a logo, and the distinction does not survive the phrase used in coverage.

Training a base model to follow instructions rather than merely continue text.

What it takes to observe a given stage of a process. Determines evidence density across a field independently of which stage decides the outcome, so precision accumulates about capability and does not accumulate about consequence.

Eight-bit integer quantization. The workhorse compression for serving models cheaply.

Attribution computed by accumulating gradients along a path from a baseline input to the actual input. Satisfies stated axioms that simple gradient methods do not, and depends heavily on the baseline chosen.

The word underneath "artificial intelligence," used constantly and defined by nobody. Over seventy published definitions and counting.

The proposed escalation where a system capable enough to improve its own design produces a more capable successor, repeatedly. Described in 1965. The empirical record contains zero instances of sustained, open-ended, autonomous self-improvement.

The specific clinical purpose a device is authorised for. Regulation attaches to it, which is why a general model answering arbitrary clinical questions sits awkwardly in a framework built for devices with a defined one.

How often your human labellers agree with each other. The real ceiling on your model, and the number most projects never compute.

The waiting list grid operators maintain for new generation and large-load connections, now the effective rationing mechanism for data centre construction. Around 2,300 GW sits in US queues, more than the country's installed capacity, and waits in the densest markets run 4 to 7 years.

A party with a stake in the result taking part in assessing it. Disclosed participation is normal and often necessary, and it is not a finding of error; it is a reason to want an independent synthesis of equal scope, which frequently does not exist.

A source whose data may be sound and whose emphasis is not disinterested, because it sells the solution to the problem it describes. Off-grid campus operators and turbine vendors document grid constraints accurately and benefit from the perception that grid connection is hopeless.

A measurable proxy standing in for the outcome that matters. Diagnostic accuracy against a reference standard is the endpoint of essentially all LLM clinical evaluation, and no study measures whether patients did better.

The model size at which a network becomes just able to fit the training data exactly. The peak of the double-descent curve, past which bigger starts helping again.

Working out what's actually happening inside a model. Distinct from explainability, much harder, and the only approach that could tell you what a system will do before it does it.

How much independent human judges agree with each other. The ceiling on what any automated scorer can be expected to match, and its omission is why an agreement figure cannot distinguish a good scorer from an ambiguous rubric.

A cancer diagnosed between screening rounds that screening missed. Carries higher disease-specific mortality than screen-detected cancer, which makes it the endpoint a screening trial is judged on rather than detection counts.

How often a person corrects, resets or rescues a machine per hour of operation. The figure that separates autonomy from supervised teleoperation, almost never published, and the first thing to ask for when a deployment claim is made.

Inferring the reward function from observed behavior. Learning what someone wants by watching what they do.

A signal that directs where to look, not a conclusion about what was found. A face recognition system returns the database faces that most resemble a probe image; reading that as an identification is where the harm in Williams v Detroit began.

How much a predicted region overlaps the true one. The standard metric for detection and segmentation.

A variant of direct preference optimization designed to resist the overfitting to preferences that DPO can suffer, by changing its objective.

An error that cannot be corrected on a later attempt. A weeding robot at 95% accuracy is excellent because the missed 5% is caught next pass; a harvesting robot at 95% damages one fruit in twenty, permanently. The same accuracy figure means different things depending on whether the error can be undone.

The uneven boundary of AI capability, where systems excel at some tasks and fail at others of similar apparent difficulty, and nobody can tell in advance which is which.

Getting a model to do what it was trained to refuse. Persists because refusals are tendencies, not rules.

An input crafted to bypass a model's refusals. The cat in an endless cat-and-mouse.

Google's composable ML framework: NumPy plus autodiff plus compilation. The researcher's choice for TPU-scale work. ::

A symmetric measure of distance between distributions. KL divergence's better-behaved sibling. ::

Efficiency gains increasing total consumption rather than reducing it, because cheaper use drives more use. Named for Jevons on coal in 1865 and visible in compute now: one operator cut data centre emissions 12% in a year while absolute data centre electricity grew 27%.

Computing similarity in a high-dimensional space without ever constructing the coordinates, by using a function that returns inner products directly. The trick that made support vector machines practical on non-linear problems.

Computing similarity as if the data lived in a much higher-dimensional space, without ever going there. What made SVMs powerful on non-linear problems.

How much one distribution diverges from another. The regularizer keeping RLHF models near their base.

The default clustering algorithm. Assumes round, similar-sized groups, and finds them whether or not they're there.

Predict by looking at the most similar examples you've seen. No training at all, and the ancestor of every vector search you use.

Getting rules out of an expert takes months, and most of what they know they cannot articulate. The thing that killed expert systems.

A structured store of facts and rules a system reasons over. The heart of expert systems then; the retrieval corpus behind RAG now.

The date after which a model knows nothing. Why it's confident and wrong about recent events.

Training a small model to mimic a large one's outputs. How capability gets compressed into deployable sizes.

Facts as a network of entities and relationships. Symbolic AI's one commercial survivor, and RAG is rediscovering it.

The old central question of AI: how to encode what a system knows so it can reason over it — logic, frames, semantic networks, and today, knowledge graphs and weights.

A chance-corrected measure of how much annotators agree, working across rating scales and missing data. The standard answer to "can we trust these labels?"

A preference-tuning method that learns from simple good/bad labels on individual outputs, rather than paired comparisons — drawing on prospect theory's model of how people weigh gains and losses.

Stored attention keys and values reused while generating. Grows with context length, which is why long prompts get expensive to serve.

The correct answer attached to a training example.

The old ceiling on supervised learning: you could only train on as much data as humans could afford to annotate. Self-supervision removed it, and foundation models followed.

What a model is actually trained to predict, as distinct from what it is described as predicting. Rearrest is not offending, default is not inability to pay, attrition is not performance, and every fairness argument is conditioned on a label somebody selected.

Wrong answers in the training labels. Ubiquitous, and models are more robust to it than intuition expects. ::

Softening one-hot targets slightly. A small regularizer with an outsized following.

The label for models trained to spend inference-time compute thinking before answering — o1-style. Same architecture family as LLMs; different training and different cost profile.

An embedding method that runs the whole document through a long-context embedder first, then splits the token embeddings into chunks — so each chunk's vector already reflects the surrounding text. Distinct from Anthropic's text-level contextual retrieval. Günther et al., 2024.

Keeping one embedding per token and matching them at query time — between bi-encoder speed and cross-encoder accuracy. ColBERT's mechanism.

The response-time ceiling a system must meet. The constraint that decides which models you can afford to serve. ::

Running the diffusion process in a compressed latent space rather than on raw pixels, which is what made high-resolution image generation affordable. The basis of Stable Diffusion.

Blei, Ng & Jordan (2003): models each document as a mixture of topics and each topic as a distribution over words. The dominant topic model for a decade, and still useful where an interpretable decomposition matters more than raw accuracy.

The compressed internal representation a model works in. Where diffusion models do their denoising.

A hidden factor a model infers but never observes directly. The machinery under VAEs and topic models.

One transformation stage in a network. Depth is how many.

Rescaling activations within a layer to keep training stable. Quiet, essential, invisible.

Normalizing across features within each example. The stabilizer transformers use instead of batch norm.

Years between order and delivery, which converts a purchasing decision into a scheduling constraint. Large power transformers run 3 to 5 years, up from 24 to 30 months before 2020, and medium-voltage switchgear is effectively sold out through 2028.

Building a feature from information unavailable at prediction time. Produces brilliant validation scores and useless models.

ReLU with a small slope for negatives instead of zero. The fix for dead neurons.

How big a step to take downhill. The single most important dial in training.

Varying the step size over training: warmup, then decay. Standard practice with surprisingly deep consequences.

Training a model on which results users preferred, to order search results. The machine-learning core of every serious search engine.

Giving a system only the permissions its current task requires, enforced in infrastructure rather than described in a prompt. The one control that caps the damage of every other failure.

Reducing a word to its dictionary form (better → good) using vocabulary and grammar — more accurate but slower than stemming.

A recorded account of what went wrong in a procurement or deployment. The cheapest governance intervention available and the one almost nobody performs, because a candid record of failure is uncomfortable to produce and has no natural owner.

An agent that simultaneously holds private data, ingests untrusted external content, and can communicate or act externally. Named by Simon Willison. When all three hold, a successful indirect injection can exfiltrate data with no human interaction, and most deployed agents satisfy all three because all three are what makes an agent useful.

The 1973 review that dismantled British AI. Its combinatorial explosion argument was correct, and its conclusion wasn't.

Attention variants that scale linearly with sequence length instead of quadratically, by approximating the softmax. The trade: efficiency for some recall.

A single linear layer trained on frozen features. The standard test of what a representation contains.

The foundational model: fit the best straight line through data to predict a number. Interpretable, and the baseline every other model is measured against.

Assembling a photo array from a system's own top candidates, so the witness is shown the face a machine already selected for resemblance alongside faces it did not. The identification is no longer independent, and no improvement in model accuracy repairs it.

Judging whether a connection should exist between two entities that are not currently connected. Underlies recommendation, knowledge graph completion and drug target discovery.

An evaluation refreshed with new items after models are released, so contamination cannot accumulate. Solves the staleness problem and introduces a comparability problem, since scores across versions measure different tests.

Meta's open-weight model family. The release that made local LLMs an ecosystem.

A model trained on enormous text corpora to predict what comes next, and capable of far more than that suggests.

Using a model to grade another model's output. Cheap, scalable, correlates decently with humans, and it prefers its own writing.

The coined term for falling cost per unit of model capability, popularised as roughly 10x per year against a fixed quality bar. A serviceable planning approximation and a compression of a range spanning two orders of magnitude.

The practices for running LLM applications in production — evaluation, monitoring, versioning, cost and latency control. MLOps adapted to models you didn't train.

Training pressure that stops an MoE router collapsing onto a few favourite experts and starving the rest into dead parameters. Finicky, and a common source of instability.

One that many others depend on, measured by how much becomes unreachable without it. Frequently unglamorous: in this corpus the top two are training data and supervised learning.

Harm whose severity depends on where it occurs. A tonne of CO2 has the same effect anywhere; a million gallons a day means something entirely different in Iowa than in Arizona. Aggregate figures systematically miss constraints that bind at basin or grid level.

The distinction between what a platform observed and what a participant recalled. In teacher AI research the adoption and usage figures are logged and the time savings are recalled, which is why the first are corroborated and the second vary fivefold across sources.

Writing programs as facts and rules, and letting an inference engine derive answers — computation as deduction. Prolog is the family's famous member.

Linear regression's classifier cousin — squashes the output through a sigmoid to predict the probability of a yes/no class. The default first model for binary classification.

Projecting a model's intermediate activations directly into output space to see what it would predict at each layer. Reveals how a prediction forms across depth, and is an approximation rather than a faithful decoding.

The raw scores a model outputs before they're turned into probabilities. What temperature is applied to.

The model's confidence in a token, in log space. What perplexity aggregates and APIs expose.

The model's own probability for each token it considered. The raw material for confidence estimates, ranking, and detecting when a model is guessing.

Windows of hundreds of thousands of tokens. Impressive on paper; attention quality degrades before the limit does.

Low-rank adaptation. Fine-tuning by training a small add-on rather than the whole model. The sensible default.

The dimensionality of a LoRA update. The dial trading adapter capacity against size.

Training loss plotted over time. The first thing practitioners look at and the last thing they trust.

The number saying how wrong the model is, and therefore the definition of what it's trying to become.

A sudden jump in training loss at scale. The instability large-model teams learn to survive. ::

Liu et al.'s finding that models retrieve information best from the start or end of a long context and worst from the middle — a U-shaped curve. Later benchmarks find the effect varies by task and model.

Frankle & Carbin (2019): a randomly initialised network contains a much smaller subnetwork that, trained alone from the same initialisation, matches the full network's accuracy. Suggests overparameterization helps by supplying many candidate subnetworks rather than by the parameters all being needed.

An RNN with gates deciding what to remember and forget. The fix that made sequence learning work, and it held for twenty years.

Lucas & Kanade (1981): estimates motion between frames by assuming brightness is constant and motion is small and locally uniform. Sparse and fast, where Horn-Schunck (1981) solves the dense version. Both predate and still inform neural optical flow.

Getting computers to learn patterns from data instead of being programmed with rules.

Translating between languages automatically. Invented modern NLP, and fluency arrived long before reliability.

Removing specific data's influence from a trained model. Demanded by regulation, unsolved in general. ::

Inventory built to carry ads rather than to be read. Fell from 21% of member budgets in 2023 to 0.6% by late 2025, then rose to 1.1% in Q1 2026, the first increase since counting began, with cheap generated content named as a contributing sub-type.

Mean absolute and mean squared error. MSE squares mistakes, so it obsesses over outliers.

A state-space architecture with linear-time sequence processing. The most serious recent challenge to attention.

Average precision across recall levels, averaged over classes. The headline metric for object detection, and easy to misread across differing IoU thresholds.

Price falling because competition removes profit rather than because cost fell. Indistinguishable from genuine efficiency in published prices, identical to a buyer today, and not durable, which matters for forecasting and not for deciding whether something is affordable now.

A process where the next state depends only on the current one. The foundational abstraction beneath language models, reinforcement learning, and much of probability.

Sampling from a distribution that cannot be computed directly, by constructing a random walk whose long-run behaviour matches it. Asymptotically exact and slow, which is why variational inference is often preferred at scale.

The formal frame underneath all of reinforcement learning. Built on an assumption that's almost always false, and it works anyway.

Predicting hidden tokens from both directions of context. BERT's pretraining objective. ::

A single model covering a hundred or more languages, on the argument that transfer lifts even languages with almost no data of their own. The competing strategy to regional models.

A legal obligation to report significant financial events to shareholders. It produced better documentation of an AI failure than any AI-specific mechanism has, because it attaches to money rather than technology and non-compliance is a securities offence.

Vectors whose prefixes are themselves usable embeddings. Truncate to trade accuracy for cost.

Resources, recognition and influence accruing to those who already have them. Documented in AI adoption: a tool free to everyone was taken up disproportionately by already-productive researchers, exacerbating citation polarisation rather than levelling it.

Maximum likelihood with a prior: choose the parameters most probable given both the data and what you believed beforehand. Regularization is MAP estimation in disguise, with the penalty term acting as the prior.

Fisher: choose the parameters that make the observed data most probable. The default fitting principle across statistics and machine learning, and what cross-entropy loss is doing underneath.

The open standard connecting AI models to tools and data. The USB port the agent ecosystem standardized on.

Where the only complete record sits with the subject of the measurement, so every external figure is a panel estimate or a proxy however carefully built. The platform knows exactly how many searches end without a click, is not obliged to say, and does not.

Research effort settling at the stage of a causal chain that is cheapest to measure rather than the stage that determines the outcome. Model discrimination is measured retrospectively without deploying anything; override rates and outcomes are not.

How far out an outcome is measured. Thirty-day surgical outcomes are abundant because they are cheap to collect; survival and recurrence need years and much larger trials. The outcomes people care about most tend to be the ones covered least.

The reliability of the figures about a subject, considered separately from the subject itself. It can fail in patterned ways that have nothing to do with whether the underlying thing is large or small, and across ten subjects it tracked disclosure obligation rather than importance.

A failure mode usually attributed to machine learning that occurs identically in deterministic software. If the mechanism does not require AI, then fixing AI does not fix the mechanism.

Reverse-engineering the circuits inside networks. Interpretability's most ambitious branch.

A frequency warping matching how human hearing resolves low tones better than high. 1937 psychoacoustics, sitting under modern audio AI.

Determining whether a datapoint was in the training set. The canonical privacy attack on models. ::

Determining whether a specific record was in a model's training data by probing its behaviour. The canonical privacy attack on trained models.

When a model reproduces training data rather than generalising. A privacy problem as well as a quality one.

A model storing training data verbatim rather than generalizing. Where copyright and privacy risk concentrate.

Summarising or discarding older context to fit within a window. The point at which information is silently lost, and what gets discarded is rarely logged, which makes failures hard to diagnose afterwards.

The layered arrangement from registers through caches to high-bandwidth memory to host memory, each larger and slower than the last. Most performance work in AI systems is moving data to the right level rather than making arithmetic faster.

The widening gap between how fast chips compute and how fast data reaches them. Modern inference waits on memory, not arithmetic, which is why fetching fewer bytes is the whole optimisation game.

The core operation of a graph neural network. Each node collects vectors from its neighbours, aggregates them with a permutation-invariant function such as sum or mean, and combines the result with its own state. Permutation invariance is required because a graph has no canonical node ordering.

Ordinary image pipeline behaviour that removes embedded data during upload, transcoding and re-encoding, including provenance manifests along with EXIF and IPTC. Not hostility to the standard, and the reason signed content commonly reaches viewers unsigned.

Learning how to learn: adapting quickly to new tasks from few examples. In-context learning made the classic methods feel quaint. ::

An evaluation measure losing validity as systems improve past the range it was validated on. Every automatic translation metric followed this pattern, and nothing announces when it happens.

Distinct safety measures moving in opposite directions on the same events. A shift from rare-and-severe to frequent-and-less-severe injuries raises recorded incident rate while lowering serious injury rate, so two parties can both report accurately and appear to contradict each other.

A compressed cepstral feature that ruled speech recognition for thirty years. Deep learning made it obsolete; seeing it in new work is inherited habit.

Which capability level a cost or progress rate describes. One study measured inference price declines of 9x to 900x per year across six benchmarks, so a single rate is a selection among those, and the selection almost never travels with the figure.

An open-source vector database built for scale. One of the names behind semantic search infrastructure.

Choosing the model that most compresses the data, counting the model's own description as part of the cost. A formalisation of Occam's razor, and one of the older answers to why simpler models generalise.

The smallest fragment of a finding that can be reported without misleading, which no author currently states and every author could. Would remove selective transmission at the point where it starts, for the cost of one sentence.

Keeping only tokens whose probability is at least a set fraction of the top token's. Adapts to the model's confidence — tight when it's sure, permissive when it isn't — and increasingly the default over top-p.

Perceptrons showed a single-layer network cannot learn XOR, a function requiring a boundary no straight line can draw. The result was correct and narrower than its reception: multi-layer networks were not ruled out, only untrainable at the time. Widely credited with the first AI winter.

A jailbreak mechanism: a model's capabilities generalise further than its safety training did, so inputs outside the trained range still work while the refusal fails to fire.

A failure or success rate published without the corresponding rate for the incumbent. A 95% pilot failure figure means little without the rate at which comparable non-AI pilots show measurable return in the same window, which nobody states.

Training in low precision (FP16 or bfloat16) for speed while keeping a high-precision copy for stability. Standard for modern deep learning.

A transformer variant that lets each token use a different number of layers, spending compute where it's needed rather than uniformly. A dynamic-compute cousin of mixture-of-experts.

A model with many specialist sub-networks that wakes only a few per token. How frontier models got enormous without getting proportionally slow.

Training on weighted blends of pairs of examples and their labels. Improves calibration and robustness, and works for reasons that remain partly unexplained.

Choosing parameters that make the observed data most probable. The principle behind most training objectives. ::

The engineering around a model that makes it a system rather than a notebook. The model is a few percent of it.

Stacked fully connected layers. The plain network, and most of a transformer's parameters.

The 57-subject knowledge benchmark models are headlined on. Saturating, contaminated, and still quoted.

A kind of data: text, image, audio, video. What "multimodal" counts.

Image and text embeddings occupy separate cones rather than mixing. The shared space isn't as shared as the diagram implies.

When a generator finds a few outputs that work and produces only those. The characteristic GAN failure.

Whether an agent learns a model of how the environment responds and plans against it, or learns which actions pay off without any such model. Model-based is more sample-efficient and inherits the model's errors; model-free is simpler and needs far more experience.

Documentation of what a model is, what it was trained on, and where it fails.

Degradation when models train on their own output across generations. Real — and the dramatic version needs you to throw away your real data, which nobody does.

An open standard for connecting models to tools and data. Solves a real integration problem, and a standards fight is a political event.

Reconstructing a model's behaviour — or the model itself — by querying it enough. Why API providers rate-limit and watermark.

A public repository of pretrained models anyone can download. Democratised AI, and it's a software supply chain nobody is securing.

A formal register of every model an institution runs. The most common examination finding in banking is that it is incomplete, because machine learning keeps lowering the cost of producing something model-shaped and no workable definition separates a governed model from a spreadsheet with a regression in it.

Watching a deployed model for failures that don't raise errors. It returns 200 and lies, and the thing you need to watch needs labels you don't have.

Splitting one model across devices. What you do when it doesn't fit on one. ::

Versioned storage of trained models and their metadata. The deployment ledger of MLOps.

Governance for the possibility that a model is wrong or misused. Formalised in US banking from 2011, scaling intensity with complexity, uncertainty, breadth of use and impact, and superseded in April 2026 by guidance that deliberately excludes generative and agentic systems.

Sending each request to the cheapest model that can handle it — easy queries to small models, hard ones to frontier ones. Cost control as an architecture.

Getting a trained model to answer requests reliably. The model is the easy part; the queue is the hard one.

Averaging weights of several fine-tunes into one model. Ensembling's free-lunch cousin, when it works. ::

A published statement of how a model is intended to behave. The policy document behind the assistant. ::

Tracking versions of models, data, and code together so a deployed model can be reproduced, audited, or rolled back.

The learned gate choosing which experts process each token. Where mixture-of-experts models succeed or collapse.

Estimating a compound's characteristics from its structure, typically with graph neural networks treating atoms as nodes and bonds as edges. The stage of drug discovery where machine learning has shown its clearest measured advantage.

Accumulating velocity across training steps, damping oscillation.

A feature that responds to exactly one human-interpretable concept. What sparse autoencoders recover from the tangled, polysemantic neurons of a model.

Planning by simulating many futures from the current state and expanding the promising branches. The search half of AlphaGo.

Moore (1965): transistor count per chip roughly doubles every two years. An observation about economics and manufacturing rather than a law of physics, and the reason compute became cheap enough for deep learning to be attempted at all.

How words are built from meaningful parts. Predicts degradation in agglutinative and morphologically rich languages, where one word carries what English spreads across a phrase and tokenizers split morphemes into pieces that are not morphemes.

Discretising continuous movement into a vocabulary of skill tokens so a transformer can model sequences of them and generate control by next-token prediction. Framed as next-token prediction for humanoid locomotion in 2024 and scaled with quantisation refinements since.

A benchmark suite for embedding models across many retrieval and classification tasks. Useful as a shortlist, misleading if you skip testing on your own data.

Several agents working on one problem. Strong in demos, awkward in production; coordination costs are real.

Having models argue opposing positions and a judge pick the survivor. Improves factuality on some tasks; also multiplies cost and can converge on confident nonsense.

Several agents cooperating or competing on a task. Powerful in demos; coordination costs bite in production.

The simplest exploration problem: repeatedly choose between options with unknown payoffs. The formal core of A/B testing and recommendation.

Running many attention computations in parallel and concatenating. The transformer's standard form of attention.

Answering questions that need a chain of lookups — find A to learn what to ask about B. Where single-shot RAG quietly fails.

Handling more than one kind of input — text and images, sometimes audio or video.

Retrieval over images, tables, and audio as well as text — finding the diagram that answers the question, not just the paragraph.

The problem that testing enough configurations guarantees some will beat a baseline by chance. The reported result is the maximum of a search rather than a draw from it, which error bars on the winner do not address.

The extreme of grouped-query attention: all heads share one set of keys and values. Smallest cache, slight quality cost.

Training one model on several tasks at once. Tasks share structure; sometimes they fight. ::

Training a model to predict several future tokens at once rather than only the next. Improves sample efficiency and provides draft tokens that speculative decoding can verify cheaply.

Models that produce music from a description. Good enough for background, sitting on an unresolved argument about whose work it learned from.

How much knowing one variable tells you about another — any dependence, not just linear. Zero exactly when the two are independent.

A classifier that assumes every feature is independent given the class. The assumption is false and the method works anyway — a running lesson in useful wrong models.

A specific person, place, organization, or thing in text. What NER exists to find.

Pulling names, dates and places out of text. Reported as solved, reliably disappointing on anything that isn't news.

A state where no player gains by changing strategy alone. Game theory's anchor, and multi-agent RL's. ::

Finding the stored vectors closest to a query vector. The core operation behind semantic search and every vector database.

A failure caught before it reached anyone. Worth recording and worth labelling separately, since counting near-misses as incidents overstates harm while excluding them hides the cases where testing worked.

Hiding a sentence in a long document and asking the model to find it. The industry's favourite long-context test, and close to string matching.

Telling an image generator what to avoid. Subtraction as a creative control.

An ordinary tort requiring a duty of care, an inaccurate statement, negligence, reasonable reliance and resulting loss. Applied to a chatbot's output in 2024 with no new doctrine created: the analysis would have been identical had a human agent given the same wrong answer.

A specialist GPU cloud provider outside the established hyperscalers. Absorbs capital expenditure transferred from customers with stronger balance sheets, so spending visible in aggregate becomes invisible company by company.

Fukushima (1980): a hierarchical network with alternating feature-detecting and pooling layers, inspired by the visual cortex. The direct architectural ancestor of the convolutional network, built almost a decade before backpropagation made such networks trainable.

Representing a 3D scene as a network that maps position and viewing angle to colour and density, trained from photos. Novel views from ordinary pictures.

Finding names, places, and organisations in text. Standard first step in PII detection.

Automating network design itself. Expensive, occasionally brilliant, rarely worth it since transformers standardized. ::

Layers of simple units whose connection strengths are adjusted until the output is useful.

A model that learns mappings between function spaces rather than between points, allowing a single trained model to solve a family of differential equations at any resolution.

Reconstructing a 3D scene from photos by training a network to be the scene. A beautiful idea, replaced in three years.

Jacot et al. (2018): in the infinite-width limit, a neural network behaves like a fixed kernel method and training becomes linear. Gives rigorous results about a regime real networks only approximate, which is both its value and its limitation.

Hardware that mimics the brain's spiking, event-driven style rather than clocked matrix math. Long promising, still niche — efficiency is the draw.

A unit computing a weighted sum then a nonlinearity. The metaphor the field is named after and has outgrown.

The training objective of LLMs: guess what comes next, billions of times. Simple enough to scale, rich enough to work. ::

A run of n consecutive tokens. The unit of pre-neural language modeling and of BLEU.

The most common reason businesses give for not adopting AI, reported by 77% of small firms and concentrated in construction, food service and the trades. Worth taking at face value: those firms looked and concluded nothing on offer addresses their work.

Labelling an individual entity from its position in a network, such as flagging an account that looks ordinary alone and suspicious in the company it keeps. One of the three standard graph learning tasks.

Averaged over all possible problems, every algorithm performs identically. It means your advantage comes entirely from your assumptions matching reality.

A decision widely cited without formally binding anyone. The best-known AI liability ruling is a small-claims decision made on documents, without counsel, with the contracts never filed, and it binds no court anywhere.

Service accounts, API keys, workload identities and agents. Now outnumbering human identities by somewhere between 45:1 and 100:1, unable to use multi-factor authentication, never logging out, and rarely retired.

A trial designed to show a new approach is not worse rather than better. Succeeding means demonstrating an absence, which is hard to headline, and with a true difference of zero the point estimate favours the intervention about half the time by chance, leaving a favourable number available for misreporting.

The 40% of teacher working hours the OECD found spent on tasks other than teaching, which is the burden the workload case rests on. Lesson planning, which accounts for 60 to 99% of AI platform usage, is not part of it.

Keeping the best of many overlapping detections of the same object and discarding the rest. The cleanup step nearly every detector runs.

Rescaling values to a common range so no feature dominates by magnitude alone.

The rate at which a failure type stops being reported as it becomes normal. The first fabricated legal citation was news and the thousandth was not, so an incident count falls for any failure mode that is becoming routine rather than rarer.

A neural processing unit in consumer devices. The chip making on-device AI a spec-sheet line. ::

A community-developed RoPE interpolation that extends context length by scaling frequencies unevenly — preserving high-frequency detail. Preceded and inspired YaRN.

Sampling only from the smallest set of tokens whose probabilities sum to p. Better text than pure likelihood.

The condition where an absence of evidence cannot be interpreted, because the regulated party controls whether they are subject to the requirement. Under NYC Local Law 144 a missing bias audit could mean non-compliance or a scope determination, and nothing distinguishes them. The default failure mode of any regulation asking parties to self-identify into it.

How many alerts a clinician must review to find one true case. At 109 the rational response is to stop reading them, which degrades response to every other alert in the same system.

How many patients must be flagged or treated to produce one benefit. A sepsis model with an NNT of 21 to 35 fires on twenty-one patients per real case, which is why alert fatigue is an operational problem that accuracy metrics do not capture.

A high-bandwidth interconnect between accelerators in one machine, faster than PCIe by roughly an order of magnitude. Determines whether a model too large for one device can be split across several without the interconnect becoming the bottleneck.

Finding where things are in an image and what they are. Boxes, not outlines.

A policy coming to be judged by whichever of its effects is easiest to count. Compute production is reported quarterly and moved as intended; adoption is harder to measure and moved the other way, and the first became the way the policy is discussed.

The formal thing being optimized. Where alignment problems are born.

Following the same object across video frames. A 200-line algorithm from 2016 still beats most deep learning.

Turning pictures of text into text. Solved for clean scans, genuinely hard for everything else.

Running a fixed set of cases against a system before it ships. The regression guard, and limited to what you thought to include. Pairs with online evaluation, which supplies the cases you did not.

Learning a policy from a fixed dataset of past interactions, no new exploration. RL when trying things is expensive or unsafe.

A single model trained natively on text, audio, and images together rather than bolted-on encoders — speech in, speech out, no pipeline. The direction flagship models moved in 2024–2025.

Representing a category as a vector of zeros with a single one. The naive alternative to embeddings.

Scoring samples of live production traffic rather than a fixed test set. Surfaces the failures real users produce that no curated set anticipated, and should be asynchronous so it does not add latency.

Updating a model continuously as data arrives rather than retraining in batches. Powerful and risky — feedback loops and poisoning arrive with it.

An open format for exchanging trained models between frameworks. Write once, run anywhere, mostly. ::

A formal specification of concepts and their relations. Knowledge graphs' schema layer. ::

An input space that cannot be enumerated in advance. Symbolic tasks are controlled by default because theirs can be; physical tasks are made controlled at cost, which is why automation concentrates where a workspace can be rebuilt around a machine.

Weights plus training code plus data information, under terms preserving the freedom to use, study, modify and share. Far stricter than open weights, and almost nothing qualifies.

A model whose weights you can download and run. Usually not open source, whatever the announcement says.

Downloadable model parameters, license permitting use. Not the same as open source — the training data and code usually stay closed.

A specification precise enough that two observers of the same system would agree on whether a condition holds. Superintelligence has none, which is why disagreement about its arrival is frequently disagreement about thresholds conducted as though it were about facts.

Estimating the motion of every pixel between two frames. One equation, two unknowns — local motion is fundamentally underdetermined.

The algorithm that decides how to apply gradients. Adam unless you have a reason.

The algorithm that decides how to apply the gradient. Adam is the default, AdamW is what you should use, SGD still wins sometimes.

The extra per-parameter memory optimizers like Adam carry. Why training takes several times the model's own memory.

Supplying ground-truth speech regions to a diarization system during evaluation rather than letting it detect them. Removes an entire stage's errors, which is why figures reported this way are not comparable to real deployment.

Coordinating models, tools, and steps into a working system. Where agent engineering actually lives. ::

The component that routes work between models, tools, and subagents — deciding who does what and in what order. Where most multi-agent complexity actually lives.

The supervising agent that decomposes a task, delegates to specialists and assembles the result. A single accountability point and a single point of failure, whose own context fills as the workflow proceeds.

A preference-tuning method that folds alignment into the supervised fine-tuning step using an odds-ratio term, removing the need for a separate reference model. Part of the post-RLHF family alongside DPO.

Charging per result rather than per seat. Published 2026 rates run from $0.50 per resolved conversation to $2.00 per conversation, and the alignment is genuinely better than per-seat, which charges for the humans an organisation is trying to redeploy.

A reward model that scores only whether the final answer is right. Cheaper to build than process rewards, easier to game.

Reporting a different primary outcome than the one pre-registered, which changes the question after seeing the answer. Detectable by comparing the registration with the publication, and common enough that the comparison is worth making.

Inputs unlike anything in training. Where model confidence and model competence part ways.

Detecting disease that would never have caused symptoms, where treatment causes harm and no benefit. A 29% increase in cancer detection is a benefit only to the extent the additional cancers would have progressed, which is why detection alone is an ambiguous screening result.

Learning the training data instead of the pattern. Perfect on what it's seen, useless on what it hasn't.

Two or more people talking simultaneously. The failure case that broke clustering-based diarization, since clustering assigns each moment to exactly one speaker, and it concentrates at interruptions where the content usually is.

Far more parameters than datapoints. Classical statistics says disaster; deep learning says Tuesday.

The share of alerts a clinician dismisses, documented between 49% and 96% across clinical decision support studies. Bounds the value of any accuracy improvement, since a better model entering a saturated channel is a better model nobody reads.

Node representations converging toward each other as graph network depth increases, until they carry no distinguishing information. The reason most deployed graph networks are shallow, and why stacking layers does not extend reach.

Information from an exponentially growing neighbourhood compressed into a fixed-size vector, so distant dependencies are lost through a bottleneck rather than through insufficient depth. Distinct from over-smoothing and not fixed by the same remedies.

Managing KV-cache memory in virtual-memory-style pages. vLLM's core idea, and a serving-throughput landmark. ::

Running a survey and a measurement on the same population, which converts a contradiction into an estimate of displaced cost. Ask how a change felt, then track what happened to that specific output. One organisation, one quarter, and commissioned by nobody.

Judging outputs by comparing two at a time rather than scoring in isolation. More reliable for humans and LLM judges alike — comparison is easier than calibration.

An organisation producing fraudulent manuscripts and selling authorship. Predates language models, since the cause is an incentive structure that pays for publication; cheaper drafting lowers a cost that was already being paid rather than creating the demand.

One learned number inside a model. Counted in billions, and a poor proxy for capability.

Matching on small chunks for precision, then returning the larger parent section for context. A standard fix for fragmentary RAG answers.

The set of options where improving one metric requires worsening another. Where model selection actually happens. ::

The probability an agent succeeds on all k independent attempts at a task, introduced by tau-bench. Distinct from pass@k, which counts a task solved if any one attempt works. One model scored 61% at pass^1 and 25% at pass^8 on the same retail tasks, and only the first figure circulates.

The chance at least one of k generated solutions passes the tests. Code generation's honest metric.

Principal component analysis. The classic way to reduce dimensions while keeping variance.

The family of methods — LoRA, adapters, prompt tuning — that adapt a model by training a tiny fraction of its parameters. What made fine-tuning affordable.

The distance between measured and self-reported productivity. A randomised trial found developers 19% slower with AI tools while they reported being 20% faster, a 39-point gap from people who had just done the work on their own code.

The first trainable neural network, 1958. The story that a book killed it is the most repeated wrong story in AI.

Rosenblatt (1958): if the data can be separated by a straight line, the perceptron learning rule will find such a line in finite time. The first proof that a machine could be guaranteed to learn, and the reason the early optimism was not unfounded.

How surprised a model is by text. Lower is better, and it hides exactly the capabilities you care about.

Answering a question about a total with a figure about a unit. Falling cost per token does not bound total spend, and falling energy per query does not bound total consumption, because usage grows faster than efficiency in both cases.

The level of linguistic analysis concerned with sound structure, and by extension written form. Predicts the AI failures defined over characters, since subword tokenization discards this level: counting letters, reversing strings, rhyme.

Newell and Simon's claim that symbol manipulation is necessary and sufficient for intelligence. Falsifiable, taken seriously, and the evidence went against it.

A network trained with physical laws included in the loss function, so solutions must satisfy known equations as well as fit data. Useful where data is scarce and the governing physics is known.

Python's serialisation format. Executes arbitrary code when loaded, which makes torch.load on an untrusted file equivalent to curl | bash.

Personally identifiable information. The category that turns a data question into a legal one.

The state of an AI project that demonstrates well and never reaches production. Describes most agent implementations: 79% of enterprises report adoption against 11% running agents in production.

A managed vector database service. The name that made "vector DB" a product category.

Splitting a model's layers across devices, passing activations along. The assembly line of large-model training.

The agent pattern that separates deciding what to do from doing it — one component plans, another carries out steps. Easier to inspect and interrupt than one loop doing both.

Working out a sequence of steps before taking them. What agent demos imply models can do, and the evidence says they mostly can't.

We know more than we can tell. Why you cannot interview an expert into a rulebook, and why learning from data works where that didn't.

Learning the behaviour directly instead of learning values. The method behind RLHF, and its entire difficulty is variance.

The mapping from states to actions. The thing reinforcement learning learns.

When a single neuron fires for several unrelated concepts, so you cannot read a network neuron by neuron. The obstacle that superposition creates for interpretability.

A Markov decision process where the agent cannot observe the true state, only noisy evidence about it, so it must act on a belief rather than on knowledge. The realistic version of the MDP formalism and the standard model for decision-making under uncertainty.

A meta-analytic average hiding individual failures. A pooled AUROC gap of 0.89 against 0.82 across dermatology models sits alongside a specific deployed model at 0.50, which is chance, and patients meet models rather than meta-analyses.

Collapsing many values into one: max, mean, attention-weighted. How token vectors become one sentence vector.

Responsibility for whether an organisation's AI investments make sense in aggregate, as distinct from reviewing each one. An audit found several entities overseeing individual use cases and none owning the portfolio, which is the difference between reviewing decisions and having a strategy.

Finding the joints of a body in an image. One of the few vision tasks where synthetic data works, because the label is geometry rather than judgement.

How a model knows word order, since attention alone sees a set rather than a sequence.

A cap on how much exposure a model is permitted to accumulate before someone reviews whether it is working. The one control that bounds a decision system regardless of how wrong it turns out to be, and the only one that does not require knowing the model is wrong in advance.

Correcting machine output rather than translating from scratch. The dominant professional pattern in translation, with well-documented productivity gains, and the editor is doing a different job rather than a smaller version of the old one.

Monitoring a device after authorisation. It matters more for software than for instruments, since 5.8% of reviewed AI devices were eventually recalled, mostly for software defects.

Everything done to a model after pretraining: instruction tuning, preference alignment, RL. Where capability becomes product.

Reducing numerical precision after training is complete, without retraining. Cheaper than quantization-aware training and generally less accurate, with the gap widening at lower bit widths.

Predicting which combination of speakers is active in each frame rather than each speaker independently, turning multi-label diarization into single-label classification over a larger space and modelling speaker dependency explicitly.

The policy gradient method that trains language models. A careful study found its gains came from implementation details, not the idea in the paper.

How context and speaker intention determine what is communicated beyond literal content. Predicts literal reading of indirect requests and systematic over-informativeness. The level where most prompt frustration actually lives.

Declining to appeal losses so that no binding ruling is created. An institution can absorb a stream of individual defeats indefinitely while continuing the practice, because a first-tier loss cancels one decision and settles nothing.

Applying inputs at the resolution of the individual plant or square metre rather than the field. The commercial success story of agricultural robotics, delivering reported chemical reductions of 60 to 80% in targeted spraying, and it acts on the environment rather than on the crop.

The two ways to be right and the two ways to be wrong. The trade-off that accuracy hides from you.

How many bits represent each number. The dial between accuracy and cost that quantization turns.

How an AI-enabled device may be modified after clearance without a new submission, finalised by the FDA in December 2024. Governs post-clearance modification rather than what evidence was required to clear the device, which is a distinction routinely lost in coverage.

The already-marketed product a new device is compared against for clearance. Asking what the predicate was is the fastest way to understand what a clearance actually certifies.

Training a model on attorney-reviewed samples to rank the remaining documents in a discovery set. Court-accepted since the early 2010s, and it works because the corpus is closed: every document surfaced exists, since it came from the production set.

Whether a given score carries the same probability of the outcome for every group. The criterion the COMPAS vendor measured and satisfied. Also called calibration by group, and mathematically incompatible with equal error rates when base rates differ.

Human judgments of which output is better. The raw material of alignment.

The phase where a model processes the entire input prompt before generating anything. Compute-bound and parallelisable across tokens, which makes it fundamentally different from decode and is why the two are optimised separately.

Reusing the computed KV cache for a prompt prefix shared across requests — the same system prompt, the same document. Large real-world savings.

Prepending trainable vectors to every layer's attention, rather than just the input. A PEFT method between prompt tuning and full fine-tuning.

A working paper cited as settled fact without a word changing. The cover says preliminary, the limitations section lists the sample constraints and the short observation window, and all of it survives in the document and none of it in the citation.

Committing predictions before running an analysis, so that finding an expected pattern can be distinguished from having encoded it. Cheap, and it makes confirmation bias falsifiable rather than deniable.

A directed record of what must be understood before each concept, distinct from associative "related to" links. Because it has direction it can be sorted, which yields learning order, depth and reach.

The evidential rule that a computer system is taken to be working correctly unless the challenging party shows otherwise. It places the burden on whoever has least access to the system, in favour of whoever has most, and applies with more force to probabilistic and proprietary systems than to deterministic ones.

The first, enormous training run on broad data, before any task-specific work.

The proportions of web, code, books, and more in training. Quietly one of the most consequential design decisions.

A criminal case brought by a body other than the state. Where that body also owns the system supplying the evidence and controls disclosure of material that would undermine it, the arrangement is a structural conflict rather than a procedural detail.

Testing what information a representation contains via simple classifiers. Interpretability's measuring stick.

A small model trained to predict some property from a network's internal representations. Shows that information is present and not that the network uses it, which is the distinction most probing results elide.

An agent's record of how to do things: conventions, workflows and patterns that worked. In coding agents this is frequently a plain instruction file in the repository, which turns out to be durable and maintainable.

Constraining what may be done with a system's output rather than requiring the output to be better. Enforceable in a way an accuracy threshold is not, because a threshold needs auditing, changes with every model update, and still permits the failure.

A reward model that scores each step of a reasoning chain, not just the final answer. The supervision signal behind much of reasoning-model training.

Prompting the model to write code that computes the answer instead of reasoning in prose. Delegates arithmetic to an interpreter, which doesn't make arithmetic mistakes.

The classic logic-programming language: state what's true, ask a query, and the runtime searches for proofs. The workhorse of 1980s symbolic AI.

What you send a model.

Reusing computation for a repeated prefix. Often the single biggest cost saving available.

Writing inputs that get better outputs. Less mystical and more empirical than the name suggests.

Instructions hidden in content a model reads, which it follows as if you'd written them. Unsolved in the general case.

The fixed scaffold user input is slotted into. Unglamorous, and where many production bugs live.

Learning a handful of continuous "soft prompt" vectors while the model stays frozen. The lightest form of parameter-efficient fine-tuning.

Tracking prompts like code — versions, diffs, rollbacks — because a one-line prompt edit can change product behaviour as much as a model swap.

Whether a consequence matches the scale of what triggered it. Its absence turned incorrect fraud flags into life-destroying events in the Dutch benefits scandal, where any unproven cost voided an entire allowance with no discretion to reduce or stage recovery.

Predicting a protein's three-dimensional structure from its amino acid sequence. The problem where structure prediction reached experimental accuracy, and a case where a solved prediction task did not immediately solve the downstream design task.

Attaching a verifiable signature at the moment of capture and carrying it forward, so the question becomes whether a chain verifies rather than whether something looks real. Unlike detection, it does not degrade as generation improves.

The RL algorithm that keeps updates conservatively close to the current policy. The workhorse behind classic RLHF.

A measure standing in for a measure standing in for the thing that matters. Training provision stands in for teacher capability, which stands in for the designed and supervised condition the outcome evidence supports, with neither step measured.

A model finding features that correlate with a protected characteristic without using the characteristic itself, such as a word associated with women's activities. Any system reading free text has thousands of unenumerated correlates, so removing the ones you thought of does not address the ones you did not.

A measure standing in for an experience it turns out to represent poorly. Time-in-note was assumed to capture documentation burden; burnout fell thirteen points in thirty days while time savings may be sixteen minutes, which suggests the aversiveness of the task mattered more than its duration.

Removing weights that don't matter to shrink a model.

Training on labelled data, predicting on unlabelled data, and promoting confident predictions to labels. Effective, and its failure mode is that confident errors compound across rounds.

The tendency for positive results to reach print while null results do not. Runs in a predictable direction in education technology, where a study finding no effect is harder to publish and less likely to be funded by anyone selling a product. Trim-and-fill is the standard correction.

Whether an automated system's answers agree with the organisation's own published documents. The cheapest control in AI governance and almost never run: take the most-asked questions, take the pages that answer them, and check weekly that the two still agree.

Whether every problem whose solution can be checked quickly can also be solved quickly. Unresolved since 1971 and one of the Millennium Prize problems. It bounds what any reasoning system can achieve efficiently, however capable, since a model cannot make an intractable problem tractable by being larger.

The framework that won by being easier to debug. A lesson about developer experience the industry keeps having to relearn.

Learning the value of every action in every state by bootstrapping off your own estimates. Converges beautifully in theory and diverges in practice.

LoRA on a 4-bit quantized base model. The trick that put fine-tuning on consumer GPUs.

The standard agreement metric in automated scoring, penalising larger disagreements more heavily. Reported at 0.75 to 0.86 for K-12 essays fine-tuned, and 0.023 to 0.327 for the same task zero-shot, which is a larger gap than the one between configured models and humans.

A condition surviving transmission as vocabulary while losing its function. A non-inferior twelve per cent reduction retains the technical word and reads as a reduction that is also good, rather than as a difference that was never established.

Predicting how reliable a translation is without a reference. Useful precisely where the reader cannot check the source themselves, which is the actual requirement in clinical and legal deployments.

Training with quantization simulated, so the model learns around it. Better accuracy than compressing after the fact.

The precision below which quality drops sharply, around 4-bit. A large model crushed past it is often worse than a smaller model at good precision in the same memory.

Storing model weights at lower precision to shrink and speed them up. GPTQ and AWQ are post-training methods; GGUF is the common file format for running quantized models locally.

A study design comparing groups that were not randomly assigned. Weaker than a randomised trial, common in education research, and mixing the two in a meta-analysis raises the sample size while lowering the average evidentiary quality.

Enriching a search query with related terms before retrieval. Old IR wisdom, newly relevant to RAG.

The three projections attention uses to decide what attends to what.

Having a model reformulate the user's query before retrieval — expanding, disambiguating, or splitting it. Often the cheapest RAG accuracy win.

Getting a machine to answer a question in natural language — extractive (find the span) or generative (compose the answer, ideally grounded via RAG).

Carrying the qualifying word attached to a figure rather than the figure alone. Not "99.98% success rate" but "99.98% success reproducing training clips": four extra words, and every downstream hop inherits the scope.

Fetching relevant documents at query time and letting the model answer from them.

Measuring retrieval and generation separately, then together. The discipline that finds where your pipeline actually fails.

Hundreds of deliberately mediocre trees, averaged. The strongest default in machine learning and almost impossible to misuse.

Breiman (2001), named for the film: the set of models that perform about equally well while disagreeing about which features matter. Its existence means a single explanation of a model is one story among many, and it undermines any claim that an interpretation is the interpretation.

Work cadence set by a system rather than negotiated by a person. Automation rarely requires it; what automation changes is that pace becomes precisely measurable, and measurable things become enforceable.

Think, act, look at what happened, think again. The loop underneath essentially every agent, and it's four lines of pseudocode.

Adoption outpacing the foundation it depends on. One survey of 1,270 IT leaders found 96% integrating AI into core processes, 85% claiming a clear data strategy, and around 80% constrained by limited data access, all from the same population.

Models that think before answering. A large real capability gain, and the visible thinking is not a reliable account of what happened.

A setting controlling how long a reasoning model thinks before answering. The largest cost component in such models, and usually the cheapest lever available.

A model trained to think at length before answering. The o1/R1 generation, and the test-time-compute era's face.

The intermediate thinking a reasoning model produces before its answer. Useful for debugging; not a faithful window into the computation — models can reason one way and explain another.

Making something cheaper causes more of it to be used. Why per-query efficiency gains in AI keep being absorbed by growth rather than reducing total energy demand.

Of the things you should have found, how many you did. The ceiling on any RAG system.

Whether the right answer appears in the top k retrieved. Retrieval's first-order health metric.

The patch of the input a single neuron can see. Deeper layers see wider patches, which is how local filters end up recognising whole objects.

Where purchases far exceed the expense recognised against them, because cost spreads over years. Four hyperscalers bought $433.9 billion of property and equipment in four quarters against about $149 billion of depreciation, and the remainder arrives later regardless of revenue.

The AI that decides what you see next. Probably the most economically significant machine learning on earth, and the least discussed.

The share of customers who return within a period about the same issue. The best available correction to an inflated resolution number, since a customer who comes back was not resolved whatever the ticket status records, and most dashboards do not show it.

A generative method that learns to move samples along straight paths between noise and data, which makes generation fast. The technique behind several recent image and video models.

An AI improving its own ability to improve, compounding. The mechanism classic intelligence-explosion arguments rest on — and a live research question, not a fact.

Deliberately attacking your own model to find failures before users or adversaries do. Standard practice for safety-critical deployments.

A frozen sample of inputs with known-good outputs, re-run on a schedule. The highest-value monitoring you can build, because it turns an invisible problem into a line on a chart. Refreshing it destroys the comparison.

A business resting on traffic another party chooses to send. Publishers under 10,000 daily page views saw 60% declines over two years, because a small site was almost entirely search-dependent while a large brand retains direct, app and newsletter traffic.

Asking a model to critique and fix its own output. Works when there's external feedback, and mostly doesn't when there isn't.

A model trained on ten to twenty related languages, usually by groups close to those languages. Frequently beats much larger general models on its targets, at a fraction of the size.

Predicting a number rather than a category. The oldest tool in the box and still the right answer more often than anyone admits.

Checking that a new model doesn't break what the old one did well. The eval suite as safety net. ::

Anything that discourages memorisation — dropout, weight decay, early stopping.

Learning by trial and error through rewards — the way you'd train a pet, applied to software.

Watching the ratio between what a system emits and what its environment contains, rather than whether the system performs as validated. Sepsis alerts rising 43% while hospital census fell 35% was invisible to absolute monitoring and obvious to relational monitoring.

max(0, x): the activation that made deep networks train. Simplicity that beat decades of cleverness.

Guidance requested by a system that retains control, distinct from teleoperation where a person drives. Its frequency per distance is the number that would separate autonomy from highly capable supervision, and no deployed programme publishes it.

A small per-attempt rate becoming a near-certainty across volume, in whichever direction the requirement runs. Reliability needs every attempt to succeed and decays; security needs every attempt to fail and decays identically. Both are conventionally reported at one attempt.

Reducing the probability of tokens the model has already produced, to stop loops. A blunt instrument — too high and the model avoids legitimately repeated words.

Stored past experience sampled during training, breaking the correlation between consecutive observations. Standard in reinforcement learning, and used in continual learning to counter catastrophic forgetting.

Reaching the same conclusion from a new experiment of the same design, which is stronger and rarer than reproducing a result from the original artifacts. Most reform effort targets reproducibility because it is checkable; replicability is what science actually wants.

The count of publicly known cases, which is a minimum rather than a total. A wrongful arrest that ends without charges, or where nobody names the technology in the file, enters no register at all.

Letting a model learn its own useful features from raw data instead of having humans hand-craft them. The core shift that made deep learning work.

Demographic balance in an output achieved without the underlying knowledge. One text-to-image model matched census skin tone distribution at 38.1% while depicting the requested skin condition in 0.94% of its images, so a pipeline tuned for balance can pass an audit while producing images no clinician would recognise.

Getting the same result from the same code and data. ML's quiet crisis, between nondeterminism and unshared details. ::

Retrieving a wide net, then reordering with a sharper model and keeping the best few. The best value in a RAG pipeline.

A shortcut letting signal skip a layer. The reason very deep networks train at all.

Add a shortcut around every couple of layers and suddenly a hundred-layer network trains. One line of arithmetic that unlocked depth.

The share of tickets where the customer's problem was actually solved end to end. The only one of the three common support metrics that measures an outcome rather than a workload, and the one vendors report least often.

Robinson (1965): prove a statement by assuming its negation and deriving a contradiction, mechanically. The inference engine behind automated theorem proving and Prolog, and the reason logic programming was possible at all.

Where a language sits in the distribution of available digital text. Tracks wealth and publishing history rather than speaker numbers, which is why Bengali is worse served than Icelandic.

The gap between learning and testing, and whether the aid is available at the test. Nearly all AI education research measures performance during the session, which the platform instruments automatically, rather than weeks later without the tool.

Finding relevant items from a collection. The R in RAG and the half of it that usually fails.

Withdrawing a certificate invalidates what was already signed with it, not only future output. One camera line's signing vulnerability voided every credential those cameras had produced, and a broken signature reads worse than no signature at all.

How many attempts a workflow makes before giving up, set by reliability engineers to hit a success target. It is also a security parameter: every retry that raises the chance a task eventually succeeds raises the chance an injection eventually lands, and no published analysis treats them as one decision.

What undoing a technical choice would cost in three years, as distinct from what making it costs today. Rarely estimated, usually obtainable in an afternoon by the team that built the system, and required by no process.

Requiring the person affected to disprove an institution's automated claim, usually using evidence the institution holds. It multiplies every other flaw in a system, because most people cannot discharge it and pay instead.

Who pays to evaluate a contribution. Stable while producing and checking cost roughly the same; unstable once one submission takes seconds, one review takes an hour, and the reviewer is an unpaid volunteer.

Time a generative tool saves returning as time spent checking its output. Documented in clinical documentation, software and teaching, where 62% of surveyed teachers reported savings partially offset by review, and absent from every headline savings figure.

The number that tells an agent what you want. The hardest thing to write correctly in all of AI.

Optimising the measure instead of the goal. Not a bug — what optimisation does.

A learned stand-in for human judgement. RLHF's weak link by construction.

A machine-readable signal that content is not available for commercial AI training. The mechanism EU-style opt-out regimes depend on, and useless against anyone who ignores it.

Reinforcement learning from preferences labelled by an AI model instead of humans. The scalable cousin of RLHF, and the mechanism behind constitutional AI.

Training on human preferences rather than correct answers. The step that turned text predictors into assistants.

Training against answers a machine can check instead of preferences it has to learn. Behind the reasoning-model era, and confined to what's verifiable.

Layer normalization minus the mean-centering. Slightly cheaper, now standard in open models.

The sequential architecture transformers replaced. Processed text one step at a time, which is why it was slow.

A curve showing every threshold at once, summarised into one number. The most-reported classification metric, and it has a coherence problem almost nobody knows about.

A language written in Latin script rather than its own. Falls between the two distributions a tokenizer actually learned, and is often handled worse than either the original script or English.

A diagram plotting achievable performance against arithmetic intensity, showing whether a workload is limited by compute or by memory bandwidth. Explains at a glance why inference and training have different bottlenecks.

Encoding position by rotating query and key vectors. The positional scheme modern LLMs settled on.

Recall-oriented n-gram overlap for summaries. Weak signal, long service. ::

The small network that picks which experts handle each token. Learned during training, and the reason only a slice of a huge model wakes up per word.

Combining ranked lists using only each item's rank, not its score. Sidesteps the problem that BM25 scores and cosine similarities aren't on the same scale.

Annualising a recent month or quarter rather than counting a full period. Always larger than booked revenue in a growing company, and the faster the growth the larger the gap: one documented case shows $13.1 billion booked against a $21.4 billion year-end run rate, 63% apart and both accurate.

An RNN architecture designed to train in parallel like a Transformer while running with an RNN's constant memory at inference. One of the leading attention-free language models. Peng et al., 2023.

See sparse autoencoder.

Protective features disabled one at a time for individually defensible reasons until nothing remains. Emergency braking, driver alertness detection and sign detection were each switched off for ride smoothness, and together they were every layer that could have caught a classification failure.

A model format that stores tensors and cannot execute code. Prefer it — pickle runs arbitrary code on load, by design.

A structured argument, with evidence, for why a system is safe enough to deploy — borrowed from aviation and nuclear practice, now applied to frontier AI.

A visualisation of which input regions most affect an output, usually via gradients. Intuitive and unreliable: several published saliency methods produce similar-looking maps for a trained network and a randomly initialised one.

How much data a method needs to learn. Where humans still embarrass machines. ::

Choosing the next token from a probability distribution rather than always taking the likeliest.

See temperature: the dial between deterministic and diverse output.

The gap between what an organisation approved and what its people use. Pilot studies measure the first, so a company where every project failed and every employee uses AI daily scores as a failure, and the productivity exists while the accounting does not.

Running an agent where it can't do damage. The only agent safety measure that doesn't depend on the model behaving.

A benchmark score being a property of an assembled system rather than of a model. Tool access, retry budget, evaluator version and orchestration architecture all move the number materially, and one browser benchmark rose from 14.41% to 61.7% mainly through planner-executor-memory design.

The prompts, guardrails and evaluations built around a model, which are simultaneously where the performance lives and where the lock-in lives. Nobody builds lock-in; they build quality, and lock-in is what quality looks like from outside.

Supervising models on tasks too hard for humans to check directly. The problem of controlling systems that outperform their evaluators.

Automation rarely creates a new failure; it raises the volume of an existing one past the point where the surrounding process can absorb it. A manual version of the same flawed rule produces disputes; the automated version produces a scandal.

The bet that capability keeps coming from scale. The most expensive scientific wager in history, still paying.

The observed relationship between compute, data, model size, and performance. Corrected once already.

The judgement about whether a system falls under a regulation. Where the regulated party makes it, absence of compliance evidence becomes uninterpretable, which is why a registry of tools in use would change what the data means and why no jurisdiction has adopted one.

Learning the gradient of a probability density rather than the density itself, which avoids computing an intractable normalising constant. The theoretical foundation under diffusion models.

Systematically exploring possibilities to find a good one. AI's oldest technique, its most complete success, and nobody calls it AI anymore.

How many configurations were tried before the reported one. Reporting it does more for a reader's calibration than an error bar, because it says how much of the result is selection.

Several parties jointly compute a function over their private inputs while learning nothing except the result. Lets organisations train on combined data without any of them disclosing their share, at substantial communication cost.

Run-to-run variation from random initialisation, data order and augmentation. Around 0.03% on standard image benchmarks and far larger for derived metrics, shifted distributions and short training. Roughly 80% of total run-to-run variance comes from sources seeding does not control.

Meta's 2023 model that segments any object in any image from a click or box prompt. Turned segmentation from a per-task model into a promptable tool.

Bias from who chooses to participate. A follow-up coding study found a speedup and published that it was unreliable, because developers reluctant to work without AI opted out and the tasks people most wanted AI for went unsubmitted.

Acting on a prediction selects for the errors that favour action. Buying when a model says underpriced means buying disproportionately where it overestimated. A model unbiased across all cases is not unbiased across the subset it causes you to act on.

The fraction of a group that passes a screening step. Two groups at 3% and 3.5% and two at 30% and 35% produce the same impact ratio and are very different situations, which is why the underlying rates matter as much as the ratio.

The mechanism behind Mamba: letting a state-space model's parameters vary with the input, which is what gave SSMs the content-based reasoning they previously lacked.

Attention within one sequence — every token weighing every other.

Where the party subject to a rule decides whether it applies. The common cause behind incomplete inventories in hiring, banking and government, and the reason absence of a record cannot be read as absence of a system.

Sampling several reasoning paths and taking the majority answer. Ensembling a single model's thoughts.

A model reviewing and revising its own output. Works when there's a checkable signal; without one, models mostly repeat their mistakes with more confidence.

Training against copies of yourself. How AlphaGo surpassed its teachers.

A model rating text more favourably when it resembles its own output. Demonstrated quantitatively as higher scores for lower-perplexity text, and separately as automated essay scoring rating model-generated writing above human writing.

Injury or performance figures produced by the party being assessed. Not necessarily wrong, and uninterpretable on a contested question without independent measurement of comparable scope, which frequently does not exist.

Asking the operator and measuring the artefact giving different answers, systematically favouring the report. Developers estimated 20% faster and measured 19% slower; 97% of executives report benefit against 29% reporting organisational return; 59% report better code quality while refactoring fell to 3.8%.

Inventing labels from the data's own structure. How foundation models are actually trained.

Serving a stored answer when a new query is close enough in meaning to a past one. Cuts cost and latency, at the risk of returning stale or near-miss results.

Serving cached answers to questions that mean the same thing. Embedding similarity as a cost lever.

Splitting documents at meaning boundaries — topic shifts — instead of fixed sizes. Costs more at index time, retrieves cleaner units.

The distance between raw pixel values and the high-level concept they depict. The core reason computer vision stayed hard for so long.

An agent's store of general facts distilled from experience — what's true, minus when it was learned. The complement of episodic memory.

Concepts as nodes, relationships as edges — 'a canary is a bird, a bird can fly'. The 1960s ancestor of today's knowledge graphs.

Searching by meaning rather than exact words. Similar is not the same as relevant.

Labelling every pixel in an image with its class (road, car, sky). The finest-grained vision task, and critical for self-driving.

Training on a small labelled set plus a large unlabelled one. The pragmatic middle most real projects live in, and the logic behind pretrain-then-fine-tune.

Withholding an inventory entry on operational security grounds. Frequently legitimate, since publishing a fraud detection system's details tells people how to evade it, and the same exclusion removes any external check on whether the system works.

One vector for a whole sentence or passage. What semantic search actually indexes.

Deciding whether text is positive or negative. The most deployed NLP task, and the one whose target may not exist.

The argument that an operator is not answerable for its own chatbot's statements. Tested once, at the British Columbia Civil Resolution Tribunal in 2024, and rejected on the ground that a company is responsible for all information on its website however it is produced.

Reconstructing an exact execution sequence from a recorded trace. Necessary because these systems are non-deterministic, so running it again is not a debugging strategy.

Stochastic gradient descent. Computing the gradient on a small random batch — noisy, cheap, and the noise helps.

AI tools used without IT or security review, approval or visibility. Found in 20% of studied organisations as a factor in a data breach, adding as much as $670,000 to average cost, with 97% of those affected having no AI access controls and 63% of organisations having no AI policy at all.

Running a new model on live traffic without showing its outputs to users, to compare against the current one safely.

A feature attribution method. Better grounded than most, and its independence assumptions break on correlated data.

Splitting model, optimizer, or data across devices. The generic word for making big things fit. ::

Lowe (1999): describes image keypoints in a way that survives scaling, rotation and lighting change, so the same object can be matched across photographs. Dominated vision for a decade and is still used where training data is unavailable.

The S-curve squashing any number into (0,1). Probability's activation function.

The S-curve squashing any number into 0–1 — turns a score into a probability. Historically the neural-net activation, until its saturation caused vanishing gradients.

Two systems reading one statistic in opposite directions. An AI grader scores low-perplexity text higher while an AI detector flags it, so conventional writing earns a better mark and an accusation. Constructed from separate studies rather than observed in a deployment, and unstudied.

Degradation that raises no error. The model keeps returning confident, plausible output that is wrong slightly more often, which is why detection has to be built in advance rather than discovered.

The tendency of networks trained by gradient descent to learn simple, general patterns before memorising specifics. Why they generalise on real, structured data.

Preference optimization without a reference model. The DPO simplification wave, continued.

Training in simulation and transferring to the physical world — robotics' answer to expensive real-world data. The gap between the two is the field's central engineering problem.

The degradation when a policy trained in simulation meets physical dynamics, sensor noise and material properties the simulator approximated. The defining difficulty in learned robotics, and a distribution shift by another name.

Optimization that sometimes accepts worse moves, cooling that tolerance over time — borrowed from metallurgy — to escape local optima.

Two large causes sharing a start date and acting on the same population, which controls alone cannot separate. Generative AI adoption and the sharpest monetary tightening in four decades both began in 2022, which is why entry-level employment effects remain contested.

An account resting on one report rather than corroborating evidence. The canonical AI bias case is single-source: one news investigation, five anonymous people, no technical report, no published figures and no named institutions.

A horizon past which change becomes unpredictable. A physics metaphor resting on the intelligence-explosion argument, with a poor record of dates.

Factorising any matrix into a rotation, a scaling and another rotation. The basis of PCA, low-rank approximation and the LoRA family, and one of the few results in linear algebra that appears everywhere in machine learning.

Junior roles as the mechanism by which senior workers are produced. A sustained gap in entry-level hiring has consequences that arrive years later and are absent from every present-day employment series, which makes it the hardest effect to measure and the easiest to defer.

Any shortcut past layers. Residual learning's general form.

Simultaneous localization and mapping — a robot building a map of an unknown space while tracking its own position in it. The foundational capability of autonomous navigation.

A deliberately planted backdoor that survived supervised fine-tuning, RLHF and adversarial training. The removal tools failed.

Each token attending only to a local window. Linear cost, at the price of global sight.

See small language model.

Merriam-Webster's 2025 word of the year, defined as low-quality digital content produced in quantity by artificial intelligence. Useful as a label for volume rather than origin: the same tools that produced a bounty flood also surfaced more than 100 real bugs in the same project.

A model small enough to run on your own hardware. Usually chosen for cost, latency and privacy rather than accuracy — and it wins more often than expected.

Quantization that shifts difficulty from activations to weights before quantizing both, addressing the problem that activation outliers make naive quantization fail in large models.

A teacher's full probability distribution over answers (85% cat, 10% dog), far richer than a one-or-zero hard label. The signal that carries dark knowledge to a student.

Turning raw scores into probabilities that sum to one. Where temperature is applied.

State of the art. Claimed in every release, true on the chart they chose, for a few weeks. ::

Pulling one voice out of many — the cocktail party problem, named in 1953. Time-domain models beat spectrograms, because the spectrogram discarded the phase.

Infrastructure operated by an entity incorporated under local law, which addresses the jurisdiction question a regional endpoint leaves open. Costs model selection.

One operation inside a trace: a model call, tool invocation, retrieval or logic block, recording inputs, outputs, duration, tokens and cost. Nesting matters more than the individual span.

Running only a small, input-dependent slice of a model's parameters per token. The mechanism that lets a model be enormous in size and cheap per word.

Attending to a subset of positions by pattern or learned choice. The long-running project of making attention cheaper.

A tool that decomposes a model's activations into many sparse, more interpretable features. A leading method for reading what a network represents.

Working out who spoke when. The unglamorous half of transcription, and usually the half that's wrong.

A vector meant to capture voice characteristics rather than content, so segments from the same person land near each other. Recording conditions leak into it, so two people on one microphone can look more alike than one person across two.

Satisfying the literal objective while violating its intent. Reward hacking's broader family, and a core alignment worry.

Sound turned into a picture so a vision model can look at it. Nearly all audio AI runs on it, and it throws the phase away.

A small model guesses ahead, the big one checks in parallel. Two to three times faster with mathematically identical output — an actual free lunch.

Detecting how someone feels from their voice. Deployed at scale, and the psychology says the thing it measures may not exist.

Turning spoken audio into text. Solved for clear speech in quiet rooms, still genuinely hard for everything real.

Learned sparse retrieval: neural term weighting in a keyword-shaped index. The middle path between BM25 and dense vectors.

The open-weight image generator that put diffusion on consumer GPUs. The release that made image generation an ecosystem.

Naming, alongside any claimed benefit, the stage of the causal chain at which it was measured. Costs a clause, prevents a well-evidenced stage from being read as evidence about the whole chain, and is practised nowhere.

A benefit at one stage of a pipeline read as a benefit overall. AI-designed drugs clear Phase I at 80 to 90% against a historical 50 to 65%, where failure is molecular, and show no established advantage at Phase II, where failure is biological.

The standard deviation of an estimate rather than of the data. Venues now require papers to state which of the two an error bar represents, because the distinction changes the claim by a factor of the square root of the sample size.

An effect size measured in standard deviations rather than in test marks. Only interpretable against a baseline, a duration, a population and a cost, and quoted without those four it is a marketing figure that happens to be true.

An agent acting continuously on its own behalf with no user behind it, requiring its own scope and its own owner. An agent given the wrong one either loses attribution or silently carries whichever employee configured it.

Recurrence rebuilt with control theory. Constant memory, linear cost, and the most credible challenger the transformer has.

The probability of detecting an effect that is really there. Routinely unreported in model evaluations, which means a null result frequently cannot distinguish no effect from a sample too small to see one.

Whether an observed difference is unlikely under a null hypothesis. Fits machine learning awkwardly, since repeated training runs are not independent in the relevant sense and the question is generally about magnitude.

Crudely chopping words to a root form (running → run) by rule. Faster and rougher than lemmatization; a classic text-preprocessing step.

The critique that a language model arranges likely words without understanding. One half of an unresolved debate about how much these models truly grasp.

A string that halts generation. The humble control every API exposes. ::

Common, low-information words (the, is, at) often filtered out before text processing, so models focus on meaningful terms. A staple of classic NLP preprocessing.

Publishing performance by subgroup rather than as a single figure. Aggregate accuracy conceals exactly the gaps that matter, and in dermatology AI the studies that found disparities stratified by skin type while those reporting headline accuracy mostly did not.

Fikes & Nilsson (1971): describe actions by their preconditions and their effects, then search for a sequence reaching the goal. The formalism underneath every classical planner, and the ancestor of how agent frameworks decompose tasks today.

Where the measured signal genuinely overlaps between groups, so improving the classifier cannot separate them. Perplexity-based detection reads predictable text as machine-written, and second-language writing is predictable for unrelated reasons, which makes this a construct problem rather than a calibration one.

The specific failure where a familiar element occupies an unfamiliar syntactic position, such as a noun seen only as a subject appearing as an object. Distinct from topical novelty, and invisible to evaluation that varies content while holding structure fixed.

The digitised records a sector needs before AI is useful at all. Low-adoption industries frequently still work from paper, photographs and phone calls, so the barrier is an earlier wave of software rather than this one.

Making a model return JSON that always parses. Solved at the format layer, still wide open at the correctness layer.

Determining a protein's three-dimensional shape from its amino-acid sequence. The strongest result in AI for science, recognised with the 2024 Nobel Prize in Chemistry, with over 200 million structures released and around 40% of new database depositions in 2024 and 2025 involving AI-assisted techniques.

Repainting one image in another's style. Made neural networks feel like magic in 2015 and got quietly absorbed into everything.

An agent spawned by another agent for a bounded task — isolated context, narrower tools, results reported back. How large agent tasks are decomposed.

A cost penalty a state absorbs because it values the capability rather than the margin. Yields of 30 to 40% against 80% elsewhere are commercially disqualifying and strategically tolerable, so analysis assuming commercial logic mispredicts what gets built.

The standard a 510(k) submission must meet: that a new device is sufficiently like one already on the market. A reasonable proxy for physical instruments, and a weaker one for software that learns a decision boundary from data.

The test for whether output infringes a protected work. Applies to AI output exactly as to human work, and is entirely separate from whether the training was lawful.

Automation replacing one injury profile with another rather than removing harm. Warehouse robots eliminated ten to twenty miles of daily walking on concrete and left stationary repetitive motion at machine cadence, and a comparison measuring only the first shows improvement.

Removing the hardest step from a specification rather than solving it. A delivery drone that releases by parachute and never lands has deleted arrival, which is where nearly all the difficulty in autonomous delivery lives.

Splitting rare words into frequent pieces so a fixed vocabulary covers any input. The reason models never see individual letters, and cannot reliably count them.

The bar that decides a result before any measurement happens. Defining pilot success as measurable P&L impact within six months produces a 95% failure rate; the same study reports 83% pilot-to-implementation for general-purpose tools under a different bar.

Producing a shorter text that keeps what matters. Extractive picks sentences; abstractive writes new ones — and can hallucinate what wasn't there.

An intellect far beyond human across the board. Rests on a 1965 argument whose key premise is exactly what the compute-hungry last decade questions.

A network representing more concepts than it has neurons by packing them into overlapping directions. Why individual neurons rarely mean one thing.

Making a low-resolution image bigger and sharper by inventing the detail. Why "enhance" is a lie in every police procedural.

Training on curated input-output pairs. The SFT in every post-training pipeline.

Find the boundary with the widest possible gap. Ruled machine learning before deep learning and still wins when data is scarce.

Many simple agents producing collective behavior. Biology's pattern, occasionally software's.

Complex collective behaviour emerging from many simple agents following local rules — ant colonies, bird flocks. The classic field behind today's agent-swarm ideas.

Agents coordinating peer-to-peer with no central orchestrator. Scales past the supervisor bottleneck and gives up the audit trail and the single place where a decision can be traced.

A benchmark of real GitHub issues the model must fix in real repositories. The standard test of agentic coding, and increasingly the headline number in model releases.

The gated activation function used in most recent LLMs (LLaMA onward) — a GLU variant that beat plain ReLU/GELU in practice, with Shazeer's honest note that nobody fully knows why.

What leaving a provider actually costs. Estimated at 19 to 34% of implementation, with one documented case at $315,000 and three months for 40 workflows after a vendor collapse. Every published figure comes from a party selling protection against it.

A model agreeing with you because agreement was preferred by labellers. Learned, not accidental.

Agents converging on agreement because preference training rewards it, producing consensus that carries no evidential weight while arriving with apparent validation.

The problem that symbols manipulated without any causal contact with the world may carry no meaning. The core of the case that text-only models do not understand.

The idea that intelligence is symbol manipulation and you build it by writing down what you know. Ruled AI for thirty years and lost.

Work whose valuable product is a document, image or record rather than a physical thing. AI has been adopted almost entirely in domains with this property, which explains why healthcare adopts heavily around its documentation while its clinical work stays manual.

A monopolist constrained by the concentration of its own customers. A sole supplier whose top two buyers are around 38% of revenue, and who has only three technically capable customers, is not in the position the word monopoly usually implies.

How words combine into phrases and sentences. Probing finds it well represented in models nobody taught it to, which is why supplying parse trees stopped helping. Predicts failures on long-distance dependency and structural generalisation.

Training on data a model generated. Increasingly standard, genuinely useful, and carrying a failure mode with a Nature paper attached.

Fodor & Pylyshyn (1988): anyone who understands "John loves Mary" can understand "Mary loves John", because the parts recombine. Their argument was that neural networks lack this by construction. Whether large models have acquired it is unsettled and actively argued.

Documentation of a deployed system rather than a model: the model plus its guardrails, oversight, user disclosure and behaviour in context. A model can be documented perfectly and deployed inside a system that changes everything.

Standing instructions given before the conversation. Not a secret, and not a security boundary.

A grid of multiply-accumulate cells where data flows between neighbours. Weights load once and stay, so intermediates never touch memory.

What practitioners know but cannot articulate, learned by doing. Systematically absent from training data for the same reason it is hard to teach, which is why senior judgment resists automation.

The activation squashing to (-1,1). Sigmoid's zero-centered sibling from the RNN era.

Establishing that acting on a biological target changes the disease. The harder and more valuable half of drug discovery, unaffected by molecular design quality, and the stage where AI-derived candidates have not demonstrated an advantage.

Breaking a big job into small ones. Reliably helps, and reliably multiplies your failure rate.

Work an AI could plausibly touch. The broadest and most alarming measure, and the one behind most scary headlines; it implies nothing on its own about whether a job changes.

The proportion of routine to complex work, which determines measured AI benefit more than the tool does. One study of 4,500 developers found 46% time savings on routine tasks and under 10% on complex ones, which explains most of the spread in this literature.

Moving work to an environment that can be engineered rather than engineering the one you are in. Not a separate form of task redefinition but environment engineering by another route, and in construction it appears in the statistics as prefabrication growth rather than as robotics adoption.

Whether a task is continuous, partially completable and forgiving, or discrete, all-or-nothing and unforgiving. Floors, lawns and pools are the first kind and have consumer robots; folding laundry and loading dishwashers are the second and do not. Shape predicts feasibility better than apparent difficulty.

Training a generator on ground-truth prefixes rather than its own outputs. Fast to train; the gap it leaves is exposure bias. ::

The randomness dial. Low is predictable, high is varied, and it isn't a creativity setting.

Post-hoc calibration by dividing logits by a learned constant. One parameter that fixes most miscalibration.

Sutton (1988): update a value estimate using the next estimate rather than waiting for the final outcome. Learning from a guess by way of a better guess, and the principle Q-learning and every modern value method rest on.

Degradation in a model's performance as the world moves away from its training distribution. In one fraud benchmark a gradient-boosting model lost 0.0017 of AUC where a deep model lost 0.0626, about thirty-seven times more, which is a large part of why banks keep using simpler architectures.

A multi-dimensional array. The data structure everything in deep learning is made of. ::

A hardware unit that performs a small matrix multiplication as one operation instead of assembling it from many. Specialisation that followed once graphics chips turned out to fit AI.

Google's production-first framework. Lost research to PyTorch; lives on in deployment. ::

Splitting individual weight matrices across devices. How single layers outgrow single chips.

Data you look at once, at the end. Check it repeatedly and you no longer have one.

Spending compute when the model answers rather than when it trains. The second scaling axis, and it runs out where verification does.

Automated analysis of large bodies of content. Several jurisdictions have statutory exceptions permitting it for AI training, which is why training legality varies so sharply by country.

Sorting text into categories — spam/not, sentiment, topic. The most widely deployed NLP task; a logistic-regression baseline is often hard to beat.

Type a description, get a picture that didn't exist. The capability that made AI visible to everyone, with the most unresolved argument underneath it.

Turning text into speech that sounds human. The remaining gap isn't the voice, it's knowing which word to stress.

Generating video from a description. Not image generation with more frames — the hard part is that things must stay themselves.

Weighting words by frequency here versus rarity everywhere. Fifty years old and still the intuition under keyword search.

A measure of the working relationship in psychotherapy, developed and validated for relationships between people. Its application to a person and a chatbot produces a number, and whether that number means what it means in its original setting is contested in the literature that uses it.

A cap on how many reasoning tokens a model may spend before answering. The knob that trades answer quality against latency and cost.

Choosing an action by sampling from your current belief about which is best, rather than always picking the current favourite. A simple and strong solution to the exploration-exploitation trade-off in bandit problems.

Mandated and penalised figures such as securities filings and regulatory incident reports; voluntary structured ones such as sustainability reports and pre-registered studies; and unstructured ones such as press releases and analyst estimates. Which tier a number occupies explains most conflicts between sources.

Tokens or requests served per second. The metric batching exists to maximize. ::

Predicting what comes next in a sequence over time. Simple methods beat sophisticated ones for forty years and only recently stopped.

Machine learning on microcontrollers with kilobytes of memory and milliwatts of power. A distinct engineering discipline where quantization, pruning and architecture search are necessities rather than optimisations.

The chunk of text a model actually reads — usually part of a word. What you're billed for.

An agent presenting its own identity together with a claim recording the authority it acts under, so the delegation chain survives into downstream logs. The mechanism for an agent acting on a user's behalf.

The average number of tokens a tokenizer produces per word in a given language. High fertility means fragmentation, and it predicts downstream performance independently of everything else.

Cutting text into the pieces a model actually reads. Cause of a surprising share of its stupidest failures, including counting letters.

The thing that splits text into tokens. Explains why models miscount letters.

The extra tokens a non-English script consumes because the vocabulary's merges were learned from English-heavy text. Often three to four times, which raises cost, shrinks usable context and degrades quality at once.

The hard cap on input plus output length. The wall context engineering works around.

The extra tokens a non-English script consumes because the tokenizer's merges were learned from English-heavy text. Often three to four times, which multiplies cost and consumes context.

Generation speed as users feel it. The latency number that shapes perceived quality. ::

Useful output per unit of energy. A more honest efficiency measure than facility-level power ratios, which can look excellent while the compute inside wastes energy.

Sending tokens to the user as they're generated rather than waiting for the full answer. Why chat AI feels fast even when it isn't.

Accepting a worse result far more often, where frequency is worth more than quality. The fifth form of task redefinition and the narrowest: it requires a task that is continuous, partially completable and forgiving. A robot vacuum cleans daily and adequately rather than weekly and well, which works for floors and not for folding.

A model invoking external functions with structured arguments. The mechanism that turns chat into action.

Letting a model call real software instead of answering from memory.

Measuring whether an agent selects, calls and interprets external tools correctly, as distinct from whether its reasoning is sound. Frequently the actual failure point, and less measured than reasoning quality.

Discovering the themes a document collection contains, unsupervised — each document as a mixture of topics, each topic as a cluster of words.

Sampling only from the k likeliest tokens.

Sampling only from the k most likely tokens. The blunt ancestor of nucleus sampling.

Ordering a directed graph so nothing appears before what it depends on. What turns a prerequisite relation into a learning sequence.

Google's chip built only for neural networks. A systolic array, and the case that specialisation beats general-purpose hardware.

The complete tree of spans for one request, preserving which step triggered which. The call stack for an AI system, and the artefact that makes a failure attributable to a component rather than to the system.

How faithfully a controller reproduces reference motions it was fitted on, in simulation. A reconstruction measure rather than a capability measure, and the headline figure attached to motion tokenization work is one of these.

Building the model. Expensive, memory-hungry, and something most teams should never do.

The examples a model learns from. Where almost all of its capability and almost all of its failures come from.

A model revealing verbatim or near-verbatim training data in its outputs. The mechanism behind privacy exposures and copyright disputes alike.

Provision of AI training to teachers by district poverty level. 67% of low-poverty districts against 39% of high-poverty by autumn 2024, with about 61% of primary teachers in mostly nonwhite schools having received none, against 35% in mostly white schools.

One end-to-end training of a model. At frontier scale, a months-long, eight-figure bet. ::

Features computed differently in training than production. The classic silent killer of deployed models.

A sequence of states, actions, and rewards. The unit of experience RL learns from.

Starting from a model that already learned something general. Why small teams can ship real AI.

The legal instrument permitting data to move between jurisdictions, such as an adequacy decision or standard contractual clauses. Assumes movement and sets terms, where localisation assumes none.

The architecture behind almost every modern model, built around attention.

Everything reachable by following a relation repeatedly. Computed rather than authored, which is what keeps a prerequisite graph readable: record only direct dependencies and derive the rest.

Prompting a model to explore multiple reasoning branches and evaluate them, rather than committing to one chain. A search over reasoning paths.

OpenAI's language for writing GPU kernels in Python. How FlashAttention-class speed became accessible. ::

Cutting input that exceeds the context window. Silent data loss with a polite name.

The line between instructions and data that ordinary security depends on. A language model has none, which is why prompt injection cannot be patched and controls must live outside the model.

Limiting how far each policy update may move. The stability idea PPO approximates.

Where a chain of attestation terminates, and therefore a single point of failure for everything that depends on it. Every image from a device shares one signing key, so a compromise there moves all of them together rather than affecting them independently.

Projecting high-dimensional data to 2D for visualization. Beautiful maps; distances between clusters mean little. ::

Methods for visualising high-dimensional data. Cluster sizes and distances on the plot are not meaningful.

Turing (1936): the abstract model defining what can be computed. Together with the Church-Turing thesis, it fixes the outer boundary of what any algorithm, including any AI system, can do. Everything about capability sits inside this limit.

The 1950 proposal that a machine counts as thinking if it passes for human. A test of deception — Turing said so plainly and everyone forgot.

Bloom's finding that one-to-one tutoring moved average students roughly two standard deviations above classroom instruction, and the forty-year search for a scalable equivalent. Established tutoring systems reach 0.18 to 0.29 SD at scale; generative tutors reach 0.73 to 1.3 SD in controlled conditions.

The faster, structure-preserving successor to t-SNE for visualization. Same caveat: read clusters, not distances. ::

Where the value of a system is that nobody was present rather than that the result was good. It is why comparing a robot vacuum to a person with an upright vacuum is the wrong comparison, and why a category grew 20% in a year on a device that misses corners.

A number in circulation that does not originate with the party it describes. Claims of 50,000 deployed units of a robot whose maker has never published a production count sit alongside that maker's own statement that the robots are not in material use. Unattributed is not the same as false, and the two need different responses.

A failure that produces no artefact naming it. Sepsis missed by both a clinician and an alert is recorded as sepsis, so a warning system that quietly does not warn is invisible to every incident register, which counts events somebody noticed.

The model is too simple to capture the pattern. Bad on training data and test data alike.

The encoder-decoder with skip connections from medical imaging. Diffusion's backbone before transformers took over.

What counts as one billable event, which is half of any metered contract and usually sits in documentation rather than in the agreement. Negotiating a rate without negotiating the unit settles the visible half of a deal.

The size of the thing being scored. Machine translation reached human parity when raters judged sentences and lost it when they judged documents, because inconsistent terminology and lost referents are invisible at sentence level. Whatever is measured in pieces looks better than the whole.

A regulator assessing one product, for one use, by one sponsor. Every FDA AI mechanism operates at that level, and the recurring evidence failures in clinical AI sit above it: synthesis across settings, transmission after publication, aggregation across a category, and choice of benchmark or comparator.

The result that a big enough neural network can approximate any continuous function. Often cited, often oversold — it says networks can represent, not that training will find.

Removing a specific record's influence from a trained model. No reliable general method exists.

An answer given confidently where the honest response was that the facts are unestablished. Worse than fabrication in a news context, since a fabricated fact can be checked and an unmarked uncertainty gives the reader no signal that checking is needed.

A variable that plausibly changes a result, is known to be unevenly sampled and is cheap to record, and was not recorded. Of 232 studies pooled into a 90% accuracy figure for AI skin cancer detection, 1.3% described skin type, which makes the aggregate uninterpretable rather than imprecise.

A control that had to be held out beforehand and cannot be reconstructed after. No uncontaminated benchmark evaluation exists retrospectively, because the test sets entered the training corpora before anyone preserved one, and the cost of not holding something out is invisible until it is needed.

A library making LoRA fine-tuning dramatically faster on single GPUs. The hobbyist fine-tuning stack's favorite.

Finding structure in data nobody labelled. Hard to know whether you got it right.

Increasing image resolution with a generative model. Super-resolution as a product feature.

A published register of the AI systems an organisation runs, mandatory for US federal agencies. One agency listed 126 active cases with 65 not publicly detailed, and auditors then found tools missing from it entirely. Read inventories as samples rather than censuses.

The acceptable error rate of a model is a property of what it is used for, not of the model. A valuation carrying a few percent of error is a good consumer estimate and a ruinous basis for a purchase price, and moving between the two changes the requirement by orders of magnitude without changing the system.

A wrong figure that produces action a correct one would not have. The bottle-of-water claim is off by a factor of tens from its own source and generated regulatory attention that 0.26 ml would not have. Whether that trade is acceptable is a real question, and this corpus answers no.

How long an asset is assumed to remain productive, which determines how its cost spreads across periods. An estimate rather than a measurement: in early 2025 one hyperscaler shortened server life from six years to five while another extended to 5.5, weeks apart, on substantially the same hardware.

The fraction of predictions that anyone has physically tested. A materials model flagged around 380,000 stable candidates and 736 have been synthesised, roughly 0.19%. The ratio of predictions to validations is the most useful statistic about any AI-for-science claim and is almost never reported.

The data you tune against, kept separate from the test set.

Expected future reward from a state. What the critic in actor-critic estimates.

The signal dying on its way back through a deep network. The problem that kept deep learning impossible for twenty years.

The training signal shrinking as it travels back, until early layers learn nothing.

An autoencoder whose latent space you can sample from. Lost to GANs on looks and won by being useful — it's inside your image model.

Approximating intractable distributions by optimizing a simpler family. The math VAEs are named after.

Vapnik & Chervonenkis: the largest set of points a model class can label in every possible way. Bounds the gap between training and true error, and is the classical explanation of why generalization is possible at all. It also fails to explain modern overparameterized networks, which is why double descent needed inventing.

A store built to find the nearest embeddings fast. Often reached for before it's needed.

The data structure (HNSW and relatives) that makes nearest-neighbour search fast at the cost of exactness. What a vector database is built around.

Finding the nearest vectors to a query, fast — by not actually finding them. Almost nobody measures the recall they're getting.

A supplier funding the customers who buy from it. Ordinary industrial practice where supply is constrained and buildouts are large, with precedents in aerospace, shipping and semiconductors, and the mechanism behind the late-1990s telecoms equipment episode. Both facts are true and neither settles the other.

Dependency deep enough that switching providers requires significant engineering effort or business disruption. In AI the dependency sits in prompts, guardrails, evaluations and orchestration tuned to one model, not in the API, which is why gateway abstraction addresses the cheapest layer.

Checking an answer is often far cheaper than producing one, where a cheap check exists. The engine behind synthetic data, reasoning training and hallucination mitigation. Note the opposite regime, refutation cost, where checking requires reproducing the work and runs the other way.

The delay between making a prediction and learning whether it was right. Sets a floor on how quickly concept drift can be detected, and where it runs to months no detection method helps.

A separate model or procedure that checks another model's output. Effective where verification is cheaper than generation, and it inherits the checking-cost problem where it is not.

A model trained only to judge whether an answer or step is correct. Pairing a generator with a verifier is the pattern behind much of test-time scaling.

Building software by prompting and accepting what runs. Named half in jest; adopted in earnest. ::

Recognising what's happening in video. Many benchmarks are solvable from a single frame, which tells you what the models actually learned.

Whether an ad rendered in view. A delivery confirmation rather than a quality judgement, which is why machine-generated low-value inventory scores 77.2% against 74.9% for clean supply and passes as premium more than 70% of the time.

Viola & Jones (2001): the first face detector fast enough to run in real time on ordinary hardware, using simple rectangular features and a cascade that rejects most regions immediately. It is why cameras started drawing boxes around faces.

Spreading risk where it is discussed rather than where it binds. Building fabs in several countries addresses tariffs, labour, energy and conflict exposure, and does nothing about a tool available from one supplier worldwide.

A model that reads images and text together. Behind every AI that can answer questions about a screenshot, and worse at counting than it looks.

Cut an image into patches, treat them as words, run a transformer. Works, and only if you have enough data.

Tying language to specific image regions — pointing at what "the red car" refers to. What separates describing an image from actually parsing it.

A transformer applied to image patches. Beats CNNs given enough data, and only then.

See vision-language model — a model that takes images and text in the same input.

How many distinct tokens a tokenizer knows. A quiet trade-off between sequence length and embedding cost.

Turning a spectrogram back into sound. The step that made synthetic speech stop sounding synthetic — and quality was never the bottleneck, speed was.

Copying a person's voice from a short sample. Technically impressive, ethically unresolved, and already being used against people.

Changing who a recording sounds like while keeping what was said. Useful, and the same technology as the fraud.

Shipping more units than any competitor while profit falls. The humanoid volume leader shipped roughly 5,500 units in 2025 and saw Q1 2026 net profit fall 52%, which is the pattern of buying market share ahead of an established product-market fit.

Memory on the graphics card. The binding constraint on running a model locally: weights plus overhead plus a context cache that grows with use all have to fit.

A comparison group receiving nothing, which captures expectancy, attention and natural fluctuation alongside any real effect. Conventional for first trials, known to produce larger effect sizes than active controls, and establishes that something happened rather than that the intervention was responsible.

Listening for one phrase, always, on milliwatts. The privacy guarantee is architectural — the always-on chip cannot transmit.

Starting training with a tiny learning rate and ramping up. The ritual that prevents early-training blowups.

Hiding a detectable signal in AI output. Technically clever, deployed almost nowhere, and there's a proof it can't do what people want from it.

Litres of water consumed per kilowatt-hour of IT load, the standard data centre water metric. Reported fleet figures range from about 0.12 to 0.30 L/kWh across major operators, and the EU now requires it from facilities above 500 kW.

The 2016 model that closed the quality gap on synthetic speech and needed 16,000 sequential passes per second of audio. Unusable, and it proved the point.

The finding that a weaker supervisor can still elicit most of a stronger model's capability — studied as a stand-in for humans supervising superhuman models. An OpenAI research direction.

One learned number determining how strongly one unit influences another.

Penalising large weights to discourage memorisation.

The learned numbers that are the model. What "open weights" opens.

Sharing the input embedding and output projection matrices. A parameter saving most LLMs quietly use. ::

A graph isomorphism heuristic that bounds what standard message-passing networks can distinguish. They are at most as expressive as it, so certain non-isomorphic structures are indistinguishable to them, including differences that matter in chemistry.

OpenAI's open speech-recognition model. The release that commoditized transcription.

How often a model's output beats a rival's in head-to-head comparison. The standard headline of preference evaluation; sensitive to who — or what — judges.

The 2013 model that made embeddings famous, and the source of king − man + woman ≈ queen.

An agent's active context during a session, volatile by design and discarded when the session ends. Agents fail across sessions rather than within them because most systems never promote anything from it into durable storage.

A cryptographically attested identity issued to a running process rather than to a person. The correct model for an agent acting continuously on its own behalf, as opposed to on a user's.

Where the environment being worked in is the thing being produced, so it cannot be engineered around the machine. Construction is the clean case: a building site is what is being built. A structural barrier to automation rather than a maturity one.

A learned model of how an environment changes, good enough to plan inside. Works in reinforcement learning; whether LLMs have one is the field's live argument.

A result below what random guessing produces, which indicates structured error rather than noise. Human accuracy on high-quality synthetic video is 24.5% against a coin's 50%, because the errors point one way: toward believing what is seen is genuine.

An orchestrator applying reasoning from one agent's situation to another's. A failure mode with no single-agent equivalent, and the reason context isolation matters more than context sharing.

The gradient-boosting library that won a decade of Kaggle. Tabular data's reigning tool. ::

A method for stretching a model's RoPE positional encoding so it handles contexts far longer than it was trained on, with minimal fine-tuning (2023).

The detector family that predicts every box and class in one pass over the image — 'you only look once'. What made real-time object detection practical.

Sharding optimizer state, gradients, and parameters across devices. DeepSpeed's memory arithmetic for huge models. ::

A search that ends without a visit to any site. The most quoted statistic in this subject and among the least settled: reported at 60%, at 69%, and at 22.4% on strict clickstream data, with one provider behind both the highest and lowest figures, because they count different populations.

Doing a task with no examples, from instructions alone.

Asking a model to do a task with no examples — just the instruction. The baseline every fancier prompting method is measured against.