Artifipedia
GlossaryMapStudyBlogMy Library
Home/Blog/Foundations

Foundations

Theory, history and the parts that will still be true in twenty years. 27 articles.

27 articles

July 24, 2026 Why neural networks generalize when theory says they can't A network with more parameters than training examples can memorize random labels perfectly. The same network, on real data, generalizes. Classical learning theory has no account of why,… July 23, 2026 What an AI confidence score actually means A model that says it is 90% sure can be right 60% of the time, and a perfectly calibrated one still tells you nothing… July 23, 2026 What actually caused the AI winters The book that supposedly killed neural networks proved a true theorem and attached a false conjecture. The field… July 23, 2026 I spent a month writing down what every AI concept depends on 256 concepts, 350 prerequisite links, one rule. I wrote down my predictions first so I could not quietly retrofit the… July 22, 2026 What AI cannot do, no matter how capable it gets Frontier models now rank just behind specialist verification tools at deciding whether programs terminate, which is… July 22, 2026 What linguistics predicts about where AI fails Linguistic features stopped improving models around 2019, because models learn the structure themselves. The categories… July 20, 2026 Neurosymbolic AI: what the symbolic half gets right On a structured manipulation task in early 2026, a neurosymbolic system scored 95% against 34% for a fine-tuned… July 18, 2026 Where the attention mechanism actually came from Attention was a fix for a specific engineering failure in 2014, three years before the paper that made it famous. It was… July 16, 2026 What is reinforcement learning? Learning from reward Reinforcement learning went from a niche corner of AI obsessed with games and robots to the paradigm that shapes how every… July 16, 2026 Who invented deep learning, and why it took so long Backpropagation was invented at least four times before it stuck. The ideas behind deep learning were mostly in place by… July 15, 2026 Regularization: L1, L2, dropout and what unites them L2 regularization is not analogous to a Gaussian prior. It is exactly a Gaussian prior, and the identity is provable. Six… July 15, 2026 Why AI aces the test and fails the variation Transformers scored 96 to 99% on a semantic parsing benchmark and 16 to 35% on the same task with the pieces recombined.… July 14, 2026 Why AI is bad at math, and which failures are permanent Change how a number is split into tokens and accuracy shifts substantially, which means the arithmetic was following the… July 9, 2026 Does it understand? The argument, properly stated Both sides of this debate are usually presented by their opponents. Here is the sceptical case at full strength, the case… June 24, 2026 How to tell if your AI actually works Most teams ship AI features on vibes, then argue about whether changes helped. An evaluation set is an afternoon of work… June 20, 2026 How neural networks work: the idea under all of it Under every transformer, every image generator, every language model, sits one idea: the neural network. Here's how it… June 17, 2026 AI scaling laws: why bigger wins, and whether it's ending One empirical discovery explains most of the last six years of AI and the hundreds of billions spent on it: model… June 15, 2026 The model was right. Acting on it would have killed people. A pneumonia model learned that asthma lowers your risk of dying. It was correct about the data and dangerously wrong as… June 14, 2026 What is generative AI? The complete guide Generative AI is the technology behind the chatbots, image makers, and video tools of the last few years, and it is… June 12, 2026 What is deep learning? The complete guide Almost every AI system that impresses today, from chatbots to image generators to voice assistants, runs on deep learning.… June 10, 2026 What are embeddings? How AI turns meaning into numbers When a search returns the right result without sharing a single keyword, when a chatbot pulls the relevant document, when… June 9, 2026 Why does deep learning work? The generalization mystery The most important technology of the decade rests on a foundation we do not fully understand. By the textbook, deep… June 8, 2026 What is computer vision? How machines learn to see You unlock your phone with your face, a car reads the road, a scan flags a tumor: all computer vision, the branch of AI… May 31, 2026 Supervised vs unsupervised learning: the four types Machine learning is usually taught as three types: supervised, unsupervised, and reinforcement learning. That map is still… May 22, 2026 AI vs machine learning vs deep learning: the difference These three terms are used interchangeably and are not interchangeable. They are nested: deep learning sits inside machine… May 20, 2026 What can AI do, and what can't it? A predictive map Any list of what AI can do is out of date before you finish reading it. What does not go out of date is the set of task… May 19, 2026 What is the Turing test? And why it stopped mattering Turing never proposed the imitation game as a definition of thinking. He proposed it to replace a question he considered…

Other subjects

Language & meaning 21 How language models handle language, and the places they do not. Agents 11 Building agents, deploying them, and why most attempts do not reach production. Evaluation & evidence 12 Measurement, benchmarks, and how to tell whether a claim is worth anything. Building with AI 18 Retrieval, cost, infrastructure and the practical business of shipping something. Safety & governance 11 Alignment, security, regulation and who is accountable when it goes wrong.