What is a large language model (LLM)? Complete guide

Large language models are the systems behind ChatGPT, Claude, Gemini, and the wave of AI tools that made machines suddenly seem able to talk, write, translate, reason, and code. They are the most visible form of modern artificial intelligence, and the interface through which most people first encountered it. Yet for all they can do, a large language model, or LLM, rests on one deceptively simple task: given some text, predict the token that comes next. Everything else, the fluent essays, the working code, the step-by-step reasoning, grows out of that single objective carried out at enormous scale and then shaped to be useful.

This guide is a complete, honest explanation of what an LLM is and how it works. It covers the simple objective at the model's core and why that simplicity is misleading, what makes an LLM "large" and why scale changes everything, how one is built from raw text through the stages that turn it into a helpful assistant, how it actually generates a response, what it can and cannot do, and the debates about how much it really understands. By the end, the systems that feel like magic should resolve into something you can reason about clearly: a very large next-token predictor, trained on much of the written world and then aligned to be helpful.

What a large language model is

A large language model is a very large neural network, built on the transformer architecture, that has been trained on a vast amount of text to predict the next piece of text. "Language model" is an old idea: any system that assigns probabilities to sequences of words, that can guess what comes next, is a language model. What is new is the "large" part. Modern LLMs contain billions of internal parameters, the adjustable numbers that store what they have learned, and are trained on datasets measured in trillions of words drawn from the web, books, and code. This scale is not a minor difference of degree; it is what separates today's LLMs from the smaller language models that came before and what gives them their surprising range of abilities.

The important thing to hold from the start is that an LLM is a deep learning system, a many-layered neural network, whose entire training reduces to one objective repeated over an immense amount of text. Understanding that objective is understanding the model.

The deceptively simple objective: predict the next token

At its core, an LLM is trained to do exactly one thing: given a stretch of text, predict what comes next. The text is broken into tokens, which are words or word fragments, and the model learns, over and over across its training data, to predict the next token given all the previous ones. That is the whole training objective. It sounds almost too simple to matter, and it is the single most misunderstood fact about these systems.

The reason it produces such capable models is that predicting the next token well, across the full variety of human writing, secretly requires learning almost everything about language and much about the world. To reliably continue a sentence about history, the model must absorb historical facts. To finish a line of code correctly, it must learn programming. To predict the resolution of an argument, it must pick up patterns of reasoning. To complete "the capital of France is," it must know geography. The simple objective is a demanding teacher precisely because doing it well is not simple: the model is forced, by the pressure of predicting text accurately, to build rich internal representations of grammar, facts, and relationships. This is why a system trained only to guess the next word can end up writing essays and debugging code. The task is trivial to state and enormous to master, and mastering it is what produces the capability. Everything an LLM does is next-token prediction; the depth comes not from the objective but from what the objective forces the model to learn.

Why "large" matters: scale and emergence

If the objective is old, the magic is in the scale, and this deserves emphasis because it is where LLMs departed from everything before them. Early language models were small and could manage local fluency but little more. As researchers built them larger, trained on more data with more compute, something notable happened: capabilities appeared that were not present in the smaller versions and were never explicitly programmed. These are often called emergent abilities. Among them are in-context learning, the ability to learn a new task from a few examples given in the prompt, without any change to the model's weights; chain-of-thought reasoning, the ability to work through problems in steps; and broad generalization to tasks the model was never specifically trained on.

This behaviour is governed by scaling laws, the finding that model performance improves predictably as size, data, and compute grow together, which is what justified spending tens of millions and eventually hundreds of millions of dollars to train ever-larger models. It is worth noting honestly that the nature of emergence is debated: some researchers argue these sudden jumps in capability are partly an artifact of how abilities are measured rather than truly discontinuous. But the practical reality is not in dispute. Scale turned a next-token predictor from a curiosity into a system with broad, general competence, and the pursuit of that scale has defined the field.

How an LLM is built: from raw text to helpful assistant

A usable LLM is not produced in one step. It is built in stages, and knowing them clears up most confusion about how these systems come to behave the way they do.

The first stage is tokenization, in which text is converted into the tokens the model works with, using an algorithm that breaks words into common fragments so that any text, including words never seen before, can be represented.

The second and heaviest stage is pretraining. Here the model reads an enormous corpus, web pages, books, code, and trains on the next-token objective across all of it, adjusting its billions of parameters to get better and better at prediction. This is where the model absorbs its broad knowledge of language and the world, and it is staggeringly expensive, with the largest models costing many millions of dollars in computation for this phase alone. The result is what is called a base model: a system that is very good at predicting and continuing text, but that is not yet an assistant. A base model will happily continue a prompt in any direction, including unhelpful or unsafe ones, because it has only learned to predict plausible text, not to be helpful.

The third stage, post-training, is what turns that raw predictor into the polite, instruction-following assistants people actually use, and it has two main parts. In supervised fine-tuning, also called instruction tuning, the model is fine-tuned on curated examples of instructions paired with good responses, teaching it to respond to requests rather than merely continue text. Then comes reinforcement learning from human feedback, or RLHF: humans rank different model responses, those rankings train a reward model that captures human preferences, and the LLM is adjusted to produce responses people rate as more helpful, harmless, and honest. This combination, pretraining followed by supervised fine-tuning and RLHF, is what transformed early raw models that were powerful but awkward into the fluent, cooperative assistants that broke into the mainstream. The knowledge comes from pretraining; the helpfulness comes from post-training.

How an LLM generates a response

When you send a prompt, the model does not compose a whole answer at once. It generates the response one token at a time, autoregressively: it predicts the next token, appends it, then predicts the next given everything so far, repeating until the answer is complete. At each step it produces a probability distribution over possible next tokens, and a sampling step chooses one, with settings like temperature controlling how much randomness is allowed, which is why the same prompt can yield different answers. The fuller mechanics of this, from prompt to output, are covered in the piece on how LLMs work end to end.

One practical limit shapes everything the model does: the context window, the maximum amount of text it can consider at once, including your prompt and its own response. Early models could hold only a few thousand tokens; current frontier models can hold hundreds of thousands to over a million, enough to read entire documents, though attention to that context is not perfectly uniform across its length. The context window is the model's working space, and it is finite, which is why techniques for managing what goes into it matter so much in practice.

The transformer underneath

None of this would work without the architecture that made training at scale feasible. Introduced in 2017, the transformer processes a whole sequence of tokens in parallel and uses a mechanism called self-attention, in which every token weighs its relationship to every other token, building context-aware representations of the text. Earlier sequence models processed text one step at a time, which made them slow to train and poor at capturing long-range connections. The transformer's parallelism and its ability to relate distant parts of a text are what allowed models to be scaled to billions of parameters and trained on trillions of tokens, which is the precondition for everything else in this guide. Every major LLM is a transformer.

What large language models can do

The range of an LLM follows from the generality of its training. Because predicting text well requires such broad competence, a single model can write and edit prose, summarise long documents, translate between languages, answer questions across countless domains, generate and explain code, hold a conversation, and increasingly reason through multi-step problems. The in-context learning ability means it can often take on a new task from nothing more than a description and a few examples in the prompt, without being retrained. This generality is why LLMs became a kind of universal interface to computing, a single system that can be pointed at an enormous variety of language tasks, and why they now sit inside search engines, coding tools, writing assistants, customer support, and much else.

Where large language models fall short

An honest account has to be as clear about the failures as the feats, and the limitations of LLMs follow directly from how they work. The most important is that an LLM has no built-in notion of truth. It generates what is plausible according to its training, not what is verified, which is why it hallucinates, producing fluent, confident statements that are simply false. The same next-token machinery that makes it articulate makes it able to be articulately wrong, and there is no internal alarm distinguishing the two. A related limit is knowledge: a model knows only what was in its training data, frozen at the point training ended, so it lacks awareness of recent events and specific private information, which is why systems often pair it with retrieval to supply current or proprietary facts. It also has no persistent memory of its own between conversations, only the context in front of it, and it can reproduce the biases present in its training data, raising the fairness concerns that attend any consequential use.

Beneath these practical limits sits a deeper, still-open question: how much does an LLM actually understand? One view holds that a system trained only to predict text is a kind of sophisticated pattern-matcher, arranging words in statistically likely orders without comprehension, sometimes called a stochastic parrot. Another view holds that to predict text as well as these models do, they must have learned real structure about the world, something that functions as a form of understanding even if it is unlike ours. The honest position acknowledges both: LLMs have demonstrably learned deep regularities that let them handle novel problems, which is more than shallow mimicry, yet they also lack grounding in the world, consistent reasoning, and any reliable sense of their own limits, which is less than understanding as we usually mean it. The debate is unresolved, and being clear that it is unresolved is more useful than confidently picking a side.

LLM versus chatbot

A common confusion is worth settling directly, because it clarifies the whole picture. An LLM is not the same as a chatbot. The LLM is the underlying model, the trained neural network that predicts text. A chatbot like ChatGPT is an application built around an LLM, adding a conversational interface, a system prompt that shapes behaviour, often tools and retrieval, and safety measures. The same LLM can power many different applications, and a chatbot is just the most familiar one. When people say they are "using ChatGPT," they are using a product built on an LLM, in the same way that using a website means using an application built on a database rather than the database itself.

The short version

A large language model is a very large transformer-based neural network trained on a vast amount of text to predict the next token, and that single objective, carried out at enormous scale, is the source of everything it does. Predicting the next token well forces the model to learn grammar, facts, reasoning, and world knowledge, so a simple task produces broad competence, and scaling the model up produces emergent abilities like in-context learning and chain-of-thought reasoning that smaller models lack. An LLM is built in stages: tokenization, then expensive pretraining that yields a raw base model good at predicting text, then post-training with supervised fine-tuning and reinforcement learning from human feedback that turns the raw predictor into a helpful assistant. It generates responses one token at a time within a finite context window. It can write, translate, reason, and code, but it hallucinates because it models plausibility rather than truth, knows only its frozen training data, and understands the world in a way that remains debated.

The idea to hold onto is that a large language model is a next-token predictor scaled up until prediction becomes competence and then aligned to be useful, which is why it can do so much from such a simple objective and also why it confidently invents falsehoods, since predicting plausible text is the same operation whether or not the result is true. Once you see the essay-writer and the code-generator as the same next-token predictor pointed at different prompts, the whole technology stops being magic and becomes something you can reason about.

Common questions

What is a large language model? A large language model, or LLM, is a very large neural network built on the transformer architecture and trained on a vast amount of text to predict the next token, meaning the next word or word fragment, given the preceding text. It contains billions of parameters and is trained on datasets of trillions of words. Through this training it absorbs broad knowledge of language and the world, which lets it write, translate, answer questions, reason, and generate code. LLMs are the technology behind assistants like ChatGPT, Claude, and Gemini, and they represent the most visible form of modern artificial intelligence.

How does a large language model work? An LLM works by predicting text one token at a time. During training, it repeatedly learns to predict the next token across an enormous body of text, which forces it to internalise grammar, facts, and reasoning patterns. When you give it a prompt, it generates a response autoregressively: predicting the next token, adding it, and predicting again, until the answer is complete, sampling each token from a probability distribution. It is built on the transformer architecture, which uses self-attention to relate every token to every other, and it operates within a finite context window that limits how much text it can consider at once.

How is a large language model trained? Training happens in stages. First, tokenization converts text into tokens. Then pretraining, the most expensive phase, trains the model on the next-token objective across a massive corpus of web text, books, and code, producing a base model that predicts text well but is not yet a helpful assistant. Finally, post-training aligns the model: supervised fine-tuning teaches it to follow instructions using curated examples, and reinforcement learning from human feedback uses human preference rankings to make its responses more helpful, harmless, and honest. This combination turns a raw next-token predictor into the polished assistants people actually use.

What is next-token prediction? Next-token prediction is the single objective an LLM is trained on: given a sequence of text, predict the next token that should follow. Text is split into tokens, and the model learns, across vast training data, to assign probabilities to what comes next and pick likely continuations. Although the task sounds trivial, doing it well across the full range of human writing requires learning grammar, facts, reasoning, and world knowledge, because accurately predicting the next word often depends on all of them. This is why a model trained only to predict text ends up able to write, translate, and reason: the simple objective secretly demands broad competence.

What is the difference between an LLM and a chatbot? An LLM is the underlying model, the trained neural network that predicts text, while a chatbot is an application built around an LLM. A chatbot like ChatGPT adds a conversational interface, a system prompt that shapes the model's behaviour, often tools and retrieval, and safety measures on top of the model. The same LLM can power many different applications beyond a chatbot, such as coding assistants or writing tools. So saying you are using ChatGPT means using a product built on an LLM, much as using a website means using an application built on a database rather than the database itself.

What can large language models do? Because predicting text well requires broad competence, a single LLM can perform a wide range of language tasks: writing and editing text, summarising documents, translating languages, answering questions across many domains, generating and explaining code, holding conversations, and increasingly reasoning through multi-step problems. A notable ability is in-context learning, where the model takes on a new task from just a description and a few examples in the prompt, without being retrained. This generality is why LLMs became a near-universal interface to computing and now sit inside search, coding tools, writing assistants, customer support, and many other applications.

What are the limitations of large language models? The most fundamental is that LLMs generate what is plausible, not what is true, so they hallucinate, producing fluent but false statements with no internal signal separating fact from fabrication. They know only their training data, frozen when training ended, so they miss recent events and private information unless paired with retrieval. They have no persistent memory between conversations beyond the context provided, and they can reproduce biases in their training data. There is also an unresolved debate about how much they truly understand, since they have learned deep patterns yet lack grounding in the world and reliable awareness of their own limits.