Context engineering: the skill that replaced prompt engineering

There's a phrase making the rounds in 2026 that sounds like it was invented to sell a course: context engineering. It wasn't. It names a real and specific shift in what actually determines whether an AI system works, a shift so pronounced that Gartner called 2026 "the year of context," a State of Context Management report found 82% of IT leaders saying prompting alone is no longer sufficient, and researcher Andrej Karpathy, who popularised the term, described it as "the delicate art and science of filling the context window with just the right information for the next step."

This piece unpacks that shift: what context engineering is, why the ground moved out from under prompt engineering, and the actual discipline hiding behind the buzzword. Because there is one, and it's more like information architecture than like writing clever instructions.

The one-line difference

Start with the distinction that makes everything else click: *prompt engineering is about what you say to the model; context engineering is about what you provide to it.*

Prompt engineering optimises the instruction, the phrasing, the role, the "think step by step." Context engineering optimises the entire information environment the model sees before it generates a single word: the retrieved documents, the memory of what happened earlier, the tool outputs, the examples, the data formats, the order it all arrives in, and how much of the context window each piece consumes. One is a writing task. The other is an engineering task, designing the flow of information into the model with the same rigour you'd apply to designing a database schema.

Karpathy's analogy is the sharpest way to hold it: the model is like a CPU, and its context window is like RAM. The context window is the model's working memory, everything it can reason about in a single response has to fit in that buffer first. Prompt engineering fiddles with the instruction loaded into RAM. Context engineering is the memory management: deciding what gets loaded, in what order, how much, and what gets evicted to make room. As tasks get bigger and agents run longer, that memory management becomes the thing that determines success, and no amount of clever phrasing fixes a context window loaded with the wrong information.

Why prompt engineering stopped being enough

The takeover happened for a concrete reason, and it's the same mechanism that thinned out prompt tricks generally: models got good enough that instruction-phrasing stopped being the bottleneck.

There's real data on this. A Stanford evaluation found that chain-of-thought prompting improved an older model's reasoning accuracy by around 12%, but the same technique improved a newer frontier model by under 2%. As models get smarter, the marginal return on optimising how you ask collapses, because the model already understands ambiguous requests without hand-holding. Meanwhile, the marginal return on supplying the right source material, scoped memory, and relevant examples went sharply up. The bottleneck didn't disappear, it moved. It moved from the instruction to the information around it.

The clearest way to see why is to picture an agent 47 steps into a complex task. What determines whether it succeeds at step 47 has almost nothing to do with the phrasing of the original prompt. It depends on what the agent still remembers, what tools it can reach, what it retrieved, and how much room is left in its context window. Prompt engineering has no answer for step 47, the original instruction is long gone from what matters. Context engineering is entirely about step 47. As the industry moved from one-shot chatbots to multi-step agents, the questions that mattered became questions of context management, and the discipline followed.

prompt engineering didn't die, it got absorbed. Writing a clear instruction is still necessary; it's just no longer sufficient. It became one component of a bigger job.

What context engineering actually involves

Behind the buzzword is a genuine set of engineering decisions. They're worth naming, because "manage the context" is vague until you see what it decomposes into.

Retrieval, what to pull in. The largest source of context is usually external: the right documents, records, or facts fetched at the moment they're needed. This is where RAG lives, and the key reframe of 2026 is that RAG is not a rival technique to context engineering; it's one tool within it. Retrieval is how you get the right source material into the window, and doing it well (good chunking, good embeddings, a reranker to surface the best passages) is a core context-engineering skill.

Memory, what to carry across steps. In any multi-turn or agentic system, the model needs the right slice of history: what was decided, what was tried, what the user prefers. Too little and the agent repeats itself or loses the thread; too much and it drowns. Designing agent memory, what persists, what gets summarised, what gets dropped, is context engineering's hardest sub-problem.

Tools and their outputs, what the model can reach and what comes back. When an agent calls a tool, the result lands in the context window, and how that result is formatted and scoped matters enormously. This is where MCP and tool use meet context engineering: a tool that dumps ten thousand tokens of raw output into the window is a context-engineering problem waiting to happen.

Structure and order, how it's all arranged. The same information helps or hurts depending on how it's laid out and where it sits. Which brings us to the two failure modes that define the craft.

The two things that go wrong

Context engineering exists because context fails in specific, predictable ways, and knowing them is most of the skill.

Too little context, and the model guesses. If the right information isn't in the window, the model does what it always does with a gap: it produces a fluent, confident answer from its training-data memory, which is exactly the recipe for a hallucination. A huge fraction of "the AI made something up" is really "the right context never reached the window." The fix isn't a better prompt; it's better retrieval.

Too much context, and the model drowns. The opposite failure is subtler and newer. Context windows in 2026 are enormous, and the naive instinct is to stuff everything in, but bigger isn't automatically better. More tokens mean more cost and more latency, and worse, there's a well-documented "lost in the middle" effect: models attend most reliably to information at the start and end of the context, and can miss things buried in the middle. Feed a model a jumbled mass of everything and ask for precision, and you'll get garbage, not because it can't reason, but because the signal was drowned in noise. This is why curation and ordering are real skills: putting the most important context where the model actually looks, and leaving out what doesn't earn its place in the window.

The discipline lives between these two failures: enough context to answer well, not so much that the answer gets lost, arranged so the model sees what matters. That's a genuine optimisation problem, and it's why it's called engineering.

Why this is winning the enterprise

There's a specific reason context engineering, not fine-tuning, is becoming the default way enterprises make models useful on their own data, and it's the same reason retrieval beats fine-tuning for knowledge generally: context is live; weights are frozen.

If your company's pricing changes tomorrow, a fine-tuned model keeps confidently repeating the old prices until you spend weeks and real money retraining it. A context pipeline just pulls the new price from the database the moment it changes, update on Tuesday morning, correct answers Tuesday afternoon, no retraining. For businesses whose data changes constantly, that flexibility is decisive. Context engineering lets you integrate proprietary data, business logic, and policy guardrails into the model's environment at runtime, turning unpredictable experiments into reliable workflows, without ever touching the model's weights.

There's an honest gap most enthusiasts skip, though, and it's worth stating plainly: context has to come from somewhere. All the talk of managing context windows and orchestrating tools quietly assumes the underlying information is clean, current, and governed. It often isn't. The upstream question, where enterprise context actually lives, who owns it, what happens when it's stale or incomplete, is the real frontier, and it's less glamorous than prompt tricks ever were. Context engineering doesn't work on a foundation of messy, ungoverned data; it just relocates the hard problem to where it always belonged, which is the data itself.

The same task, engineered badly and well

To make the discipline concrete, take one task: a support agent answering "why was I charged twice this month?" for a specific customer.

The badly-engineered version dumps everything into the window and hopes: the entire product documentation, the customer's complete three-year history, every billing record in the system, a long generic system prompt, and the question, a hundred thousand tokens of mostly-irrelevant material. The model has to find the one duplicate charge somewhere in that pile. It's slow, it's expensive, and thanks to lost-in-the-middle, it may well miss the relevant transaction because it's buried on line 8,000. The prompt was fine; the context was a mess.

The well-engineered version does the retrieval and curation for the model: it pulls only this customer's transactions from this month (memory scoped to the task), runs a quick check that surfaces the two matching charges to the top (ordering by relevance), includes the two-sentence refund policy that applies (retrieved, not the whole manual), and states the question. A few hundred tokens, all of them relevant, the key facts where the model looks. The model answers correctly, fast, and cheaply, not because it's smarter, but because it was given the right things in the right order.

Same model, same question, same underlying data. The only difference is context engineering, and it's the difference between a right answer and an expensive wrong one. Every principle above is visible in that contrast: retrieve the relevant subset, scope the memory, order by importance, leave out what doesn't earn its place.

The through-line

Step back and the arc of the last three years is clean. In 2023 the craft was prompt engineering, find the phrase that unlocks the model. By 2025 it was RAG architecture, get the right documents to the model. In 2026 it's context engineering, design the whole information ecosystem the model operates in, of which prompting and RAG are now components. Each stage didn't replace the last so much as contain it: a good instruction still matters, retrieval still matters, but they're now parts of a larger discipline concerned with everything the model sees.

prompt engineering asks how to phrase the request; context engineering asks what information should surround it, and on modern models, the second question is where the wins moved. The craft stopped being a writing exercise and became information architecture: what goes in the window, in what order, how much, from where, and what to leave out. Treat context design with the rigour you'd give a schema, and you're doing the thing that actually determines whether an AI system works in 2026. Keep polishing prompts in isolation, and you're optimising the one lever the models already outgrew.

The short version

Context engineering is the practice of deliberately assembling everything a model sees on a given call, the system prompt, retrieved documents, conversation history, tool results, and memory, rather than just wording a single prompt well. It has grown more important than classic prompt engineering because modern models follow instructions reliably but are highly sensitive to what information is in front of them and where. The context window is finite and attention degrades over length, so the work is choosing what to include, ordering it so key material sits where the model attends best, and trimming noise that would dilute or distract. Context engineering treats the context window as a scarce resource to be curated, not a bucket to be filled, because what a model does depends less on how you ask and more on what you show it.

Common questions

What is context engineering? Context engineering is the systematic practice of designing, curating, and managing all the information a language model sees in its context window before it generates a response, the retrieved documents, memory, tool outputs, examples, and their structure and order. Where prompt engineering optimises how you phrase a request, context engineering optimises what information surrounds it. It's an engineering discipline closer to information architecture than to writing.

Why is context engineering replacing prompt engineering? Because modern models got good enough that instruction-phrasing stopped being the bottleneck, clever prompts give diminishing returns on frontier models, while supplying the right information gives increasing returns. As systems moved from one-shot chatbots to multi-step agents, what determines success became what the model remembers, retrieves, and can reach, questions of context management, not phrasing. Prompt engineering didn't die; it got absorbed as one component.

What's the difference between prompt engineering and context engineering? Prompt engineering is about what you say to the model, the instruction, phrasing, and role. Context engineering is about what you provide, the documents, memory, tool outputs, data formats, and how much of the context window each consumes and in what order. A useful analogy: the model is a CPU and the context window is its RAM; prompt engineering tweaks the instruction, context engineering manages the memory.

Is RAG part of context engineering? Yes. Retrieval-augmented generation is one technique within the broader discipline of context engineering, it's how you get the right external source material into the context window at the moment it's needed. Context engineering also covers memory management, tool output handling, and the structure and ordering of everything in the window. RAG is a major tool in the kit, not a competitor to it.

What is the "lost in the middle" problem? It's a well-documented effect where language models attend most reliably to information at the beginning and end of their context window and can miss information buried in the middle. It's why stuffing a huge context full of everything can backfire: the important signal gets lost in the noise. Good context engineering curates and orders information so the most important content sits where the model actually looks.

Why do enterprises prefer context engineering over fine-tuning? Because context is live and weights are frozen. A fine-tuned model keeps repeating outdated information until it's expensively retrained, while a context pipeline pulls current data at runtime, update the database and the answers update immediately. This lets businesses integrate proprietary data, business logic, and guardrails into the model's environment without retraining, which is decisive for data that changes constantly. The catch is that the underlying data must be clean and well-governed.

What goes into an LLM's context besides the user's question? A model's context on any call is assembled from several sources: the system prompt setting its role and rules, the user's message, the recent conversation history, any documents or data retrieved for the task, tool definitions and results if it is an agent, and any stored memory about the user. Context engineering is the discipline of deciding what to include from each of these, in what order, and how to compress or trim it so the most relevant material lands where the model attends best. The window is finite and attention degrades with length, so what you leave out matters as much as what you put in.