How AI agents actually work: the loop behind the hype

"Agent" is the most overused word in AI right now, and one of the least understood. Vendors slap it on chatbots. Demos call a single API call an "agent." And the genuine article, the thing that can be handed a goal and actually go accomplish it across a dozen steps without you holding its hand, gets lumped in with all of it. So here is what an AI agent actually is, mechanically, stripped of the marketing: not a smarter model, not a longer prompt, but a specific piece of architecture wrapped around a model. Once you see the architecture. You can tell the real thing from the label instantly.

The whole idea compresses to one line that the major AI labs independently converged on: an agent is a language model using tools in a loop, driven by feedback from the environment. That's it. The capability isn't in the model, it's in the loop, and the rest of this explains why.

The one distinction: a chatbot responds, an agent acts

Start with what an agent is not, because the contrast is the whole concept.

A regular language model call is stateless and one-shot. You send a prompt, it returns a response, and the interaction is over. It has no memory of what came before beyond what's in the current conversation. It can't do anything in the world beyond producing text, and, crucially, it takes exactly one turn. Ask it to "book me a flight to Tokyo" and it can describe how to book a flight, but it can't check prices, compare options, or actually reserve anything. It responds; it does not act.

An agent is different in one structural way: it runs in a loop. Given a goal, it doesn't produce one response and stop. It thinks about what to do, takes an action, observes the result, and then thinks again in light of that result, over and over, until the goal is met or it hits a limit. That loop is the entire difference. As Anthropic put it, agents are typically just models using tools based on environmental feedback in a loop; OpenAI's agent runtime is documented as a literal loop, call the model, and if it asks to use a tool, run the tool, feed the result back, and call the model again. The labs landed in the same place because there's only one place to land.

This is why the common test fails: telling a model to "act like a project manager" in its system prompt does not make it an agent. That's still one-shot text generation with a costume on. The agent property doesn't come from the prompt, it emerges from the infrastructure around the model that gives it tools, memory, and a loop of action and observation. No loop, no agent, no matter what the system prompt says.

The four components

The cleanest way to understand what's inside an agent is a formula that's become the field's shorthand, originally from researcher Lilian Weng: Agent = LLM + Memory + Planning + Tool Use. Four parts, each doing a distinct job, tied together by the loop. Take them one at a time.

The LLM is the reasoning core. At the centre sits a language model, but its role here isn't to answer, it's to decide. At each step of the loop, the model looks at the goal, the history so far, and the tools available, and decides what to do next. It's the brain making choices, not the mouth producing final text. The quality of an agent depends less on raw model power than on how well the loop and tools are built around it, a strong model in a bad loop is a worse agent than a decent model in a good one.

Tools are how it acts on the world. A model alone can only produce text. Tool use is what lets it do things, search the web, run code, call an API, read a file, query a database, send an email. Each tool comes with a description the model reads, so it knows what's available and when to reach for each. Tools are the hands; without them, the model can plan a flight booking neatly and accomplish nothing. This is the component that turns a conversationalist into an executor.

Memory is how it maintains context across steps. Because an agent runs many steps, it needs to remember what it's already done, otherwise every loop iteration starts from scratch. The simplest memory is just the running history inside the context window: every thought, action, and result appended as the conversation grows. More sophisticated agents add long-term memory, storing facts, preferences, and past results outside the context window and retrieving them when relevant, often via the same retrieval machinery as RAG. Memory is what lets an agent work on something for an hour, or learn your preferences over many sessions, rather than forgetting everything each turn.

Planning is how it breaks a big goal into steps. Handed a complex goal, a good agent doesn't dive in blindly, it decomposes the task. "Plan my Tokyo trip" becomes: find flights, find a hotel near the right area, check the weather, build a day-by-day itinerary. This is planning, and the important part is that good planning is dynamic: a static plan shatters the moment reality doesn't cooperate (no flights on Tuesday), while a dynamic planner observes the failure and adjusts (shift to Wednesday). For very complex goals, agents plan hierarchically, a high-level plan whose steps are themselves broken down further.

Not every agent has all four in full. Some have no long-term memory, some use a single tool, some barely plan. But the LLM-plus-loop is non-negotiable, that's the irreducible core. The other three are what make an agent capable rather than merely technically-an-agent.

The loop, watched in slow motion

The four components only come alive when the loop runs them, so here's the loop concretely, on a real task: "Find the most-cited 2026 paper on agent memory and summarise its findings." A one-shot model can't do this, it doesn't know 2026 citation counts. An agent can. Watch.

The dominant pattern is called ReAct, short for Reason + Act, where the model interleaves thinking with doing. Each iteration is think, act, observe:

Iteration 1. Think: "I need to find papers on agent memory from 2026, so I should search." Act: calls the search tool with relevant keywords. Observe: gets back 15 results with citation counts.

Iteration 2. Think: "The top result has 340 citations, that's the most cited. I need its full content to summarise it." Act: calls a document-retrieval tool on that paper. Observe: receives the full abstract and key sections.

Iteration 3. Think: "I now have enough to write the summary." Act: produces the final summary. Observe: the goal is met, so the loop stops.

Three iterations, three tool calls, one complete answer, and no human intervention between steps. That's the entire mechanism. The model never "knew" the answer; it figured it out by reasoning about what it needed, using tools to get it, and reasoning again about what it got. The loop turned a model that predicts text into a system that accomplishes a goal.

Note what each component contributed: the LLM decided what to do at each step, tools fetched what it couldn't know, memory (the accumulating history) let iteration 3 build on iterations 1 and 2, and lightweight planning sequenced search-then-retrieve-then-summarise. The formula, running.

The orchestrator: the unglamorous piece that actually matters

The part most explanations skip, and it's the one that separates people who talk about agents from people who build them. The model doesn't run itself. Something has to actually execute the loop, call the model, read its requested action, run the tool code, feed the result back into the context, and check whether to stop. That something is the orchestrator (or runtime, or control loop), and it's a regular program, not the model.

This matters because the orchestrator, not the model, is where the real engineering lives. It's what executes the tool call safely, injects the result back in the right format, and, critically, enforces the stopping conditions. Without those, an agent can loop forever, or rack up a fortune in tool calls, or spiral when a tool keeps failing. So the execution layer is also the guardrail layer: maximum iteration limits to prevent infinite loops, cost controls to stop after N tool calls, and human-in-the-loop checkpoints that pause and ask before doing something irreversible, writing to a database, sending money, deleting files. The maxim among builders in 2026 is that you're no longer just prompting the model; you're engineering the loop around it. The prompt is a small part; the loop, the tools, and the guardrails are the product.

To make the orchestrator's role concrete, consider the kind of agent many people now use daily: a coding agent asked to "fix the failing test in this project." The model can't see your files or run anything, the orchestrator gives it tools (read file, edit file, run tests) and runs the loop. Think: "I should run the tests to see what's failing." Act: orchestrator executes the test command and captures the output. Observe: the model reads a specific assertion error. Think: "The bug is in this function; I'll read it." Act: read the file. Think: "I'll change this line." Act: edit. Think: "Re-run to confirm." Act: run tests → they pass → stop. The model supplied the judgment at each step, but the orchestrator did everything real: it ran the code, captured the output, applied the edit, and would have cut the loop off after too many failed attempts. Take away the orchestrator and you have a model describing a fix it can't make; take away the guardrail and a confused agent could edit the same file forever. The intelligence feels like it's in the model, but the capability is in the machinery around it.

Beyond a single agent

Two extensions are worth knowing because they're where the field is heading.

Multi-agent systems split a complex job across several specialised agents, a researcher agent, a writer agent, a critic agent, coordinated by an orchestrator, each with its own tools and focus. The bet is that specialisation plus division of labour beats one generalist agent on complex work, the same reason human teams exist. It also adds coordination overhead and new failure modes, so it's not a free win (multi-agent systems go deeper on the trade-offs).

MCP, the Model Context Protocol, is the plumbing that's quietly making all of this practical. Every agent needs to connect to tools and data, and historically each connection was custom-built. MCP standardises that interface, so a tool built once can be used by any compatible agent, eliminating the integration busywork that used to dominate agent projects. It's not glamorous, but standard interfaces are usually what turn a promising technology into an ecosystem.

Why agents are hard, and where this connects

Understanding the architecture also explains why agents are fragile, which is the honest other half of the story. Every component is a place to fail: a tool returns something unexpected, memory fills up and important context gets pushed out, the planner commits to a bad decomposition, or a small error in an early step compounds across the whole loop because each iteration builds on the last. An agent taking twenty steps has twenty chances to go wrong, and errors don't stay put, they cascade. This is precisely why an agent that demos neatly can fail in production, and it's a big enough subject to deserve its own treatment, which is exactly what our companion piece on why AI agents fail is about. If this article is the anatomy, that one is the pathology, read them together and you'll understand both what makes an agent work and what makes it break.

The short version

An agent is a language model in a loop. The model reasons about what to do; tools let it act on the world; memory lets it carry context across steps; planning breaks a big goal into a sequence; and an orchestrator runs the loop, feeds results back, and enforces the guardrails that stop it running away. Think, act, observe, repeat, until the goal is met. Strip away the marketing and that's the entire idea, and it's powerful: it's what lets you hand a system a goal instead of a question, and get back completed work instead of a description of how the work might be done.

The one sentence to keep: the intelligence people attribute to agents lives less in the model than in the loop around it. A better model helps, but the leap from chatbot to agent isn't a smarter brain, it's the architecture of tools, memory, planning, and a control loop that lets a brain act, check, and adjust. That's why "agent" isn't a kind of model. It's a way of using one.

Common questions

What is an AI agent, exactly? An AI agent is a language model wrapped in a loop that lets it use tools, keep memory, and take multiple steps toward a goal, reasoning about what to do, acting, observing the result, and adjusting, until the task is done. The defining feature is the loop of action and observation. A single model call that just returns text is not an agent, even if its prompt tells it to "act like" one.

What's the difference between an agent and a chatbot? A chatbot responds: one prompt, one reply, and it's done. An agent acts: given a goal, it runs a loop, thinking, using tools, checking results, and adjusting, across many steps without needing input at each one. The chatbot describes how to book a flight; the agent checks prices, compares options, and books it. The loop and tool use are what separate them.

What are the components of an AI agent? A common shorthand is Agent = LLM + Memory + Planning + Tool Use, tied together by a loop and run by an orchestrator. The LLM is the reasoning core that decides what to do; tools let it act on the world; memory carries context across steps; planning breaks a big goal into sub-tasks; and the orchestrator executes the loop and enforces guardrails. Not every agent uses all four fully, but the LLM-plus-loop is essential.

What is the ReAct loop? ReAct (Reasoning + Acting) is the dominant agent pattern: the model interleaves thinking and doing. Each iteration, it reasons about what it needs, calls a tool to get it, observes the result, and reasons again, repeating until the goal is met. It's the think-act-observe cycle that turns a text model into a system that accomplishes multi-step tasks.

Does telling a model to "act like an expert" make it an agent? No. That's still a one-shot text response with a role in the prompt. The agent property comes from the infrastructure around the model, tools, memory, and a control loop of action and observation, not from the system prompt. Without a loop that lets the model act and react. You have a chatbot with a costume, not an agent.

What is an orchestrator in an agent system? The orchestrator (or runtime, or control loop) is the program that actually runs the agent: it calls the model, executes the tool code the model requests, feeds results back into the context, and enforces stopping conditions and guardrails like iteration limits, cost caps, and human-in-the-loop checkpoints. The model decides; the orchestrator executes and keeps the loop safe. It's where much of the real engineering of an agent lives.

What tools can an AI agent use? An agent uses tools to act beyond generating text: reading and sending email, searching the web, querying databases, running code, calling APIs, and increasingly controlling other software. Each tool is exposed to the model with a description of what it does and what inputs it needs, and the model decides when to call one based on the task. Standards like the Model Context Protocol have made connecting tools more uniform. The set of tools an agent has defines what it can actually do, and also its risk surface, since every tool that can take a real action is something a hijacked or mistaken agent could misuse.