When not to use an agent

On a standard coding benchmark, GPT-3.5 answering directly scored 48.1%. GPT-4 answering directly scored 67.0%.

GPT-3.5 placed inside a structured workflow scored 95.1%.

The structure was worth more than two generations of model improvement. That result is several years old now and the specific numbers have moved, but the shape has not: most of the performance gap between a naive deployment and a good one is architecture rather than model capability, and the architecture that closes it is usually simpler than an agent.

An agent is warranted when you cannot hardcode the path but can still verify progress. That is a narrow condition. Almost everything currently built as an agent fails the first half, and a substantial fraction fails the second, which is the difference between a system that is flexible and one that is merely unpredictable.

The distinction that actually matters

The word "agent" has been applied to everything from a chatbot with a database connection to a self-directing system with shell access, which makes the category useless without a definition.

The one that holds is about ownership of control flow.

A workflow orchestrates models and tools through predefined code paths. A step may call a model, and the next step happens regardless of what came back, because a human decided the order in advance.

An agent lets the model direct its own process, choosing what to do next based on what just happened, and retaining control of how the task gets accomplished.

The presence of an LLM call is not the signal. A pipeline that calls a model six times and then does exactly what it was always going to do is a workflow. A tightly scripted loop with two allowed actions and a hard step limit is closer to an agent than that pipeline, despite being simpler, because the model decides.

This matters because the two have different failure modes, different costs, different testing requirements and different debugging stories, and calling both "agentic" hides all of it.

The two-part test

Before building, answer two questions honestly.

Can you write down the steps? Not the steps for the happy path. The steps including the branches, including what happens when a lookup fails, including the conditions under which the process should stop. If you can enumerate them, you have a workflow, and implementing it as an agent means paying a model to rediscover your flowchart on every run, non-deterministically, with the possibility of getting it wrong.

Can you tell whether it is making progress? An agent that cannot be checked mid-flight is a system you have to trust entirely or not at all. The tasks where agents work best are the ones with cheap verification: code that either compiles and passes tests, a form that either validates, a query that either returns rows. Where verification is expensive or subjective, the agent's autonomy becomes a liability, because errors compound before anyone can see them.

Both yes means use a workflow. First no, second yes is the agent's actual territory. Second no means you should be very careful regardless of which you choose, because you are building something whose behaviour you cannot observe.

A worked decision on a real-sounding task

Abstract criteria are easy to agree with and hard to apply, so here is one task taken through the test.

The task: a customer emails about an order. The system should work out what they want, look up the order, and either answer, issue a refund, or escalate.

Can you write down the steps? Attempt it. Classify the intent into a handful of categories. Look up the order. If the intent is a status question, answer from the record. If it is a refund request, check eligibility against policy. If eligible and under a threshold, issue it. If over the threshold or ineligible, escalate with a summary.

That is a flowchart. It has branches and it has a failure path, and every branch was knowable in advance. This is a routing workflow with a classification step and a policy check, and building it as an agent means the model rediscovers that flowchart on every email, at several times the cost, with a chance of deciding something else.

Now change one thing. The customer's email describes a situation the policy does not cover, and there are hundreds of such situations, and new ones arrive weekly.

Now the branch count is unbounded rather than large, and the correct response depends on details visible only at runtime. The first test now fails, which is the condition an agent exists for. And the second test still passes, because there is a check: does the proposed resolution comply with policy, which a deterministic component can evaluate.

That is the boundary, and it moved because of one property of the input distribution rather than anything about the technology. The useful discipline is that the question is answered by the task, not by the architecture you find interesting, and the way to find out is to try the flowchart rather than to reason about whether one exists.

Five cases where the answer is no

The path is knowable. Intake, classify, route, respond, log. Everyone involved can draw it on a whiteboard. A state machine is cheaper, faster, deterministic, testable with ordinary tools, and debuggable by reading a stack trace. Wrapping it in a model adds cost and non-determinism to something that had neither.

One model call would do. A single prompt with a few tools, one context, one decision. This is not an agent and does not benefit from being described as one. Adding an orchestration layer to a single call introduces coordination overhead to a problem with nothing to coordinate.

The real problem is retrieval. The task is finding the right document and answering from it. That is a retrieval system. Framing it as an agent grants autonomy the task does not need and moves the failure from a retrieval miss you can diagnose to an agent decision you cannot.

Determinism is a requirement. Regulated calculations, financial postings, anything where the same input must produce the same output and be defensible afterwards. Non-determinism is the agent's defining property, and where it is unacceptable, no amount of temperature tuning makes it acceptable. Use the model to draft and a deterministic system to decide.

Failure is irreversible and cheap to prevent. If a wrong action cannot be undone and a rule would have prevented it, encode the rule. The blast radius argument applies here too. The argument that an agent could learn not to do it is an argument for accepting occasional catastrophic outcomes in exchange for flexibility you did not need.

What the workflow patterns actually cover

The case against agents is stronger when you can see what replaces them, and the replacement set is small and well documented.

Chaining. Output of one step becomes input to the next, in a fixed order. Covers most document processing, most content generation with review stages, most extract-transform-load work with a language component.

Routing. Classify the input, then send it down one of several fixed paths. Covers most support triage, most intake, most anything with categories. The classification uses a model; the routing does not.

Parallelisation. Split into independent subtasks, run them concurrently, combine. Covers evaluation, multi-aspect analysis, anything where the parts do not depend on each other.

Orchestrator-workers. A model decides which subtasks are needed, fixed workers execute them. This is the closest to an agent and remains a workflow, because the workers do not choose their own behaviour.

Evaluator-optimiser. Generate, critique, revise, in a bounded loop with a fixed exit condition. Covers most quality-sensitive generation.

Between them these cover the overwhelming majority of what gets built as agents. The production evidence supports this: workflows rather than autonomous agents were the dominant pattern behind successful deployments, with fully autonomous multi-agent systems remaining largely exploratory outside narrow domains.

What agents are actually for

The territory is real and it is smaller than the discourse implies.

Open-ended problems with cheap verification. Software engineering against a test suite is the canonical case, because the path cannot be specified in advance and correctness is checkable at every step. Debugging is similar. So is anything where the environment supplies ground truth.

Environments too large to enumerate. Browsing, exploring a codebase, navigating an interface nobody documented. You cannot hardcode a path through a space you have not mapped.

Tasks where the branch count is unbounded. Not "many branches", which a router handles, but a space where new situations arise that nobody anticipated and the correct response depends on details only visible at runtime.

Long-horizon work with intermediate checkpoints. Where the task takes many steps, the steps depend on each other, and progress can be assessed along the way rather than only at the end.

Notice what these have in common: the verification question comes back every time. Agents work where the world tells you whether you are winning. They work badly where the only feedback is the agent's own account of itself.

How the wrong decision gets made

The decision is rarely made on the merits, and the mechanism is worth naming because recognising it is most of the defence.

Someone in a design discussion says the task feels too dynamic for a workflow, and suggests an agent. It sounds reasonable, because agents are flexible and flexibility sounds free. Everyone nods. Nobody asks whether the steps could be written down, because that would require someone to try, and trying takes an afternoon that the meeting does not have.

Three months later the system loops in unexpected places, the logs are unreadable, costs are several times the estimate, and nobody can reconstruct who proposed the architecture or what problem it was solving.

The specific failure is that "we might need flexibility later" is treated as free, and it is not. It costs determinism, testability, debuggability, and a multiple on inference. The correct response to that suggestion is to spend the afternoon attempting the flowchart. If it can be drawn, the question is settled. If it cannot, you have discovered something and the agent is justified.

There is a related failure with frameworks. Starting with an agent framework rather than direct model calls hides the prompts, adds abstraction before anyone knows what is needed, and makes the agentic path the default because it is the path the framework is built around. Building from basic components first tends to reveal that less is needed.

The three checks worth running

If you have built something and are unsure whether it needed to be an agent, these surface the answer quickly.

Read the system prompt as the model. Paste it in and ask what is ambiguous or underspecified. Prompts that a person finds clear are frequently full of assumptions the model cannot resolve, and the resulting behaviour looks like poor reasoning when it is poor instruction.

Do the task with only what the agent can see. Restrict yourself to the observations available to the system and attempt the task by hand. If you cannot, the agent has an information problem rather than a capability problem, and no model upgrade will fix it.

Feed a failed trajectory back and ask why the step failed. Not for a fix, for a diagnosis. This distinguishes the cases where the model reasoned badly from the far more common ones where the tool returned something confusing, the context was assembled wrongly, or the task was underspecified.

All three take under an hour and they redirect most investigations away from the model, which is where most investigations start and where the fault usually is not.

The cost of choosing wrong, in each direction

The two errors are not symmetric, which is the argument for defaulting to the simpler option.

Over-engineering: building an agent where a workflow would do. You pay a multiple on inference for the same outcome. You lose determinism, so the same input can produce different results and you cannot reproduce a bug. You lose stack traces, replacing them with trajectories that need interpreting. Testing requires running each case repeatedly because a single run tells you little. And every future change has to be validated against a system whose behaviour is a distribution rather than a function.

The failure is expensive and it is gradual. Nothing breaks. The system works, costs more than it should, and slowly becomes something nobody wants to modify.

Under-engineering: forcing an open-ended problem into a rigid pipeline. The system works on the cases you anticipated and fails on the rest, usually silently, by taking the closest available branch rather than the correct one. Each new situation requires a code change, so the backlog fills with special cases and the flowchart accretes conditions until nobody can read it.

The failure is visible and it is recoverable. You can see which inputs fail, and the fix is to loosen the structure at the point where it binds.

The asymmetry matters. Under-engineering produces a legible problem with an incremental fix. Over-engineering produces a working system that quietly costs more and resists change. Given uncertainty, the cheaper mistake is the simpler architecture, which is the reverse of how these decisions usually get made, because the simpler architecture looks like the less ambitious answer in the room where it is decided.

What is unresolved

Where the boundary moves as models improve. Tasks requiring agents today may be hardcodeable tomorrow, if models become reliable enough that a workflow with one model step handles what currently needs deliberation. Alternatively, better models make agents viable in more places. Both are plausible and the evidence is mixed, which means any specific architecture is a bet on a moving line.

Whether the workflow advantage is about structure or about constraint. The result at the top of this article is usually read as structure adding value. It could equally be read as constraint removing failure modes, which would predict something different about where agents help. Nobody has cleanly separated these.

Whether hybrid systems are a stage or a destination. Most production systems that work are workflows with agentic components in specific slots. It is unclear whether this reflects current model limitations or is the stable answer, and the two readings imply different investment.

The counter-argument

Workflow advocacy can become an excuse not to try. The problems that most need agents are the ones where the path cannot be specified, and an organisation that always chooses the workflow will never build those. There is a real risk of using "simplest thing that works" to avoid the harder thing that would work better.

Some of the anti-agent evidence is dated. Model capabilities have moved substantially since much of this guidance was written, and the boundary has moved with them. Advice calibrated to a weaker generation may be too conservative now.

Frameworks are not the enemy. The argument for starting with direct model calls is sound for learning and can be wrong for shipping. A team that would otherwise build a worse orchestration layer themselves is better off with one someone else maintains.

And the failure numbers cut both ways. High agent project failure rates are cited as evidence agents are overused. They are also consistent with agents being hard and worth doing, in the way that most difficult things have poor success rates before the practice matures.

The short version

The distinction that matters is ownership of control flow. A workflow orchestrates models and tools through predefined code paths; an agent lets the model choose what to do next based on what just happened. The presence of a model call is not the signal, and calling both "agentic" hides that they have different failure modes, costs, testing requirements and debugging stories.

Two questions settle most cases. Can you write down the steps, including branches and failure conditions? If yes, an agent means paying a model to rediscover your flowchart non-deterministically on every run. Can you tell whether it is making progress? Agents work where the environment supplies cheap verification, like code that either passes tests, and work badly where the only feedback is the system's own account of itself.

Five cases where the answer is no: the path is knowable, one model call would do, the real problem is retrieval, determinism is required, or failure is irreversible and a rule would have prevented it. Five workflow patterns cover most of what gets built as agents: chaining, routing, parallelisation, orchestrator-workers, and evaluator-optimiser. Workflows rather than autonomous agents were the dominant pattern behind successful production deployments.

The agent's real territory is open-ended problems with cheap verification, environments too large to enumerate, unbounded branch counts, and long-horizon work with intermediate checkpoints. All four share the verification property.

The decision is usually made badly for one reason: "we might need flexibility later" is treated as free. It costs determinism, testability, debuggability and a multiple on inference. The correct response is to spend one afternoon attempting the flowchart. If it can be drawn, the question is answered. If it cannot, you have learned something worth knowing and the agent is justified.

Common questions

What is the difference between an AI workflow and an AI agent? Ownership of control flow. A workflow orchestrates models and tools through predefined code paths, so a human decided the order in advance and the next step happens regardless of what the previous one returned. An agent lets the model direct its own process, choosing what to do next based on what just happened. A pipeline that calls a model six times and then does what it was always going to do is a workflow; a two-action loop where the model decides is closer to an agent.

When should I not use an AI agent? When the path is knowable, since a state machine is cheaper, deterministic and debuggable. When one model call with a few tools would do. When the real problem is retrieval, which framing as an agent only obscures. When determinism is a requirement, as in regulated calculations, because non-determinism is the agent's defining property. And when failure is irreversible and a rule would have prevented it, since accepting occasional catastrophe in exchange for unused flexibility is a poor trade.

How do I decide between a workflow and an agent? Two questions. Can you write down the steps, including branches and what happens when things fail? If yes, use a workflow. Can you tell whether the system is making progress mid-task? Agents need cheap verification, like a test suite or a validating form. If you cannot hardcode the path but can verify progress, that is the agent's territory. If you cannot verify progress, be careful whichever you choose, because you are building something you cannot observe.

What workflow patterns replace most agents? Five. Chaining, where output feeds the next step in fixed order. Routing, where a model classifies and fixed logic dispatches. Parallelisation, where independent subtasks run concurrently and combine. Orchestrator-workers, where a model selects subtasks and fixed workers execute them. And evaluator-optimiser, a bounded generate-critique-revise loop with a fixed exit. Between them these cover the large majority of what gets built as agents.

Are agents ever the right choice? Yes, in a narrower territory than the discourse suggests. Open-ended problems with cheap verification, software engineering against a test suite being the canonical case. Environments too large to enumerate, like browsing or exploring an undocumented codebase. Tasks where the branch count is unbounded rather than merely large. And long-horizon work where progress can be assessed at checkpoints. All four share the property that the environment tells you whether you are winning.

Why do teams choose agents when they should not? Because "we might need flexibility later" is treated as free, and nobody asks whether the steps could be written down, since answering that takes an afternoon a design meeting does not have. Three months on, the system loops unexpectedly, logs are unreadable, costs are several times the estimate, and nobody remembers who proposed it. Frameworks compound this by making the agentic path the default and hiding the prompts behind abstraction.

How much does architecture matter compared to model choice? Substantially more than most teams assume. On a standard coding benchmark, a weaker model inside a structured workflow scored 95.1% against 67.0% for a stronger model answering directly, so the structure outperformed two generations of model improvement. The specific numbers have moved since, and the pattern holds: most of the gap between a naive and a good deployment is architecture, and the architecture that closes it is usually simpler than an agent.

How can I check whether my agent needed to be one? Three checks, each under an hour. Paste the system prompt into the model and ask what is ambiguous, since prompts humans find clear are often full of unresolvable assumptions. Attempt the task yourself using only the observations available to the agent, which distinguishes an information problem from a capability one. And feed a failed trajectory back asking why the step failed, which usually reveals a confusing tool response or badly assembled context rather than poor reasoning.