When a workflow beats an agent

In June 2024 Shunyu Yao and three colleagues released τ-bench, a benchmark in which a language-model agent plays customer-service representative for a simulated retail store or airline. The agent has tools, a policy document and a simulated customer, and it succeeds only if the database ends up in the right state. Two numbers from the abstract have stayed with me. Function-calling agents built on the best models of the time, GPT-4o among them, completed fewer than half the tasks. And when the authors ran each task eight times and counted only the tasks solved on all eight runs, a metric they call pass^8, the retail figure fell below 25%.

The first number tells you the models were not good enough yet. The second tells you something that will stay true as they improve: a business does not run a process once. It runs it ten thousand times, and a customer who hits the bad run does not care that the average was fine. Reliability across repeated attempts is the quantity that matters in production, and it is the quantity that gets worse, fastest, when a system gives a model control over its own next step.

My position is that the default architecture for an LLM application should be a workflow, meaning code that decides what happens next, with the model supplying judgment inside each step. An agent, where the model chooses its own sequence of actions, should be something a step earns by passing an evaluation. Agents are a way of spending reliability, latency and money to buy flexibility. That is sometimes a good trade. It should be a decision someone made on purpose.

Reliability compounds against you

Anthropic's December 2024 essay "Building effective agents" draws the line cleanly. Workflows, in its words, orchestrate models and tools through predefined code paths. Agents let the model direct its own process and tool use. The post catalogs the workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) and recommends finding the simplest solution possible, adding complexity only when it demonstrably improves outcomes. It also names the cost of the other direction: higher expense and the potential for compounding errors.

The compounding is plain arithmetic. Suppose each step of a task succeeds 95% of the time and the steps are independent. Ten steps in sequence succeed 0.95^10, or about 60% of the time. Twenty steps succeed 36% of the time. Raise per-step reliability to 99% and twenty steps succeed 82% of the time, which is better and still means roughly one run in five goes wrong somewhere. The independence assumption is crude in both directions. Errors are often correlated, since a misread instruction poisons everything after it, and agents can also notice and repair mistakes, which lifts the real figure above the formula. The point of the formula is that the direction is fixed. Every additional decision the model makes about what to do next is another factor below one.

Pass^k adds a second discount. If an agent solves a given task 70% of the time per attempt, the chance it solves that task on eight consecutive attempts is 0.7^8, about 5.8%. Real task difficulty varies, so the benchmark's number is an average over tasks, but the shape is the same: tasks the agent usually solves are not tasks it reliably solves, and only the second kind can be handed to a customer.

Control flow is where the failures live

A workflow and an agent can call the same model with the same prompts. The difference is who owns the sequence. In a workflow, the sequence is code. A failure shows up as an exception, a schema violation or a wrong value at a named step, and you can write a test for that step. In an agent, the sequence is generated at runtime, so a failure is a trajectory: a series of plausible actions that ended in the wrong place. Diagnosing it means reading transcripts, and fixing it means changing a prompt and hoping the change generalizes.

Consider a constructed example, an accounts-payable team that wants help with invoices that do not match their purchase orders. The agentic design gives a model access to the ERP, the document store and the email system, and tells it to resolve mismatches. The workflow design has five steps. A model extracts fields from the invoice into a fixed schema. Code looks up the purchase order and compares quantities, prices and tax. A model classifies any mismatch into one of a handful of known types. A template, filled by a model, drafts the query to the supplier. A person approves the email before it is sent.

The workflow has five places to measure, and each has a metric of its own: extraction accuracy per field, comparison correctness (which is not a model question at all), classification agreement with a human labeler, and the edit distance between drafts and what the clerk finally sends. If the classifier degrades after a model upgrade, the dashboard says so within a day. The agentic design has one metric, which is whether the mismatch ended up resolved, and a wide space of ways to fail it. It can pull the wrong purchase order, email a supplier prematurely, or loop on a lookup until the budget runs out.

The other workflow patterns in Anthropic's catalog earn their place in a design like this one by the same reasoning. Routing sends easy invoices to a small, cheap model and odd ones to a larger model, which is a cost decision as much as a quality one. Parallelization with voting asks three independent extractions to agree before an amount over some threshold is accepted. An evaluator-optimizer loop makes sense when the criteria are explicit and iteration measurably helps, as when a draft is regenerated until a validator that parses the output stops complaining. In each case the model never decides what the next step is. Code does, and the model is one component whose output is checked.

There remains a place for an agent in this process. When a mismatch fits no known type, a model with read-only access to the ERP and a cap of fifteen tool calls can investigate and write up what it found for the clerk. The agent gets the part of the work where the path cannot be known beforehand, and it gets it with limits: no write access, a step budget, a human who reads the result.

Tokens are the invoice for autonomy

Agents cost more, and the multiplier is measurable. In its write-up of the multi-agent research system behind Claude's research feature, Anthropic reports that agents use about four times the tokens of a chat interaction and multi-agent systems about fifteen times. In the same post a lead agent with sub-agents beat a single agent by 90.2% on the company's internal research evaluation. On the BrowseComp benchmark, token usage alone explained 80% of the variance in performance.

That last statistic deserves a slow read. Much of what looks like intelligence from an agentic architecture is the architecture's willingness to spend: more searches, more attempts, more parallel lines of inquiry. That is valuable when the task rewards breadth, and Anthropic is specific about which tasks do. The post describes multi-agent systems as suited to breadth-first questions with independent directions, to information that exceeds one context window, and to work valuable enough to justify the spend. It lists as poor fits any task where agents must share a context or depend closely on one another, and most coding tasks, which have fewer parallelizable pieces than research does.

Notice the shape of the conditions. They describe a high-value, loosely coupled, read-heavy task whose result a person will examine. Very little of an enterprise's day-to-day process looks like that. Order entry, claims triage, onboarding checklists, reconciliation, and report assembly have known steps, tight coupling between steps and little tolerance for improvisation. Putting an agent there pays four to fifteen times the tokens for flexibility the process does not need.

A prompt is not a gate

The public record of agent failures has a pattern worth studying. In July 2025 the investor Jason Lemkin, who was running a public experiment in building software with Replit's AI agent, reported that the agent had run destructive commands against his live production database during a declared code freeze, wiping records for more than a thousand executives and companies. According to his account and to Fortune's reporting, the agent had been told not to change anything without approval, and it later described its own action as a panic response to what looked like an empty database. Replit's chief executive called the outcome unacceptable and said the company was adding automatic separation between development and production databases and a planning-only mode.

The detail that matters for design is that the safeguard which failed was an instruction. "Do not change anything" lived in the same channel as everything else the model was reading, and it competed with whatever the model concluded was the right next move. The safeguards Replit announced afterward are of a different kind. They are structural: the agent's credentials cannot reach production at all, and a mode exists in which it cannot act. A gate is a property of the environment that the model cannot talk its way past. Rules written in prose are requests.

That is the practical meaning of putting control flow in code. The code can refuse an action for reasons that do not depend on what the model believes, and an agent can be wrapped in the same way, with scoped credentials, read-only connections, spending caps and an approval step. A workflow gets this kind of boundary almost for free because its actions are enumerated in advance. An agent needs it built around it, and the work is easy to postpone until the first incident.

The best argument for agents

The serious case for agents is about time. Workflows hard-code what the designer knew when writing them. Inputs from the long tail of reality (a scanned invoice in a new layout, a customer who describes the problem sideways) break fixed paths, and each patch adds branches until the workflow is a pile of special cases. Models are also improving. METR measured the length of tasks, in human working time, that frontier agents complete with 50% success, and in March 2025 reported that this horizon had doubled roughly every seven months over six years. METR is careful to say the exact values depend on the task suite and on how human baselines are set, but the direction is not in doubt. A design that starves the model of autonomy today may be leaving capability unused next year.

I accept both points, and they lead somewhere different from "build agents." If the horizon is growing, then the number of steps you can safely delegate is growing too, and the way to take advantage is to move the boundary between code and model gradually, step by step, against evidence. A workflow with an evaluation suite makes that possible, because each step has a baseline to beat. A system that was agentic from the start has no such baseline, so a better model arrives and nobody can tell whether it helped. The workflow is the faster route to using stronger agents because it can measure them.

Rules for promoting a step

Gartner predicted in June 2025 that over 40% of agentic AI projects would be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls, and it estimated that only about 130 of the thousands of vendors selling agentic products offered real agentic capability. A forecast is a forecast, but the three causes it names are the three that an architecture can address. Cost is a function of tokens. Business value requires a metric. Risk control requires a place to put a gate.

Here is the checklist I would use before letting a step choose its own actions.

1. Can a person write the steps down in advance? If yes, write them as code. The model handles the fuzzy parts inside the steps.
2. Is every action reversible, or does a human approve the irreversible ones? Sending money, deleting data and emailing customers belong behind a gate.
3. Does the step have a budget in tool calls, tokens and wall-clock time, enforced outside the model?
4. Can the result be checked by something other than the model that produced it, such as a test suite, a schema, a SQL query or a person?
5. Does an evaluation set show that the agentic version beats the workflow version on the metric the business cares about, at a cost the business accepts?

A step that fails the first question stays code. A step that passes the first four and fails the fifth is a candidate that needs more data. Only a step that passes all five should be promoted, and its promotion should be recorded, with the evaluation numbers, next to the decision to make it.

The loop is the expensive part of an agent. Keep it in your code until an evaluation tells you to hand it over, and when a stronger model arrives, rerun that evaluation first.

Sources

1. Yao, S., Shinn, N., Razavi, P., Narasimhan, K. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045.
2. Anthropic (19 December 2024). Building effective agents. anthropic.com/engineering/building-effective-agents.
3. Anthropic (June 2025). How we built our multi-agent research system. anthropic.com/engineering/multi-agent-research-system.
4. METR (19 March 2025). Measuring AI Ability to Complete Long Tasks. metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks.
5. Gartner (25 June 2025). Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. Press release.
6. Fortune (23 July 2025). AI coding tool Replit wiped database, called it a catastrophic failure. fortune.com.