When an AI agent fails on the way from pilot to production, the first reaction is usually to improve the prompt or switch models. Often the useful question is a different one: why are we asking the model this at all?
This piece argues a thesis that is easy to state and hard to apply: many agents get complicated because they ask the model for decisions that already had a rule. Designing an agent means deciding what the model decides and what the code decides. What repeats, has one right answer or is expensive when it goes wrong goes into code; what needs judgement goes to the model, with a person deciding where there is risk.
Workflow or agent: two ends of the same axis
In December 2024, Erik S. and Barry Zhang published a distinction at Anthropic that has become a reference. In a workflow, models and tools are orchestrated through predefined code paths. In an agent, the model directs its own process and decides which tools to use. Their starting advice: find the simplest solution possible and only increase complexity when needed.
The same text names the trade-off. Workflows offer predictability and consistency for well-defined tasks; agentic systems often trade latency and cost for better task performance. Neither is better in the abstract. What matters is where on that axis you put each decision, not what you call the whole system.
How much of the loop to make deterministic
Phil Winder puts it in practical terms in an October 2026 article on what he learnt building agents: a harness is a loop with one or more models inside it, and harness engineering is deciding how much of that loop to make deterministic. To decide, he suggests one question: what does a wrong answer cost, and who catches it?
His criterion for what goes into code is concrete: the steps that repeat, that have one right answer and that sit at the centre of your problem. Judgement, such as understanding what someone meant or whether an argument holds up, stays with a model or a person. One of his examples: in an agent that navigates phone menus, moving the choice of the next option into code cut the exploration of one test menu from 46 steps to 6.
Dex Horthy gets to the same place from another side. In his «12-Factor Agents» he notes that many products sold as agents are mostly deterministic code, with LLM steps placed at just the right points. One of his principles says it plainly: own your control flow.
| If the decision… | It goes to | What it needs |
|---|---|---|
| Has one right answer (a calculation, a valid format) | Code | Automated tests |
| Repeats the same way every time (an approval route by amount) | Code | Someone who maintains it when the rule changes |
| Is expensive if wrong and nobody reviews every output | Code or a person | A record of who decided and why |
| Requires interpreting what someone meant | Model | Evals with known-answer cases |
| Fits no known rule | Model, with a way out to a person | An exception path |
Criteria compiled by onext from Phil Winder (2026) and Anthropic (2024)
An example: reviewing travel expenses
An illustrative example, with no client behind it: an agent that reviews a company's travel expense claims. If everything is asked of the model, each claim depends on how it reads the prompt that day. If the work is split, it looks like this:
- The model reads the receipt and extracts the date, amount, supplier and type of expense. This is the ambiguous part: crumpled receipts, languages, badly written descriptions.
- The code checks that the amount is within the limit for its category and that the invoice has the mandatory tax details. It has one right answer.
- The code decides who approves based on the amount. Company policy already says so; there is nothing to interpret.
- A person receives what does not fit: an expense outside policy, a receipt the model cannot classify with confidence.
The model still does the work only it can do, and what could go wrong through a misreading no longer depends on it. What that person receives and in what order is covered in the exception path.
Every decision that stays with the model has to be measured. Every decision that moves into code has to be maintained. Choosing where each one goes is choosing which cost you would rather pay.
Measuring what the model decides means having evals: known-answer cases that run on every change, as we explain in how to build a golden set and evals on every pull request. And some decisions should not depend on the model for security reasons: which tools it can use and with which permissions, as we cover in prompt injection and tool permissions.
More code is not always better
The opposite temptation exists too: writing rules for everything until the agent is a rigid program with a decorative model. Winder cites Han Lee, who argues that most harness code will dissolve into the next generation of models. Winder's advice: add to the harness only to fix a failure you have already seen.
The data does not point one way either. The same article reports that when the SWE-agent team removed the syntax checker from their agent's edit command, the share of issues it resolved fell from 18.0% to 15.0%. A well-placed deterministic check helped the model; it did not replace it.
Risk A written rule can be wrong too: when company policy changes and nobody updates the code, the agent confidently applies a rule that no longer exists. Moving a decision into code does not remove it; it changes who has to watch it.
What we would do on Monday
If your agent is in pilot, there is an exercise that fits in one morning. Write down in a table every decision the model takes today, from classifying an input to deciding it has finished. Next to each one, two columns: whether it has a clear rule, and what happens when it goes wrong. The ones with a rule that fail often are the first candidates to move into code, starting with the cheapest to write. The ones that stay with the model are the ones that need evals and an exception path before they reach production. If the model you use also returns a probability alongside the decision, you can set a threshold, as we explain in the model that decides instead of writing.
How we approach it at onext
It is the criterion we work with on our agent projects: the model decides only what needs judgement, what has a rule goes into code with its tests, and a person decides where there is risk. What we build stays with your team, including the list of what each part decides and why. From pilot to production, the rest of the way is in AI engineering: from pilot to production.
Frequently asked questions
What is the difference between a workflow and an AI agent?
Anthropic puts it this way: in a workflow, models and tools are orchestrated through predefined code paths; in an agent, the model directs its own process and decides which tools to use. They are not two closed categories but the two ends of one axis: almost every real system mixes both.
Which decisions of an agent should be code?
The ones that repeat, have one right answer and sit at the centre of the problem: a calculation, a format check or a fixed approval route. Phil Winder sums it up with that rule, and adds a question to decide how much to make deterministic: what a wrong answer costs and who catches it.
And which decisions should stay with the model?
The ones that need judgement: understanding what someone meant, writing a reply or classifying a case that fits no rule. Those decisions need evals that measure how the model decides and an exception path for the doubtful case, with a person deciding where there is risk.
What is an agent harness?
Phil Winder defines it as a loop with one or more models inside it, and harness engineering as deciding how much of that loop to make deterministic. It is everything around the model: the code that decides what it is asked, which tools it has, what is checked before and after, and what happens when it fails.
Does more code always make an agent more reliable?
No. A written rule can be wrong too, and it has to be maintained when the business changes. Winder also cites Han Lee's argument that most harness code will dissolve into the next generation of models. That is why he advises adding code only to fix a failure you have already seen.
Where do I start if my agent is already in pilot?
List the decisions the model takes today and mark the ones with a clear rule or an expensive failure. Those are the candidates to move to code, starting with the ones that fail often and are cheapest to write. The ones that stay with the model need evals and an exception path before they reach production.
Sources
- Erik S. and Barry Zhang, «Building effective agents», Anthropic, 19 December 2024.
- Phil Winder, «Harness Engineering: What We Learnt Building AI Agents», Winder.AI, 11 October 2026.
- Dex Horthy (HumanLayer), «12-Factor Agents», GitHub repository (no publication date visible; accessed 18 October 2026).
- The criteria table, the travel expenses example (illustrative, no client) and the exercise are onext's own work.
Bernat López is founder and CEO of onext, an AI boutique. He helps development and product teams work with AI with method —specification before building, a person who decides where there is risk and Spec-Driven Development— and applies to his own company what he proposes: onext runs on its own agentic system.
LinkedIn →