Single Agent vs Multi-Agent: 66 of 70 Bugs, or 5.1x the Cost?

In the same October week two results came out that seem to contradict each other.

A multi-agent workflow on Claude found 66 of 70 planted bugs where a single agent found at most 27.

A benchmark from Vals AI found agent teams costing up to 5.1 times more for a score gain that was statistically significant in one of four comparisons.

Both are right. The single agent vs multi-agent question has no general answer.

It depends on whether the work splits into independent pieces, and on whether the capacity behind the agents can grow with them.

I learned the second part on my own desk.

On my Mac mini, four parallel requests to a local model produced 23% fewer decisions per hour than one request at a time.

The two results

The first number comes from Anthropic's launch of dynamic workflows in Claude Managed Agents on 9 October.

In its launch thread on X, Anthropic's developer account reported that 70 bugs were planted in a codebase of about 116,000 lines.

The Decoder covered it the same day.

Over three runs a single agent found 14, 15 and 27. The dynamic workflow found 66 in every run.

I have not reproduced it, and neither the model, the token count nor the cost of those runs has been published.

The second result is Vals AI's multi-agent Vibe Code Bench, also from 9 October.

Fifty apps, built either by one agent or by a lead running up to five subagents at once. Each setup ran once.

Model and effortSoloTeamCost of teamMedian time solo to team
GPT 6 Sol, medium77.6%84.9%2.5x12.4 to 11.9 min
GPT 6 Sol, max89.0%90.4%1.8x38.4 to 27.6 min
Opus 5.5, medium91.5%91.2%2.2x20.3 to 26.6 min
Opus 5.5, max89.8%93.2%5.1x74.7 to 174.1 min

Only the first row was significant. At Opus max the team cost $122 per app against $23.77 for one agent, and took more than twice as long.

Most of the extra spend was cached input: the subagents read a median of 224 million cached tokens per app.

What Claude Managed Agents dynamic workflows are

The docs describe a workflow as "a program that an agent writes to run many agents and combine what they return." The lead agent writes code that fans work out in phases and merges the results.

The limits are generous: up to 64 agents working at once, 1,000 started over a run's lifetime, a 24-hour default lifetime.

A run has no price of its own. Every agent's tokens are billed like any other session tokens.

Anthropic's own docs add the sentence that matters: keep runs to the tasks that need them, "because every agent in a run uses tokens."

Single agent vs multi-agent on two kinds of work: breadth-first bug hunting where a workflow found 66 of 70 bugs, and coupled app building where teams cost 1.8x to 5.1x more for a gain that was significant once
Two October results side by side, and the task structure behind them

Single agent vs multi-agent: the task structure decides

Seventy independent bugs in a large codebase is the best case for fan-out.

Each agent can search its own part of the code. Nobody has to agree with anybody else. A found bug is easy to verify.

Building one app is the opposite. Every piece depends on decisions made in another piece. That is where Vals found almost nothing.

Google Research measured the split across 180 configurations earlier this year. Centralized coordination gained 80.9% on parallelizable financial reasoning.

On sequential planning every multi-agent variant they tested lost between 39% and 70%.

Error behaviour differed too. Independent agents amplified errors 17.2 times. A central coordinator brought that down to 4.4 times.

Anthropic's own guidance from January lands in the same place.

Multiple agents help when context pollution hurts, when tasks can run in parallel, or when specialization improves tool choice.

"Outside these situations, the coordination costs typically exceed the benefits."

Cognition made the counterpoint last year: actions carry implicit decisions, and conflicting decisions carry bad results.

Two subagents building parts of one game produce parts that do not fit.

Even a small pipeline shows the handoff loss. In one experiment on dev.to the planner wrote up to 584 characters and the executor received a 60-character hint.

The pipeline still won, but the author traced the gain to re-reading the source, not to the plan.

Speed is the product, and it needs elastic capacity

Anthropic's Opus 5.5 system card frames large teams as a speed purchase.

Scores rise with team size, with diminishing returns. Larger teams reach a given score earlier, at a higher cost.

On short research tasks without a time budget, the coordination overhead made teams slower than one agent.

Under a tight latency budget the same teams were up to 2.8 times faster.

The latency numbers in that card come from fixed reference rates for reading and generating tokens.

That is deliberate, to remove batching, queuing and hardware contention from the comparison.

On a cloud API that assumption roughly holds. On your own hardware it does not, and that is where my Mac mini comes in.

I measured a local 4B and a 9B model on a Mac mini M4 with 10 GPU cores, on the same 150 texts.

Four concurrent requests gave about 23% fewer decisions per hour than one request at a time, for both models.

The reason is that reading the prompt, the prefill, is compute-bound. The GPU was already busy.

Four agents on one machine share the same compute and wait for each other.

So the same fan-out that buys speed in Anthropic's data costs speed on a machine without spare capacity.

Whether parallel agents are faster is a property of the infrastructure, not of the agents.

How I decide

My orchestrator runs up to three builders per project. Each gets its own task and its own branch, with a review gate before anything merges.

They do not share mutable state and they do not negotiate with each other.

That is fan-out at the level of separate tasks, not inside one task.

I explained the trade-offs against Anthropic's agent teams in Claude agent teams vs building your own.

Inside one task I keep a single agent with a clean context.

When that context fills up I reset it rather than splitting the work, for the reasons in context compaction vs clearing.

The one place I would use a wide fan-out is breadth-first search with a cheap check: reviewing many independent files, hunting bugs, researching sources.

Exactly the kind of work in the 66-of-70 test.

One of the largest public examples shows both sides.

Nous Research refactored Hermes Agent with 1,393 subagents, up to 218 at once, for about $19,300 in model cost on the main run.

The write-up is frank that the agents introduced regressions the existing tests had missed.

📖 Read also: Async quality gates: why AI agents do not get to decide when they are done

A practical rule set

  1. Classify the task first. Independent and verifiable: fan out. Coupled or sequential: one agent.
  2. Give every subagent its own ownership. A file, a module, a question. Never two agents writing the same thing.
  3. Check the capacity behind the agents. On an API, parallel buys wall-clock time. On one local machine, measure before you assume it does.
  4. Compare against more effort on one agent. Vals' own conclusion is that higher reasoning effort is an alternative worth weighing.
  5. Budget the tokens before you start. Anthropic measured multi-agent research at about 15 times the tokens of a chat in 2025. On their BrowseComp evaluation, token use alone explained 80% of the variance in performance.

Why the architecture around the agents matters more than the model count is the theme of why architecture beats models.

Frequently Asked Questions

Vincent van Deth

AI Strategy & Architecture

I build production systems with AI — and I've spent the last six months figuring out what it actually takes to run them safely at scale.

My focus is AI Strategy & Architecture: designing multi-agent workflows, building governance infrastructure, and helping organisations move from AI experiments to auditable, production-grade systems. I'm the creator of VNX, an open-source governance layer for multi-agent AI that enforces human approval gates, append-only audit trails, and evidence-based task closure.

Based in the Netherlands. I write about what I build — including the failures.

Comments

Your email address will not be published. Comments are reviewed before publication.

Loading comments...