Multi-Agent: Work You Can Split, Not Free Speed

What exactly are we parallelising?

In the previous post we wrote the flow in code: chain, router, planner–worker. In the third pattern, a longer plan means a longer run. “Last month’s rejected withdrawals” meant 34 records, and each record got the same treatment: fetch the record, read the rejection reason, look at the customer’s earlier withdrawals, assign a label.

That looks like work you can parallelise. Three questions decide whether it really is:

I should have said “partly” to the third question, and in the first run I did not. I paid for it in the merge step.

From the field: 34 withdrawals, 4 agents, two mistakes

The first setup was simple: the orchestrator split 34 records four ways, four sub-agents ran with the same prompt, each reviewed its share and appended its result to summary.md.

The 3.4× token number is no surprise: every sub-agent carries its own system prompt, tool definitions and instructions from scratch. 4 agents means paying the fixed input cost 4 times. I could live with that. Two other things I could not.

Mistake one: writing to a shared file

Four agents appended lines to summary.md at the same time. Two read it in the same second, both added their line and wrote it back; the second one deleted the first one’s line. This is exactly the story in why stock goes negative — read, modify, write. The only difference is that there it was two threads and here it is two models. The model being clever does not win this race. What wins it is every agent writing its own output somewhere separate, and code doing the merge.

# WRONG: four agents, one file
for part in parts:
    spawn(agent, prompt=part, output="summary.md")   # race

# RIGHT: each agent returns its record, code merges
results = run_parallel(
    [lambda p=p: sub_agent(p) for p in parts], concurrency=3
)
report = merge(results)     # code: sort, deduplicate, check the count

Mistake two: duplicated reads

13 tool calls were duplicates. The cause was the split rule: I had cut the record list into four equal slices, but two withdrawals from the same customer landed in two different agents. Both fetched that customer’s history. The fix was to change the split key: split by customer, not by record. All records for one customer go to one agent. This is the same rule as in worker sharding: the partition key is the natural key of the work.

Sub-agent context: the other half of the win

The benefit of multi-agent that nobody talks about is not speed; it is isolation. When one agent reviews 34 records, everything piles into the same window: the detail of 34 records, 71 tool replies, notes in between. By record 20, the detail of the early ones has either dropped out or is muddying the decision. (A post of its own.)

In a sub-agent the window is clean: 8–9 records, its own tool replies, nothing else. We measured the quality difference against my own hand-labelling of all 34: the single agent got 27 labels right, the four-way split got 31. Same model, same prompt; the only difference is the stack put in front of it.

The price shows up here too: the detail a sub-agent saw does not come back up. Only its summary does. So the sub-agent’s output must be a record with fixed fields, not free text:

{
  "record_id": "WD-24817",
  "label": "kyc_incomplete",        # closed set: 6 labels + other
  "reason": "Address document is older than 6 months.",  # one sentence, 200 chars
  "confidence": "high",             # high | low
  "records_read": 3                 # how many tool calls it made
}

The confidence field was added later and turned out to be the most useful one: the merge step pulled out the 5 low-confidence records and gave them to a person. Instead of asking the model to be certain, it is cheaper to give it a field where it can say it is not.

Merging: this is the real work

Four summaries arrive and the report does not write itself. What broke in the merge:

• The count did not add up. 34 went out, 33 came back. The first job of the merge step is counting: does the number of parts sent equal the number of records returned? If not, no report is produced and the run fails loudly.
• Two names for one thing. One agent wrote kyc_incomplete, another wrote document_missing. A closed label set fixed this; as long as the set is open, every agent invents its own vocabulary.
• Duplicate records. When two withdrawals of the same customer landed in two agents, the customer was reported twice. Fixing the split key ended it.
• Order. In a parallel run, results come back in random order. If the report is sorted differently every run, you cannot diff two reports. The merge step sorts deterministically, by record id.

None of this is model work: counting, mapping, deduplicating, sorting. The first time I gave the merge to a model, 2 of the 34 records never made it into the report, and it took me two days to notice. When code merges and the same thing happens, the program stops.

Topology: a star, not a chain

In most multi-agent examples online, the agents talk to each other: a “critic agent”, a “writer agent”, a “manager agent”. I tried it once. Two agents — one labelling a record, one reviewing the label — wrote to each other for 9 turns and ended up agreeing on the same wrong label. The reviewer read the reasoning behind the thing it was reviewing, and was convinced by it.

Fix the single agent first

The most important point came last: I went through the 11-minute single-agent run. 6 minutes of it went into fetching the same report three times. With repeat protection and a better tool contract, the single agent went from 11 minutes to 5 minutes 40 seconds — with no parallelism and no extra tokens.

Then I split it four ways: not 3 minutes 12 seconds but 1 minute 50 seconds. In other words, the fix before parallelising was worth more than the parallelising itself, and it was free. Order matters: parallelise broken work and you get four copies of the breakage and four times the bill.

In the last row the tokens are still 2.8 times the single agent. Was it worth it? This job runs once a month and has to be ready for the morning meeting: yes. If it ran 400 times a day, the answer would be no. Cost per task exists to answer exactly this question.

What to watch

• Parallel efficiency. Single-agent time ÷ (number of sub-agents × parallel time). Close to 1 means a good split; under 0.5 means the parts are not independent.
• Duplicate tool call rate. The one number that tells you the split key is wrong. Ours went 18% → 0.
• Tokens per sub-agent. If one is double the others, the parts are unbalanced.
• Records rejected in the merge. If the count does not match, no report is produced; anything above zero has its root cause in the sub-agent contract.
• Share of low-confidence records. The work that reaches a person. If it is rising, the input got harder, not the model.
• Same input, same report. After deterministic sorting, the diff should be clean.

Checklist

Conclusion

Four agents took 11 minutes down to 3 and I was pleased. That was the wrong place to be pleased: the real win came when we found the 6 wasted minutes in the single agent. Parallelism was added on top of that and the result was good — but in the other order we would have made four copies of broken work and tripled the bill.

What I learned when two agents wrote to the same file was not new either: it is a race condition, and model intelligence does not solve it. Sub-agents return their own records, code does the merge, and if the count does not match the program stops. There is no model in that part, and that is a good thing.

The sentence to remember: multi-agent is not a speed technique, it is a splitting technique. If you can split the work, the number of agents is an engineering decision. If you cannot, then however many agents you add, you are doing the same work several times.