Agent Orchestrator: A Guided Flow, Not One Clever Agent

What does the agent loop actually do?

In the in-house agent post there were three parts: the model that decides, the MCP server that runs things, and the agent that manages both. The agent’s job was a loop:

messages = [system_prompt, user_question]
turn = 0
while turn < TURN_LIMIT:
    reply = model.ask(messages, tools)       # the model decides
    if reply.tool_calls:
        for call in reply.tool_calls:
            result = mcp.run(call)           # MCP runs it
            messages.append(result)          # the agent collects
        turn += 1
        continue
    return reply.text                        # plain text: the job is done

This loop worked for report questions: “how many dollars were deposited today?” is one tool, one turn, done. Then we asked it to do work. Reconciliation is not one tool: read the file, fetch the records, match them, label the differences, write a summary. Five steps. When I looked at the loop again, I saw the problem: the order of the steps is not written anywhere. The model picks the order every turn, and not by remembering the earlier turns — by reading the whole message stack again.

From the field: a 17-turn night

I went through that night’s log turn by turn. One agent with 11 tools, one request: find yesterday’s reconciliation difference, work out why, leave a summary.

Total: 17 turns, 9 minutes 12 seconds, 11 tool calls, two of them identical. 9 turns moved nothing forward. The next night I ran the same request again: 12 turns this time, and a different path. The answer was wrong again, but for another reason. That is what really bothered me: the two runs could not be compared. There was nothing to fix, because there was no flow to fix.

Who should decide: the model or the code?

When I sat down to write the flow, I asked one question for each step: should the model decide this? Over time it turned into a three-part test.

For the reconciliation job, the test gave this:

• Which date range is “yesterday”? Finite, and the rule is one sentence: business day 00:00–23:59, Europe/Istanbul. Code. Turn 3 of that night existed because of this; in code it would never have happened.
• Match 2,300 records against 2,287. Reference number, amount, date. Code. One JOIN and one tolerance value. Four model turns went into this.
• Why did 13 records not match? There is a free-text description field: “wrong MT103 ref”, “customer refund”, “fee difference”. I cannot write that rule in one sentence; today it is 13 rows, tomorrow it is a new phrase. Model.
• The summary left for the morning. Natural language, no fixed template. Model.
• If the difference is over $5,000, call the on-call person. A threshold comparison. Code — and because it cannot be undone, the code holds the limit, not the model.

Three patterns

1. Chain: the order is fixed

The steps and their order are fixed, and each step takes the output of the one before. The reconciliation job is this. The model appears in two steps and nowhere else.

# flow.py - the order lives in code, not in the model
def nightly_reconciliation(date):
    bank_file = read_bank_file(date)                # code
    records   = list_transactions(date)             # code
    diff      = match(bank_file, records, tol=0.01) # code: JOIN
    if not diff:
        return "No difference."
    labels = model_label(diff)                      # model: 13 rows, free text
    if diff.total > 5000:
        call_on_call(diff)                          # code: threshold
    return model_summary(diff, labels)              # model: natural language

Here model_label and model_summary are not agents; they are calls: no loop, no tools, limited input. Three of the five steps are already code, and the other two take one turn each.

2. Router: the type of request is unknown

You do not know what the next question in the chat window will be: a report question, a reconciliation request, a KYC lookup? Here the model decides once — which flow to enter — and then the flow takes over.

• The router is one turn, and its output comes from a closed set: report, reconciliation, kyc, unknown.
• “Unknown” must be a real option. Without it the model forces the request into the nearest box; in our first week 3 questions went to the wrong flow, all because the closed set was too narrow.
• If the router picks wrong, the cost is one step, not the whole run: the flow replies “I do not have that data” and comes back.

3. Planner–worker: the number of steps depends on the input

When you say “review every rejected withdrawal from last month, reason by reason”, you cannot know in advance how many steps that needs: it could be 4 reasons or 40. Here the model produces a plan and the code runs it.

• The plan is a data structure, not free text: a list of steps, each with a tool name and arguments.
• The code validates the plan: does the tool exist, are the arguments the right type, is the step count under the limit (ours is 12)? An invalid plan is not run; it goes back to the model once.
• Workers run the plan; when running them in parallel pays off is a post of its own.
• The plan is saved at the start of the run. If the run stops halfway, it continues from that step instead of starting over.

I did not delete the last row, because a free agent is not a bad thing. The bad part is using it for a night job. When I am exploring — “what is in this data?” — with my hands on the keyboard, a free agent works well. At 02:40 every night, nobody is watching.

Four brakes

A guided flow still asks the model somewhere, and the model can still answer wrong. That is why the brakes live in the flow itself, not in the model prompt.

STEP_TURN_LIMIT  = 3        # how many times a step may ask for tools
RUN_TURN_LIMIT   = 8        # total turns in the run
RUN_TOKEN_BUDGET = 120000   # over this, the run stops
STEP_TIMEOUT_SEC = 45       # for a single step

# repeat protection: same tool + same arguments
#   2nd call -> return from cache, do not count a turn
#   3rd call -> end the step with an error

The four work together, because each one catches a different failure:

• The turn limit stops the loop. Not enough alone: you can burn 400,000 tokens in 8 turns.
• The token budget stops the cost. Not enough alone: three steps that each wait two minutes will eat the night without touching the budget.
• The step timeout stops hanging. Not enough alone: it does not see a fast, wrong loop.
• Repeat protection stops the third visit to the same place. That night, turns 9–12 had two identical calls; this brake would have made them one.

When a brake fires, the run does not end quietly. Three things are saved: the partial result so far, which brake fired, and the last step with its arguments. The message in the morning is not “I could not do it” but “I hit the turn limit in step 3, and I have these 13 records”.

From the field: the same job, four steps

After the flow was written. Same night job, same model, same tools:

It ran every night for two weeks. 13 of the 14 runs followed the same 4 steps; one hit the timeout in step 3 (the model server had restarted), the partial result was saved, and I finished it by hand in 2 minutes. The real win is not the time: “why was it slow yesterday?” now has an answer, because there is a fixed flow to compare against.

What to watch

• Turns per run (p50 / p95). A wide spread means the flow is not written down. Ours is 2 / 3.
• Model calls per step. A step above 1 is really two steps; split it.
• Brake hit rate. At 0% the brakes are loose and you are getting more expensive without noticing; above 10% the flow is wrong.
• Same input, same path. The single number for repeatability. Ours is 13/14.
• Tokens and time per run. The input for cost per task; keep it broken down by step.
• Share of runs that reach a person. If it is rising, you are asking too much of the model; expect a falling curve.

Checklist

Conclusion

That night the agent took 17 turns, spent 9 minutes and left a wrong summary. Not because the model was bad, but because we gave it an empty room instead of a task. “Find the reconciliation difference” is one sentence and five steps. Three of those five had a clear rule, and we asked the model all three.

In the guided flow the same job takes 38 seconds, contains two model turns, and follows the same path every night. The speed is nice; what really changed is that it became measurable. When the answer is wrong, I know which step was wrong and I fix that step. In the free agent there was no step to fix.

The sentence to remember: an agent’s intelligence starts after you write the flow. Put everything whose rule you can write into code, leave the rest to the model, and put the brakes in the flow rather than in the model. What is left is exactly where the model is genuinely good.