Home → Engineering
Tool Design: A Decision Surface, Not an API Wrapper
In three weeks we grew from 11 tools to 23. Each one had been asked for, each one worked. Then I measured: the rate of picking the right tool had fallen from 96% to 71%. Not one line of code had broken. The menu we put in front of the model had grown — and the menu is the prompt.
- A tool is not an API. A person reads an API and can open the docs; a model reads a tool and picks in one shot. Its name, description and parameters are part of the prompt.
- Every new tool makes selection harder. 20/20 with 5 tools, 26/27 with 11, 17/24 with 23. Same code; the only thing that grew was the number of options.
- Separate decision, or parameter? That is the merge rule. We cut 23 tools to 9, accuracy reached 94%, and no capability was lost.
- A description is four lines: what it does, when to use, when not to use (and which one instead), one example question. The “when not to use” line helped most.
- An error message is an instruction. A model that sees “400 Bad Request” repeats the call; a model that reads “the range cannot exceed 31 days, narrow it and call again” fixes it.
- The tool does the trimming. A tool that returns 2,300 rows poisons the rest of the run. A tool’s job is not to move data; it is to enable a decision.
The difference between a tool and an API
In the in-house agent post, the MCP server ran the tools, and the first rule was there: on a small model, the docstring decides tool choice. Back then there were 11 tools and they routed 26 of 27 questions to the right one. Then the report requests came in, and we wrote a new tool for each request. That was the mistake.
| API | Tool | |
|---|---|---|
| Who calls it | The developer writing the code | The model, in one shot, at question time |
| Can it read the docs | Yes, on a separate page | No; all it sees is name + description + schema |
| If it picks wrong | A compile or test error | A quietly wrong answer |
| Are extra ones harmless | Yes, they go unused | No: they take space every turn and can be picked |
| Naming | Consistency is enough | Distinctiveness is required |
From the field: 23 tools, 71% accuracy
The measurement was simple: 24 real questions, each with the correct tool written down by hand. Run the agent and look at the first tool it picks.
| Number of tools | Correct tool choice | Most common mistake |
|---|---|---|
| 5 | 20/20 | — |
| 11 (after the docstring fix) | 26/27 | Balance tool used for a volume question |
| 23 | 17/24 | list_transactions confused with transaction_summary |
| 9 (merged) | 22.5/24 — average of two runs | Missing date range in KYC |
When we hit 23 tools I looked at the pairs that got confused; they all had the same disease:
list_transactions/transaction_summary/transaction_detail— all three start with the same word, and the difference is in the third sentence of the description.daily_deposits/deposit_report— one takes a day, the other a range. To see that, the model has to read the parameter list; it does not.pending_withdrawals/withdrawal_queue— the same thing, written twice by two people two weeks apart.
The merge rule: separate decision or parameter?
I decided whether to merge two tools with one question: is the user choosing between these two, or asking the same question with a different scope? If it is the same question, it is one tool with a parameter.
# BEFORE: three tools, all starting with the same word
daily_deposits(date)
deposit_report(start, end)
deposit_summary(month)
# AFTER: one tool, closed-set parameter
def deposits(start: date, end: date,
group_by: Literal["day", "month", "total"] = "total"):
"""Returns deposit totals for the given date range.
WHEN TO USE: "how many dollars were deposited today", "how much came in
last month", "weekly deposit breakdown".
WHEN NOT TO USE: for withdrawal questions; use withdrawals() instead.
EXAMPLE: "how many dollars were deposited today?"
-> deposits(start=today, end=today, group_by="total")
"""
With this change, 23 tools became 9: deposits, withdrawals,
positions, volume, balance, kyc,
reconciliation, customer, transactions. No capability was
lost; the same capabilities come through fewer doors. Accuracy went from 71% to 94%.
The opposite is also true: if two jobs really are different decisions, forcing them into one tool
is also wrong. withdrawals(...) and approve_withdrawal(...) cannot be
the same tool, because one reads and the other writes; calling one wrongly is a reporting error,
calling the other wrongly moves money.
Five parts of a good tool contract
1. Name: verb + object, distinctive
Not get_data, not withdrawals alone — list_withdrawals.
If three tools start with the same prefix, the model treats them as the same door. Use a different
verb instead of a shared prefix.
2. Description: four lines
What it does · when to use · when not to use (and which one instead) · one example question. The “when not to use” line was the most valuable one for us: for every confused pair we named the other tool in both descriptions, and the confusion stopped.
Write the example in the user’s words, not the engineer’s. Not “aggregate deposit volume” but “how many dollars were deposited today?”
3. Parameters: closed sets and defaults
A free-text parameter is a field the model can invent. When the grouping is
Literal["day","month","total"], it cannot invent a fourth value. Give every
parameter a sensible default: the fewer fields the model has to fill, the better it does. Adding
a default to group_by alone fixed 2 questions for us.
4. Reply: trimmed, shaped, counted
A tool’s reply goes straight into context. The rule: numbers first, then a sample, then a continuation key. Never the raw list. Always include “how many records exist”, so the model does not see 5 sample rows and write “there were 5 transactions”.
5. Errors: what was wrong, which field, what to do now
# BEFORE
{"error": "400 Bad Request"}
# the model retried the same call, then gave up
# AFTER
{"error": "date_range_too_long",
"message": "The date range can be at most 31 days. It is currently 92 days.",
"action": "Pick a start date within 31 days and call again.",
"field": "start"}
# the model made the correct call on the next turn
There is one more rule for error messages: never echo user text back as is. An
error that says “{input} is invalid” carries whatever the user wrote into
the model. That is a prompt injection path; truncate the
input, quote it, or leave it out.
Tools with side effects are a separate class
Eight of our nine tools only read. The ninth — approve_withdrawal —
moves money, and it lives under completely different rules:
- The model may call them as often as it likes.
- Repeats are harmless; they can come from cache.
- An error costs one turn.
- No approval needed.
- Marked in the schema:
"side_effect": true. - An idempotency key is required.
- They do not run without human approval; the approval lives in the flow, not the prompt.
- Every call is written to the audit log: who, when, with which arguments.
In the first week, before we separated them, the agent read “review the pending withdrawals” as “approve” and approved 3 records. It was a test environment and there was no money in it; the lesson was free. Putting reads and writes side by side on the same menu is not the model’s fault.
An unused tool is not harmless
Six of the 23 tools were never called in three months. The ones we kept “in case we need them one day”. What they cost:
- Their definitions enter the window every turn: 6 tools × ~210 tokens = 1,260 wasted tokens per run.
- Each one carries a chance of being picked wrongly; two of them actually were.
- A new person reading the menu cannot tell which ones are live.
The rule is simple: a tool not called once in 30 days leaves the menu. The code stays, the tests stay; it is just not offered to the model. Adding it back is one line.
What to watch
- Tool-choice accuracy. On a hand-labelled set of 20–30 questions; rerun it whenever anything changes (the eval post).
- Error rate per tool, and recovery after an error. Can the model make the right call after a failure? If not, the error message is bad.
- Empty-result rate. A tool that keeps returning nothing is answering the wrong question.
- Reply size, p95. This number names your fattest tool; trimming starts there.
- Number of uncalled tools. The input for the 30-day rule.
- Confused pairs. Which two show up in wrong choices; write the “do not use” line for that pair.
Checklist
- Is this a separate decision, or a parameter of an existing tool?
- Does its name share a prefix with any existing tool?
- Does the description have a “when not to use” line? Does it name the alternative?
- Is the example question a sentence a user actually wrote?
- Are the parameters closed sets? Do they have defaults?
- Is the reply trimmed? Does it include the total count?
- Does the error message say what to do? Does it echo user text?
- Does it have a side effect? If so, is it marked, keyed and approved?
- Did you rerun the accuracy measurement after adding it?
- Is there a tool on the menu that has not been called in 30 days?
Conclusion
In three weeks we added 12 tools and made the system worse. None of them was faulty; together they were. The model reads the menu we wrote and picks in one shot; as the menu grows and the headings start to look alike, the picking breaks down.
When we cut back to 9 tools we lost no capability: the same jobs are done with parameters. Accuracy went from 71% to 94%, tokens per run dropped, and a new person can read the menu and see what the system can do. None of this was a model change; all of it was text.
The sentence to remember: a tool’s code decides what it does, and its description decides whether it is chosen. Writing the first without the second gives you a function that works and is never found.