Silent. Nobody could get the same number twice. The finance lead ran one supplier query through the agent on a Tuesday morning, ran it again after lunch to check the figure, and the second run cost several times the first. Same question, same documents, same specialists, same answer. A third run landed between the two. No error to investigate and no alert to acknowledge, so the matter was closed as a rounding oddity and the forecast you were given stayed in the plan.
What went wrong
The saving on what an agent reads is a saving on repetition, and repetition is judged strictly.
The Cabinet, ArkOne’s reference design for an executive agent, hands the model a Brief before every turn: who the agent is, what it must always do, where it is working, and the list of what it may ask for. Providers charge a much lower rate to re-read an opening stretch of that material when it arrives byte for byte as it did last time. The stretch runs from the first word to a marker, and the design’s budget for those markers, recorded on the Register, spends one of them on the tool list.
That list is where the determinism goes missing, in a way no reviewer catches by reading code. It is assembled at the moment of the request from whatever the agent has available. Gather it into a structure with no order of its own, or build it up as separate parts finish loading, and the contents are the same every run while the sequence is not. A capability that took a moment longer to become available sits further down the list, and the request is then a different request.
The Register’s answer is one phrase long: sorted by name, always, before every request. The paper on the ten times mistake sets out what the cheap rate is worth and how a build keeps qualifying for it.
The record
Suppose a wholesaler of about three hundred people whose agent answers questions about supplier terms. The company, the runs and the amounts are invented for this exercise and modelled from the ratio the Register carries, not taken from an invoice. One question, asked three times on one Tuesday, with the tool list arriving unsorted.
| Run | The order the list arrived in | What the provider saw | Modelled reading bill |
|---|---|---|---|
| Morning | Documents, calendar, supplier lookup, outbox | New material, nothing to match | Illustrative baseline |
| Midday | Documents, supplier lookup, calendar, outbox | New material again; the opening stretch differs from the morning | Roughly the baseline |
| Afternoon | Documents, calendar, supplier lookup, outbox | Matches the morning, which is now too old to be held | Roughly the baseline |
Exhibit 1. Illustrative. One invented conversation run three times on a Tuesday, with the tool list arriving in a different order each time.
Every row did the same work and produced the same answer. What changed was the sequence of four names in a list, settled by whichever part of the system finished loading first.
The right-hand column is why nobody found this. Three amounts, all roughly alike, all higher than expected, with no pattern to hold on to. Had one run come in cheap, the investigation would have gone straight to whatever was different about it. The design failed evenly instead, and an even failure reads as the price of the thing.
The third row matched the morning’s order exactly and still paid the full rate, because a provider holds its copy for a while rather than forever. So a build that leaves the order to chance is never really in a position to earn the discount at all.
What happens
Two identical conversations produce byte-identical requests, so both are billed the same and either can be reproduced.
What your company sees
The action was refused and the reason recorded.
What it means for you
The instinct is to treat this as a billing question and ask for a cap. The cap is worth having, and it answers a smaller question than the one in front of you.
An unreproducible bill is a statement about the request. If one conversation run twice sends the provider two different requests, something in the build is deciding by accident what somebody ought to be deciding on purpose. Today it shows up in money, which is harmless and measurable. The same indifference to sequence can sit in the order material is presented to the model, and what varies then is the judgement.
You cannot forecast what you cannot reproduce. A cost per conversation that moves for reasons nobody can name will not multiply by a volume and go into a plan.
So ask whoever built your agent to run one identical conversation twice and show you both bills. The pass mark is not approximate: two identical conversations should cost the same to the cent, because the cheaper rate either applied to the same material or it did not. A difference of a few percent is the answer, not noise. Then ask what, in code, guarantees the two requests were byte for byte the same up to the marker. A build that has thought about it will say the list is sorted before it is sent.
Next: One check, run twice: the finding that was counted two ways.
Asked plainly
Why does the same AI agent task cost a different amount each run?
Because something in what the agent is handed is not the same each run. Providers charge a much lower rate to re-read an opening stretch of a request that arrives character for character identical to last time, so any part of that stretch which varies withdraws the discount. A list of capabilities assembled in whatever order the parts finished loading is a common source: the contents are identical, the order is not, and the request is therefore not identical.
Does the order of tools in an AI agent's prompt matter?
For cost, yes, considerably. The list of what the agent can do sits inside the stretch of the request that qualifies for the cheap re-reading rate, and that rate depends on the text arriving in exactly the same sequence as before. Sorting the list by name before every request costs nothing and makes the sequence the same every time. Left unsorted, it can differ between two runs a minute apart.
Should I worry if my AI agent's costs cannot be reproduced?
Yes, and not mainly because of the money. An unreproducible bill means two runs of the same work sent different requests, which means something in the build is not deterministic. That same lack of determinism can affect what the agent decides, not only what it costs. The bill is simply the first place it becomes visible, because a bill is written down.