- A multi-step agent consumes ten to fifty times more tokens than a simple chat call, because it re-reads its whole context at every step.
- Three items make the bill slip: tool loops, growing context and silent retries.
- Our formula: cost per task = average tokens per task × model price × number of tasks, plus a 20% margin for failures.
- Caching, a smaller model for simple steps and a cap per task divide the bill by three without hurting quality.
Why the demo says nothing about cost
An agent demo handles one task, in front of everyone, with a polished prompt. In production, the same agent handles two thousand tasks a day, some of them badly worded, incomplete or contradictory. It re-runs tools, asks for clarification, starts over. Every retry costs tokens, and tokens are billed.
We saw a client go from a forecast of 300 euros a month to a real invoice of 4,200 euros in the second month. Nothing was broken: the agent did exactly what it was asked, on a volume nobody had measured.
What a chat call costs, and what an agent costs
An exchange with a classic assistant is a question, some context and an answer: a few thousand tokens. An agent chains steps. At each one it receives its instructions again, the history of what it has already done and the results of the tools it called. The context grows, and so does the price.
The three items that slip
Tool loops
An agent that queries a database, does not find what it needs and tries again with another wording can loop five or six times. Each turn replays the full context. Without a cap, one hard task costs as much as fifty easy ones.
Growing context
Tool results are often verbose: a two-hundred-row database extract, a whole web page, a complete PDF. All of it stays in the context window until the task ends. Summarising or truncating intermediate results is the most profitable measure we know.
Silent retries
Orchestration libraries automatically retry a call that fails or times out. Useful, but every retry is billed. At one client we found an 18% retry rate that nobody was watching: nearly a fifth of the invoice for calls that produced nothing.
Set a token cap per task and a maximum number of steps. When the limit is reached, the agent stops, logs the failure and hands over to a human. An agent that knows when to give up costs less than a stubborn one.
The method to estimate before building
We now refuse to price an agent project without measuring three things on a real sample of fifty tasks: the average number of steps, the average context size at each step and the failure rate. One day of prototyping is enough to get them.
| Data point | How to measure it | Observed range |
|---|---|---|
| Steps per task | Prototype log over 50 real tasks | 4 to 12 |
| Tokens per step | Average input + output, from the tracing tool | 8,000 to 20,000 |
| Failure or retry rate | Tasks abandoned or retried out of 50 | 10 to 25% |
| Monthly volume | With the business team, on last month's real data | Often underestimated by a factor of 2 |
The calculation becomes simple: average tokens per task times model price, times monthly volume, plus 20% for failures and retries. If the result makes you jump, that is the right moment to change the architecture, not after three months of billing.
Dividing the bill by three without losing quality
Four levers come back in all our deployments. The first is caching the system prompt and reference documents, which most providers bill far less when repeated. The second is routing: a small model to classify, extract or rephrase, the large model only to reason. The third is summarising intermediate results. The fourth, the most neglected, is measurement: a cost-per-task dashboard, checked every week by someone with the authority to stop the agent.
"An agent whose invoice nobody reads is not in production. It is freewheeling."
Conclusion
Inference cost is not fate, it is a design variable. The projects that last are those where the budget was measured on real tasks before launch, where the agent has a cap, and where someone reads the invoice. The rest is optimism.
FAQ
What monthly budget for a first agent?
For a bounded agent handling 500 to 2,000 tasks a month, expect 150 to 800 euros of inference, depending on context length and the model. Below 500 tasks, a simple automation flow is often cheaper than an agent.
Do we need an open-source model to cut costs?
Not necessarily. An open model you host yourself costs infrastructure and team time. For volumes below a few million tokens a day, a proprietary model with caching and routing usually remains cheaper.
How do we track costs day to day?
Tracing tools such as LangSmith, Langfuse or Arize give the token count per call. Add the model price and group by task: you get the unit cost, the only figure that matters.
What if the budget explodes mid-way?
Cut automatic retries first, cap the steps, then summarise tool results. These three measures take a day to implement and bring most invoices back in line.
