The conversation founders avoid is not the one about whether to build with agentic AI. That conversation has largely been settled. Most teams have approved a build or are close to it. The conversation they avoid is the one about what it will cost to run after it ships. Most production cost models for agent systems are built from demo economics and adjusted upward slightly. That is the wrong starting model, and the adjustment is usually by the wrong factor.
Token prices have fallen by more than 85% since GPT-4 launched in 2023. At the same time, the monthly bill for running an agent in production has increased substantially for many teams. Both of those things are true and they are not contradictory. The cost that surprises teams is not the price per token. It is the number of tokens an agent consumes per task compared to what a chat product consumes per interaction. Those numbers are structurally different, and understanding that difference is the infrastructure conversation most founders skip.
Why token prices falling and agent costs rising are both true
The mechanism is the reasoning loop. A chatbot sends a message, receives a response, and stops. An agent executes a loop – plan, select a tool, call the tool, read the result, update its context, decide the next step, and repeat until the task is complete. Every step in that loop sends the entire accumulated context to the model. By step ten, the agent is paying for the same system prompt and all previous tool call results ten times.
EY’s 2026 analysis of enterprise agentic AI token costs quantified this shift precisely – a simple 2023 linear workflow cost approximately $0.04 per interaction. A 2026 orchestrated system involving tools, reasoning, and iterative loops costs approximately $1.20 per interaction, roughly 30 times higher, even though per-token prices fell over that same period. EY’s characterisation of this dynamic is worth the attention of any founder building a cost model – these costs stay structurally invisible until specifically designed for visibility.
The 30x figure represents a typical orchestrated workflow. The range across agent types is wider. Simple tool-calling agents consume 5,000 to 15,000 tokens per task. Complex multi-agent systems can consume 200,000 to over 1,000,000 tokens per task. A 10-turn research agent accumulates 500,000 or more input tokens per task if context is not actively pruned, because long-context models bill linearly for every token in the window. Gartner estimates that agentic systems consume 5 to 30 times more tokens per task than chat, and a 2026 study of coding-agent tasks found the variance across runs of the same task can reach 30x, because the number of tool calls and retries is determined by the task’s complexity at runtime, not at planning time. That variance is what makes agent cost forecasting genuinely difficult and why models built from average-case assumptions produce systematically wrong budgets.
The multi-agent cost multiplier
Every agent added to a system adds its own token consumption. More critically, the orchestrator that coordinates them adds a token cost on top of each worker agent’s token cost – it must reason about which task to delegate, parse the worker’s output, decide whether the output is sufficient, and determine the next step. In a five-agent system where the orchestrator calls each worker once, the effective token cost is the sum of all five workers’ costs plus the orchestrator’s reasoning cost across five decision cycles. If any worker produces an output the orchestrator judges insufficient and retries, that cost is added again.
The retry factor is the most underestimated variable in multi-agent production cost. A single tool call timeout, a partial retrieval result, or an ambiguous output from a worker agent triggers a retry that re-sends the full context. In a ten-step workflow with a 20% retry rate per step, the expected number of LLM calls is not ten. It is closer to fourteen. Over millions of tasks per month, the difference between a 10% and a 20% retry rate can double the monthly inference bill.
The output token cost amplifies this further. Output tokens are priced at 3 to 5 times the rate of input tokens across all major providers in 2026, with some premium reasoning models reaching an 8 to 1 ratio. An agent that thinks out loud in a verbose chain-of-thought before producing a structured output is paying the premium output token rate for every reasoning step, not just for the final response. Structured output schemas that constrain the model to produce concise, typed responses instead of verbose free-text reasoning are one of the most effective cost controls available, and one of the least consistently implemented.
The four cost buckets most agent budgets miss
Token cost is the visible part. Most agents in production carry three or four additional cost buckets that rarely appear in a demo cost estimate.
Vector database and retrieval infrastructure
A RAG-grounded agent requires a vector database that is continuously updated, queried at every task execution, and scaled with the knowledge base. At low volume this is negligible. At production scale, vector database hosting costs $500 to $2,500 per month depending on corpus size and query volume, and query latency at scale can become a meaningful fraction of total task time if the retrieval infrastructure is not right-sized.
Observability and tracing
Running a multi-agent system in production without cross-agent tracing is running it blind. A distributed tracing infrastructure that captures every agent invocation, every tool call, every handoff, and every token spent across the full execution graph is not optional once the system is handling real users. The tooling to build and maintain that infrastructure carries both engineering time and hosting cost that almost no pre-launch budget accounts for.
Human review and quality sampling
A responsible production deployment includes a regular accuracy sampling cadence – a named owner reviews a sample of agent outputs against a quality standard on a scheduled basis. If that review catches systematic errors that require rework of the agent’s knowledge base, prompt structure, or routing logic, the engineering time to remediate is a real operational cost. The same budgets that account for model API costs rarely account for the engineering time to keep the system accurate after the first deployment.
Retry and rework costs
When an agent produces an output that is wrong, the cost of that failure is not just the tokens consumed on the failed run. In a customer-facing system, a wrong output may require a human support intervention that costs multiples of the original task cost. In an internal system, a wrong decision that propagates before it is caught requires manual correction that may exceed the efficiency gain the agent produced. These downstream costs are real parts of total cost of ownership that a token-count model cannot capture.
How to model production cost before the first invoice
The formula that produces a realistic production cost estimate is not units times token price. It is expected task volume per month, multiplied by steps per task, multiplied by a retry factor calibrated to the specific workflow’s complexity, multiplied by the token cost per step, then add retrieval, observability, human review, and rework as separate line items.
Before that formula can produce a useful number, one measurement is essential – the actual token cost per task on your specific workflow with your specific prompting and tool configuration, measured on real production representative inputs rather than on the clean inputs used during development. Demo inputs are shorter, cleaner, and less ambiguous than production inputs. They consume fewer tokens and trigger fewer retries. The gap between demo measured and production measured token cost per task is consistently larger than teams expect, and closing that gap before launch is significantly cheaper than discovering it on the first month’s invoice.
The measurement protocol is straightforward – run a representative sample of 50 to 100 real inputs through the agent in a staging environment instrumented with token counting. Log the total token cost including all retries and tool calls, not just the primary LLM call. Use the 90th percentile of that distribution as the planning number, not the mean. Production cost is driven by the high cost tail, not by the average case.
Budget and mid tier models cover 70 to 80% of real agent workloads on accuracy. For the majority of high volume tasks in a production agent, a less expensive model produces equivalent results at a fraction of the flagship model’s cost. Routing high volume standard tasks to a smaller model and reserving the most capable model for the specific query types that genuinely require it is one of the most effective cost controls available, and one that requires a measurement baseline to implement responsibly.
What changes when you go multi-agent on cost
A multi-agent architecture does not simply multiply the cost of a single agent. It introduces coordination overhead that a single agent model does not carry, and it creates a new class of cost risk – orchestrator failures that consume significant token budget before producing no useful output.
An orchestrator that misdecomposes a task sends worker agents on incorrect subtasks. Each worker consumes its full token budget executing a task that contributes nothing to the correct final output. The orchestrator then processes each worker’s output, determines the decomposition was wrong, and potentially retries, consuming token budget again on a second orchestration pass. This failure mode is most common in the early weeks after a multi-agent system reaches production, when the orchestrator encounters input distributions that were not well represented in testing.
Setting per task token budgets with hard limits, a maximum token cap per agent call that triggers an escalation to a human rather than an unlimited retry loop, is the single most effective control against runaway cost in a multi-agent system. It is also the control most commonly absent from first production multi-agent deployments because the developers who built the system know how it behaves on the inputs they tested and do not expect the unexpected inputs that production surfaces.
Gartner projects that 40% of agentic AI projects will be canceled by the end of 2027, with escalating costs cited as one of the primary causes. The teams that avoid that cancellation are not the ones that built cheaper systems. They are the ones that understood their cost structure before production, set hard budget limits per task, and measured actual production cost against a pre-built model rather than discovering the gap on an invoice.
Plan your AI infrastructure before production
The infrastructure conversation founders avoid is not technically difficult. It is just uncomfortable to have before the first production deployment, when optimism about the build is at its highest and the evidence for the cost is still in the future. The teams that have that conversation early, measure actual token cost on real inputs before they commit to production volumes, and set hard per-task limits before the first invoice are the ones most likely to build AI systems that remain practical, predictable, and economically sustainable over the long term.
If you’re planning to deploy an agentic AI system and want to validate your production cost model before launch, get in touch with our team. We can help you estimate token consumption, evaluate infrastructure requirements, and implement practical cost controls that support long term scalability.
Your queries, our answers
Output tokens are generated by the model one at a time during inference, which is computationally more expensive than processing input tokens in parallel. Providers pass that cost difference to consumers. The implication for agent design is that every chain-of-thought step, every verbose reasoning trace, and every long free-text response is being charged at the higher output rate. Structured outputs that constrain the model to produce concise, typed responses rather than verbose free-text reasoning materially reduce the output token bill.
Run 50 to 100 real production-representative inputs through the system in a staging environment with full token instrumentation. Log the total token cost per task including all tool calls and retries, not just the primary LLM call. Use the 90th percentile as your planning number. That figure, multiplied by expected monthly task volume, plus vector database, observability, and an assumed human review rate, is a realistic production cost estimate.
Model routing: identify which query types genuinely require the most capable model and which can be handled by a smaller, less expensive model at equivalent quality. In most production deployments, 70-80% of tasks fall into the latter category. Structured output schemas to prevent verbose chain-of-thought output. Context pruning to prevent accumulated tool call results from inflating the context window over multi-turn tasks.
Yes, and it usually should tilt toward single-agent for first deployments. A single-agent system has one retry factor, one token budget to manage, and one orchestration layer fewer. The coordination overhead in a multi-agent system adds cost and cost variance that a single agent does not carry. The architecture decision and the cost model should be made together, not sequentially.
It depends entirely on the task complexity and business value of the task. A support triage agent resolving routine queries might cap at 10,000 to 15,000 tokens per task. A research synthesis agent might cap at 200,000. The cap is not about the absolute number. It is about defining a threshold at which continuing is not worth the cost, and ensuring that threshold is encoded as a hard limit before the system goes live, not discovered as a budget line item after the fact.
What happens after you fill-up the form?
Request a consultation
By completely filling out the form, you'll be able to book a meeting at a time that suits you. After booking the meeting, you'll receive two emails - a booking confirmation email and an email from the member of our team you'll be meeting that will help you prepare for the call.
Speak with our experts
During the consultation, we will listen to your questions and challenges, and provide personalised guidance and actionable recommendations to address your specific needs.
