The demo that worked too well
The pattern repeats across DACH mid-market organisations with depressing regularity. A promising agentic AI proof-of-concept delivers impressive results in the lab: a customer-service agent that resolves tier-one tickets without human escalation, a procurement assistant that negotiates supplier terms, a compliance reviewer that flags regulatory gaps in contract drafts. The business case looks compelling. The Geschäftsführung approves production deployment. Three months later, the finance team flags a problem: the agent's monthly cloud bill has quadrupled, and the cost per successfully completed task now exceeds what the organisation paid a junior employee to do the same work manually.
The failure mode is not model quality. The agent performs as advertised. The failure is that no one modelled the economic structure of an autonomous system that calls multiple models, retries failed steps, retrieves context from vector stores, invokes external APIs, and occasionally loops into expensive reasoning cycles before arriving at an answer. The innovation budget covered the pilot. No one built a unit-economics model that could predict what happens when the agent runs ten thousand tasks a month instead of fifty.
What the leaderboard does not tell you
Foundation-model benchmarks measure accuracy, reasoning depth, and task completion on curated datasets. They do not measure cost per successfully completed business task in a production environment where the agent must handle ambiguous inputs, incomplete context, and edge cases that require multiple tool calls or escalation to a more expensive model. Organisations across industries are discovering that AI infrastructure costs can spiral as agents proliferate across enterprise systems, with usage growth outpacing declining unit costs even as token prices fall.
The economic structure of an agentic system is fundamentally different from a chatbot. A chatbot receives a prompt, generates a response, and stops. An agent receives a goal, plans a sequence of steps, executes each step (which may involve calling other models, querying databases, or invoking external APIs), evaluates the result, and decides whether to continue, retry, or escalate. Each decision point consumes tokens. Each tool call adds latency and cost. Each retry loop multiplies both. When agentic systems construct prompts dynamically and chain model calls, organisations lose direct control over prompt length and token consumption, making cost prediction significantly more complex.
The result is that the cost per task becomes a distribution, not a fixed number. Some tasks complete in one pass with a small model. Others trigger escalation to a larger model, multiple retrieval steps, and several retry loops before succeeding. A few never succeed and must be routed to a human, incurring both the failed agent cost and the human review cost. The organisation that budgets for the median case will be surprised by the tail.
The hidden cost structure of a production agent
A CFO-grade cost model for an agentic system must account for at least six cost components, each of which can vary by an order of magnitude depending on task complexity and system design. Model inference costs are the most visible: the per-token charge for input and output across all models the agent invokes. In a well-designed system, the majority of tasks route to a small, fast, inexpensive model. In a poorly designed system, every task hits the largest available model because no one implemented routing logic. Tool-call and API costs accumulate whenever the agent invokes an external service: a vector database query, a web search, a CRM lookup, a payment gateway check. Each call has a unit cost, and some APIs charge per request regardless of whether the request succeeds. Retrieval costs include both the vector-database query (charged per embedding and per result returned) and the cost of passing retrieved context into the model's input window. Retrieval is often the largest hidden cost in RAG-augmented agents because the system must retrieve context for every step in a multi-step plan, not just once per task.
Retry and rework costs emerge when the agent's first attempt fails a validation check or produces an output that does not meet the task's success criteria. The agent must re-plan, retrieve additional context, or escalate to a more capable model. Each retry consumes the full cost stack again. Human-review and escalation costs include both the time a human spends reviewing flagged tasks and the opportunity cost of pulling that human away from other work. If the agent escalates 15 per cent of tasks and each escalation takes twenty minutes of a mid-level employee's time, the blended cost per task must account for that labour. Vendor minimums and reserved capacity appear when the organisation commits to a minimum monthly spend or reserved inference capacity to secure lower per-token rates. These fixed costs must be amortised across all tasks; if utilisation drops, the cost per task rises even though the per-token rate stays flat.
The organisations that discover these costs in production, rather than modelling them before deployment, face an unpleasant choice: accept a cost structure that makes the agent uneconomical, re-architect the system to reduce costs (which delays ROI and demoralises the team), or quietly throttle the agent's usage and return to manual processes.
Model routing is not an optimisation; it is the business model
The single most effective cost-control lever in an agentic system is intelligent model routing: the ability to send each task to the smallest, fastest, cheapest model capable of completing it, and escalate only when necessary. A well-designed routing layer can reduce total inference cost by 40–70 per cent compared to a naïve system that routes all tasks to the largest model, based on typical tier-one task volumes of 40–60 per cent. The challenge is that routing decisions must be made in real time, based on task characteristics that are not always obvious from the initial input.
A useful heuristic is to classify tasks into three tiers. Tier-one tasks are high-volume, low-ambiguity requests that follow a known pattern: checking an order status, extracting structured data from a standard document, answering a frequently asked question from a knowledge base. These tasks can be handled by a small, fast model with a tightly scoped prompt and minimal retrieval. The cost per task should be measured in fractions of a cent. Tier-two tasks require moderate reasoning, multi-step planning, or retrieval from multiple sources: drafting a response to a non-standard customer inquiry, summarising a complex document with specific extraction requirements, validating a contract clause against regulatory guidance. These tasks justify a mid-sized model and a more expensive retrieval step, but still aim for single-digit cents per task. Tier-three tasks involve open-ended reasoning, ambiguous goals, or high-stakes decisions where error cost is unacceptable: negotiating terms with a supplier, drafting a legal opinion, designing a custom solution for a client. These tasks may require the largest available model, multiple reasoning passes, and human review. The cost per task can reach several euros, which is acceptable only if the task's value justifies it.
The routing logic must be explicit, instrumented, and continuously refined. If the system cannot explain why it routed a given task to a given model, the organisation cannot optimise the decision. If the system does not log the cost and outcome of every routing decision, the organisation cannot measure whether the routing logic is working. Foundation-model vendors are increasingly competing on cost efficiency for agentic workloads, positioning newer models as capable of handling autonomous tasks that previously required larger, more expensive alternatives. The implication is clear: the organisations that master routing will enjoy a structural cost advantage over those that do not.
Why the first production agent is the hardest
The first production agent forces the organisation to confront questions it could defer during the pilot phase. What is the acceptable cost per successfully completed task? If the answer is "we don't know," the agent will be built without a cost constraint, and the organisation will discover the answer by accident when the bill arrives. What is the acceptable error rate, and what is the cost of each error? If the agent misroutes a customer inquiry, the cost is a follow-up ticket and minor customer friction. If the agent misclassifies a regulatory filing, the cost could be a five-figure fine and reputational damage. The acceptable error rate must be set in relation to error cost, not in isolation. What is the escalation threshold, and who owns escalated tasks? If the agent escalates too often, it becomes an expensive triage tool rather than an autonomous system. If it escalates too rarely, it will produce low-quality outputs that require rework. The escalation threshold must be tuned to the organisation's risk tolerance and the availability of human reviewers.
What is the retry budget? An agent that retries failed steps indefinitely will eventually succeed at most tasks, but the cost per task will be unbounded. A retry budget sets a maximum number of attempts (and a maximum cost) before the agent escalates or fails. What is the model-selection policy? If the policy is "always use the best available model," costs will be high and routing will be trivial. If the policy is "use the cheapest model that meets the success criteria," costs will be lower but the system must implement success criteria and measure compliance. What is the instrumentation and observability strategy? If the organisation cannot see which tasks are expensive, which routing decisions are suboptimal, and which failure modes are common, it cannot improve the system. Observability is not a nice-to-have; it is the feedback loop that makes unit economics manageable.
The organisations that answer these questions before deploying the first production agent build systems that scale economically. The organisations that defer the questions build systems that work in the lab and fail in the finance review.
The CFO conversation you will have eventually
The conversation happens three to six months after the first production agent launches. The finance team presents a chart showing monthly AI infrastructure costs rising faster than task volume. The Geschäftsführung asks whether the agent is delivering ROI. The AI lead explains that the agent is performing well by technical metrics—accuracy, task completion, user satisfaction—but the cost per task is higher than expected because of retry loops, escalation rates, and unanticipated retrieval costs. The CFO asks why these costs were not modelled in the business case. The AI lead explains that the pilot ran on a small dataset and did not encounter the edge cases that drive costs in production. The CFO asks what the plan is to bring costs in line with the original business case. The AI lead does not have a plan because the system was not instrumented to identify cost drivers or test routing alternatives.
This conversation is avoidable. It requires building a unit-economics model before the agent enters production: a spreadsheet or simulation that estimates cost per task across the expected distribution of task types, models the impact of routing decisions and retry policies, and stress-tests the economics against realistic utilisation and error rates. The model does not need to be precise—production will always surprise—but it must be explicit. It must force the team to state their assumptions about task mix, model costs, escalation rates, and human-review time, and to calculate the blended cost per task under those assumptions. If the blended cost is acceptable and the assumptions are reasonable, the agent can proceed to production with a cost budget and a monitoring plan. If the blended cost is unacceptable, the team can redesign the system, adjust the task scope, or conclude that the agent is not yet economical.
The alternative is to discover the unit economics by running the system at scale and reading the bill. That discovery is expensive, demoralising, and often leads to the agent being quietly throttled or retired. The organisations that treat unit economics as a design constraint, not a post-launch audit, build agents that survive contact with the finance team.
What a unit-economics budget looks like in practice
A practical unit-economics budget for an agentic system begins with a task taxonomy: a list of the task types the agent will handle, the expected volume of each type, and the target cost per task. For a customer-service agent in a DACH Mittelstand firm, the taxonomy might include: order-status inquiries (40 per cent of volume, target cost €0.02 per task), product-information requests (30 per cent, target cost €0.05), returns and refunds (15 per cent, target cost €0.10), complaints and escalations (10 per cent, target cost €0.50 including human review), and edge cases (5 per cent, no cost target because these will be routed to humans). Edge cases (5 per cent) are excluded from the blended cost as they route immediately to human handling at existing labour costs, adding no incremental AI infrastructure expense. The blended target cost per task, weighted by volume, is approximately €0.09. If the agent handles 10,000 tasks per month, the target monthly cost is €900.
The budget then allocates that €0.09 across cost components. A reasonable starting allocation might be: €0.03 for model inference (assuming most tasks route to a small model with occasional escalation), €0.02 for retrieval (one or two vector queries per task), €0.01 for tool calls and API access, €0.01 for retry and rework (assuming a 10 per cent retry rate with an average of 1.5 retries per failed task), and €0.05 for human review (5 per cent escalation rate, 10 minutes at €60/hour blended cost). The allocation is a hypothesis. The system must be instrumented to measure actual costs per component and per task type, and the allocation must be revised as production data accumulates.
The budget must also include a variance analysis: what happens if retry rates are higher than expected, if escalation rates double, if a vendor raises API prices, or if task volume spikes. A robust budget models at least three scenarios—base case, pessimistic case, optimistic case—and confirms that the agent remains economical even in the pessimistic case. If the pessimistic case breaks the business case, the system design must change before deployment.
The organisations that build and monitor unit-economics budgets discover cost problems early, when they can be fixed with routing changes or retry-policy adjustments, rather than late, when the only options are expensive re-architecture or project cancellation. The budget is not a constraint that limits innovation; it is a feedback mechanism that ensures innovation is economically sustainable.
A Fit Call walks through your first production agent's task taxonomy, cost structure, and routing logic—before the finance team asks why the bill is climbing faster than task volume.