Where the AI money actually goes

Ask engineering leaders what their number one AI topic is right now and the answer is nearly unanimous: cost. Not capability, not safety, not adoption — cost. The technology crossed the usefulness threshold; the invoices crossed a different one. In large enterprises, model spend has become a line item measured in the tens of millions, and it is growing faster than the value story is being written.

This piece is about where that money actually goes, because the honest answer is different from the assumed one — and the difference is where the savings live.

The waste isn’t where you think

The instinct, when the AI bill spikes, is to look at efficiency: are our agents using too many tokens? Could prompts be tighter? Is the harness wasteful? Those questions have answers worth pursuing, but they’re the small money.

Here’s what we see in practice. When an organization rolls AI tools out broadly — hundreds or thousands of people — the vast majority of wasted spend has nothing to do with whether any given agent run is 5% more or less efficient. It comes from ungoverned access: expensive frontier models being used, by default, for tasks that don’t need them. Reformatting a config file with the most expensive reasoning model on the market. Answering a lookup question with a model priced for novel research. Multiply small mismatches by ten thousand users by every working day, and the mismatch is the bill.

The frontier models are genuinely expensive because they’re genuinely expensive to run. The subsidized pricing individual developers enjoy doesn’t extend to enterprise scale — enterprises pay something much closer to true cost. Which means the central cost question isn’t “how do we use AI more efficiently?” It’s “how do we make sure each task runs on the cheapest model that does it well?”

Routing is the answer — and it’s a harness problem

The mechanism for that is model routing: the system decides, task by task, which model to use, instead of every request defaulting to the most expensive option.

The design question is where that decision should live. The tempting answer is a gateway — a layer between your tools and the model APIs that swaps models based on rules. Gateways can capture some gains. But routing done well has to live in the harness — the system actually running the task — for a reason that becomes obvious once stated:

Only the harness knows the state of the work. It knows whether the agent is three steps from finishing a task — in which case finishing on the current cheap model is right, even if the task is hard, because escalating now would waste everything cached and in progress. It knows whether the agent is clearly far from solving the problem — in which case escalating to a stronger model immediately is the economical move, because every further cheap token is a wasted one. It knows what’s in the cache, and that breaking the cache to switch models is sometimes worth it and sometimes ruinous. It knows that switching within a model family preserves more continuity than switching across families, where reasoning traces don’t carry over and the context must be compacted to move.

A gateway sees none of this. It sees a request. The harness sees a task in progress — and cost-optimal routing is a decision about the task, not the request.

The dial, not the switch

In Cortex, routing is configurable rather than binary: administrators choose a point on the cost-quality curve, from conservative (frontier models by default, route down only when confidence is high) to aggressive (cheaper models by default, escalate only when needed). Savings scale with aggressiveness — in aggressive configurations they can reach substantial fractions of total spend — and the honest framing is that this is a trade-off you are choosing, not a free lunch. What the routing removes is the need for every employee to become a model-selection expert; what it preserves is the organization’s right to decide how the trade-off is made.

There’s a further step this enables, and it’s where we think cost governance is heading: every organization has its own task distribution — the actual mix of work its people and agents do. Learn that distribution, measure model performance against it, and the routing decision stops being a rule of thumb and becomes an empirical answer: for your tasks, this is the cost-quality frontier, and this is the right point on it. That’s the direction of travel for our tooling.

The psychology tax

One finding from running these systems deserves its own section, because it explains a lot of enterprise AI spend that otherwise looks irrational.

When people don’t know which model they’re using, they rate the work on the work. When they do know, their assessment changes: quality ratings drift toward the famous models, and preferences fly to the frontier — even for tasks where cheaper models perform identically. Part of what organizations are paying for, at scale, is brand preference operating inside their own workforce.

This isn’t a criticism of anyone’s judgment; it’s marketing doing what marketing does. But it has a practical implication: blind routing isn’t just a cost optimization, it’s a de-biasing mechanism. The organization that routes by measured performance rather than by model fame keeps the quality and drops a large share of the premium. Open models in particular have reached a level where ignoring them is expensive — and where much of the resistance to them is recognition, not performance.

What about the big jobs?

At the other end of the spectrum from ten thousand small tasks sit Missions — large, bounded, verified units of work: a legacy migration, a full product surface. These do cost real money; a single Mission can run from around a hundred dollars into the thousands, driven by orchestration, parallel workers, validation, and repair loops.

The right economic frame for a Mission isn’t a chat subscription — it’s an engagement. You’re commissioning a defined outcome with verification attached, the way you’d commission a firm: with a scope, a quote, and evidence of completion. Cost controls and budget caps exist today; where we’re heading is full cost prediction — being able to tell you, before a Mission runs, what the outcome will cost, and standing behind the number. That’s an engineering problem, not a business-model problem, and it’s one we’re actively working on.

Math is going to win

The through-line of all of this: enterprise AI cost is not a mystery to be endured; it’s a governance problem with known mechanics. Route by task, not by fame. Put the routing where the task state lives. Measure your own distribution instead of trusting benchmarks. Frame big jobs as engagements with verified outcomes.

The organizations doing this are watching their cost curves bend while their output holds. The ones that aren’t are paying a premium composed of defaults, brand preference, and ungoverned access. At tens of millions of dollars a year, that premium doesn’t survive contact with a CFO. As our founder likes to put it: math is going to win.

Questions this paper answers

Why are enterprise AI costs so high? Because enterprises pay close to true inference cost, and by default, expensive frontier models get used for every task — including the majority that cheaper models handle equally well. The waste is concentrated in ungoverned access, not in agent inefficiency.

What is model routing? Automatically selecting which model runs each task based on the task’s needs, instead of defaulting to the most expensive option. Done well, it’s configurable: organizations choose their point on the cost-quality curve rather than accepting a fixed rule.

Why should routing live in the harness instead of a gateway? Because only the system running the task knows its state: whether the work is nearly done, whether escalation is needed, what’s cached, and what a model switch would cost mid-task. A gateway sees isolated requests; cost-optimal routing is a decision about the task in progress.

Do cheaper and open models actually hold up for real work? For a large share of enterprise tasks, yes — and blind evaluations show people often can’t tell the difference until they’re shown the model name. The gap between measured performance and perceived performance is itself a significant cost driver.