Token Usage Monitoring and LLM Cost Observability in 2026

LLM spend can jump even when traffic barely moves.

Maybe Tuesday’s prompt release added another 1,000 input tokens to every support request. Or even a tool starts timing out and an agent now needs five model calls to finish a task that used to take two. The provider dashboard will show the increase.

It won’t tell you which application change caused it.

Token totals have the same problem. 

1,000,000 tokens spent resolving customer cases might be perfectly healthy. 50,000 burned retrying failed tool calls are waste.

The useful question is what happened around those tokens. Think which workflow created them, and whether anything useful came out the other end.

That’s what cost observability needs to make visible.

When LLM spend moves, an engineer should be able to confidently point to the workload or routing decision that moved it.

LLM cost observability explains why spend changed, not just how much

Start with one expensive request and work backwards:

• Which feature sent it? 

• What model actually handled it? 

• Which prompt version was running? 

• Were there retries before the user got an answer?

• Did routing escalate it to a more expensive provider? 

Those details are what turn a cost number into something an engineer can investigate.

Say support-search spend rises after a knowledge-base release. Token usage alone shows the increase. Once each request also carries its prompt version and outcome, you can see the new retrieval setup is stuffing twice as much context into every call.

Or perhaps token volume barely changes. Yet a fallback rule starts pushing more traffic to a premium model.

In both cases, the useful data sits around the model call and not just inside the token count. Keep enough context to connect spend back to the workload and the decision that created it.

We think cost regressions should be investigated as application regressions first. Something in the retrieval path or agent behavior usually changed before the bill did.

Track the metrics that explain LLM unit economics

A cheap model call can still be part of an expensive workflow.

Take an example where a support agent costs $0.03 each time it calls a model. That sounds healthy until you find that difficult tickets trigger four calls, a fallback, and eventually a human handoff. 

Cost per request stayed low. Cost per resolved case didn’t.

So measure the unit the application is actually trying to produce. For support, that could be a resolved case. For document processing, a validated extraction. If you already evaluate answer quality, a research workflow can track the cost of responses that actually clear that bar.

Then work backwards to see what is moving that number.

Input tokens tell you when prompts, conversation history, or retrieved context are getting heavier. Output tokens expose a different kind of creep. Longer answers, changed formats, or agents generating more intermediate text than they used to. 

Keeping the two separate makes a spike much easier to explain.

You can also break usage down by the model and provider serving a particular project.

Orq.ai’s Reporting API can query daily usage like this:

That breakdown makes it easier to distinguish a workload-driven increase in token usage from a shift in which models or providers are serving the workload.

Track the cost drivers that token totals miss

Cached input deserves the same treatment. 

A prompt can consume roughly the same number of tokens and become much cheaper if more of that context is served from cache, so token volume alone can hide a meaningful change in economics.

Currently, OpenAI prices cached GPT-5.6 Sol input at one-tenth of ordinary input. Caching economics vary by provider. 

Anthropic's explicit prompt caching charges a premium when context is written to cache and discounts subsequent cache reads.

Orq.ai lets you mark reusable Anthropic prompt context for caching directly in the request:

Retries, fallbacks, and routing shifts fill in the rest of the picture. 

A route can look cheap on a price-per-call chart and become expensive because one user task now needs five calls to finish.

That’s the number worth watching.

What the application spends to get useful work done.

Attribute every model call before aggregating the cost

Once 50,000 model calls have been rolled into one provider bill, figuring out who spent the money becomes detective work.

Capture ownership while the request is still happening.

The platform team may run the gateway, but the product team owns the prompt that suddenly doubled in size. A customer-specific workflow may be responsible for a spike even though every request uses the same provider account.

In Orq.ai, that context can travel with the model request itself. 

For example:

Here, name, metadata, tags, thread, and identity attach attribution context to the request before it reaches the model. Those fields can then be used to investigate or aggregate spend without reconstructing ownership afterwards.

Once that context travels with each call, the rollups become useful. You can see spend by team or workflow and put a budget against the part of the system that can actually change it.

Try adding that context after the calls have already been aggregated.

You’ll be left matching invoices back to logs that were never designed to answer the question.

Cost runaways often start with changes in application behavior

A surprisingly small application change can add a lot of model spend.

Add the full conversation history to every support request and input tokens climb immediately. 

Increase the amount of retrieved context sent with each request and the same thing happens.

In a sample of OpenAI coding-agent evals, leaner system prompts reduced total token use by 41–66% and cost by 33–67%. Yet eval scores improved by 10-15%.

Traffic can stay almost flat through all of this.

That’s why the useful warning signs sit closer to the workflow. 

• If tokens per request suddenly jump, inspect the prompt and retrieval path. 

• If one completed task now needs six model calls instead of three, look for retries or loops. 

• If premium-model traffic starts climbing, check which requests are being escalated and why.

These changes are easier to catch when you can line them up against releases and configuration changes.

Alert on changing unit economics before the provider invoice moves

One expensive request usually tells you very little. A support workflow getting 40% more expensive over three days tells you a lot.

Build alerts around the normal economics of each important workflow.

If cost per resolved case jumps, investigate even when overall spend is flat. 

If input tokens per request climb after a release, check what changed in the prompt or retrieval path.

Relative changes are often more useful than a fixed spend threshold. A $500 day might be perfectly normal during heavy traffic. The same spend on half the usual workload is much more interesting.

You also don’t want an alert every time someone uploads an unusually large document. 

Look for changes that persist. Think along the lines of a customer workload drifting away from its usual pattern, or a feature whose cost per completed task keeps climbing.

The best alert tells you that the economics changed while there’s still time to find out why.

We rarely judge a cost improvement in isolation. A route getting 20% cheaper isn’t much of a win if successful task completion falls with it.

Multi-provider cost visibility requires normalized routing context

Using multiple model providers doesn’t automatically create optionality. 

Without a common record of why each request was routed, cost data becomes a collection of incompatible invoices rather than a view of the system.

"The goal is to give teams enough context to understand why a request was routed somewhere and whether that decision was actually better" - Sohrab Hosseini, Orq.ai co-founder

Normalized routing data also exposes hidden spend. A low-cost default model can appear efficient while a poorly configured fallback route sends difficult requests to a premium provider. 

Be careful comparing provider averages, too. If your router sends easy requests to the default model and difficult ones to a premium model, their average cost and success rates are measuring different workloads.

Trace every expensive request from owner to outcome

A $2 model request? Seems perfectly reasonable. 

The real problem is when nobody can explain why it cost $2.

Take an agent that normally completes a task for $0.20. One request suddenly costs ten times as much. 

To find out why, you need the whole path. Who triggered the task, where it retried, and whether the task eventually succeeded.

Keep those events under one trace ID.

For each request, preserve enough context to answer a few basic questions:

• Who owns the spend? Team, application, feature, workflow, and possibly a privacy-safe customer or tenant identifier.

• Where did the request go? Requested model, model actually served, provider, route, fallback, and retries.

• What did it consume? Input and output tokens, plus other billable token categories relevant to the provider and the price applied at the time.

• What happened in the end? Success, latency, evaluation result, escalation, abandonment, or the business event the workflow was trying to complete.

Now an expensive trace can tell a story. 

Maybe one retry became four. 

Maybe a fallback pushed the request onto a premium model. 

Maybe the model call was expensive but resolved a case that would otherwise have gone to a person.

Without that chain, all you have is a large number attached to a model name.

The cheapest model call isn’t always the cheapest application outcome

A cheap model gets expensive when the workflow has to keep fixing its mistakes.

We’ll take a case with 100 support tasks. 

Model A costs $0.02 per call and completes 70 correctly on the first attempt. Model B costs $0.08 and gets 95 right. 

The initial model spend is $2 for Model A and $8 for Model B. On a provider bill, you’d think Model A is the obvious winner.

Now let’s assume every failed task needs a $0.50 fallback or human review before it can be completed.

Model A creates 30 of those failures, adding $15. The 100 completed tasks now cost $17 in total, or $0.17 each. Model B creates five, adding $2.50. Its total comes to $10.50, or about $0.11 per completed task.

The model that costs four times more to call ends up cheaper once you include the work required to finish the task.

Retries and fallbacks add to the cost of completion

The same accounting applies when extra model calls happen for reliability rather than output quality. A single application request can trigger several paid attempts before the workflow gets a usable response.

In that calculation, the following also belong there:

• Retries

• Fallbacks

• Human review

• Extra tool calls

• Additional retrieval

Retries are one way a single application request can accumulate several model attempts. 

In Orq.ai, you can configure the AI Gateway to retry transient provider failures automatically:

Those retries improve resilience, but they still belong in the cost of completing the task.

Filter out routes that miss the workflow’s quality or policy requirements first. Then compare what the remaining routes actually cost once the job is done.

We think of cost optimization as operating inside an eligibility envelope. First establish which routes meet the application’s requirements. 

Only then should price decide between them.

A price-per-token table can tell you which model is cheaper to call. It can’t tell you which one is cheaper to run your application with.

Orq.ai connects model spend to the decisions that created it

In Orq.ai, cost data can stay attached to the request that produced it.

If a workflow suddenly gets more expensive, you can inspect the model and trace behind the increase instead of starting from a provider total and working backwards.

That makes routing decisions easier to judge properly. A premium model might cost more per call and still reduce total spend if it avoids retries or completes more tasks successfully. 

A cheaper route can have the opposite effect.

The useful comparison is the one engineers can make from the trace: what changed, what did it cost, and did the application perform better afterwards?

Make every LLM cost explainable

Maybe a prompt got longer. A fallback started firing more often. An agent began retrying a broken tool.

Or maybe the expensive route was completely justified since it completed more tasks successfully.

Once cost stays connected to the request and outcome, those questions stop being guesswork. Engineers can find what changed and decide whether to fix it or keep paying for it.

See how Orq.ai helps teams trace LLM spend from request to outcome.