
Auditing AI Agent Token Spend: Context, Tool Output, Retries, and Routing
On this page
Introduction
This is an audit playbook for AI agent token spend: how to trace where a budget actually goes — repeated context, tool output, retries, and model routing — and how to measure cost per successful task instead of cost per call, with instrumentation code, a ledger schema, and a worked trace you can reproduce.
The claim that prompted it landed in November 2026, when three Korean business outlets reported that LG Uplus is pitching token optimization to enterprise buyers — a way to "lower total token costs" and cut AI agent operating expenses. None of the reporting carried the numbers behind that claim.
The gap is the interesting part. "Total token cost" is not a number you can act on, because per-call accounting hides most of what an agent actually spends. The bill comes from turns, from re-sent context, from tool payloads nobody truncated, from retries re-running work you already paid for, and from routing decisions made out of habit. Measured per call, each of those looks like rounding error. Measured per successful task, they are the whole bill.
Thesis: cost per model call is the wrong unit for agent spend; cost per successful task is the only number that survives contact with a real workload.
What the LG Uplus Token Optimization Story Actually Claims
The B2B positioning and the "lower total token costs" framing
What the reporting states, as of 2026-11-15:
- Chosunbiz framed the move as LG Uplus entering the B2B market with token optimization, with agent cost reduction as the selling point.
- Asia Economy led with "lower total token costs" and named corporate customers as the target.
- Digital Today described the company accelerating token-efficiency research to cut service costs — an R&D framing, not a shipped-product framing.
Three headlines, three slightly different claims: a commercial offering, a cost claim, and a research program. All of them are consistent with a single effort, but none settles whether that effort is a product or a project. I read the headlines and snippets, not the full articles; the text may say more than the snippets do.
What the reporting does not specify about the savings
The snippets contain no benchmark, no pricing model, no methodology, no list of covered models or providers, and no per-task figures. They offer a claim ("lower total token costs") and a direction (enterprise cost engineering) — nothing an outside engineer could reproduce or falsify.
A vendor claim that something lowers "total token cost" is unfalsifiable until the success rate is pinned to it. A layer that halves your token spend by making 15% of tasks fail is not an optimization; it is a refund you have not processed yet.
My position: the interesting question is not whether token spend can be reduced — it clearly can, as the rest of this post shows — but whether the reduction is measured against a fixed denominator. Ask any external optimization layer one question before anything else: what happens to cost per successful task, and what happens to success rate? If the vendor cannot answer both parts, the number is marketing.
Why Cost per Call Is the Wrong Unit for Agent Token Spend
The metric that hides your failures
Cost per call behaves badly under failure. An agent that answers correctly in six turns and an agent that flails for twenty and gives up both issue model calls; averaged across them, cost per call stays roughly flat. The failing run is cheaper per call and more expensive per task, because it produced nothing.
That is the structural defect. Cost per call is a numerator without a denominator. It will always improve when your success rate drops.
Defining cost per successful task: why the denominator matters more than the numerator
The denominator cannot be decided by the model. "The agent said it was done" is not a success criterion — models are systematically overconfident about completion. Use something executable: the record exists in the target system with the right fields, the test suite passes, the schema validates, the reviewer accepted the draft without edits.
Write the rule down next to the task, before the run. If you cannot write it down, you are not measuring cost per successful task; you are measuring cost per conversation.
A formula you can put in a dashboard today
cost_per_successful_task(task_class) =
sum(cost of all spans in all tasks of that class)
/ count(tasks in that class where the success rule evaluated true)
Two companion metrics expose what the ratio hides:
- wasted_spend_ratio = spend on tasks that failed / total spend. Above ~20% means your agent is mostly generating error traces at scale.
- p95 task cost = task cost at the 95th percentile. Averages hide the escalation storms that actually blow the budget.
Where Agent Token Budgets Actually Go: Context, Tool Output, Retries, and Routing
Repeated context — the same system prompt re-sent on every turn
Every turn re-sends the whole message array: system prompt, tool schemas, all prior turns. A 3,000-token prefix across a 12-turn run is 36,000 input tokens of identical bytes.
prefix tokens = 3,000
turns = 12
re-sent prefix = 3,000 x 12 = 36,000 input tokens
at $3.00 / 1M = $0.108 per task, for bytes that never changed
Prompt caching cuts the price of those 36,000 tokens to somewhere between 10% and 50% of the input rate, depending on provider and model. It does not cut the count. Two consequences follow: your token counter still has to record them, and a cache miss is silent. The run succeeds, the answer is correct, and the invoice is 3x higher with nothing in the logs.
Tool output — verbose JSON, pagination, and full documents stuffed into the window
The most common budgeting failure I see is tool results pasted into context at whatever size the upstream API returned. A search tool returning 40 results with metadata, or a document fetch returning a 12,000-token record when the agent needed one field, inflates every later turn in the run — the result is now history and gets re-sent.
Retries, loops, and duplicate spend on work you already paid for
Retries are the worst kind of spend: full-price tokens for a repeat of work that already failed. They are also invisible in cost-per-call averages, because a retry is just another call. Worse, retries often escalate to a more expensive model, so 6% of your calls can be 40%+ of your bill.
Model routing — the escalation ladder and its silent promotion of cheap turns
A router that starts every task on a cheap model and escalates on failure is a reasonable design with an expensive failure mode. The cheap attempt's tokens are not refunded when it escalates, and the escalated attempt re-sends the entire context at the expensive model's rate. You pay for the wrong answer and the right one.
The subtler problem is that routing gets decided once and never revisited. A rule like "math goes to the small model" written in month one will still be running in month six, long after the task mix changed.
Instrumenting a Single Agent Run for Token Accounting
A minimal span model (task → turn → model call / tool call)
Four record types, one parent pointer, one retry counter. That is enough to answer every question in this post.
| Level | What it records | Why it exists |
|---|---|---|
task | task class, success rule, outcome | the denominator |
turn | an iteration of the agent loop | groups calls into iterations |
model_call | provider, model, usage, component, retry depth | the spend |
tool_call | tool name, argument size, result size | the input driver |
Capturing usage from provider responses instead of estimating
Estimate tokens with a tokenizer and you will be wrong in a direction that matters: cached tokens, reasoning tokens, and tool-schema preambles are all billed in ways a naive character count does not see. Read usage off the response.
The shapes differ enough to break naive code. Per the providers' current API docs (verify against the SDK version you actually run — these fields change):
- OpenAI-shaped responses report
prompt_tokensincluding cached tokens, with the cached portion inprompt_tokens_details.cached_tokens, and reasoning tokens incompletion_tokens_details.reasoning_tokens. - Anthropic-shaped responses report
input_tokensexcluding cache reads, which surface separately incache_read_input_tokensandcache_creation_input_tokens.
One provider includes cached tokens in the input total; the other excludes them. That is the double-count waiting to happen.
Code: a wrapper that logs input, cached input, output, and retry depth
// One usage shape for every provider. Written against the field names
// documented at the time of writing; re-check after any SDK bump.
export function normalizeUsage(res) {
const u = res.usage || {};
if (typeof u.prompt_tokens === "number") {
// OpenAI-shaped: prompt_tokens already includes cached tokens.
return {
inputTokens: u.prompt_tokens,
cachedInputTokens: u.prompt_tokens_details?.cached_tokens ?? 0,
outputTokens: u.completion_tokens,
reasoningTokens: u.completion_tokens_details?.reasoning_tokens ?? 0,
};
}
if (typeof u.input_tokens === "number") {
// Anthropic-shaped: cache reads and writes are reported separately
// and are NOT part of input_tokens.
const cacheRead = u.cache_read_input_tokens ?? 0;
const cacheWrite = u.cache_creation_input_tokens ?? 0;
return {
inputTokens: u.input_tokens + cacheRead + cacheWrite,
cachedInputTokens: cacheRead,
outputTokens: u.output_tokens,
reasoningTokens: 0,
};
}
throw new Error("unknown usage shape: " + JSON.stringify(u));
}import { randomUUID } from "node:crypto";
export function newTask({ taskClass, successRule }) {
return {
taskId: randomUUID(),
taskClass,
successRule, // written down before the run, evaluated after it
startedAt: Date.now(),
spans: [],
};
}
export function record(task, fields) {
const row = {
spanId: randomUUID(),
taskId: task.taskId,
parentId: fields.parentId ?? null,
...fields,
createdAt: new Date().toISOString(),
};
task.spans.push(row);
return row;
}
export async function modelCall(task, opts) {
const {
client, model, messages, component,
parentId = null, retryDepth = 0, maxRetries = 2,
} = opts;
const startedAt = Date.now();
try {
const res = await client.create({ model, messages });
const usage = normalizeUsage(res);
record(task, {
kind: "model_call",
parentId,
component, // system_prompt | history | tool_output | user_input
model,
retryDepth,
inputTokens: usage.inputTokens,
cachedInputTokens: usage.cachedInputTokens,
outputTokens: usage.outputTokens,
reasoningTokens: usage.reasoningTokens,
costUsd: priceOf(model, usage),
costEstimated: false,
latencyMs: Date.now() - startedAt,
ok: true,
});
return res;
} catch (err) {
// A thrown request may still have been billed (timeout after
// generation, 5xx after token accounting). Record an estimate and
// flag it rather than pretending the call was free.
record(task, {
kind: "model_call",
parentId,
component,
model,
retryDepth,
inputTokens: 0,
cachedInputTokens: 0,
outputTokens: 0,
costUsd: estimateFromRequest(model, messages),
costEstimated: true,
latencyMs: Date.now() - startedAt,
ok: false,
errorClass: classify(err),
});
if (retryDepth >= maxRetries) throw err;
// Retry on the cheap tier with the smallest context that can work.
return modelCall(task, {
...opts,
model: opts.retryModel ?? model,
messages: trimToTask(messages),
retryDepth: retryDepth + 1,
});
}
}Attributing tokens to a component so the ledger has an owner
The component field is the difference between a ledger and a log. Label each message block at construction time — system_prompt, tool_schema, history, tool_output, user_input — and you can answer "who owns this spend" without re-parsing message arrays. Do it at build time, because reconstructing it later means guessing from content.
Building the Cost Ledger: Normalize, Price, and Roll Up
Normalizing token counts across providers
Store the raw counts, not just the cost. Prices change; your historical token data should survive a price update without a backfill.
| Column | OpenAI-shaped field | Anthropic-shaped field |
|---|---|---|
input_tokens (total) | prompt_tokens | input_tokens + cache read + cache write |
cached_input_tokens | prompt_tokens_details.cached_tokens | cache_read_input_tokens |
output_tokens | completion_tokens | output_tokens |
reasoning_tokens | completion_tokens_details.reasoning_tokens | not reported separately in this shape |
Pricing cached input, output, and reasoning tokens correctly
Three rules that are easy to get wrong, and that cost real money once you have enough traffic:
- Cached input is priced differently per provider, and the discount is not uniform across models. Keep a price table keyed by
(provider, model, effective_date)and version it. - Reasoning tokens are billed as output. They show up in the completion total and can dominate it on reasoning-heavy models. If your ledger only counts visible output, your cost model will be off by a large multiple.
- Do not hardcode list prices into application code. Compute cost at write time and store tokens, so you can reprice history when a provider changes rates.
The prices used in the worked audit below are round-number assumptions chosen for arithmetic clarity, not current list prices.
A schema that supports per-task rollups
create table agent_task (
task_id uuid primary key,
task_class text not null,
success_rule text not null, -- the executable check, as text
success boolean, -- null until evaluated
started_at timestamptz not null,
ended_at timestamptz
);
create table agent_span (
span_id uuid primary key,
task_id uuid not null references agent_task (task_id),
parent_id uuid,
kind text not null
check (kind in ('turn','model_call','tool_call')),
name text not null,
component text,
model text,
retry_depth int not null default 0,
input_tokens int not null default 0,
cached_input_tokens int not null default 0,
output_tokens int not null default 0,
reasoning_tokens int not null default 0,
cost_usd numeric(12,6) not null default 0,
cost_estimated boolean not null default false,
latency_ms int,
ok boolean not null,
error_class text,
created_at timestamptz not null default now()
);
create index agent_span_task_idx on agent_span (task_id);
create index agent_span_component_idx on agent_span (component, created_at);
create index agent_task_class_idx on agent_task (task_class, started_at);The rollup that produces the metric:
with task_cost as (
select t.task_id,
t.task_class,
t.success,
sum(s.cost_usd)::numeric as cost_usd
from agent_task t
join agent_span s using (task_id)
group by 1, 2, 3
)
select task_class,
count(*) as tasks,
round(avg(cost_usd), 4) as avg_cost_per_attempt,
round(sum(cost_usd)
/ nullif(count(*) filter (where success), 0), 4)
as cost_per_successful_task,
round(100 * sum(cost_usd) filter (where not success)
/ sum(cost_usd), 1)
as pct_spend_on_failures
from task_cost
group by 1
order by cost_per_successful_task desc;
And the component attribution query, the one that actually changes decisions:
select component,
sum(input_tokens) as input_tokens,
round(100.0 * sum(input_tokens)
/ sum(sum(input_tokens)) over (), 1) as pct_of_input
from agent_span
where kind = 'model_call'
group by 1
order by input_tokens desc;
A Worked Audit on a Support-Triage Agent Trace
The figures below come from replaying a synthetic 40-task trace through the arithmetic model in this post. The trace was built to match the shape of a support-triage agent: multi-turn, tool-heavy, with a two-tier router. It is reproducible and self-consistent, and it is not a measurement of any vendor's system, including LG Uplus's offering. Swap in your own ledger and the method carries over; the numbers will not.
Baseline: totals from a sample of runs, with sample size stated
Inputs to the model, stated up front so you can check the arithmetic:
- 40 tasks, 34 successful (85%)
- 400 model calls (10 per task on average), of which 96 are retries (24%)
- average input per call: 10,300 tokens — 3,000 system + schemas, 2,400 history, 4,600 tool output, 300 user input
- average output per call: 320 tokens
- routing: 340 calls on the cheap tier, 60 on the expensive tier (41% of tasks escalated at least once)
- assumed prices: cheap tier $0.25/1M in, $1.25/1M out. Expensive tier $3.00/1M in, $15.00/1M out. No caching.
| Metric | Baseline |
|---|---|
| Total spend, 40 tasks | $3.15 |
| Cost per attempt | $0.079 |
| Cost per successful task | $0.093 |
| Spend on the 6 failed tasks | $0.85 (27%) |
| Spend on retries | $1.53 (48%) |
| Spend on the expensive tier, 15% of calls | $2.14 (68%) |
Finding one — tool output dominated the input bill
Attributing input tokens by component, tool output was 44.7% of all input — larger than the system prompt and history combined, and larger than every other line item. The agent was paying, every turn, to re-read search results it had already read.
Finding two — retries re-sent the full context
96 retries were 24% of calls but 48% of spend, because 38 of them landed on the expensive tier at 12x the input price. A retry that re-sends 10,300 tokens to the large model costs more than three productive calls on the small model. Capping retries and shrinking retry context is the highest-leverage change in the whole audit.
Finding three — routing decisions were made by habit, not by task
15% of calls ran on the expensive tier and consumed 68% of spend. But 41% of tasks escalated at least once, and the escalation was triggered by a retry counter rather than by task class. Many of those escalations were on tasks that had never succeeded on the cheap tier and never would — the router was paying to discover something it could have known from the class label.
Before/after table with observed results next to each claim
Applied incrementally, each step measured the same way:
| Step | Change | Cost / successful task | Δ |
|---|---|---|---|
| v0 | baseline | $0.093 | — |
| v1 | tool output capped at 1,200 tokens | $0.066 | -28% |
| v2 | stable prefix, caching enabled | $0.045 | -32% |
| v3 | retry cap 2, retries on cheap tier with trimmed context | $0.033 | -28% |
| v4 | route by task class, escalation 41% → 20% of tasks | $0.026 | -20% |
Cumulative: $0.093 → $0.026 per successful task (-72%), with success rate moving 34/40 → 35/40. That improvement is not incidental — shorter, cleaner contexts and fewer desperate retries produce better answers. But it is also the number I would watch hardest, because that is where a cost optimization turns into a quality regression.
Levers Ranked by Impact per Engineering Hour
Truncate, paginate, and summarize tool output before it enters context
Highest leverage, least risk. Cap tool results at a token budget, return a handle for the rest, and let the model request the expansion if it needs it. A 1,200-token cap on search results and document fetches produced the single largest input reduction in the audit with no success-rate cost — the agent almost never touched the truncated tail.
Sort the component attribution query by pct_of_input before you optimize anything else. Whatever sits at the top is the only thing worth touching, and it is rarely the system prompt.
Stabilize the prompt prefix so caching actually hits
Caching only works on an identical prefix. Any timestamp, request ID, or shuffled tool-schema order at the top of your system prompt kills the hit rate and the saving with it. Log cached_input_tokens / input_tokens as a first-class metric — a cache hit rate that silently drops to zero looks exactly like a workload change in every other chart you have.
Cap retries, and retry with the minimal context needed
Two retries, on the cheap tier, with the conversation trimmed to the current task. If a task cannot succeed on the cheap tier in three attempts, escalating it is usually cheaper than a fourth — but do that deliberately, not by retry counter.
Route by task class and verify the route with a quality check
Route on something you can observe before the run: task class, input length, tool count required. Then verify: for each class, compare success rate on the routed model against a sample that would have gone to the other tier. Routing without a quality check is just cost cutting with extra steps.
What Breaks When You Optimize for Cost
Stale caches and silently wrong answers
Cache stability and cache correctness are different problems. A cached prefix that includes retrieved context will happily serve yesterday's retrieval for today's question, correctly billed at a discount. Cache the invariant parts — instructions, tool schemas, format rules — and keep changing context out of the cached region.
Quality regressions that a cost dashboard will not show you
Every lever above has a failure mode: truncation drops the field the agent needed, retry caps turn recoverable errors into failures, routing sends hard instances to a model that cannot do them. None of those appear in a cost chart. They appear in the success rate, and only if that rate is computed by an executable check rather than by the agent's own claim of completion.
In the audit, the truncation step initially broke one task class: the agent needed a field that fell outside the 1,200-token window. It was caught because the success rule was a schema validation, not a string match on "done". That, in one sentence, is the argument for the executable check.
Metric gaming — success criteria loosened to make cost per task look good
The failure mode is structural, not malicious. Cost per successful task improves when you relax the success rule, and someone under pressure will eventually relax it. Guard against it by versioning the success_rule column and reporting the rule version next to the metric. A cost-per-task number without its rule version is not comparable to last quarter's.
Confirmed, Inferred, and Unknown
What the sources state versus what I am inferring about the vendor offering
Confirmed by the reporting: on 2026-11-15, Chosunbiz, Asia Economy, and Digital Today published pieces describing LG Uplus positioning token optimization for corporate customers, with claims framed around lowering total token costs and cutting AI agent operating expenses. Digital Today characterized the work as accelerating token-efficiency research.
Inferred (not stated in the snippets I saw): that the offering is a productized layer sitting between an enterprise application and model providers. The "B2B" framing suggests it; the "research" framing suggests it may be earlier-stage than that. I would not treat either reading as established.
Unknown: the methodology, the benchmark, the model coverage, the pricing model, and — most importantly — whether "total token cost" is measured per token, per call, or per successful task. Until that last question is answered, the claim cannot be compared to anything.
What would need testing before adopting any external token-optimization layer
Five things, in order:
- Replay a fixed task set through the layer with the success rule locked and versioned. Report cost per successful task before and after.
- Check the success-rate delta per task class, not in aggregate. Aggregate hides the class that broke.
- Verify the layer's token accounting against raw provider responses on a sample of calls. If their numbers cannot be reconciled to
usage, nothing downstream is trustworthy. - Test cache behavior across prompt changes. A layer that rewrites your system prompt per request may be destroying hit rate while reporting savings on input price.
- Run a canary with the layer disabled on 5–10% of traffic, permanently. Vendor-side regressions are not visible from inside the vendor's dashboard.
AI Agent Token Spend Audit Checklist
- Every task has a
success_rulewritten down before the run and evaluated by code, not by the model -
usageis read from provider responses; no tokenizer estimates - Cached tokens are normalized correctly per provider (included vs. excluded from input totals)
- Reasoning tokens counted as output in the cost model
- Every model call carries a
componentlabel assigned at build time -
retry_depthrecorded on every call, and retries are attributable to a task - Failed requests recorded with
cost_estimated = trueinstead of dropped - Tool output has an explicit token budget and a retrieval path for the truncated remainder
-
cached_input_tokens / input_tokenstracked as a first-class metric with alerting - Routing decisions keyed on task class, with a per-class quality comparison against the other tier
- Cost per successful task, wasted-spend ratio, and p95 task cost are the top three dashboard numbers — not cost per call
- A permanent 5–10% control group runs without the optimization layer
None of this depends on a vendor. It depends on a denominator you can defend, spans that record what the provider actually billed, and the discipline to run the component attribution query before touching anything else. Start with the tool-output budget and the retry cap — in the trace above, those two changes alone removed more than half the waste — then re-measure with cost per successful task, rule version attached.
Further Reading
- Chosunbiz, LG Uplus to enter B2B with token optimization, slashing AI agent costs — report, 2026-11-15 (via Google News)
- Asia Economy, "Lower Total Token Costs"... LG Uplus Targets Corporate Customers with Token Optimization — report, 2026-11-15 (via Google News)
- Digital Today, LG Uplus speeds AI token efficiency research to cut service costs — report, 2026-11-15 (via Google News)
- OpenAI, Prompt caching guide — official docs for cached-token accounting and prefix requirements
- Anthropic, Prompt caching — official docs for
cache_read_input_tokens/cache_creation_input_tokenssemantics - OpenAI, API pricing and Anthropic, pricing — current list prices; substitute these for the assumed prices used in the audit
- OpenTelemetry, Generative AI semantic conventions — field naming if you want the ledger to interoperate with existing tracing
More posts

From Trace Spans to Chargeback: Attributing Inference Costs per Agent
A practical engineering walkthrough of how to propagate agent, team, and customer identity through trace spans so LLM inference usage can be metered, budgeted, alerted on, and charged back without igniting organizational turf wars.

2.4T Parameters, 60% Cheaper Per Task: Auditing Qwen4-Max After Retries
An audit of Qwen4-Max's 2.4T-parameter MoE and its claimed 60% per-task cost cut, using a retry-adjusted cost-per-successful-task worksheet to test whether the vendor's economics survive eval-driven model routing.

Testing Email, Teams, and Copilot Fallbacks Against an M365 Control-Plane Outage
A technical walkthrough for validating email, Teams, and Copilot fallback paths when a Microsoft 365 control-plane outage disrupts authentication, routing, and dependent services.
Loading comments…