Where Provider Routing, Caching, and Batching Absorb the 80% LLM Price Cut

A Price Cut Is a List-Price Event, Not a Bill Event

A provider announces an LLM price cut, and the reflex is to redo the business case: reopen the spreadsheet, drop the per-seat assumption, maybe dust off the build-vs-buy deck. That reflex is premature. List price is one input in a cost equation that has five more, and four of those sit on your side of the table — which is why this post works through what the two source reports actually claim, an explicit cost equation, and constructed arithmetic where the same 80% cut lands three workloads at 16.0%, 0.7% and 15.6% of their pre-cut list bill. The spread is workload shape, not the discount.

My position up front: an 80% cut is real money, but it is a linear multiplier, handed to every one of your competitors on the same day. The multipliers you own — cache hit ratio, routing mix, calls per task, retry rate, batch share — are the ones that compound. They are also what a price cut should pay for. Banking it as margin is the weakest move available, because the next announcement wipes it out.

What the Two Reports Actually Claim About the 80% LLM Price Cut

Two items sit in the source material, and both are aggregator reports rather than vendor announcements:

1. news.lavx.hu, 26 October 2026 — OpenAI and Anthropic "slash LLM prices up to 80%" as open-weight competition intensifies.
2. Startup Fortune, 25 October 2026 — DeepSeek's cheap new model is "making GPU scarcity worse, not better."

That is the whole public record I have. No per-model price sheet, no effective date table, no statement of which models moved.

The Up-to-80% Cut Claim and Where It Stops Being Useful

"Up to 80%" is a bound chosen to be defensible, not a rate you can compute with. It leaves open:

• which models and which tiers moved;
• whether the cut applies to input, output, or both (output typically runs 2-5× input, so the split matters more than the headline);
• whether cached-input pricing moved proportionally, or at all;
• whether this is a list price or a committed-spend/enterprise rate.

Without the per-model sheet, the number is a signal that the competitive floor moved. It is not an input to a budget. I have not verified it against any vendor pricing page, and I am not going to quote a rate I did not read.

The Counter-Claim That Cheap Models Worsen GPU Scarcity

The Startup Fortune argument, as the headline frames it, is that cheap open-weight-style models expand total inference demand and therefore tighten GPU supply rather than relieve it. The mechanism is plausible — cheaper inference brings more workloads over the viability line, and reasoning-heavy models burn far more tokens per request — but the report is an argument, not a measurement I can reproduce.

If it holds, two things follow. First, list price on a hosted API is a rental rate on hardware that is currently scarce, so a cut reflects oversupply in one tier, not a structural decline in the cost of compute. Second, self-hosting economics do not improve when an API price falls: your GPU-hour cost is unchanged, and if spot capacity is tight it gets worse. Inference: a price cut is weak evidence for reopening build-vs-buy, and it may be evidence for the opposite.

The Effective Cost Equation

Here is the equation I actually use. An announced LLM price cut touches exactly one term in it. Everything below is normalized so the frontier input price is 1.00 and the frontier output price is 4.00 — a ratio I picked for the arithmetic, not a quote from anyone.

effective_cost = Σ_routes  share_r
               × list_price_r
               × (1 − cache_ratio_r × (1 − cache_multiplier))
               × batch_multiplier_r
               × (1 + retry_rate_r)
               × calls_per_task_r

The Five Multipliers Between List Price and Invoice

The cut moves row one. Rows two and four are the ones I have watched move costs by 5-20× on real systems.

Provider Routing — The Discount Becomes a Portfolio Decision

Once price gaps between tiers are wider than capability gaps on your task, provider routing is where the discount actually reaches the invoice. The failure mode is routing on price alone: extraction traffic goes to the cheapest model, it silently drops a field, and you ship a wrong invoice total.

Building a Routing Table from Capability Tier, Latency Budget, and Price

The rule that keeps this honest: a route only gets traffic if it clears a fixed golden set at the required pass rate. Price breaks ties inside a capability tier, never across one.

Instrumenting Routed Cost Instead of Quoting Advertised Cost

Every dashboard I have seen that "quotes" cost is wrong somewhere, because it multiplies tokens by a price someone remembered. Charge each response in one adapter instead, and make an unpriced model a hard failure:

const PRICES = loadPrices(); // per 1M tokens, keyed "provider:model"

export function charge({ route, usage, meta }) {
const key = route.provider + ":" + route.model;
const p = PRICES[key];
if (!p) throw new Error("unpriced model: " + key); // never default to 0

const uncached = usage.inputTokens - usage.cacheReadTokens;

const billed =
  (uncached / 1e6) * p.input +
  (usage.cacheReadTokens / 1e6) * p.cacheRead +
  (usage.cacheWriteTokens / 1e6) * p.cacheWrite +
  (usage.outputTokens / 1e6) * p.output;

return billed * (meta.batch ? p.batchMultiplier : 1);
}

usage field names differ per provider. Normalize them to inputTokens/cacheReadTokens/cacheWriteTokens/outputTokens in the same adapter, and log the raw object next to the normalized one. The throw matters: a missing price that quietly becomes zero is how a cost dashboard reports a 90% saving that never happened.

Prompt Caching — The Multiplier That Can Dwarf the Headline Cut

Prompt caching is the only multiplier that can beat an 80% cut, because cached input is typically billed at a small fraction of input while the cut applies uniformly to everything. It is also the one most often left on the table.

I model cache reads at 0.1 × input price in the arithmetic below. That ratio is provider-specific, and on some providers a cache write is priced above the base input rate — which means a low hit ratio with high churn makes caching more expensive than not caching. Measure net, not gross.

Cache-Friendly Prompt Layout and Hit-Rate Measurement

The rule is boring and absolute: stable prefix first, volatile content last.

[ system instructions ]        <- stable
[ tool / function schemas ]    <- stable, versioned
[ output schema + examples ]   <- stable
[ retrieved document header ]  <- semi-stable
[ user turn, IDs, timestamps ] <- volatile, last

The classic bug is a new Date().toISOString() or a request UUID at the top of the system prompt. Every request becomes a cache miss, and nobody notices because the feature works.

Measure it explicitly, per route, and alert when it drops:

const hitRate = usage.cacheReadTokens /
  (usage.cacheReadTokens + (usage.inputTokens - usage.cacheReadTokens));

A prompt-template deploy that reorders two sections is a cost incident. Review template changes the way you review performance changes.

Batching — The Discount That Rewrites the Latency Contract

Request batching is usually the deepest discount on the sheet, and it is not a discount on your interactive traffic. It is a different product with a different SLO. If a feature tolerates minutes-to-hours, take it: extraction backfills, nightly classification, embedding refresh, eval runs. If a human is waiting, it does not apply.

Where Batching Quietly Costs More (queue time, rate limits, retry amplification)

Three ways batch turns into a loss:

1. Queue time with dependents. The batch job finishes in 40 minutes, but a dashboard reads its output, so the user-visible latency is 40 minutes. Budget queue time into the SLO, not just the model call.
2. Submission bursts and rate limits. Enqueueing 200k requests at 09:00 gets you 429s that look like failures and are not.
3. Retry amplification on ambiguous failures. This is the expensive one.

Three Workloads, One Price List, Three Different Bills

Everything in this section is constructed arithmetic, not measured production results. The prices are normalized assumptions; the workload shapes are the kind I have seen. Run the script with your own numbers.

const M = 1e6;
const CACHE_READ = 0.1; // assumption: cached input billed at 10% of input
const BATCH = 0.5;      // assumption: batch billed at 50% of interactive

const TIERS = {
frontier: { in: 1.0, out: 4.0 }, // normalized: frontier input = 1.00
mid:      { in: 0.2, out: 0.8 },
};

function cost({ calls = 1, prompt, output, cacheHit = 0, tier = "frontier",
              batch = false, retries = 0, cut = 1 }) {
const p = TIERS[tier];
const pin = p.in * cut, pout = p.out * cut;
const inCost  = (prompt / M) * ((1 - cacheHit) * pin + cacheHit * pin * CACHE_READ);
const outCost = (output / M) * pout;
return calls * (inCost + outCost) * (batch ? BATCH : 1) * (1 + retries) * 1000;
}

const W = {
"chat (A)":       { prompt: 1500, output: 300, cacheHit: 0.4, retries: 0.02 },
"extraction (B)": { prompt: 6000, output: 200, cacheHit: 5/6, retries: 0.01,
                    tier: "mid", batch: true },
"agent loop (C)": { calls: 5, prompt: 3000, output: 800, cacheHit: 0.5, retries: 0.08 },
};

for (const [name, w] of Object.entries(W)) {
const list      = cost({ ...w, cacheHit: 0, tier: "frontier", batch: false });
const cutOnly   = cost({ ...w, cacheHit: 0, tier: "frontier", batch: false, cut: 0.2 });
const stackOnly = cost(w);
const both      = cost({ ...w, cut: 0.2 });
console.log(name.padEnd(16), list.toFixed(2), cutOnly.toFixed(2),
            stackOnly.toFixed(2), both.toFixed(2),
            ((both / list) * 100).toFixed(1) + "%");
}

Output, in dollars per 1,000 requests (per 1,000 user requests for C):

workload          list   cut-only  stack-only   both   both/list
chat (A)          2.75       0.55        2.20   0.44      16.0%
extraction (B)    6.87       1.37        0.23   0.05       0.7%
agent loop (C)   33.48       6.70       26.19   5.24      15.6%

Read that honestly:

• A, a latency-bound chat feature with a short prompt, gets more from the list cut (2.75 → 0.55) than from the caching work (2.75 → 2.20). A 1,500-token prompt only leaves so much room.
• B goes the other way: caching a 5,000-token stable prefix and routing to mid-tier beats the cut by roughly 6×, because the stack changes apply to a much larger base. Batch on top is close to a rounding error.
• C is dominated by the cut (33.48 → 6.70), and the stack changes barely register alone (33.48 → 26.19), because 5 calls × 1.08 retries means you pay for 5.4 effective calls per task. The cut multiplies the waste down; it does not remove it. The remaining headroom in C is call-count reduction, not price.

What to Re-Measure Before Reopening the Build-vs-Buy Decision

If you are going to let a price cut reopen that decision, measure these first — a list price on its own decides nothing:

• Effective cost per successful task, not per token. Include retries and validator failures.
• Cache hit ratio per route, read from usage fields, not from what your prompt design was supposed to do.
• Retry and ambiguous-timeout rate, plus duplicate-request rate after reconciliation.
• Batch-eligible share of traffic, by product surface.
• GPU-hour cost and utilization, if self-hosting is on the table at all. The scarcity claim, if true, means the hosted price is a rate on constrained hardware; a cut in one tier does not change your utilization math.
• Tokens per task after prompt changes. A template edit that adds 800 tokens of instructions can eat a large share of a nominal cut.

Position — Reinvest the Cut in the Stack, Don't Bank It

Banking the cut survives exactly until the next press release, because list price is the one lever your competitors moved on the same day you did. The multipliers in your control do not behave that way.

So take the delta between your old modeled cost and the new one, and spend it. In priority order:

1. Kill retries, especially ambiguous ones. At an 8% retry rate, idempotency keys and request-ID reconciliation pay for themselves.
2. Raise cache hit ratio. Move volatile strings out of the prefix, version prompt templates, alert on hit-rate drops.
3. Route with an eval gate, not by price. The cut is what makes routing worth building; without it, tier differences are too small to justify the complexity.
4. Batch everything that tolerates it, with queue time in the SLO.

If a provider cuts list prices 80% and your invoice drops 80%, you were paying for tokens, not for a system. Most production stacks are not that shape. The ones I have looked at land closer to the 16% row than the 0.7% row.

Further Reading and Verification Notes

The source material for this post is two aggregator reports, not vendor pricing pages. I did not verify any per-model rate, effective date, or which tiers moved, and no rate here should be read as a quote — all figures are normalized assumptions chosen to make the arithmetic legible.

• Aggregator report, 26 October 2026 — OpenAI and Anthropic said to cut LLM prices up to 80% (publisher: news.lavx.hu)
• Aggregator report, 25 October 2026 — DeepSeek's cheap new model is making GPU scarcity worse, not better (publisher: Startup Fortune)
• Vendor pricing pages are the primary source for the numbers used in PRICES: OpenAI pricing and Anthropic pricing. Not inspected for this post; check current values before you hardcode anything.