Gating a Bedrock Coding-Agent Router on Cost per Accepted Patch

Introduction: The Default-to-Biggest-Model Tax

Most coding-agent traffic does not need a frontier model. The bill is usually the only thing that says so out loud. Renaming a symbol across 40 files, scaffolding a test file from an existing pattern, fixing a lint error — that work has roughly the same difficulty distribution as a shell script. Yet the default setup on Amazon Bedrock, one hardcoded model ID and usually the largest one, prices all of it as if every turn were an architectural refactor.

This post shows how to evaluate a NeMo Switchyard-style router for coding-agent requests by task difficulty, and how to gate rollout on latency, task-level pass rate, and cost per accepted patch instead of cost per request. Routing is the obvious fix. The gate is where most implementations get it wrong.

The position: a model router is an inference-economics change, so gate it on cost per accepted patch — not on cost per request, and not on a benchmark score. Cost per request is trivially improved by picking a cheaper model. A benchmark tells you about capability in someone else's harness with someone else's prompts. Neither tells you whether your agent produced more code that survived your review process per dollar. If you cannot compute cost per accepted patch on a frozen task set, you are not ready to route.

This piece is about the measurement harness, the four numbers, and the rollout gates. It is not a vendor comparison and not a capability benchmark. I did not run NeMo Switchyard against a Bedrock estate for this article, so the arithmetic below is illustrative and labeled that way, and everything I say about the router's internals is flagged as inference.

Scope note

What follows assumes an agent framework calling Bedrock, a repo you control, real merged pull request history to sample from, and a CI pipeline that can label a patch as accepted. If you do not have the last one, build it before you build the router.

What AWS Published, and What the Source Does Not Establish

Confirmed from the source

AWS published guidance on routing coding-agent requests to the right model on Amazon Bedrock using NVIDIA NeMo Switchyard. The stated goal is matching each request to an appropriate model rather than defaulting to the largest one. That is the whole of the confirmed claim: a routing layer sits in front of Bedrock model invocations, and its purpose is model selection per request. The announcement is dated 2026-11-13 and attributed to AWS (the announcement).

Explicitly unverified

The seed material does not establish any of the following, so I am not asserting it:

• the router's concrete API surface (SDK, sidecar, proxy, or gateway integration)
• its configuration schema, and whether policy is expressed as thresholds, labels, or model pairs
• which classifier modes are supported, and whether classification is heuristic, embedding-based, or model-based
• which Bedrock model IDs the guidance pairs as the cheap and strong tiers

Treat every one of those as unknown. They matter for implementation and not at all for the methodology, which is why the rest of this post is written to survive the answer being any of the above.

How to read the rest

The architecture and measurement sections are general: they apply whether your router is NeMo Switchyard, a 60-line classifier inside your agent framework, or a static rule on task class. Anything that leans on NeMo Switchyard internals is labeled as inference and is not load-bearing for the rollout decision.

Why Task Difficulty Alone Is the Wrong Routing Signal

Difficulty is not token count

Prompt length is the tempting classifier, and the worst one. "Make the retry test stop flaking" is a short prompt that can be genuinely hard, because it requires understanding a race in code the model has never seen. A 40,000-token prompt describing a mechanical rename is easy no matter how long it is. Token count measures context volume, not reasoning depth, and the two decouple exactly in the cases where routing matters.

Better signals, roughly in order of value: how many files the task plausibly touches; whether the task text names an error class or a failing symbol rather than an intent ("improve reliability"); prior turn count in the session; and whether the previous turn produced a failing tool result.

The Misrouting Asymmetry That Makes Cheap-Tier Errors Expensive

Misrouting is not symmetric. Send an easy task to the expensive model and you waste cents. Send a hard task to the cheap model and you pay for a full agent run — every turn, every tool call, every repo-context token — that produces a plausible-looking patch which fails CI, plus the escalation run behind it, plus the human minutes spent reading the failure.

That asymmetry is the argument for conservative routing. A router tuned to minimize spend per request will happily make the expensive mistake, because the expensive mistake looks cheap on the request it was made on.

Three different workloads inside one agent loop

An agent loop is not one workload:

Optimizing the whole loop as one unit hides the fact that most of the token volume sits in tool-result turns, where the cheap tier is indistinguishable, and most of the risk sits in patch synthesis, where it is not.

The Architecture: Where a Router Sits in a Bedrock Coding Agent

Request path

agent framework
  -> router / classifier        (decides tier for this turn)
  -> Bedrock Converse invocation (model chosen per turn)
  -> tool execution             (shell, file read, test runner)
  -> tool result back to framework
  -> router / classifier again

The router sits on the path of every turn, not once per task. That gets you per-turn decisions and per-turn cost attribution — and it also puts a new single point of failure on the critical path.

What the classifier can see, and what it cannot

Visible at classification time without extra plumbing: task text, the repo identifier and branch, turn index, the previous turn's tool result status, and accumulated token count for the session.

Not visible without building it: whether the produced patch will pass CI, whether the human reviewer will reject the approach, whether the failing test is flaky, and how much of the repo the change actually needs. A classifier that has to guess at outcomes will systematically under-estimate difficulty on exactly the tasks where the label matters.

The Statefulness Problem: Pick Once or Pick Every Turn

A decision made on turn 1 usually locks the session, because switching models mid-conversation changes prompt formatting, and in practice teams pin the tier at task start. That is a design choice, not a law. Make it explicit in your config: "the router picks once" and "the router picks every turn" produce very different cost profiles and very different failure modes.

Escalation is a cost multiplier, not a rescue

The cheap-model-fails-then-escalate loop is the standard mitigation, and it quietly doubles or triples effective cost on the hardest tasks: a failed cheap run, sometimes a retry with a reformulated prompt, then the full strong-model run. On tasks that succeed on the cheap tier it is free money. On tasks that do not, you paid for both tiers.

Which is fine — provided you count both tiers in the numerator. A dashboard that attributes escalation to a separate budget will report a saving that did not happen.

Defining the Unit of Account: Cost per Accepted Patch

The Cost-per-Accepted-Patch Formula

cost per accepted patch =
  (Σ all routed spend across all attempts, all turns, all tiers)
  / (count of patches that met the acceptance criterion)

One number, two halves, both easy to fudge.

The Numerator: Count Retries, Escalations, and Classifier Calls

Include retries, escalations, classifier calls, embeddings if the classifier uses them, context-compaction and summarization turns, and all tool-call-adjacent model tokens. Exclude only what genuinely sits outside the routing decision. A numerator that counts "main" invocations only produces an optimistic figure nobody should ship on.

The Denominator: Pick One Acceptance Criterion and State It

Acceptance is not self-defining. Each row below yields a different number, and they are not interchangeable:

Report the criterion alongside the number. Always. A router that lifts the local-test pass rate while dropping the merged-and-survived rate has improved nothing, and only the fourth row would show it.

A worked example with illustrative prices

Illustrative prices: cheap tier $0.50 per million input tokens, $2.00 per million output tokens. Strong tier $3.00 input, $15.00 output. Task set: 100 tasks, average 6 turns, 12,000 input tokens and 900 output tokens per turn.

Baseline, always-strong:

per task  = 6 × (12,000×3.00/1e6 + 900×15.00/1e6)
          = 6 × (0.0360 + 0.0135) = $0.297
100 tasks = $29.70
accepted  = 84
cost per accepted patch = 29.70 / 84 = $0.354

Router, 70 tasks to the cheap tier, 30 to the strong tier, 8 cheap-tier tasks escalating after failure:

cheap runs    = 70 × 5 × (12,000×0.50/1e6 + 900×2.00/1e6) = 70 × 0.039 = $2.73
escalations   =  8 × 0.297                                     = $2.38
strong runs   = 30 × 0.297                                     = $8.91
classifier    = 100 × 1,500 tokens × 0.50/1e6                  = $0.08
total                                                          = $14.10
accepted      = 82
cost per accepted patch = 14.10 / 82 = $0.172

Roughly half the cost per accepted patch, with a two-task pass-rate cost. Now the same arithmetic under escalation thrash — 25 cheap tasks escalate, each after two failed cheap attempts:

cheap runs    = $2.73
cheap retries = 25 × 2 × 0.039 = $1.95
escalations   = 25 × 0.297      = $7.43
strong runs   = $8.91
classifier    = $0.08
total         = $21.10

Break-even against the $0.354 baseline is 21.10 / 0.354 ≈ 60 accepted patches. Land at 72 accepted and the router still wins. Land at 58 and it lost money while adding a dependency — even though every individual request was cheaper.

The Guardrail: Cost per Request Is Not the Gate

If the router improves cost per request but worsens cost per accepted patch, it failed. Write that as a test assertion, not as a slide.

Building the Evaluation Harness

Task set selection

Sample from real merged pull requests in your own repositories. Reconstruct the task from the issue text or commit message plus the pre-change repo state, and stratify: roughly a third mechanical (lint, rename, docs), a third moderate (a bug fix with an existing failing test), a third hard (multi-file change with ambiguous intent). Then freeze the set. A task set you can edit mid-evaluation is a task set you will edit until it agrees with you.

Instrumentation per request

Per invocation, capture taskId, turn, modelId, inputTokens, outputTokens, latencyMs, escalation: boolean, and a routing reason. Capture the outcome per task separately, then join.

import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";


const client = new BedrockRuntimeClient({ region: process.env.AWS_REGION });
const LOG = process.env.EVAL_LOG ?? "runs/out.jsonl";

// Model IDs come from the environment on purpose: never hardcode an ID you
// have not verified against your account's enabled models.
const MODELS = {
strong: process.env.MODEL_STRONG,
cheap: process.env.MODEL_CHEAP,
};
if (!MODELS.strong || !MODELS.cheap) {
throw new Error("MODEL_STRONG and MODEL_CHEAP must be set");
}

const CONFIGS = {
baseline: () => ({ tier: "strong", reason: "always-strong" }),
router: (ctx) => {
  // Placeholder policy. Replace with your classifier. The harness does not care.
  const hard = ctx.turn === 0 && /refactor|race|intermittent|flaky/i.test(ctx.task);
  const tier = hard ? "strong" : "cheap";
  return { tier, reason: hard ? "keyword-hard" : "keyword-easy" };
},
};

export async function callTurn({ config, taskId, task, turn, messages, toolConfig }) {
const decision = CONFIGS[config]({ task, turn });
const modelId = MODELS[decision.tier];
const started = performance.now();

let res, error;
try {
  res = await client.send(new ConverseCommand({ modelId, messages, toolConfig }));
} catch (e) {
  error = e.name ?? "UnknownError";
}
const latencyMs = Math.round(performance.now() - started);

await appendFile(LOG, JSON.stringify({
  ts: new Date().toISOString(),
  taskId,
  turn,
  config,
  modelId,
  tier: decision.tier,
  routeReason: decision.reason,
  escalation: turn > 0 && decision.tier === "strong",
  inputTokens: res?.usage?.inputTokens ?? 0,
  outputTokens: res?.usage?.outputTokens ?? 0,
  latencyMs,
  error: error ?? null,
}) + "\n");

if (error) throw new Error(error);
return res;
}

// Join request-level logs with task-level acceptance labels.
export async function summarize({ logPath = LOG, labelsPath }) {
const rows = (await readFile(logPath, "utf8")).trim().split("\n").map(JSON.parse);
const labels = JSON.parse(await readFile(labelsPath, "utf8")); // { taskId: "accepted" | "rejected" }

const spendByTask = new Map();
const latencies = [];
let escalations = 0;

for (const r of rows) {
  // Price table is illustrative; swap in live rates.
  const rate = r.tier === "strong" ? { in: 3.0, out: 15.0 } : { in: 0.5, out: 2.0 };
  const cost = (r.inputTokens * rate.in + r.outputTokens * rate.out) / 1e6;
  spendByTask.set(r.taskId, (spendByTask.get(r.taskId) ?? 0) + cost);
  latencies.push(r.latencyMs);
  if (r.escalation) escalations++;
}

const totalSpend = [...spendByTask.values()].reduce((a, b) => a + b, 0);
const accepted = Object.values(labels).filter((v) => v === "accepted").length;
latencies.sort((a, b) => a - b);
const pct = (p) => latencies[Math.min(latencies.length - 1, Math.floor(p * latencies.length))];

return {
  tasks: spendByTask.size,
  accepted,
  passRate: accepted / spendByTask.size,
  totalSpend: +totalSpend.toFixed(4),
  costPerAcceptedPatch: +(totalSpend / accepted).toFixed(4),
  p50Ms: pct(0.5),
  p95Ms: pct(0.95),
  escalations,
};
}

The reproducible run

#!/usr/bin/env bash
set -euo pipefail

## Pin the two tiers you want to compare, then run the identical frozen task set
## under both configs. Nothing else changes between runs.
export MODEL_STRONG="${MODEL_STRONG:?set to a model enabled in your account}"
export MODEL_CHEAP="${MODEL_CHEAP:?set to a model enabled in your account}"
export AWS_REGION="${AWS_REGION:-us-east-1}"

mkdir -p runs
node router-eval.js --tasks eval/tasks.frozen.jsonl --config baseline --out runs/baseline.jsonl
node router-eval.js --tasks eval/tasks.frozen.jsonl --config router   --out runs/router.jsonl

node summarize.js runs/baseline.jsonl eval/labels.json
node summarize.js runs/router.jsonl   eval/labels.json

The shape of the output summarize should give you — illustrative, since I have not run this against your repositories:

{ tasks: 100, accepted: 84, passRate: 0.84, totalSpend: 29.7041,
  costPerAcceptedPatch: 0.3536, p50Ms: 4210, p95Ms: 18930, escalations: 0 }

{ tasks: 100, accepted: 82, passRate: 0.82, totalSpend: 14.1032,
  costPerAcceptedPatch: 0.1720, p50Ms: 2380, p95Ms: 21040, escalations: 8 }

Look at the second line: p95 got worse while p50 improved. That is normal. Escalated tasks carry the cheap-tier latency plus the strong-tier latency.

Sample size honesty

Twenty tasks is a smoke test. At 100 tasks and a baseline pass rate near 84%, the confidence interval on that rate is roughly ±7 points, which makes a 2-point regression indistinguishable from noise. Put the task count and the interval next to the result. If someone asks whether 82 versus 84 is a real regression, the honest answer is "not with this n."

Determinism control

Pin temperature to 0 where the model supports it, set a seed if the provider supports one, and pin model versions rather than aliases during evaluation. Then run the baseline twice against itself. The delta between baseline run A and baseline run B is your noise floor, and no router improvement smaller than that is worth a rollout.

The Four Numbers That Decide Rollout

Task-level pass rate

Pass or fail per task, not per request. A task that took eleven requests and eventually passed is one pass, not ten failures and a success. Name the most common failure mode explicitly: "9 of 18 failures were the agent editing the wrong file after failing to locate the definition" is actionable; "pass rate 82%" on its own is not.

Latency: p50 and p95

The agent loop amplifies tail latency, because a task's wall-clock time is the sum of its turns. A 3-second p95 at the model layer becomes an 18-second p95 at the task layer after six turns. Cheaper tiers are usually faster per token, but escalation puts a slow tail behind the fast body.

Cost per accepted patch

The primary gate. Compare against the always-strong baseline on the same frozen set, with the same acceptance criterion, on the same day.

Misroute rate on hard tasks

Take the hard stratum, count how many were sent to the cheap tier, and report how those failed. In the illustrative run: 30 hard tasks, 6 routed cheap (20% misroute rate), 4 of those 6 produced patches that failed CI.

Strategy Comparison for Router Rollout

Every cell marked fill from your run is deliberate. Published numbers from my environment would be a benchmark, and you already know what those are worth.

Failure Modes to Test Before You Trust the Router

Silent downgrade

A hard task classified as easy. The agent produces a plausible patch that fails CI. This is the most expensive failure class, because it consumes a full run, an escalation, and reviewer attention, and because nothing in your logs marks it as a routing error unless you join route decisions to outcomes. If your harness does not produce a "hard task routed cheap → rejected" count, you cannot see this failure at all.

Escalation thrash

Repeated cheap-model retries before escalation. Cap it: at most one cheap retry per task, then escalate. Each additional cheap retry has a low marginal success probability and a real marginal cost.

Classifier drift

Routing behaviour shifts as your repo grows, as your prompts change, or as the models behind an alias change. A router calibrated in March against a 40k-file repo is a different router in September. Re-run the frozen set weekly and pin the model version in production.

Flaky acceptance

Tests that pass locally and fail in CI turn router noise into false regression signals. Before attributing a pass-rate delta to routing, run the baseline twice and measure how much of the delta is flake.

Context truncation on the cheap tier

A smaller context window silently dropping repo context and changing patch quality in ways that task-level pass rate only partly reveals. Log input tokens per turn and count the turns where the framework truncated. If the cheap tier truncates on more than a few percent of tool-result turns, it is not a cheaper version of the strong tier — it is a different task.

Gating the Rollout

Stage 0 — shadow mode

Route in parallel and log what the router would have chosen. Ship nothing. Run for at least a week of real traffic, then compute the hypothetical cost per accepted patch against actuals. It costs nothing, and it catches misconfiguration before it costs a bad merge.

Stage 1 — canary with pre-committed thresholds

Small traffic share, with the numbers written down before the run. Block rollout if task-level pass rate drops more than 5 points below baseline, for example, or if cost per accepted patch fails to improve by at least 20%. Pre-committing is the whole point; thresholds chosen after seeing the data are not gates.

Stage 2 — per-task-class enablement

Start with low-risk classes: test scaffolding, lint fixes, doc updates, dependency bumps. Mechanical acceptance criteria, short blast radius. Expand to bug fixes, then multi-file changes, and only then consider anything touching migrations or auth.

Hard Controls for Production Routing

Three, minimum:

1. Kill switch to always-strong, flippable in config without a deploy.
2. Per-task-class override, so a single class can be pinned to the strong tier during an incident.
3. Weekly drift re-run of the frozen task set in CI, alerting on cost-per-accepted-patch movement.

Reporting Results Honestly

The fixed report shape

Use the same four-field shape every time, and keep the result next to the claim it supports:

Task set:   frozen 100-task set from merged PRs (34 mechanical / 33 moderate / 33 hard).
Criterion:  merged and not reverted within 7 days.
Result:     baseline 66/100 accepted at $0.449 per accepted patch;
            router 63/100 accepted at $0.221 per accepted patch.

Three fewer accepted patches for a 51% cost reduction is a decision someone can actually make. "Router cut inference costs in half" is not.

Observed versus inferred

Keep them apart in the document. Confirmed: what you ran, what the harness printed, what the AWS announcement states. Inferred: that NeMo Switchyard's classifier behaves a certain way, that a specific tier pairing is optimal, that the model behind an alias will not change under you. The last two are worth flagging as untested in your own write-up rather than inheriting someone else's confidence.

Publish the failure breakdown

The headline improvement is the least useful artifact. The failure taxonomy — how many silent downgrades, how many escalations, which task class produced them — is what the next team can actually use.

Further Reading

Primary source

• AWS, Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard, published 2026-11-13 — announcement.

Pricing and model reference

• Amazon Bedrock pricing — use this for the real per-token rates before running the harness.
• Amazon Bedrock supported models — check which model IDs are enabled in your account and what their context windows are before pairing tiers.

Conclusion: Gate on Accepted Patches, Not on Requests

Routing coding-agent traffic is worth doing. The tier gap on input-token-heavy agent workloads is large, most turns in a loop are not reasoning-hard, and defaulting to the largest model is genuinely wasteful. None of that is in dispute.

The metric is. A router that reduces spend per request while reducing accepted patches per dollar is a regression with better dashboards. Gate on cost per accepted patch, name your acceptance criterion, run a frozen task set, and pre-commit your thresholds.

Two things would change this conclusion. One: a large pass-rate regression concentrated on hard tasks. If the misroute rate on your hard stratum is high enough that escalations dominate spend, the router stops paying for itself, and the honest move is per-task-class enablement only, or no router at all. Two: a provider price change that shrinks the tier gap. A 2x difference between tiers is a routing problem; a 1.2x difference is a rounding error that does not justify a new dependency on your critical path. Re-check both quarterly.