Instrumenting Agent Loops for Cost per Successful Task
Cost per token is a vendor metric. Cost per request is an infrastructure metric. This walkthrough instruments an agent loop so the number on the dashboard is cost per successful task instead. What the video covers: - Why input tokens grow roughly quadratically with step count when every call re-sends the whole transcript - A hand-computed ledger at a placeholder price sheet: 3 steps at about 2.07 cents per run, 9 steps at about 9.45 cents, a 4.6x multiplier - Why failed runs belong in the numerator: at a 30% failure rate, cost per success is about $0.135 against $0.023 for the short loop - The instrumentation plan: one span per model call and one per tool call, a three-level token budget with reserve and settle semantics, a cache-key scheme, a model-routing ladder with an explicit stop condition, and an NDJSON harness that turns a synthetic workload into a cost-per-success figure with p50/p95 spread The post also separates the reported claim about token usage doubling monthly from your own measured baseline, covers the tool-output term you don't control, and includes the jq and PromQL queries that turn spans into an alert when cost per success regresses. Read it here: https://hackyjs.com/posts/instrumenting-agent-loops-for-cost-per-successful-task