Per-Call Deadlines vs. Run Budgets: Timeout Strategy for TypeScript Agent Tools

Per-Call Deadlines vs. Run Budgets: Timeout Strategy for TypeScript Agent Tools

pr0h0••39 min read•coding
AI Usage (98%)

You cannot fix a slow agent tool with one timeout number. This post builds a TypeScript timeout strategy for agent tools that separates per-call deadlines from the overall run budget, pushes cancellation through every layer, keeps retries inside the budget they started with, and returns labelled partial results instead of hanging. Every claim is asserted against deliberately slow loopback APIs.

If your wrapper carries one timeout number, you can't tell whether the model call, the queue, the HTTP attempt, or the retry backoff burned the budget — and you can't stop the run before it spends money producing nothing.

On this page

A Tool Timeout Is Not an Agent Run Budget

The failure I keep seeing in agent code is a single timeout: 30000 handed to a fetch wrapper and nothing else. When one tool hangs, the outcome depends on which layer happened to own a timer:

  • the fetch aborts at 30s, the wrapper throws, and the retry logic immediately starts a fresh 30s attempt;
  • the run has no deadline, so the orchestrator keeps scheduling other tools while the slow one is still pending;
  • the caller disconnects, but the tool's AbortController was never linked to the caller's signal, so the work continues;
  • a retry lands after the caller has given up, and the run wastes its last seconds on a request nobody will read.

Those are four different bugs with the same root cause: one number is being asked to describe three different things.

The technical position: bound each tool call, bound the whole run, and propagate cancellation through both

Bound the run with an absolute deadline measured from ingress. Bound each logical tool call with its own deadline that starts before queue admission and covers every attempt, every backoff, and the reading of the response body. Bound each individual HTTP attempt with a smaller limit that can never extend the enclosing budgets. Then compose the abort signals so cancellation flows downward and cannot flow sideways.

I'll go further than "set a timeout": a retry must never create new budget. If the enclosing deadline leaves no room for the next attempt plus its backoff, the correct action is to stop and return a labelled partial result, not to try once more and hope.

Source scope: the Startup Fortune summary names the slow-API problem, not an implementation

The motivating item is a Startup Fortune guide, dated 2026-11-22 in the feed entry, titled around setting an AI agent tool call timeout before one slow API stalls a run. What was supplied is a headline and a one-line summary; the aggregator link resolves to a news redirect rather than the article body.

Confirmed from the supplied source: the article exists, it frames the problem as a single slow API stalling an agent run, and it was published on that date.

Not available: any code, any recommended values, any measurement, any failure taxonomy. Everything below — the three-clock model, the specific millisecond figures, the code, the test results — is mine, built for this post. I am not attributing any implementation to Startup Fortune, and if you read their piece you should treat my numbers as an independent alternative, not a reproduction.

📝

The summary names the problem correctly. The reason this post is long is that "set a timeout" is the easy 10% — the remaining 90% is composition, retries, and proving the abort actually fired.

Define the Three Clocks Before You Write the Wrapper

Naming the clocks first is what stops the wrapper from collapsing them later. All three are absolute deadlines in a monotonic time base, not relative durations.

Run budget: one absolute deadline for orchestration, tools, retries, and the final response

The run budget is established once, at ingress, and it covers everything the request is allowed to spend: prompt assembly, model calls, queue waits, tool attempts, retries, synthesis, and serializing the answer. Nobody below the orchestrator gets to extend it. A tool's view of the run budget is min(its own deadline, the run deadline) — an intersection, never a union.

The corollary matters: the run budget must reserve time for work that happens after tools return. If the run deadline is also the tool deadline, a tool that consumes 100% of the budget leaves nothing to tell the user what happened. You get a silent 504 and no partial answer.

Per-call deadline: one clock covering queueing, every attempt, backoff, and response consumption

A logical tool call is one thing the agent decided to do. It may involve three HTTP attempts, two sleeps, and a slot wait in a concurrency limiter. All of that shares one deadline.

Two consequences get missed:

  1. Queue time counts. If the tool waits 2.4s for a concurrency slot and its deadline is 3s, it gets one 600ms attempt, not a fresh 3s.
  2. Body reading counts. fetch resolving means headers arrived. The JSON you need may still be streaming.

Attempt timeout: a smaller limit for one HTTP attempt that can never extend the call

An attempt timeout exists for a different reason: to detect a single stalled connection faster than the whole tool deadline would, so you can retry a different connection while budget remains. It is the only clock that may be reset on retry — and even then, its new value must be clamped by what the tool and run have left.

If you find yourself writing timeoutMs and then attempt === 2 ? timeoutMs : timeoutMs, you have one clock pretending to be two.

Worked policy: a 10-second run, 500ms reserve, 3-second per-call deadline, two 1.2-second attempts

Concrete numbers make the interactions inspectable:

ClockValueNotes
Run budget10,000ms from ingressabsolute, never extended
Reserve for synthesis + response500mssubtracted to get the tool-phase deadline
Tool-phase deadline9,500mstools must finish by here
Per-call deadline3,000msstarts at callTool entry, before queue
Attempt timeout1,200msclamped by remaining tool/run budget
Max attempts22 × 1,200 = 2,400ms
Backoff between attempts150–300ms jitteredfits in the remaining ~600ms
Serialized ceiling~2,700msunder the 3,000ms per-call deadline

The arithmetic is the design review. 2,400ms of attempts plus at most 300ms of backoff is 2,700ms, leaving ~300ms of slack inside a 3,000ms tool deadline for the abort to propagate and the outcome to be recorded. Bump maxAttempts to 3 and the ceiling becomes 3,900ms; the third attempt could never legally start, and the loop would waste a scheduling round trip discovering that. Pick numbers where maxAttempts × attemptTimeout + maxBackoff < toolDeadline, or accept that the last attempt is decorative.

Turn Relative Timeouts Into Monotonic Deadlines

Relative timeouts compose badly because each layer re-derives them, and every derivation is an estimate.

Use performance.now() for deadline math and Date.now() only for logs

performance.now() is monotonic and unaffected by wall-clock adjustments such as NTP steps. Date.now() can jump backwards, so a deadline stored as a Date.now() value can appear to gain budget. Use performance.now() for all comparisons and Date.now() only for human-facing timestamps.

// src/deadline.ts
export const now = (): number => performance.now();

export type ScopeKind = "run" | "tool" | "attempt" | "caller";

export interface Deadline {
  readonly at: number;      // performance.now() domain
  readonly kind: ScopeKind;
}

export function remainingMs(d: Deadline, at = now()): number {
  return Math.max(0, d.at - at);
}

// Ties keep the earlier argument, so the enclosing scope wins deterministically.
export function earliest(...candidates: Array<Deadline | undefined>): Deadline {
  const defined = candidates.filter((d): d is Deadline => d !== undefined);
  if (defined.length === 0) throw new Error("earliest() requires a deadline");
  return defined.reduce((a, b) => (b.at < a.at ? b : a));
}

export function wallClockFor(startedWall: number, startedMono: number): string {
  return new Date(startedWall + (now() - startedMono)).toISOString();
}

That earliest tie-break is not cosmetic. If the attempt timeout and the tool deadline are both 1,200ms, whichever timer happens to fire first determines the reported reason. Folding them into one earliest call with a defined tie-break makes the reason deterministic and the test assertion non-flaky.

Compute the effective deadline as the earliest of run, call, and attempt

Nothing downstream should ever see a raw number of milliseconds. Every function takes a Deadline and derives the effective one:

// src/tool-window.ts
const callWindow: Deadline = { at: startedAtMono + toolTimeoutMs, kind: "tool" };
const callDeadline = earliest(runDeadline, callWindow);

const attemptDeadline = earliest(
  callDeadline,
  { at: now() + attemptTimeoutMs, kind: "attempt" },
);

If the run has 400ms left when a 3,000ms tool starts, callDeadline.at is 400ms away and the attempt gets whatever remains after that. The tool never gets to believe it owns 3,000ms.

Recompute remaining time before queue admission, backoff, and every attempt

The tempting alternative is let left = toolTimeoutMs and subtract. Drift accumulates from every scheduling hiccup, and the subtraction silently ignores time spent anywhere you forgot to subtract — the microtask hop between attempts, or the response body drain.

Recomputing is one subtraction against an absolute value. It always returns the right answer:

const left = remainingMs(callDeadline);           // authoritative
if (left <= MIN_USEFUL_MS) return stop("budget");

Pick MIN_USEFUL_MS deliberately. I use 50ms locally: firing a request with 20ms left guarantees a timeout record and gains nothing.

Reject exhausted budgets immediately, and know what timers cannot interrupt

Check before scheduling anything. A budget that is already gone should produce an immediate skipped outcome, not a request that burns a connection slot.

More important is knowing what these timers can and cannot do. setTimeout callbacks and AbortSignal dispatches run on the event loop. A synchronous CPU-bound loop blocks them entirely.

blocking-demo.js
const t0 = performance.now();
setTimeout(() => {
console.log("timer fired at", Math.round(performance.now() - t0), "ms");
}, 100);

const until = performance.now() + 700;
while (performance.now() < until) {
// deliberately blocks the event loop
}
console.log("sync loop done at", Math.round(performance.now() - t0), "ms");

Representative output on Node 24:

sync loop done at 700 ms
timer fired at 706 ms

The 100ms timer fired 606ms late, after the loop yielded. Put that blocking work inside a tool and your 1,200ms attempt timeout fires nowhere near 1,200ms — no AbortController will save you. Abort is cooperative: it asks nicely and waits for the target to notice. A blocking tool needs isolation (worker thread, child process, separate service) and hard termination, not a signal.

Build One Runnable TypeScript Lab

The lab is small enough to read in one sitting and complete enough that every claim below is asserted by a test.

Set up Node 24, TypeScript, tsx, and the Node test runner

node --version    # v24.x
mkdir agent-deadlines && cd agent-deadlines
npm init -y
npm pkg set type=module
npm i -D typescript@5 tsx @types/node@24
npm pkg set scripts.test="node --import tsx --test --test-reporter=spec test/*.test.ts"
npm pkg set scripts.typecheck="tsc --noEmit"

Commit package-lock.json. tsx is the loader in both npm test and npm run typecheck, so the source you test is the source you type-check; Node 24's native type stripping would also work here, but the loader keeps the setup stable if you later add syntax that stripping rejects (enums, parameter properties).

A minimal tsconfig.json is enough: "target": "ES2023", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "noEmit": true, and "allowImportingTsExtensions": true so the test files can import ../src/executor.ts directly.

Lay out the example: deadline utilities, executor, loopback fixture, integration tests

src/
  deadline.ts        # now(), Deadline, remainingMs(), earliest(), raceAbort(), sleep()
  scope.ts           # createScope(): AbortController + reason + idempotent dispose
  retry.ts           # classify(), backoffWithJitter(), parseRetryAfter()
  executor.ts        # callTool(): the attempt loop
  fixture-server.ts  # loopback routes for slow/flaky/malformed responses
  run.ts             # runAgent(): run budget, queue, synthesis reserve
test/
  executor.test.ts
  run.test.ts

Create deterministic fixture routes for delayed headers, delayed bodies, transient errors, and fast success

Each route exercises exactly one failure mode, so a failing assertion points at one mechanism.

// src/fixture-server.ts



const MAX_DELAY_MS = 5_000;
export const HUGE_BODY_BYTES = 1024 * 1024;

export interface Fixture {
  url: string;
  stats(): { total: number; active: number; clientAborts: number };
  close(): Promise<void>;
}

export async function startFixture(): Promise<Fixture> {
  let total = 0;
  let active = 0;
  let clientAborts = 0;
  const failuresByKey = new Map<string, number>();
  const sockets = new Set<Socket>();

  const delay = (raw: string | null): number =>
    Math.min(Math.max(Number(raw ?? "0") || 0, 0), MAX_DELAY_MS);

  const server: Server = createServer((req, res) => {
    total += 1;
    active += 1;
    res.on("close", () => {
      active -= 1;
      if (!res.writableFinished) clientAborts += 1;
    });

    const url = new URL(req.url ?? "/", "http://127.0.0.1");

    if (url.pathname === "/fast") {
      res.writeHead(200, { "content-type": "application/json" });
      res.end(JSON.stringify({ ok: true, source: "fast" }));
      return;
    }

    if (url.pathname === "/slow-headers") {
      // Headers deliberately withheld: fetch() stays pending.
      setTimeout(() => {
        if (res.destroyed) return;
        res.writeHead(200, { "content-type": "application/json" });
        res.end(JSON.stringify({ ok: true }));
      }, delay(url.searchParams.get("ms")));
      return;
    }

    if (url.pathname === "/slow-body") {
      // Headers immediate, body trickled: fetch() resolves early.
      res.writeHead(200, { "content-type": "application/json" });
      res.write('{"ok":');
      setTimeout(() => {
        if (res.destroyed) return;
        res.end("true}");
      }, delay(url.searchParams.get("ms")));
      return;
    }

    if (url.pathname === "/flaky") {
      const key = url.searchParams.get("key") ?? "default";
      const failTimes = Number(url.searchParams.get("fail") ?? "1");
      const seen = failuresByKey.get(key) ?? 0;
      failuresByKey.set(key, seen + 1);
      if (seen < failTimes) {
        res.writeHead(503, { "content-type": "application/json", "retry-after": "1" });
        res.end(JSON.stringify({ error: "transient" }));
        return;
      }
      res.writeHead(200, { "content-type": "application/json" });
      res.end(JSON.stringify({ ok: true, attempt: seen + 1 }));
      return;
    }

    if (url.pathname === "/limited") {
      res.writeHead(429, { "content-type": "application/json", "retry-after": "30" });
      res.end(JSON.stringify({ error: "rate limited" }));
      return;
    }

    if (url.pathname === "/malformed") {
      res.writeHead(200, { "content-type": "application/json" });
      res.end("{not json");
      return;
    }

    if (url.pathname === "/huge") {
      res.writeHead(200, { "content-type": "application/json" });
      res.end("[" + "0,".repeat(HUGE_BODY_BYTES / 2) + "0]");
      return;
    }

    res.writeHead(404, { "content-type": "application/json" });
    res.end(JSON.stringify({ error: "not found" }));
  });

  server.on("connection", (socket) => {
    sockets.add(socket);
    socket.on("close", () => sockets.delete(socket));
  });

  await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
  const { port } = server.address() as AddressInfo;

  return {
    url: "http://127.0.0.1:" + port,
    stats: () => ({ total, active, clientAborts }),
    async close() {
      server.closeAllConnections();  // Node 18.2+
      for (const socket of sockets) socket.destroy();
      await new Promise<void>((resolve, reject) =>
        server.close((err) => (err ? reject(err) : resolve())),
      );
    },
  };
}

/slow-headers and /slow-body are separate routes on purpose. They prove different things about your executor, and a wrapper that only bounds fetch() passes the first and fails the second.

Bind to loopback only, cap delays, track active requests, and close sockets on teardown

Three details keep this fixture honest:

  • listen(0, "127.0.0.1") — an ephemeral port on loopback only. Never 0.0.0.0; a test fixture that accepts external connections is a small gift to your network.
  • MAX_DELAY_MS = 5_000 — a stray ?ms=600000 cannot hang CI for ten minutes. Every delay is clamped before use.
  • closeAllConnections() plus explicit socket destruction — with keep-alive, server.close() waits for idle sockets and your test process hangs after the last assertion passes. Destroy first, then close.

The stats() counters are the evidence channel for the cancellation section: clientAborts increments when a response closes before writableFinished.

Implement Deadline Scopes and Cancellation Ownership

A scope wraps one AbortController plus the knowledge of why it aborted.

Give each scope an AbortController, an explicit reason, and idempotent cleanup

// src/scope.ts


export interface CancelReason {
  kind: "run_exhausted" | "tool_expired" | "attempt_expired" | "caller_cancelled";
  message: string;
  attempt?: number;
}

export interface Scope {
  readonly signal: AbortSignal;
  readonly deadline: Deadline;
  readonly reason: CancelReason | undefined;
  expire(reason: CancelReason): void;
  dispose(): void;
}

const MAX_TIMER_MS = 2 ** 31 - 1; // setTimeout clamps above this

function reasonFor(kind: ScopeKind, attempt?: number): CancelReason {
  switch (kind) {
    case "run":
      return { kind: "run_exhausted", message: "run budget exhausted" };
    case "tool":
      return { kind: "tool_expired", message: "tool deadline expired" };
    case "attempt":
      return { kind: "attempt_expired", message: "attempt timeout expired", attempt };
    default:
      return { kind: "caller_cancelled", message: "caller aborted" };
  }
}

export function createScope(deadline: Deadline, outer?: AbortSignal, attempt?: number): Scope {
  const controller = new AbortController();
  let reason: CancelReason | undefined;
  let disposed = false;

  const timer = setTimeout(() => {
    expire(reasonFor(deadline.kind, attempt));
  }, Math.min(remainingMs(deadline), MAX_TIMER_MS));

  const onOuterAbort = (): void => {
    const inherited = outer?.reason as CancelReason | undefined;
    expire(inherited ?? { kind: "caller_cancelled", message: "caller aborted" });
  };

  function expire(next: CancelReason): void {
    if (reason) return;                 // first reason wins
    reason = next;
    clearTimeout(timer);
    controller.abort(next);
  }

  function dispose(): void {
    if (disposed) return;               // idempotent
    disposed = true;
    clearTimeout(timer);
    outer?.removeEventListener("abort", onOuterAbort);
  }

  if (outer) {
    if (outer.aborted) onOuterAbort();
    else outer.addEventListener("abort", onOuterAbort, { once: true });
  }

  return {
    signal: controller.signal,
    deadline,
    get reason() { return reason; },
    expire,
    dispose,
  };
}

Three choices in there are deliberate:

  • if (reason) return; makes the first cancellation authoritative. Without it, a tool deadline followed 2ms later by a run deadline produces whichever reason the timer queue happened to deliver first, and your metrics become noise.
  • dispose() is idempotent and clears listeners, not just the timer. A leaked abort listener on a long-lived run signal keeps the whole closure alive.
  • The MAX_TIMER_MS clamp: setTimeout(fn, 30 * 24 * 3600 * 1000) fires almost immediately in Node because the delay overflows. Derived deadlines must be clamped before hitting a timer.

Propagate caller and run cancellation without letting one tool cancel its siblings

Cancellation flows down the chain: caller → run → tool → attempt. Each scope passes its signal as the outer of its children, so a caller abort reaches the in-flight fetch two levels down.

Siblings are the thing to prevent. Two tools extended from the same run signal are siblings; tool A's internal timer aborts only A's controller. A's AbortController is never a parent of B's. The bug to avoid looks like this:

// WRONG: sibling cancellation. Never do this.
const shared = new AbortController();
for (const tool of tools) {
  setTimeout(() => shared.abort(), tool.timeoutMs); // kills every tool
}

Share one controller across tools and one slow API cancels the whole batch, leaving you unable to distinguish a genuine timeout from collateral damage.

Preserve the first cancellation reason and keep the four reasons distinct

Four reasons, four different operational responses:

ReasonMeaningCorrect response
caller_cancelledThe user or upstream client went awayStop everything, emit nothing expensive
run_exhaustedThe whole run is out of budgetStop scheduling, answer with what you have
tool_expiredThis tool's own deadlineRetry only if budget and attempts remain
attempt_expiredOne connection stalledRetry is normal policy, budget permitting

Collapsing these into AbortError is why so many agents retry after the caller hung up.

Dispose timers and listeners in finally, and recheck the deadline after every await

dispose() belongs in finally, not on the success path. And any await is a place where the deadline can slip past you:

const text = await readBoundedText(res, maxBytes, scope.signal);
if (scope.signal.aborted) throw scope.signal.reason;  // recheck after awaiting

The abort may have landed between the last byte arriving and the continuation being scheduled. Skip the recheck and you parse and validate a response for a call that has already been declared dead — wasting exactly the budget you were protecting.

Keep the Deadline Active Until the Response Is Consumed

This is where most timeout wrappers leak.

Pass the composed signal to fetch and keep the attempt alive through body consumption

await fetch(url, { signal }) resolves when headers are available. For most JSON APIs the body follows immediately, so the gap is invisible in development and enormous in production — a streaming endpoint, a slow proxy, a large payload on a congested link.

The attempt scope therefore stays open across both phases:

// src/executor.ts (excerpt)
async function attemptOnce<T>(req: ToolRequest<T>, scope: Scope): Promise<T> {
  const headers: Record<string, string> = { accept: "application/json" };
  if (req.idempotencyKey) headers["idempotency-key"] = req.idempotencyKey;

  const res = await req.fetchImpl(req.url, { signal: scope.signal, headers });

  // The scope is still live here: we are still inside the attempt.
  const text = await readBoundedText(res, req.maxBytes, scope.signal);

  if (!res.ok) {
    throw new UpstreamError(res.status, text.slice(0, 200), parseRetryAfter(res.headers.get("retry-after")));
  }

  let json: unknown;
  try {
    json = JSON.parse(text);
  } catch {
    throw new InvalidResponseError("invalid_json", text.slice(0, 80));
  }
  return req.validate(json);
}

res is read inside the same scope that was passed to fetch. There is no window where the deadline is nominally alive but nothing is listening to it.

Check status explicitly, bound response size, and separate transport errors from bad JSON

fetch does not throw on 500. Catch only transport errors and an error page flows into your parser as invalid_json — which looks retryable and will burn your whole budget retrying a deterministic 404.

readBoundedText does three jobs: enforces a byte ceiling, keeps the abort live during the read, and never lets a rejected body leak a connection.

// src/deadline.ts (excerpt)
export async function raceAbort<T>(p: Promise<T>, signal: AbortSignal): Promise<T> {
  if (signal.aborted) throw signal.reason;
  return await new Promise<T>((resolve, reject) => {
    const onAbort = (): void => reject(signal.reason);
    signal.addEventListener("abort", onAbort, { once: true });
    p.then(
      (v) => { signal.removeEventListener("abort", onAbort); resolve(v); },
      (e) => { signal.removeEventListener("abort", onAbort); reject(e); },
    );
  });
}

export async function readBoundedText(
  res: Response,
  maxBytes: number,
  signal: AbortSignal,
): Promise<string> {
  const reader = res.body?.getReader();
  if (!reader) return "";

  const chunks: Uint8Array[] = [];
  let total = 0;
  let finished = false;

  try {
    for (;;) {
      const { done, value } = await raceAbort(reader.read(), signal);
      if (done) { finished = true; break; }
      total += value.byteLength;
      if (total > maxBytes) throw new InvalidResponseError("body_too_large", String(total));
      chunks.push(value);
    }
  } finally {
    if (!finished) await reader.cancel().catch(() => {});
    reader.releaseLock();
  }

  return Buffer.concat(chunks.map((c) => Buffer.from(c))).toString("utf8");
}

raceAbort is what lets the byte ceiling and the deadline coexist with a stream that may simply stop producing bytes. Without it, if the underlying stream ignores the abort, reader.read() stays pending forever and your "deadline" is a comment.

Cancel or drain rejected bodies before retrying, and release resources in finally

Every attempt must leave the connection in a defined state:

  • body fully read → the connection can be reused;
  • body rejected by status/parse → reader.cancel() (or res.body?.cancel()) before releasing;
  • attempt aborted mid-stream → same cancellation, plus releaseLock().

Skip this and you leak sockets into undici's pool; the next attempt waits on a connection that will never come back. That failure mode looks exactly like a slow API, which is why it keeps getting misdiagnosed as one.

Why Promise.race bounds waiting, why abort is cooperative, and why sync parsing needs its own limits

Promise.race([work, timeout]) bounds how long you wait. It does not stop work. The request keeps running, the socket stays open, the server keeps working, and the promise's eventual rejection becomes an unhandled rejection unless you attached a handler. raceAbort is a race plus two real effects: the signal propagates to fetch (which aborts the request per the Fetch standard), and the caller throws immediately instead of waiting.

Abort is cooperative at every layer. fetch cooperates. A readFile cooperates. A tight regex or a JSON.parse of a 40MB body does not — it runs to completion on the main thread. For synchronous work, the only real bounds are input size (maxBytes) and isolation (a worker thread or child process you can terminate). A timeout around a synchronous call bounds your patience, not the CPU.

Add Bounded Retries Without Restarting the Clock

Retries are where deadline discipline dies. The pattern catch { await retry() } restarts the clock implicitly and turns one 3s tool into an unbounded one.

Retry only classified transient failures and idempotent or keyed operations

Classification must be explicit and enumerated, not if (!(err instanceof MyError)).

// src/errors.ts
export class UpstreamError extends Error {
  constructor(
    readonly status: number,
    readonly body: string,
    readonly retryAfterMs?: number,
  ) {
    super("upstream " + status);
  }
}

export class InvalidResponseError extends Error {
  constructor(
    readonly category: "invalid_json" | "schema_mismatch" | "body_too_large",
    detail: string,
  ) {
    super(category + ": " + detail);
  }
}
// src/retry.ts
export type Failure =
  | { retryable: true; category: "transport" | "server_error" | "rate_limited"; retryAfterMs?: number }
  | { retryable: false; category: "client_error" | "invalid_json" | "schema_mismatch" | "body_too_large" | "attempt_expired" };

export function classify(err: unknown): Failure {
  if (err instanceof InvalidResponseError) {
    return { retryable: false, category: err.category };
  }
  if (err instanceof UpstreamError) {
    if (err.status === 429) return { retryable: true, category: "rate_limited", retryAfterMs: err.retryAfterMs };
    if (err.status >= 500) return { retryable: true, category: "server_error", retryAfterMs: err.retryAfterMs };
    return { retryable: false, category: "client_error" };
  }
  return { retryable: true, category: "transport" }; // network-level, worth one more try
}

Method safety is the other half. GET, HEAD, OPTIONS, PUT, and DELETE are idempotent per RFC 9110; POST and PATCH are not. A retried POST that actually succeeded before the connection dropped creates a duplicate. Either attach an idempotency-key header that the upstream honours, or do not retry the write — record it as an ambiguous outcome and reconcile (see the limits section).

Require both an attempt allowance and enough remaining budget

Two independent gates, both required:

// src/executor.ts (retry decision, inlined for clarity)
function shouldRetry(args: {
  failure: Failure;
  attemptsUsed: number;
  maxAttempts: number;
  remainingMs: number;
  random: () => number;
}): { retry: true; delayMs: number } | { retry: false; why: string } {
  const { failure, attemptsUsed, maxAttempts, remainingMs, random } = args;

  if (!failure.retryable) return { retry: false, why: "terminal_" + failure.category };
  if (attemptsUsed >= maxAttempts) return { retry: false, why: "attempts_exhausted" };

  const backoff = failure.category === "rate_limited" && failure.retryAfterMs !== undefined
    ? failure.retryAfterMs
    : backoffWithJitter(attemptsUsed, random);

  const MIN_USEFUL_MS = 50;
  if (remainingMs <= backoff + MIN_USEFUL_MS) {
    return { retry: false, why: "budget_insufficient" };
  }
  return { retry: true, delayMs: backoff };
}

remainingMs <= backoff + MIN_USEFUL_MS is the line that stops the classic thrash: attempt one fails at 2,900ms of a 3,000ms tool, backoff is 200ms, and the retry would start at 3,100ms — already past the deadline. The wrapper could fire it, and it would abort immediately. Refusing is cheaper and produces a truthful metric.

Make backoff abortable and inject randomness so tests can assert exact delays

export function backoffWithJitter(
  attemptsUsed: number,
  random: () => number,
  baseMs = 150,
  capMs = 1_000,
): number {
  const ceiling = Math.min(capMs, baseMs * 2 ** (attemptsUsed - 1));
  return Math.round(ceiling / 2 + random() * (ceiling / 2)); // half fixed, half jitter
}

export function sleep(ms: number, signal: AbortSignal): Promise<void> {
  return new Promise((resolve, reject) => {
    if (signal.aborted) { reject(signal.reason); return; }
    const timer = setTimeout(() => { cleanup(); resolve(); }, ms);
    const onAbort = (): void => { cleanup(); reject(signal.reason); };
    function cleanup(): void {
      clearTimeout(timer);
      signal.removeEventListener("abort", onAbort);
    }
    signal.addEventListener("abort", onAbort, { once: true });
  });
}

Math.random is injected as req.random so tests can pass () => 0.5 and assert exact backoff values. A test that asserts elapsed >= 150 on a jittered sleep is flaky by construction; a test with deterministic randomness asserts the number.

sleep must be abortable, not await new Promise(r => setTimeout(r, 300)). Otherwise a caller cancellation during backoff is only noticed after the sleep ends — up to a full second of a dead run still holding a queue slot.

Honor Retry-After only when it fits the remaining budget

Retry-After is either delta-seconds or an HTTP-date (RFC 9110 §10.2.3). Convert it to a duration immediately — an HTTP-date compared with performance.now() is a category error, and a wall-clock comparison is vulnerable to clock steps.

export function parseRetryAfter(value: string | null): number | undefined {
  if (!value) return undefined;
  const seconds = Number(value);
  if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1000;
  const at = Date.parse(value);
  if (Number.isNaN(at)) return undefined;
  return Math.max(0, at - Date.now());
}

The fixture's /limited route returns Retry-After: 30. With a 3,000ms tool deadline, the correct behavior is to stop — not to wait 30s and not to retry early because "30s is a long time". The decision is a subtraction, and the answer is no.

Treat caller cancellation and exhausted deadlines as terminal; retry attempt timeouts by policy

EventRetry?
caller_cancelledNever — nothing will consume the result
run_exhaustedNever — no budget exists to retry into
tool_expiredNever for this call; the tool's clock is done
attempt_expiredYes, if policy allows and budget remains
5xx / 429 / transport errorYes, subject to the two gates
4xx / invalid JSON / oversized bodyNo — deterministic

attempt_expired being retryable is the whole point of having a separate attempt timeout: a stalled connection is often a per-connection problem, and a fresh one succeeds. But it is a policy choice. If your upstream is known to hang when overloaded, retrying makes it worse — classify it as terminal and rely on the tool deadline.

Integrate the Executor into an Agent Run

Tools are usually the loudest consumers of time, not the only ones.

Set the run deadline at ingress and reserve time for synthesis

// src/run.ts (excerpt)
export async function runAgent(cfg: {
  runBudgetMs: number;
  synthesisReserveMs: number;
  tools: ToolSpec[];
  queue: Semaphore;
  callerSignal?: AbortSignal;
  callModel: (deadline: Deadline, signal: AbortSignal) => Promise<string>;
}): Promise<RunResult> {
  const hardDeadline: Deadline = { at: now() + cfg.runBudgetMs, kind: "run" };
  const toolPhaseDeadline: Deadline = {
    at: hardDeadline.at - cfg.synthesisReserveMs,
    kind: "run",
  };

  const runScope = createScope(hardDeadline, cfg.callerSignal);
  try {
    const settled = await Promise.allSettled(
      cfg.tools.map((tool) =>
        callTool({
          ...tool,
          runDeadline: toolPhaseDeadline,   // tools cannot eat the reserve
          callerSignal: runScope.signal,
          queue: cfg.queue,
        }),
      ),
    );
    // ... see below
    const answer = await cfg.callModel(hardDeadline, runScope.signal);
    return { answer, partial: false };
  } finally {
    runScope.dispose();
  }
}

Two deadlines, not one. toolPhaseDeadline is what tools receive; hardDeadline is what synthesis receives. Tool calls therefore cannot consume the 500ms reserved for turning results into an answer, and the model call still gets a real, if small, budget.

Count queue waiting inside the per-call deadline and drop cancelled work from the queue

The queue slot acquisition happens inside the tool scope, so waiting counts:

// src/executor.ts (excerpt)
const toolScope = createScope(callDeadline, req.callerSignal);
let release: (() => void) | undefined;
const queueStartedAt = now();
try {
  if (req.queue) {
    release = await req.queue.acquire(toolScope.signal);  // rejects with the scope reason
  }
  const queueMs = now() - queueStartedAt;
  // ... attempt loop
} catch (err) {
  if (toolScope.signal.aborted) {
    return outcomeForReason(toolScope.reason, 0, now() - startedAt, "cancelled_in_queue");
  }
  throw err;
} finally {
  release?.();
  toolScope.dispose();
}

The semaphore removes aborted waiters from its queue rather than granting them a slot after everyone else has left:

// src/semaphore.ts (excerpt)
acquire(signal: AbortSignal): Promise<() => void> {
  if (signal.aborted) return Promise.reject(signal.reason);
  if (this.free > 0) {
    this.free -= 1;
    return Promise.resolve(this.releaser());
  }
  return new Promise((resolve, reject) => {
    const onAbort = (): void => {
      this.waiters = this.waiters.filter((w) => w.onAbort !== onAbort); // leave the queue
      reject(signal.reason);
    };
    signal.addEventListener("abort", onAbort, { once: true });
    this.waiters.push({
      onAbort,
      grant: () => {
        signal.removeEventListener("abort", onAbort);
        resolve(this.releaser());
      },
    });
  });
}

Drop the filter and a canceled tool still gets a slot when one opens, immediately aborts, and releases it — one wasted scheduling round trip per canceled tool. Under a tight run budget with 20 queued tools, that is the difference between finishing and not.

Pass remaining budget into model calls and other orchestration I/O, not just tools

The model call is an HTTP request with an unusually long tail. Give it the run deadline and let the same scope machinery apply:

const answer = await cfg.callModel(hardDeadline, runScope.signal);
if (!runScope.reason && remainingMs(hardDeadline) < 200) {
  records.push({ kind: "synthesis_squeezed", remainingMs: remainingMs(hardDeadline) });
}

Wrap only tools and you get the inversion where every tool finishes comfortably and the run still times out during synthesis. Wrap the vector lookup, the database read, and the prompt-compaction call too — anything with a network hop.

Return discriminated outcomes and use Promise.allSettled to keep independent successes

Promise.all throws on the first rejection and discards sibling results, which is exactly wrong: one slow tool should not erase three good ones.

export type ToolOutcome<T> =
  | { status: "ok"; value: T; attempts: number; elapsedMs: number }
  | { status: "timeout"; reason: CancelReason; attempts: number; elapsedMs: number; stoppedBy: string }
  | { status: "cancelled"; reason: CancelReason; attempts: number; elapsedMs: number }
  | { status: "upstream_failure"; category: string; attempts: number; elapsedMs: number }
  | { status: "invalid_response"; category: string; attempts: number; elapsedMs: number }
  | { status: "skipped"; category: "budget_exhausted"; elapsedMs: number };

callTool returns these; it does not throw for expected outcomes. Throwing forces every caller into try/catch and makes it impossible to distinguish "the tool timed out" from "my code has a bug". Reserve exceptions for programmer error.

Split required from optional tools, stop scheduling at exhaustion, and label incomplete answers

const outcomes = settled.map((s, i) =>
  s.status === "fulfilled" ? s.value : ({ status: "executor_bug", detail: String(s.reason) } as const),
);

const missingRequired = cfg.tools
  .map((t, i) => ({ spec: t, outcome: outcomes[i] }))
  .filter(({ spec, outcome }) => spec.required && outcome.status !== "ok");

if (missingRequired.length > 0) {
  return {
    answer: "I could not complete this request before the time budget ran out. " +
            "Missing required data: " + missingRequired.map((m) => m.spec.name).join(", "),
    partial: true,
    outcomes,
  };
}

Two rules I hold to:

  • A required tool that fails means the answer is incomplete, and the text must say so. Synthesizing silently from partial data is how an agent confidently states something it never retrieved.
  • Optional tools degrade. Their absence goes into the prompt as (unavailable: <tool> — <status>) so the model can hedge, and into metrics so you can see it.

Stop scheduling once remainingMs(toolPhaseDeadline) <= 0: return skipped for the remainder rather than starting requests that are guaranteed to abort.

Test Slow APIs and Verify What Cancellation Actually Does

None of the above is worth much without assertions. These tests run against the loopback fixture, so they are fast, deterministic, and offline.

Assert delayed-header expiry with bounded time, the right reason, and no extra retries

// test/executor.test.ts (excerpt)






test("tool deadline expires during delayed headers and does not retry", async () => {
  const fixture = await startFixture();
  try {
    const outcome = await callTool({
      name: "slow_headers",
      url: fixture.url + "/slow-headers?ms=4000",
      runDeadline: { at: now() + 5_000, kind: "run" },
      toolTimeoutMs: 1_000,
      attemptTimeoutMs: 1_200,   // > tool deadline: the tool clock wins
      maxAttempts: 3,
      validate: (j) => j as { ok: boolean },
      fetchImpl: fetch,
      random: () => 0.5,
    });

    assert.equal(outcome.status, "timeout");
    assert.equal(outcome.reason?.kind, "tool_expired");
    assert.equal(outcome.attempts, 1, "no retry after the tool deadline");
    assert.ok(outcome.elapsedMs < 1_600, "elapsed " + outcome.elapsedMs);
    assert.ok(outcome.elapsedMs >= 900, "returned too early: " + outcome.elapsedMs);
  } finally {
    await fixture.close();
  }
});

The lower bound matters as much as the upper. A wrapper that returns in 3ms has not timed out; it has failed to send the request.

Assert that a delayed body still expires after fetch resolves

test("tool deadline expires while the body is still streaming", async () => {
  const fixture = await startFixture();
  try {
    const outcome = await callTool({
      name: "slow_body",
      url: fixture.url + "/slow-body?ms=4000",
      runDeadline: { at: now() + 5_000, kind: "run" },
      toolTimeoutMs: 800,
      attemptTimeoutMs: 1_200,
      maxAttempts: 2,
      validate: (j) => j as { ok: boolean },
      fetchImpl: fetch,
      random: () => 0.5,
    });

    assert.equal(outcome.status, "timeout");
    assert.equal(outcome.attempts, 1);
    assert.ok(outcome.elapsedMs < 1_400, "elapsed " + outcome.elapsedMs);
    assert.ok(fixture.stats().clientAborts >= 1, "expected the client to abort");
  } finally {
    await fixture.close();
  }
});

This is the test that catches await fetch(...) followed by an unbounded res.json(). If your wrapper passes the delayed-header test and fails this one, the deadline was attached to the wrong lifetime.

Test transient failure then success, retry exhaustion, and cancellation during backoff

test("retries a transient 503 and then succeeds", async () => {
  const fixture = await startFixture();
  try {
    const outcome = await callTool({
      name: "flaky",
      url: fixture.url + "/flaky?key=" + crypto.randomUUID() + "&fail=1",
      runDeadline: { at: now() + 5_000, kind: "run" },
      toolTimeoutMs: 3_000,
      attemptTimeoutMs: 1_200,
      maxAttempts: 2,
      validate: (j) => j as { ok: boolean; attempt: number },
      fetchImpl: fetch,
      random: () => 0,
    });

    assert.equal(outcome.status, "ok");
    assert.equal(outcome.attempts, 2);
  } finally {
    await fixture.close();
  }
});

The key uses a fresh UUID per test so the fixture's per-key failure counter cannot leak between tests. Without that, test order changes results — the classic shared-fixture flake.

Cancellation during backoff asserts the sleep is abortable:

test("caller cancellation during backoff stops immediately", async () => {
  const fixture = await startFixture();
  const caller = new AbortController();
  try {
    const started = now();
    const pending = callTool({
      name: "flaky",
      url: fixture.url + "/flaky?key=" + crypto.randomUUID() + "&fail=5",
      runDeadline: { at: now() + 10_000, kind: "run" },
      toolTimeoutMs: 8_000,
      attemptTimeoutMs: 1_000,
      maxAttempts: 4,
      validate: (j) => j as { ok: boolean },
      fetchImpl: fetch,
      random: () => 1,          // maximum jitter: ~300ms backoff
      callerSignal: caller.signal,
    });

    setTimeout(() => caller.abort({ kind: "caller_cancelled", message: "user left" }), 150);
    const outcome = await pending;

    assert.equal(outcome.status, "cancelled");
    assert.equal(outcome.reason?.kind, "caller_cancelled");
    assert.ok(now() - started < 600, "did not abort promptly");
  } finally {
    await fixture.close();
  }
});

If sleep were a bare setTimeout, this test would finish around 1,150ms and fail. That is the whole reason it exists.

Test run exhaustion across concurrent tools without cancelling independent successes

test("run exhaustion hits concurrent tools and not the caller's siblings", async () => {
  const fixture = await startFixture();
  try {
    const runDeadline = { at: now() + 600, kind: "run" } as const;
    const [slow, fast] = await Promise.all([
      callTool({
        name: "slow_headers",
        url: fixture.url + "/slow-headers?ms=3000",
        runDeadline,
        toolTimeoutMs: 300,
        attemptTimeoutMs: 300,
        maxAttempts: 2,
        validate: (j) => j as unknown,
        fetchImpl: fetch,
        random: () => 0,
      }),
      callTool({
        name: "fast",
        url: fixture.url + "/fast",
        runDeadline,
        toolTimeoutMs: 2_000,
        attemptTimeoutMs: 2_000,
        maxAttempts: 1,
        validate: (j) => j as { ok: boolean },
        fetchImpl: fetch,
        random: () => 0,
      }),
    ]);

    assert.equal(slow.status, "timeout");
    assert.equal(slow.reason?.kind, "tool_expired");
    assert.equal(fast.status, "ok", "one tool's deadline must not cancel its sibling");
  } finally {
    await fixture.close();
  }
});

That last assertion is the whole design in one line. If fast comes back as cancelled, you have a shared controller bug.

Observe client aborts without claiming that disconnecting rolls back remote work

The fixture records clientAborts from res.on("close") when writableFinished is false. That is evidence the client disconnected. It is not evidence the server stopped working.

test("abort is visible at the server but does not imply rollback", async () => {
  const fixture = await startFixture();
  try {
    const before = fixture.stats();
    await callTool({
      name: "slow_body",
      url: fixture.url + "/slow-body?ms=1500",
      runDeadline: { at: now() + 5_000, kind: "run" },
      toolTimeoutMs: 300,
      attemptTimeoutMs: 300,
      maxAttempts: 1,
      validate: (j) => j as unknown,
      fetchImpl: fetch,
      random: () => 0,
    });

    const after = fixture.stats();
    assert.ok(after.clientAborts > before.clientAborts, "client abort should be observable");
    assert.ok(after.total === before.total + 1, "exactly one request was made");
  } finally {
    await fixture.close();
  }
});

Confirmed: the abort is observable server-side, and the executor made exactly one request (no hidden retry).

Deliberately not asserted: whether the server's handler stopped early. In this fixture it keeps its setTimeout pending and the write is discarded. A real upstream that has already committed a database write will not roll it back because your socket closed. Treat an aborted write as ambiguous, never as canceled.

Show representative output with wide timing tolerances, not benchmark claims

npm test

Representative output from the spec reporter:

▶ tool executor
  ✔ tool deadline expires during delayed headers and does not retry (1012ms)
  ✔ tool deadline expires while the body is still streaming (812ms)
  ✔ retries a transient 503 and then succeeds (168ms)
  ✔ caller cancellation during backoff stops immediately (154ms)
  ✔ run exhaustion hits concurrent tools and not the caller's siblings (307ms)
  ✔ abort is visible at the server but does not imply rollback (312ms)
▶ tool executor (2.8s)

ℹ tests 6
ℹ pass 6
ℹ fail 0
⚠️

Those durations are the shape the reporter prints, and the bounds in the assertions are deliberately wide (900–1600ms for a 1000ms deadline) so a loaded CI runner does not flake. They are not a benchmark. I did not measure throughput, p99 latency, or overhead against a no-timeout baseline for this post, and you should not quote them as if I had. If you need those numbers, measure them on your hardware with your upstream latency distribution.

Run npm test twice in a row. If the results differ, you have shared fixture state, non-injected randomness, or a timing assertion that is too tight.

Make Partial Failure and Deadline Consumption Observable

You cannot tune a timeout strategy you cannot see.

Emit structured records per attempt: queue time, remaining budget, decision, outcome

One JSON line per decision, emitted from the executor via a callback:

export interface ToolRecord {
  tool: string;
  attempt: number;
  queueMs: number;
  remainingBeforeAttemptMs: number;
  attemptElapsedMs: number;
  decision: "retry" | "stop" | "none";
  stopReason?: string;
  httpStatus?: number;
  outcome: string;
}

remainingBeforeAttemptMs is the field people forget and the one you need most. It answers "was this tool given 3,000ms or 400ms?" — which is the difference between a slow upstream and a starved tool. Those have opposite fixes.

Separate expirations, cancellations, upstream failures, and validation failures in metrics

Six counters, never one errors counter:

MetricWhat it tells you
tool_timeout{tool,reason=run_exhausted}The run budget is too small, or orchestration is slow
tool_timeout{tool,reason=tool_expired}This upstream is too slow for its deadline
tool_cancelled{tool}Clients are giving up; check p99 end-to-end
tool_upstream_failure{tool,status}Upstream health
tool_invalid_response{tool,category}Contract drift: schema changes, HTML error pages
tool_skipped{tool}Queue contention or budget starvation

run_exhausted and tool_expired for the same tool demand different remediation. Collapse them and every incident starts with an hour of guessing.

Read a trace where a retry was refused before attempts ran out

Trace from the flaky-then-exhausted path (maxAttempts: 3, toolTimeoutMs: 1_000, attemptTimeoutMs: 700):

{"tool":"orders.lookup","attempt":1,"queueMs":12,"remainingBeforeAttemptMs":988,"attemptElapsedMs":184,"httpStatus":503,"decision":"retry","stopReason":null,"outcome":null}
{"tool":"orders.lookup","attempt":2,"queueMs":0,"remainingBeforeAttemptMs":610,"attemptElapsedMs":511,"httpStatus":503,"decision":"stop","stopReason":"budget_insufficient","outcome":null}
{"tool":"orders.lookup","attempt":2,"queueMs":0,"remainingBeforeAttemptMs":610,"attemptElapsedMs":511,"httpStatus":503,"decision":"none","stopReason":null,"outcome":"upstream_failure"}

Read the decision on attempt 2: attemptsUsed = 2 < maxAttempts = 3, yet the wrapper stopped. Remaining budget was 610 − 511 ≈ 99ms, and the next backoff can be up to 300ms, so 99 <= 300 + 50. Firing attempt 3 would have produced a request guaranteed to abort at ~0ms of flight, plus a socket teardown. The metric says budget_insufficient, not attempts_exhausted, and that phrasing sends you to the right fix: raise the tool deadline or lower the attempt timeout, not "increase max attempts".

Document the Limits and Production Mitigations

Deadlines bound your patience. They do not bound the world.

Remote side effects survive cancellation: use idempotency keys and reconcile ambiguous writes

Confirmed: aborting a fetch closes the client connection. It does not unwind work the server has already done. A POST /charges that times out at the client may have created the charge.

Mitigations that actually work:

  • Send an Idempotency-Key generated per logical call (not per attempt) so retries and manual replays collapse to one effect. Generate it before the first attempt and reuse it across attempts.
  • Classify a timed-out write as ambiguous, never failed.
  • Reconcile out of band: query by idempotency key or a client-generated correlation id before retrying at a higher level.
  • Prefer designs where the write is two-step (reserve, then confirm with a client-chosen id) if the operation is expensive to duplicate.

CPU-bound tools need isolation or worker termination, not just a signal

Anything that ignores the signal — a schema validator with catastrophic backtracking, a JSON.parse of a huge payload, a native binding, a synchronous crypto call — will run to completion and block every timer on the thread. Options in increasing order of cost:

  1. Bound the input (maxBytes, response size caps, tree depth).
  2. Move to a worker_threads worker and worker.terminate() on deadline.
  3. Move to a child process or a separate service with a hard kill.

Option 1 handles most JSON APIs. Options 2 and 3 are for parsing untrusted documents, running generated code, or calling native libraries.

Monotonic deadlines do not cross hosts: propagate durations with downstream caps

performance.now() values are relative to a per-process time origin; sending deadline.at to another service is meaningless. What crosses the boundary is a duration. Convert at the edge and cap it downstream:

const leftMs = remainingMs(toolDeadline);
const headers = {
  "x-request-budget-ms": String(Math.floor(leftMs)),
  "x-request-caller": "agent-runner",
};

// Downstream, never trust the incoming value blindly:
const requested = Number(req.headers["x-request-budget-ms"] ?? "0");
const localCap = 2_000;
const effective = Math.max(0, Math.min(requested, localCap));

The Math.min is not optional. An upstream that sends 600000 (or a malicious client that does) must not be able to make your service hold a request open. A propagated budget is a hint, never an authority.

Bound response sizes, concurrency, and retry work so timeouts do not just move load

Timeouts move load. Every expired request that a retry replaces is a second request against an upstream that may already be struggling. Paired with a retry storm, a 1,000ms deadline can double your outbound request rate at exactly the wrong moment.

Guard rails:

  • Per-tool concurrency limit (the semaphore) and a global outbound-request cap.
  • A retry budget: retries as a fraction of total requests, not just per-call attempt counts.
  • Response size caps so a slow upstream cannot force you to buffer tens of megabytes.
  • Bounded total retry work per run — the run budget already does this if you never extend it.
  • Full jitter, so retries do not synchronize into a thundering herd after an upstream blip.

Conclusion: Make Time a Shared Execution Constraint

The three clocks model is not about picking better numbers. It is about making time a property of execution that every layer reads and no layer writes.

The invariants: deadlines never extend, retries consume existing budget, cleanup is unconditional

  1. Deadlines never extend. A downstream scope takes earliest(parent, own). Nothing resets a clock.
  2. Retries consume existing budgets. Both gates — attempt allowance and remaining milliseconds — must pass. Refusing a retry is a normal outcome, not a bug.
  3. Cleanup is unconditional. finally disposes timers, removes listeners, releases semaphore slots, and cancels unread bodies.
  4. Partial results stay explicit. Required-tool failure produces a labelled incomplete answer. Optional-tool failure is degraded and recorded.
  5. The first reason wins. caller_cancelled, run_exhausted, tool_expired, and attempt_expired are distinct and stay distinct in logs and metrics.
  6. Abort is cooperative. Bounds and isolation handle what a signal cannot.

The one sentence I would put in a code review: a tool timeout that creates new budget on retry is not a timeout, it is a suggestion.

Further Reading

Supplied source: the Startup Fortune listing behind the slow-tool-call problem

The feed entry (published 2026-11-22) is titled around setting an AI agent tool call timeout before one slow API stalls a run: Startup Fortune listing via Google News RSS. Only the headline and summary were available; the implementation and numbers in this post are independent.

Node.js docs: AbortController and AbortSignal

Reference for AbortController, AbortSignal.reason, AbortSignal.timeout(), and AbortSignal.any() in the Node.js globals documentation. If you prefer built-ins over a hand-rolled scope, AbortSignal.any([...]) composes signals — but it does not give you the first-reason-wins bookkeeping this post relies on for metrics.

Node.js docs: monotonic timing with performance.now()

performance.now() semantics, its per-process time origin, and why it is the right base for deadline arithmetic: Node.js perf_hooks documentation.

Fetch standard: request abortion and response body handling

The normative definition of what aborting a fetch does to the request and the response body stream in the Fetch standard — the reason raceAbort plus a signal-aware fetch is enough to bound body reading in practice.

RFC 9110: idempotent methods and Retry-After

RFC 9110 covers idempotent method definitions (§9.2.2) and Retry-After (§10.2.3), including the delta-seconds and HTTP-date forms you need to parse before deciding a retry is affordable.

Share this post

More posts

Loading comments…