Scaling Agents on Kubernetes with acpx and ACP: Isolation, Routing, and Cost per Agent

Scaling Agents on Kubernetes with acpx and ACP: Isolation, Routing, and Cost per Agent

pr0h0••21 min read•ai
AI Usage (95%)
On this page

Introduction

This post is about scaling AI agent runtimes on Kubernetes with acpx and ACP: packaging an agent as a deployable pod contract, placing session and workspace state, routing tool calls across a fleet, isolating untrusted execution, and modelling cost per concurrent agent. It builds on the talk "Scaling Agents on Kubernetes with acpx and ACP" by Onur Solmaz of OpenClaw, surfaced through an aggregator listing dated 2026-11-18, whose description runs three lines: acpx and ACP are a way to package agent runtimes and protocols so they can be deployed beyond a single developer machine, and scaled on Kubernetes.

That listing is the entire public record I could confirm. Everything past it — pod shapes, load numbers, cost arithmetic — is either inference from the protocol or arithmetic you can redo yourself. I've marked which is which, because the distinction matters when you're the one sizing the cluster.

The position this post argues: ACP is a protocol boundary, not a runtime, and that changes what "scaling" means. An HTTP service scales when you add replicas behind a request-balancing proxy. An agent fleet doesn't, because the unit of work is a session that owns state across many turns, and often across many tool calls. Scale agents like a stateless web tier and you get session-not-found errors, split workspaces, duplicated side effects, and a bill you can't explain per tenant.

⚠️

I had no access to the acpx runtime or a recording of this talk while writing. Every number in the load and cost sections is a placeholder with a stated method, not a benchmark. Run the probes yourself before you size anything.

What the Talk Establishes, and What This Post Infers

The Three Claims Worth Building On

The listing supports three claims. All three hold up structurally.

  1. Agent runtimes need packaging before they scale. A coding agent that runs as a child process on a laptop has no defined interface, no resource contract, and no lifecycle. Deploying it means giving it one.
  2. ACP is the interface being standardised on. The Agent Client Protocol defines how a client — an editor, a CLI, a gateway — talks to an agent. It's a real, documented spec at agentclientprotocol.com, JSON-RPC 2.0 based.
  3. Kubernetes is the deployment target. Framing agent scaling as a Kubernetes problem is really a claim that agents should inherit the same isolation, quota, and scheduling machinery as everything else.

Where the Public Details Run Out

ClaimStatus
Onur Solmaz of OpenClaw presented acpx and ACP at AI EngineerStated in the source listing
The talk covers packaging agent runtimes and scaling on KubernetesStated in the source listing
ACP is a JSON-RPC protocol between client and agentConfirmed by the ACP spec
acpx is a headless ACP client / packaging layerInference from the name and framing, not confirmed
Agents are horizontally scalable with session affinityInference — my design reasoning, not a talk claim
Anything about the talk's benchmarks or cost figuresUnknown. I have no numbers from the talk.

If acpx turns out to do less than I assume, the manifests and the probe still apply; you'd just point them at a different launch command.

ACP Is a Protocol Boundary, Not a Runtime

The Client-to-Agent Contract and Why It Matters Operationally

ACP defines a session-oriented conversation between two roles: a client that owns the user interaction and the filesystem, and an agent that does the reasoning and requests actions. The documented method surface includes initialize, session/new, session/prompt, session/update notifications for streaming progress, session/cancel, and session/request_permission for cases where the agent needs the client to approve an action. Transport is stdio with JSON-RPC — no HTTP surface by default.

A turn looks roughly like this on the wire:

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":1,"clientCapabilities":{}}}
{"jsonrpc":"2.0","id":2,"method":"session/new","params":{"cwd":"/workspace/tenant-42","mcpServers":[]}}
{"jsonrpc":"2.0","id":3,"method":"session/prompt","params":{"sessionId":"s-1","prompt":[{"type":"text","text":"Summarise CHANGELOG.md"}]}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"s-1","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"..."}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}

Two operational facts fall straight out of that exchange.

First, the request/response unit is a session, not a prompt. The server returns sessionId and it scopes everything after it. Any replica receiving a session/prompt must already know that sessionId. A load balancer that picks a backend per request will send turn 3 to a pod that has never seen s-1.

Second, the agent asks the client for things. Permission requests, filesystem reads, terminal execution — all of it flows client-ward. Which makes your client part of the trust boundary, not a dumb proxy. If the gateway auto-approves every permission request so it can stay stateless, you've moved the approval decision into infrastructure that has no idea what it's approving.

Why a Wire Protocol Changes Your Deployment Shape

A stdio JSON-RPC protocol can't be exposed as a Kubernetes Service directly. You need exactly one of:

  • a gateway process per pod that owns the stdio pipe and exposes WebSocket or HTTP to the Service, or
  • a sidecar doing the same, leaving the agent container untouched, or
  • a gateway outside the pod (an ingress-level relay) multiplexing many agent pods.

Where session routing happens follows from that choice. Gateway-inside-pod makes routing a client concern and the Service can stay round-robin. Gateway-outside-pod forces the routing layer to be session-aware, and now you own consistent hashing.

I'd put the gateway in the pod. A bad route then costs you one replica instead of many, and you can drain sessions by pulling a pod out of the Service endpoints while its gateway finishes the open ones.

What acpx Adds on Top of the Protocol

Packaging an Agent Runtime as a Pod Contract

Strip the branding and acpx's job is: take an agent that expects to be spawned with stdio attached, and give it a lifecycle a container runtime can manage. Concretely that means a stable entrypoint, a defined working directory, a loud failure on bad config, and a handshake you can probe.

The pod contract I'd insist on, whatever the wrapper:

Contract itemWhy it must be explicit
Entrypoint takes config from a mounted file, not argv secretsargv is visible in ps and often in pod spec
Stdio pipe owned by exactly one processtwo writers to one JSON-RPC stream corrupt it
Workspace root is a declared pathotherwise cwd in session/new is attacker-influenced
Handshake on startup, exit non-zero on failurereadiness must mean "can complete initialize"
SIGTERM drains open sessionsotherwise every rollout truncates live turns

The Parts That Must Be Stateless Before Horizontal Scaling Works

An agent pod can be replaced at any moment only if these live outside it:

  • conversation transcripts (owned by the client or a session store)
  • workspace files the session depends on (a Volume, not the container layer)
  • credentials (projected ServiceAccount tokens or an external secret store)
  • tool registry and MCP server addresses (config, not baked images)
  • cost/usage counters (emitted, not accumulated in memory)

Everything else can be local. The trap is the agent's own scratch directory: it looks ephemeral, but a mid-turn compaction or a multi-step edit will read files it wrote two turns ago. Make that scratch an emptyDir, evict the pod, and the next replica resumes a session against a workspace that no longer has the intermediate state.

Where Agent State Actually Lives

Session State, Conversation State, and Workspace State Are Different Problems

People say "agent state" and mean three things, each with its own durability requirement.

StateLifetimeWhere it belongsLoss impact
Sessionone connection, minutes to hoursin-memory on the owning pod, plus a session record in Redisuser must restart the session
Conversationdays, survives restartsdurable store keyed by session/thread idlost context, wasted tokens on replay
Workspaceas long as the user caresPVC or object store synced per turncorrupted edits, lost intermediate files

Session state is the only one legitimately bound to a pod, and that's a feature: it lets you evict aggressively. Conversation state must never be pod-local. Workspace state is the one teams forget, and the one that produces "the agent deleted my file" bug reports.

Session Affinity and What It Costs You

If a session lives in one pod's memory, you need affinity. Two ways to get it:

  • Client-side sticky routing — the gateway is in the pod and the client holds a direct connection for the life of the session. Simple and correct, at the cost of the client handling reconnection when the pod dies.
  • Consistent-hash routing at a shared gateway — the gateway hashes sessionId to a pod index. Cheap, but a scale-up reshuffles the ring and evicts live sessions.

Either way, your scaling granularity becomes the session, not the request. You can't smooth a burst by spraying turns across ten pods. You can only move whole sessions, and moving one means either a handoff protocol or a client-visible restart.

Bounded, though. As long as sessions are short relative to the traffic pattern, affinity is nearly free. If your agents hold sessions open for hours and check whether anyone is still typing, you're running a stateful service — plan for it with a PodDisruptionBudget and a long termination grace period, not an optimistic rolling update.

Routing Tool Calls Across a Scaled Fleet

Routing Dimensions: Session, Tenant, Tool, and Model

An agent pod isn't the only thing that needs routing. Tool calls fan out to MCP servers, shell execution, and model endpoints, each with a different rule.

DimensionRouting ruleWhy
Sessionconsistent hash on sessionIdmust land on the pod holding the session
Tenanthash on tenant id, or dedicated node poolnoisy-neighbour and data-residency control
Toolper-tool Service, not per-agent sidecartool servers scale independently of agent pods
Modelmodel-aware poola pod holding a 200k-token context is not interchangeable with a small-model pod

The model dimension is the one that bites. Route a turn to a pod that has the session but not the model alias and you get a runtime error mid-turn, after you've already paid for the context.

Failure Modes When Routing Is Wrong

  • Round-robin across replicas with in-memory sessions. Symptom: intermittent session not found on turn 2 and later. Looks random; it isn't.
  • Session affinity by client IP. Symptom: every user behind one corporate NAT lands on one pod. Correlates with office hours.
  • Hash ring without a handoff. Symptom: a burst of failed sessions during a scale-up, at exactly the moment you added capacity.
  • Tool calls routed through the agent pod. Symptom: agent pod memory grows with tool concurrency; OOMKilled during long file-processing calls.
  • Retry at the wrong layer. Symptom: one user prompt produces three session/prompt calls under the same idempotency key and three tool invocations.

That last one needs its own callout.

⚠️

session/prompt is not idempotent. A gateway that retries on timeout can execute a destructive tool twice. Retry only before the first session/update notification for that turn, and only if you can prove no tool call started.

Horizontal Scaling Measured in Concurrent Agents, Not Requests

Choosing the Right Saturation Signal

CPU is the wrong signal. An agent spends most of its wall-clock time blocked on a model API response or a tool subprocess; a pod sitting at 8% CPU can be saturated at its concurrency limit.

Better signals, most useful first:

  1. Open sessions per pod — the closest thing to true occupancy. Expose agent_sessions_open as a gauge and scale on it.
  2. In-flight turns per pod — catches the case where sessions are open but idle.
  3. Queue wait time at the gateway — what the user actually feels. Scale when p95 wait exceeds your target, not when replicas exceed a number.
  4. Model-side 429 rate — if your provider is throttling you, adding pods makes it worse.

An HPA on a pod-level gauge looks like this:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: agent-runtime
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: agent-runtime
  minReplicas: 3
  maxReplicas: 40
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 900   # sessions are long; don't churn
      policies:
        - type: Pods
          value: 2
          periodSeconds: 120
  metrics:
    - type: Pods
      pods:
        metric:
          name: agent_sessions_open
        target:
          type: AverageValue
          averageValue: "6"            # concurrent agents per pod

The 900-second stabilization window isn't timidity. If sessions take twenty minutes, a fast scale-down evicts sessions that were about to finish paying off their context cost.

A Reproducible Load Probe with kubectl and a Synthetic Agent Client

You can't size a fleet from a dashboard. You need a probe that drives real sessions and records where it breaks. ACP is newline-delimited JSON-RPC over stdio, so the probe is a process wrapper — point AGENT_CMD at whatever your acpx invocation is.

probe.mjs
import { spawn } from "node:child_process";


const CMD  = process.env.AGENT_CMD ?? "acpx";
const ARGS = (process.env.AGENT_ARGS ?? "").split(" ").filter(Boolean);
const SESSIONS = Number(process.env.SESSIONS ?? 8);
const TURNS    = Number(process.env.TURNS ?? 5);

function startAgent() {
const child = spawn(CMD, ARGS, { stdio: ["pipe", "pipe", "inherit"] });
const pending = new Map();
let nextId = 1;
let buf = "";

child.stdout.setEncoding("utf8");
child.stdout.on("data", (chunk) => {
  buf += chunk;
  let nl;
  while ((nl = buf.indexOf("\n")) !== -1) {
    const line = buf.slice(0, nl).trim();
    buf = buf.slice(nl + 1);
    if (!line) continue;
    let msg;
    try { msg = JSON.parse(line); } catch { continue; }
    if (msg.id !== undefined && pending.has(msg.id)) {
      const { resolve, reject } = pending.get(msg.id);
      pending.delete(msg.id);
      msg.error ? reject(new Error(JSON.stringify(msg.error))) : resolve(msg.result);
    }
    // session/update notifications are ignored by this probe on purpose
  }
});

const call = (method, params) => new Promise((resolve, reject) => {
  const id = nextId++;
  pending.set(id, { resolve, reject });
  child.stdin.write(JSON.stringify({ jsonrpc: "2.0", id, method, params }) + "\n");
});

return { child, call };
}

const { child, call } = startAgent();
await call("initialize", { protocolVersion: 1, clientCapabilities: {} });

const started = performance.now();
const rows = await Promise.all(
Array.from({ length: SESSIONS }, async (_, i) => {
  const { sessionId } = await call("session/new", { cwd: "/tmp", mcpServers: [] });
  const lat = [];
  for (let t = 0; t < TURNS; t++) {
    const s = performance.now();
    await call("session/prompt", {
      sessionId,
      prompt: [{ type: "text", text: "Reply with exactly: OK" }]
    });
    lat.push(performance.now() - s);
  }
  lat.sort((a, b) => a - b);
  const pick = (q) => lat[Math.min(lat.length - 1, Math.ceil(lat.length * q) - 1)];
  return { i, p50: Math.round(pick(0.5)), p95: Math.round(pick(0.95)) };
})
);

const wall = (performance.now() - started) / 1000;
child.kill("SIGTERM");

console.log(JSON.stringify({
sessions: SESSIONS,
turns: SESSIONS * TURNS,
wallSeconds: Number(wall.toFixed(1)),
turnsPerMinute: Number(((SESSIONS * TURNS) / wall * 60).toFixed(1)),
p50: Math.round(rows.reduce((a, r) => a + r.p50, 0) / rows.length),
p95: Math.round(Math.max(...rows.map((r) => r.p95)))
}, null, 2));

Run it in-cluster as a Job so it measures the real pod, not your laptop:

SESSIONS=6 TURNS=5 AGENT_CMD=acpx \
kubectl run agent-probe --rm -it --restart=Never \
  --image=registry.example.com/agent-runtime:probe \
  --env=SESSIONS=6 --env=TURNS=5 --env=AGENT_CMD=acpx \
  -- node probe.mjs

The output shape you're looking for:

{
  "sessions": 6,
  "turns": 30,
  "wallSeconds": 41.3,
  "turnsPerMinute": 43.6,
  "p50": 742,
  "p95": 3120
}

Those figures are a template, not a measurement — substitute your own run. The method is the part that matters: step SESSIONS up 2, 4, 6, 8, 12 until p95 stops being flat. The first step where p95 roughly doubles is your per-pod concurrency ceiling. Feed that number, minus a margin, into the HPA's averageValue.

Two probe caveats worth writing down. It ignores session/update notifications, so it measures time-to-final-result rather than perceived latency; a streamed first token will feel faster to users than this number suggests. And it doesn't exercise permission requests, so an approval-gated agent will be slower in production than the probe reports.

Per-Agent Isolation on Kubernetes

Pod, Namespace, and Node Boundaries and What Each One Actually Buys

Be precise here, because plenty of "we isolate agents in pods" claims do less than the author thinks.

BoundaryWhat it actually gives youWhat it does not
Podseparate PID/IPC/network namespace, separate cgroupshares the host kernel
NamespaceRBAC scope, NetworkPolicy scope, ResourceQuota scopenot a kernel boundary
Nodeseparate kernel, unless you add a sandbox runtimedoes not restrict network by itself

Only a node gives kernel-level separation, and only if you aren't running untrusted code on a shared kernel. If the agent executes arbitrary shell commands — and coding agents do — pod-per-agent on a shared kernel is not a security boundary against a kernel exploit. For untrusted input, add a sandbox runtime class (gVisor, Kata) or dedicate a node pool. That's inference from how containers work, not a claim from the talk, but it's the difference between a real boundary and a naming convention.

An Isolation Checklist: ServiceAccount, NetworkPolicy, Resource Limits, Filesystem

Work through this before you scale past one node.

  • Dedicated ServiceAccount per tenant tier, with automountServiceAccountToken: false unless the agent genuinely calls the API. Most don't.
  • Default-deny NetworkPolicy in the agent namespace, with explicit egress allow-lists for the model endpoint, the MCP servers, and the package mirror.
  • ResourceQuota and LimitRange on the namespace, so a runaway agent still can't starve the cluster.
  • Requests equal to limits for CPU if you want predictable latency; burstable agents get throttled precisely during the long tool call.
  • readOnlyRootFilesystem: true with an explicit emptyDir for scratch and a PVC for workspace.
  • allowPrivilegeEscalation: false, runAsNonRoot, drop CAP_NET_RAW.
  • No secrets in env. Mount them as files or pull from an external secret store; env is readable by any process in the pod and regularly leaks into crash dumps.

Modeling Cost per Concurrent Agent

The Cost Formula and Which Terms Dominate

The useful unit is cost per concurrent agent-hour, not cost per pod-hour, because that's the number you weigh against the value of a running session.

C = (vCPU * p_vCPU + GiB * p_GiB) / (N_max * u)  +  T * c_turn  +  S
  • N_max * u = effective concurrent agents a pod sustains (ceiling times utilization)
  • T = turns per active agent-hour
  • c_turn = blended token cost per turn
  • S = storage and sidecar overhead, amortised

A Worked Cost Model with Sensitivity Ranges

Illustrative rates — swap in your actual bill: p_vCPU = $0.031611/hr, p_GiB = $0.004237/hr (a common on-demand figure used for cost allocation), pod at 2 vCPU / 4 GiB, N_max = 6, u = 0.55.

Pod cost per hour: 2 × 0.031611 + 4 × 0.004237 = $0.0802.

Effective concurrent agents: 6 × 0.55 = 3.3.

Compute cost per concurrent agent-hour: 0.0802 / 3.3 = $0.0243.

Now the token side, with illustrative pricing of $3/Mtok input and $15/Mtok output and a 6,000-in / 700-out turn: 6000×3/1e6 + 700×15/1e6 = $0.018 + $0.0105 = $0.0285 per turn. At 40 turns per active agent-hour that's $1.14/hr.

UtilizationEffective agents/podCompute $/concurrent-agent-hr
0.201.2$0.0668
0.402.4$0.0334
0.553.3$0.0243
0.804.8$0.0167

The result is unambiguous: token spend is roughly 47× compute spend per concurrent agent-hour in this model. Over-provisioning by a factor of two costs you $0.02/hr per agent; a badly chosen context window costs dollars per hour. Idle concurrency is a real cost, but a second-order one. The leverage sits in what the agent sends to the model, and in whether you're holding sessions open past their useful life.

The sensitivity that surprises teams is turn rate. Double the turns per hour and token cost doubles linearly; long-running sessions with a growing transcript push c_turn up superlinearly, because every turn re-sends history. A cost model that assumes a flat c_turn is wrong for any session past roughly 20 turns.

A Minimal Kubernetes Deployment Skeleton for Agent Runtimes

Two blocks: the workload, and the guard rails.

agent-runtime.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent-runtime
namespace: agents
spec:
replicas: 3
selector:
  matchLabels: { app: agent-runtime }
template:
  metadata:
    labels: { app: agent-runtime }
  spec:
    serviceAccountName: agent-runtime
    automountServiceAccountToken: false
    terminationGracePeriodSeconds: 300
    securityContext:
      runAsNonRoot: true
      runAsUser: 10001
      fsGroup: 10001
      seccompProfile: { type: RuntimeDefault }
    topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: ScheduleAnyway
        labelSelector:
          matchLabels: { app: agent-runtime }
    containers:
      - name: agent
        image: registry.example.com/agent-runtime:0.1.0
        command: ["/usr/local/bin/acpx"]
        args: ["--config", "/etc/agent/config.yaml"]
        ports:
          - { name: gateway, containerPort: 8080 }
        resources:
          requests: { cpu: "2", memory: "4Gi" }
          limits:   { cpu: "2", memory: "4Gi" }
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities: { drop: ["ALL"] }
        readinessProbe:
          exec:
            command: ["node", "/opt/probe/healthcheck.mjs"]
          initialDelaySeconds: 10
          periodSeconds: 20
          timeoutSeconds: 5
        volumeMounts:
          - { name: config, mountPath: /etc/agent, readOnly: true }
          - { name: scratch, mountPath: /tmp }
          - { name: workspace, mountPath: /workspace }
    volumes:
      - name: config
        configMap: { name: agent-config }
      - name: scratch
        emptyDir: { sizeLimit: 2Gi }
      - name: workspace
        persistentVolumeClaim: { claimName: agent-workspace }
guardrails.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: agent-runtime
namespace: agents
spec:
maxUnavailable: 1
selector:
  matchLabels: { app: agent-runtime }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: agent-default-deny
namespace: agents
spec:
podSelector: {}
policyTypes: ["Ingress", "Egress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: agent-egress-allow
namespace: agents
spec:
podSelector:
  matchLabels: { app: agent-runtime }
policyTypes: ["Egress"]
egress:
  - to:
      - namespaceSelector:
          matchLabels: { kubernetes.io/metadata.name: mcp }
    ports: [{ protocol: TCP, port: 8443 }]
  - to:
      - ipBlock: { cidr: 203.0.113.10/32 }
    ports: [{ protocol: TCP, port: 443 }]

The 203.0.113.10/32 is a documentation-range placeholder — pin your actual model endpoint. Default-deny with no allow rule is the correct starting state. Every egress you add should have a name attached to it in review.

Observability: What to Instrument Before You Scale Past One Node

Instrument these before the second replica exists. None of them are exotic.

MetricTypeWhy it earns its place
agent_sessions_opengaugethe HPA signal, and your occupancy truth
agent_turns_total{stop_reason}counterdistinguishes end_turn from cancelled from error
agent_turn_duration_secondshistogramp95 per model alias, not global
agent_tool_calls_total{tool,outcome}counterfinds the tool that is quietly failing
agent_tokens_total{direction,model}counterthe only way to reconcile the cost formula
agent_permission_requests_total{decision}counterauto-approve hiding in production

Two structural rules. Tag every metric with tenant and model from day one — retrofitting labels onto a counter is a re-instrumentation project. And emit one span per session, with a child span per turn and per tool call, because the question you actually ask at 3am is "which turn hung," not "which pod was busy."

Defensive Defaults and Mitigations for Scaled Agent Fleets

💪

Cap the session before you cap the cluster. A max_turns and a max_context_tokens hard limit per session stops a runaway loop from eating a node — and stops it eating your model budget, which is the larger number.

  • Deny-by-default egress. Agents make outbound calls to whatever the model suggests; an allow-list is the highest-value control you have.
  • Never auto-approve session/request_permission in the gateway. If you must, log the request payload and require an allow-list of tool names.
  • Bind cwd server-side. session/new carries a working directory. If it comes from user input, resolve it against a fixed workspace root and reject escapes.
  • Kill switch on spend, not on CPU. A per-tenant token budget enforced at the gateway degrades gracefully; an OOM kill mid-turn does not.
  • Cap concurrent sessions per tenant with a token bucket at the gateway, so one tenant can't reach your maxReplicas alone.
  • Retry only pre-tool. See the warning above.

Failure Modes and How They Surface in Practice

FailureObservable symptomLikely cause
session not found on later turnserrors spike at replica count changesaffinity not implemented, or hash ring reshuffle
Context loss mid-sessiontoken count per turn resets, agent repeats itselfconversation state was pod-local
Workspace corruptionfiles from two sessions interleavedshared PVC across concurrent sessions
OOMKilled during file workrestart count climbs under load, CPU stays lowtool calls executed in-process
Model 429 stormp95 climbs after scale-upscaling on load when the bottleneck is provider quota
Doubled tool side effectsduplicate rows, duplicate commitsgateway retry on session/prompt
Silent permission bypasspermission_requests_total at zerogateway auto-approve shipped as a "fix"

That last row belongs in a postmortem template. It's the failure mode where the system gets more reliable and less safe at the same time.

Conclusion: What to Verify Before Adopting acpx and ACP

The talk's framing is right and the direction is right. ACP being a documented protocol rather than a vendor SDK is what makes portable agent packaging possible at all, and Kubernetes is the only reasonable place to put it once you're past a handful of concurrent users.

Three things decide whether it works for you, and the talk settles none of them:

  1. Does acpx give you a drainable lifecycle? Verify SIGTERM handling, verify the readiness handshake actually calls initialize, verify a rollout doesn't truncate live turns. Run a rollout with a long session open and count the failures.
  2. Where does your session state live? "In the pod" means you have a stateful service and need a PodDisruptionBudget plus a long grace period. "Nowhere yet" means fix that before you add replicas.
  3. What actually dominates your cost? Run the probe, get N_max, measure utilization, then multiply out the formula. In every model I've built, tokens beat compute by one to two orders of magnitude. If that holds for you, optimize the context, not the node pool.

The verifiable core of the talk is small: a protocol, a wrapper, a deployment target. The hard engineering is state placement and routing, and that part is yours.

Further Reading

Share this post

More posts

Loading comments…