2.4T Parameters, 60% Cheaper Per Task: Auditing Qwen4-Max After Retries

2.4T Parameters, 60% Cheaper Per Task: Auditing Qwen4-Max After Retries

pr0h0••23 min read•ai
AI Usage (83%)
On this page

Introduction: Auditing Qwen4-Max's 60% Cheaper-Per-Task Claim

Alibaba's Qwen4-Max arrived in reports dated 16 November 2026 with two numbers attached: 2.4 trillion parameters and roughly 60% lower cost per task. The same coverage says it beats US rivals on coding and cybersecurity benchmarks and that the release bundles safety upgrades. This post separates what the release coverage states from what it leaves out, explains why MoE sparsity makes the 2.4T headline mostly irrelevant to your API invoice, and then builds a retry-adjusted cost-per-successful-task worksheet you can run against your own tasks this week — including where a 60% headline collapses to 14% and where it is plausible.

The parameter count is an architecture fact. The 60% figure is not: it describes a workload — presumably Alibaba's — on tasks you cannot inspect, under a retry policy you do not know. After working through the arithmetic below, my position is simple: a per-task cost claim means nothing until you know tokens per successful task and the retry distribution behind it. Until then, "60% cheaper" is a hypothesis about your workload, not a price list.

Everything below is arithmetic over stated assumptions. I have not run Qwen4-Max, and I have not seen an independent reproduction of its benchmark results. When I say "likely" or "untested", I mean it.

What Qwen4-Max Actually Claims — and What the Coverage Leaves Out

Confirmed, Reported, and Unknown Claims in the Qwen4-Max Release

Benchmark tables travel faster than methodology sections, so keep the categories separate. This is how the release looks from the coverage I have.

ClaimStatusNotes
Alibaba released a flagship model named Qwen4-MaxReportedConsistent across the coverage dated 16 Nov 2026
2.4 trillion total parametersReportedDescribed as an MoE; the coverage does not state active parameters per token
~60% lower per-task costReported, as a vendor claimThe definition of "task", the retry policy, and the comparison baseline are not in the coverage
Outperforms US rivals on coding benchmarksReported, as a vendor claimBenchmark names, versions, and harnesses not stated in the coverage I have
Outperforms US rivals on cybersecurity benchmarksReported, as a vendor claimWhether this means exploit-level tasks or multiple-choice knowledge is not stated
Safety upgrades bundled into the releaseReportedNo specifics in the coverage beyond the fact that upgrades exist
Active parameters per tokenUnknownNot stated in the coverage. This is the single most important missing number
Unit pricing, context window, weight availabilityUnknownNot stated in the coverage
Independent reproduction of any resultUnknown / untestedI found none

The missing number that matters most is active parameters. Without it you cannot reason about throughput, latency, or self-hosting feasibility — only total memory footprint, which is a different question.

Why "60% Cheaper Per Task" Is a Retry Claim, Not a Price Cut

A price cut changes a rate card: $X per million input tokens becomes $0.8X. A per-task drop changes a ratio:

cost per successful task = (Σ cost of every attempt, including failed ones) / successes

The denominator is where the engineering lives, and it is where most LLM economics arguments quietly go wrong. You can get a 60% per-task improvement with no rate-card change at all — by raising first-attempt pass rate and making outputs shorter. You can also get a 60% worse per-task cost on a model that is cheaper per token, if it needs more attempts or emits more reasoning tokens per attempt.

So the claim is compound: price, token volume, success rate. Coverage gives you the compound. My reading is that the vendor's 60% almost certainly includes at least one channel that is not a price cut — this is inference, not something the coverage states.

MoE Mechanics: 2.4T Total Parameters vs. Active Parameters Per Token

Sparse Activation and the Compute You Actually Pay For

A mixture-of-experts layer replaces one feed-forward block with N expert blocks plus a router. For each token, the router picks the top-k experts and only those run. Total parameters sum every expert. Active parameters per token are the subset that fires, plus shared attention and embedding weights.

Published examples show the shape:

ModelTotal paramsActive per tokenRatio
Switch Transformer (2021, largest config)up to 1.6Tnot comparable across configs—
Mixtral 8x7B46.7B~12.9B~28%
DeepSeek-V3671B37B~5.5%

Ratios vary widely by design, which is why you cannot infer Qwen4-Max's active count from "2.4T". A 5% ratio gives about 120B active; 20% gives about 480B. Those designs have very different serving economics and almost identical headlines. The active parameter count for Qwen4-Max is unknown in the coverage I have.

Forward-pass FLOPs per token scale roughly with active parameters, not total. The 2.4T number is not "how much compute a token costs"; it is "how much state the model has".

Why Total Parameter Count Is a Memory Story, Not a Token-Cost Story

Here is what does follow from the headline, as arithmetic rather than a deployment report:

2.4e12 parameters × 2 bytes (BF16)  ≈ 4.8 TB of weights
2.4e12 parameters × 1 byte  (FP8)   ≈ 2.4 TB
2.4e12 parameters × 0.5 byte (4-bit) ≈ 1.2 TB

Add KV cache, activations, and framework overhead. For scale: an H200 has 141 GB of HBM3e, so 1.2 TB of 4-bit weights needs at least nine of them just to hold parameters, and realistically more once KV cache and headroom are included. At BF16, you are past thirty.

For a practitioner, that is the entire content of the 2.4T number: this is not a single-node model, and probably not an eight-GPU model. It says nothing about your per-token bill.

💡

If you call this over an API, MoE sparsity is your provider's gross-margin story, not your invoice line item. Hosted inference is priced per token. You benefit from sparsity only if the provider passes it through, or if lower active compute improves latency and therefore your retry timing.

What a 2.4T Model Does to Your Serving Topology

If you self-host — and at 2.4T total parameters, most teams will not — the topology constraints are the interesting part:

  • Expert parallelism. Experts get distributed across devices, which means an all-to-all collective inside every MoE layer. Inter-node bandwidth becomes a first-order cost, not a detail.
  • Load imbalance. Routing is learned and uneven. Hot experts become the stragglers, so tail latency is driven by the router's worst distribution, not the average. Capacity factors and auxiliary load-balancing losses exist to bound this, and they trade quality for throughput.
  • Batch-size sensitivity. Sparse models need larger batches to amortise the all-to-all. Small interactive batches leave hardware idle.
  • Ramp cost. Cold starts on a multi-node expert-parallel deployment are measured in minutes, which interacts badly with autoscaling on bursty agent traffic.

None of this appears in a benchmark table. All of it shows up in your p99.

The Definitional Gap: Per-Token Pricing vs. Per-Task Cost

Five Channels That Can Produce a 60% Per-Task Drop

A 60% per-task improvement has to come from somewhere. There are five candidates, and only one is a price cut.

ChannelWhat changesWhere it shows up in your metering
1. Higher first-attempt pass rateRetries per success dropattempts per task in your wrapper
2. Shorter outputs per attemptFewer output tokensoutputTokens per attempt
3. Shorter inputs per attemptFewer input tokens, sometimes via prompt cachinginputTokens, cachedInputTokens
4. Routing / tieringCheap model handles most trafficWhich model served each attempt
5. Rate card cutPrice per million tokensYour price table

Channels 1–4 are engineering outcomes, and all are measurable in your own harness. Channel 5 is the only one you can read off a pricing page. If the vendor's 60% comes mostly from channels 1 and 2, it will reproduce on your workload if and only if your workload has similar retry behaviour. For structured output with a deterministic validator, that is plausible. For open-ended security triage, much less so.

Tokens Per Successful Task Is the Only Denominator That Matters

Define it precisely, because vague definitions are how cost claims survive contact with reality:

TSPST = (Σ input tokens across all attempts + Σ output tokens across all attempts) / successful tasks

Two notes. Failed attempts count in the numerator; that is the point. TSPST also is not cost per successful task, because input and output tokens are priced differently — usually by a factor of three to five. A model that trades verbose output for longer input can look better on TSPST and worse on cost. Track both.

Building the Retry-Adjusted Cost Worksheet

The Formula: Expected Attempts and Cost Per Success

For a task family with independent attempts and an uncapped retry loop, expected attempts is the mean of a geometric distribution:

E[attempts] = 1 / p_success
cost per success = (Σ_tokens priced by type per attempt) / p_success

If the loop is capped at k attempts, expected attempts are (1 - (1-p)^k) / p, and (1-p)^k of tasks never succeed — worse than uncapped, because you paid for k attempts and got nothing.

Add fixed per-task overhead for anything paid regardless of tokens: test-runner time, sandbox minutes, human review. Human review dominates for free-form analysis, and it is the term most audits forget.

cost_per_success = (Σ attempt costs + fixed overhead per task) / successes

A Metering Wrapper That Counts Attempts, Not Calls

Most cost dashboards count API calls. That is the wrong unit: it splits one task across N rows and hides the retry multiplier. Group by task instead.

task-meter.js
// USD per 1M tokens — ILLUSTRATIVE placeholders. Substitute your rate card.
const PRICES = { input: 1.0, cachedInput: 0.1, output: 4.0 };

function priceOf(u = {}, p = PRICES) {
const cachedIn = u.cachedInputTokens ?? 0;
const uncachedIn = Math.max((u.inputTokens ?? 0) - cachedIn, 0);
return (
  (uncachedIn * p.input +
    cachedIn * p.cachedInput +
    (u.outputTokens ?? 0) * p.output) / 1e6
);
}

export function createTaskMeter(prices = PRICES) {
const records = [];

async function runTask({ id, run, validate, maxAttempts = 4 }) {
  const attempts = [];
  let verdict = { ok: false, kind: "max_attempts" };

  for (let i = 1; i <= maxAttempts; i += 1) {
    const startedAt = Date.now();
    let result;

    try {
      result = await run({ attempt: i });
    } catch (err) {
      // Transport failure is not a quality signal. Record it, keep going,
      // and keep it out of the pass-rate denominator discussion.
      attempts.push({ i, kind: "infra_error", error: String(err) });
      continue;
    }

    const check = await validate(result.output);
    attempts.push({
      i,
      kind: check.ok ? "pass" : check.kind ?? "wrong",
      ms: Date.now() - startedAt,
      inputTokens: result.usage?.inputTokens ?? 0,
      cachedInputTokens: result.usage?.cachedInputTokens ?? 0,
      outputTokens: result.usage?.outputTokens ?? 0,
      costUsd: priceOf(result.usage, prices),
    });

    if (check.ok) { verdict = { ok: true, kind: "pass" }; break; }

    // Policy refusals do not converge. Retrying them burns budget on a
    // deterministic outcome. Escalate or fall back; do not re-ask.
    if (check.kind === "policy_refusal") {
      verdict = { ok: false, kind: "policy_refusal" };
      break;
    }
  }

  const spentUsd = attempts.reduce((s, a) => s + (a.costUsd ?? 0), 0);
  const tokens = attempts.reduce(
    (s, a) => s + (a.inputTokens ?? 0) + (a.outputTokens ?? 0), 0
  );

  const record = {
    id,
    solved: verdict.ok,
    attempts: attempts.length,
    spentUsd,
    costPerSuccessUsd: verdict.ok ? spentUsd : null,
    tokensPerSuccess: verdict.ok ? tokens : null,
    verdict,
    detail: attempts,
  };
  records.push(record);
  return record;
}

function report() {
  const solved = records.filter((r) => r.solved);
  const spend = records.reduce((s, r) => s + r.spentUsd, 0);
  const attempts = records.reduce((s, r) => s + r.attempts, 0);
  return {
    tasks: records.length,
    solved: solved.length,
    passRate: records.length ? solved.length / records.length : 0,
    meanAttempts: records.length ? attempts / records.length : 0,
    totalSpendUsd: spend,
    costPerSuccessUsd: solved.length ? spend / solved.length : null,
    tokensPerSuccess: solved.length
      ? solved.reduce((s, r) => s + r.tokensPerSuccess, 0) / solved.length
      : null,
  };
}

return { runTask, report };
}

Run the same task file and validator against baseline and candidate, with the same maxAttempts, then diff the two reports. If you change the validator between runs, you changed the experiment.

Scoring Success Without Grading Your Own Homework

The validator is the load-bearing part of the audit. Get it wrong and every downstream number is decorative.

  • Prefer deterministic checks. Does it parse? Does the JSON Schema validate? Does TypeScript compile? Does the existing test suite still pass? For security work in a lab harness: does the canary still hold, does the fixture still leak, does the patched build still pass the regression corpus?
  • Write the task set before you see the results. Pull 200+ tasks per task family from your real backlog, strip anything that leaks the answer, and freeze it.
  • Do not use a model as the judge for the model you are evaluating. If you must use an LLM grader, calibrate it against your own hand labels on at least 50 items and report the grader's disagreement rate alongside the result.
  • Report the validator's own accuracy. A validator that accepts 8% of wrong answers makes a pass rate of 0.85 look like 0.92.
  • Record the failure kind, not just pass/fail. Capability failure, policy refusal, and transport error have completely different cost implications and completely different retry policies.

A Worked Example With Illustrative Numbers

These numbers are constructed to show the arithmetic. They are not measurements of Qwen4-Max and not the vendor's figures. Both models use $1.00 per million input tokens and $4.00 per million output tokens, so only token volume and pass rate vary.

Where the 60% Survives the Arithmetic

ScenarioIn tok/attemptOut tok/attemptPass rateE[attempts]Cost/attemptCost/success
Baseline2,4009000.621.61$0.00600$0.00968
Candidate A (retries + brevity only)2,2006000.851.18$0.00460$0.00541

Candidate A is 44.1% cheaper per successful task with an unchanged rate card. That is real and meaningful — but not 60%.

Now add a 20% rate card cut and a modest input reduction from better instruction-following:

ScenarioIn tok/attemptOut tok/attemptPass ratePrice mult.Cost/attemptCost/success
Baseline2,4009000.621.00$0.00600$0.00968
Candidate B1,8006500.880.80$0.00352$0.00400

Candidate B lands at 58.7% cheaper. That is where a "roughly 60%" headline becomes arithmetically plausible: retries down, output shorter, input shorter, and a price cut all pulling in the same direction.

Mind the mix effect. Candidate A cuts tokens per successful task by 38% (5,323 → 3,294) but cuts cost by 44%, because the tokens it removed were disproportionately expensive output tokens. A model that trims input tokens and adds reasoning output can reduce TSPST while increasing cost. Track both.

Where It Evaporates: Escalation Ladders

The number dies when retries become structural and you escalate on failure. Take Candidate A, keep its prices and pass rate, but escalate the 20% of tasks that fail three attempts to the baseline model — the classic two-tier cascade:

Per 100 tasks:
  80 tasks handled by candidate:  94.12 attempts × $0.00460  = $0.43296
  20 tasks, 3 candidate attempts: 60    attempts × $0.00460  = $0.27600
  20 tasks escalated to baseline: 20    attempts × $0.00600  = $0.12000
                                                        total = $0.82896

Per success:  $0.82896 / 100 = $0.00829   vs baseline $0.00968  →  14.4% cheaper

The 60% becomes 14%. Same models, same prices, same per-attempt quality. The only change is that you paid for the failed attempt ladder twice: once in the cheap tier, once in the expensive one. If you also wait for three cheap attempts before escalating, you paid in latency too.

That is the failure mode I expect most teams to hit first, because escalation-on-failure is the default shape of agent frameworks. This is my inference from the arithmetic; the release coverage does not discuss it.

Routing: Cascading to Small Models vs. Sending Everything to the Giant

Break-Even Math for a Two-Tier Cascade

Let the small tier cost c_s per attempt with pass rate p_s, and the giant cost c_g with pass rate p_g. Try the small tier up to k times, then escalate. With q = 1 - p_s:

expected small-tier attempts = (1 - q^k) / p_s
P(escalate)                  = q^k
cascade cost per task        = c_s · (1 - q^k)/p_s + q^k · (c_g / p_g)
single-tier giant cost       = c_g / p_g

Setting the cascade below the single tier and simplifying:

c_s · (1 - q^k)/p_s  <  (1 - q^k) · (c_g / p_g)
c_s / p_s            <  c_g / p_g

The (1 - q^k) cancels. The comparison reduces to cost per successful attempt in each tier, independent of k under these assumptions. That is cleaner than most routing discussions admit, and it gives the rule: compare price / pass_rate per candidate tier on each task family, not per-token price.

Three assumptions carry a lot of weight, and all three break in practice:

  1. Attempts are independent. They rarely are — a model that failed once tends to fail the same way, especially on a policy refusal or a systematically ambiguous spec. Correlated retries inflate E[attempts] above 1/p.
  2. Escalation succeeds in one attempt. If the giant also needs retries, the second term grows.
  3. Latency is free. It is not. In code generation with a test runner in the loop, three cheap attempts plus an escalation can easily exceed the wall-clock budget of one giant attempt. If your constraint is latency, k = 1 and the cascade is a router, not a retry ladder.

Eval-Driven Routing and the Retry Inflation Problem

Routing on public benchmark scores is the usual mistake. Benchmark scores measure single-attempt capability on a curated distribution, often with generous scoring. Your router needs per-task-family pass rate under your retry policy, measured on your tasks.

Retry inflation is subtler. Two models with the same eventual solve rate under a 5-attempt cap can differ by 3× in cost per success. A quality-only leaderboard scores them identically. Route by quality and pay by attempts, and you over-select the model that brute-forces its way to the answer. Measure E[attempts] and route on price / pass_rate, then verify against the retry-adjusted number.

Watch the second-order effect: retries are non-deterministic, so p_success is stable only if you sample a distribution. One run of 200 tasks gives a point estimate. Re-run the task set on another day with the same seed policy and check interval overlap before you commit a routing change.

Workload Shape Matters: Coding and Security Retries Are Not Uniform

The 60% claim is not one number. It is a different number for every task family, because retry behaviour is a property of the task and the validator, not the model.

Task familyCheap validator?Do retries converge?Where the cost actually hides
Structured extractionYes — schema + exact matchUsuallyLong inputs; output tokens are few
Code patch + test suiteYes — tests, types, buildOften, if the failure is localisedTest-runner minutes and sandbox cost, not tokens
Long-horizon refactorPartially — tests may pass for the wrong reasonWeakly; errors compound across filesEscalation, and human cleanup of subtly wrong diffs
Free-form security analysisNoRarelyHuman review time, which dwarfs token cost
Exploit / CTF-style in a lab harnessYes, in a scoped harnessSometimesThe harness is the cost; tokens are noise

Structured-Output Tasks: Where Retries Usually Converge

Here a per-task improvement is most likely to be real. The validator is deterministic, failures are legible, and a retry often converges because the model gets a specific error to fix. If your workload is mostly schema-bound extraction or test-verified patches with a fast runner, a 40–60% per-task improvement over a previous generation is easy to construct arithmetically.

The trap: a passing test suite is not a correct patch. A model optimising against a visible test can satisfy the test and miss the intent. Budget time for hand-reviewed patches so you know your validator's false-accept rate.

Free-Form Analysis Tasks: Where Human Review Dominates

Here the claim usually fails, and not because the model is bad. It fails because the denominator is wrong. There is no cheap oracle for "is this threat model complete", so success means a human accepted the output — and human review costs orders of magnitude more per task than tokens. If the giant model cuts human edit time by 30%, that dominates every token-level improvement in the worksheet. If it does not, a 60% token cost reduction is a rounding error on the real per-task cost.

My position: for free-form security analysis, per-token economics are the wrong target. Measure review minutes per accepted artifact.

The Safety Upgrades Bundled Into the Release — and Their Retry Cost

The coverage reports safety upgrades without describing them. I cannot evaluate what I cannot see, so treat specifics as unknown and evaluate behaviour empirically.

Safety behaviour enters the cost worksheet in one concrete way, and it is easy to miss: policy refusals are retry-inflating. If a security workload triggers a refusal and your retry loop resends the same prompt with slight variations, you burn three to five full attempts on a deterministic outcome. Per-task cost looks terrible, but the model is not failing the task — your loop is failing at classification.

Instrument this. Add a policy_refusal failure kind to the validator as in the wrapper above, break the loop on it, and route the task to a human or a differently scoped harness. Report refusal rate separately from capability pass rate. Two numbers, because they imply different responses.

⚠️

If you A/B a model across a safety-boundary release, run the security task set against both versions with the harness frozen. A pass rate that moves because a refusal policy changed tells you nothing about capability, and it will silently corrupt any routing decision you make from it.

Failure Modes in Your Own Audit

Sample Size and the Three-Prompt Anecdote

Underpowered task sets are the usual failure. Suppose a candidate solves 26 of 30 tasks — 0.867 point estimate. The Wilson 95% interval is roughly:

p̂ = 26/30 = 0.867,  n = 30
Wilson 95% CI ≈ [0.703, 0.947]

Since expected attempts scale as 1/p, that interval maps to expected attempts between 1.06 and 1.42 — a 34% spread in cost per success from sampling noise alone. With 30 tasks you cannot distinguish a 60% improvement from a 30% one. You need a few hundred tasks per family before the interval is tight enough for a routing call you can defend.

Report the count next to the claim. "0.87 pass rate" is not a finding; "26/30, 95% CI [0.70, 0.95]" is.

Benchmark Theatre and Vendor Evals

A vendor's evaluation is a marketing artifact that happens to contain measurements. It is not independent evidence. Check three things before a benchmark number enters a design decision:

  • Who ran it. If the answer is the company selling the model, treat it as an upper bound pending reproduction.
  • What the security tasks are. Multiple-choice questions about vulnerability classes and exploit-level sandbox tasks are different measurements; only one predicts your workload.
  • Contamination. Coding benchmarks built from public repositories have had years of exposure. A saturated benchmark cannot discriminate between two good models, which means it cannot drive routing either.

I have not seen an independent reproduction of the Qwen4-Max coding or cybersecurity claims, and the coverage does not name the benchmarks. Until someone publishes the harness and task distribution, the status is "unverified vendor claim".

A Retry-Adjusted Cost Audit Checklist You Can Run This Week

The goal is a number you can defend in a design review, not a leaderboard position.

  1. Freeze a task set. 200+ tasks per family from your real backlog, answers stripped, with the validator written before you see any model output.
  2. Calibrate the validator. Hand-label 50 items and report its false-accept rate. If it is above ~5%, fix it before you spend money.
  3. Pin the retry policy. Same maxAttempts, same backoff, same escalation rule, same prompt for both models. Log the policy version with the results.
  4. Meter per task, not per call. Use the wrapper above. Emit attempts, inputTokens, cachedInputTokens, outputTokens, failureKind, and wall-clock per attempt.
  5. Record the rate card separately. Price is configuration; quality is measurement. Never let a price change hide inside a quality comparison.
  6. Compute cost_per_success and tokens_per_success for both models, with intervals. Bootstrap the ratio if you want a defensible delta.
  7. Break the result down by task family. A single blended number will hide the family where the claim collapses.
  8. Run the escalation variant. If you plan to cascade, measure the cascade — the retry ladder is where per-task savings go to die.
  9. Write down the latency budget. If p95 wall-clock is a product requirement, k is smaller than you think.
  10. Keep the raw attempt logs. Every future audit re-litigates the validator, and the logs are the only defence.

Where This Leaves Builders

The 2.4-trillion-parameter headline tells you one useful thing: most teams will not self-host this model, so your exposure is a rate card and a retry policy. The 60% per-task claim tells you something more interesting — it is a claim about retry distributions, and retry distributions belong to your tasks and your validator, not the vendor's.

So the honest reading: a 40–60% per-task improvement over a previous generation is plausible for structured-output and test-verified coding work, where retries converge and the validator is cheap. It is plausible only as a compound of four or five channels, at least one not a price change. And it evaporates to roughly 15% the moment you put an escalation ladder in front of it, because you pay for failed attempts twice.

The actionable part is not the model. It is the measurement: if you cannot state tokens per successful task and cost per successful task for your own task families, you cannot confirm or refute a claim like this one — and you will route on benchmark scores that cannot see retries. That is the gap worth closing this week, before the next flagship lands with a bigger number attached.

💪

Run the worksheet on the model you already use in production first. You need a baseline you trust more than you need a new candidate. The first time you compute retry-adjusted cost per success on your current stack, the number is usually worse than expected — and more useful than any benchmark table.

Further Reading

  • Qwen official blog — release notes and technical write-ups from the model's authors (primary source): qwenlm.github.io/blog
  • DeepSeek-V3 Technical Report — a published MoE design with explicit total vs. active parameter counts (671B total, 37B active), useful as a reference for how sparse activation is specified: arXiv:2412.19437
  • Switch Transformers (Fedus, Zoph, Shazeer) — the scaling study behind trillion-parameter sparse models, including load-balancing and capacity-factor trade-offs: arXiv:2101.03961
  • Outrageously Large Neural Networks (Shazeer et al.) — the original sparsely-gated MoE paper: arXiv:1701.06538
  • Mixtral of Experts — a published 46.7B total / ~12.9B active design, useful for calibrating what an activation ratio looks like in practice: arXiv:2401.04088
  • SWE-bench — the coding benchmark whose public-repository construction makes contamination a live concern: swebench.com
  • Binomial proportion confidence intervals (Wilson score) — reference for the interval arithmetic used in the sample-size section: Wikipedia
  • BigGo Finance coverage of the Qwen4-Max release (secondary, 16 Nov 2026): news link
  • India Today coverage of the coding and cybersecurity benchmark claims (secondary, 16 Nov 2026): news link

Share this post

More posts

Loading comments…