ERNIE 5.5 Flash Base at 1M tokens: separating FLOPs from KV-cache bandwidth
ERNIE 5.5 Flash Base is a 280B-parameter mixture of experts with 14B active parameters per token and a stated 1M-token context window. This walkthrough separates two costs that get conflated: the FLOPs per token that sparse activation reduces, and the KV-cache traffic it doesn't. Running the arithmetic, attention work overtakes the linear 2 x active x tokens term at roughly 10,700 prompt tokens and reaches about 93x it at 1M, or roughly 14 minutes of one 8xH100 node at 40% MFU under full quadratic attention. Cache bytes per token depend on layer count, KV heads, head dim and dtype, none of which are published, so the planning range spans 16x: from 305 GiB down to 19 GiB of working set at 1M tokens. Every number in the post is computed from stated parameters plus flagged assumptions. There is no measured benchmark yet, and the post says so. Full write-up: https://hackyjs.com/posts/benchmarking-ernie-55-flash-base-separating-per-token-flops-from-kv-cache-bandwidth-at-1m-context