Auditing Model Routing and Prompt Caching Before You Trust the Cost Chart

An audit framework for LLM gateway model routing and prompt caching: verify provider-side usage counters, cache read rates, and tenant isolation before trusting a savings chart. The post runs a fixture with three synthetic tenants and 1,500 replayed requests. Full write-up: https://hackyjs.com/posts/auditing-model-routing-and-prompt-caching-before-you-trust-the-cost-chart

Read the post →