Token cost accounting

An honest ledger has four buckets, not two, and the buckets have to be mutually exclusive by construction: input tokens that were processed fresh, input tokens served from a prompt cache, tokens written into that cache, and output tokens. Each is billed at its own rate, and on a cache-heavy call the four can span two orders of magnitude. The moment one bucket can contain another, the same token is priced twice or not at all, and no care downstream recovers it. This is why the invariant belongs at the provider boundary: normalise the vendor’s shape into disjoint buckets as the response is parsed, assert it there, and let everything downstream do plain arithmetic.

The disagreement that costs the most money is about cache tokens. Anthropic reports `cache_read_input_tokens` and `cache_creation_input_tokens` exclusive of `input_tokens`: the three numbers add up to what you were charged for on the input leg. OpenAI reports `prompt_tokens_details.cached_tokens` inside `prompt_tokens`, and Gemini reports `cachedContentTokenCount` inside `promptTokenCount`: the cached figure is a breakdown of the total, not an addition to it. Read the inclusive convention as if it were exclusive and you bill the cached prefix twice — once at the full input rate and once at the cached rate. Read the exclusive convention as inclusive and you subtract the cache reads from a number that never contained them, and under-bill. On traffic where most of the prompt is a repeated prefix, the error runs to somewhere between half and nine tenths of the input leg, which on agent workloads is most of the bill.

Provider usage is untrusted wire data even when a type declaration says it is a number. A negative count subtracts from a budget window; a float that is not a safe integer loses precision as soon as it is persisted or summed; a missing field becomes `NaN` and poisons every total it touches. Refusing anything that is not a non-negative safe integer, at the boundary, turns a class of silent mispricing into a loud parse failure. Streaming needs one more rule: provider usage counters are cumulative, but some compatible endpoints emit more than one usage frame and a later frame can omit or regress a bucket. Merging by taking the greatest validated value seen for each bucket is conservative — it never credits a budget, and it never mistakes a repeated total for an increment, which is the mistake that doubles a streamed call’s recorded cost.

Then there is the question of which price. A price table matched on the model name alone breaks as soon as the same model is reachable through two upstreams at different rates, so rows are keyed on model pattern and provider kind: an exact provider beats a wildcard, an exact model beats a wildcard, and the longest pattern wins among equals. Give rows an effective window rather than overwriting them, so a historic entry can still be explained by the row that produced it. Two ordering rules matter. Price against the provider that actually served the call: a failover priced at the first candidate’s rates attributes spend to a vendor that never ran the request. And pin the row before the call rather than re-reading it afterwards, since a catalogue sync landing mid-call would otherwise reprice an admitted request.

The failure worth designing against is not an inaccurate figure but a missing one. A price lookup that returns zero for an unknown model produces a ledger of $0 traces and, where a spend ceiling reads the same estimate, a per-request cap that admits everything — a control that is off while the console still shows it configured. An estate spending nothing and an estate spending unmetered emit identical bytes, and the difference arrives on the invoice. Refuse a budgeted call whose resolved route cannot be priced, rather than pricing it at nothing. Two smaller hedges belong beside the claim: a character-count estimate (roughly four characters per token) is fine for a pre-flight check and must never reach the ledger, where the figure has to come from the provider’s own reported usage; and a per-call ledger will not reconcile exactly to a vendor invoice, which carries commitments, batch rates and rounding the call record never sees.