Cap what an AI agent can spend, rather than find out afterwards
Refused before egress, priced against everything the route could reach, and never estimated at zero.
On this page
The dashboard, the token quota, and the price table nobody loaded
The first thing most teams buy is a cost dashboard, and it is genuinely useful in a monthly review and useless at two in the morning. A dashboard tells you what you spent after you spent it. The control a finance owner is actually asking for is the one that refuses the call, and refusing a call means holding a defensible price for it before it leaves the building — which is a harder engineering problem than drawing the chart, and is why so much of the market stops at the chart.
The second thing is a token quota on the gateway, which bounds volume rather than money. Those are different quantities and they diverge exactly when it matters: the same number of requests costs an order of magnitude more once a router promotes them to a reasoning model, once a retrieval loop stops truncating, or once a caller stops setting an output cap. A quota that admits a hundred requests admits a hundred requests whether they cost four dollars or four hundred.
The third is the pricing itself, and it is where most implementations are quietly wrong. Providers disagree about whether cached prompt tokens sit inside or outside the prompt total — Anthropic reports cache reads and cache writes alongside its input count, while OpenAI’s cached-token detail and Gemini’s cached-content count are already inside theirs — and treating one convention as the other misprices cache-heavy traffic by 50 to 90 per cent. Agent traffic is cache-heavy by nature, because the system prompt and the retrieved context repeat on every turn.
The failure that shaped the current design was worse than an inaccurate number, and it is published rather than implied. The shipped price rows loaded only under the demo seeder, which the setup documentation tells production operators not to run, so the documented production install started with an empty price table. An unknown model produced a zero estimate, every trace recorded nothing, and every per-request ceiling admitted every request: the control was off while appearing to be on. An estate spending nothing and an estate spending unmetered emit identical bytes, and the customer finds out from the vendor.
How to actually do it
Each step is something you can go and do. Where a step depends on a decision somebody has to make rather than a setting somebody has to change, it says so.
- 01
Put the traffic through one endpoint and give each agent a record
Ceilings are enforced per governed subject, so every agent needs its own registry row with a named owner before a limit can be attached to it. The gateway resolves that row on every call rather than a copy of it, so an edited ceiling applies to the agent’s next request without a redeploy. - 02
Confirm the price table is populated before you trust any ceiling
The 50 shipped price rows load at boot on the documented production path, additively: a row whose key of model, provider kind and effective date already exists is skipped, so a restart can never overwrite a price an operator corrected by hand. Check the table has rows for every model your agents can reach, including the fallbacks, because an unpriced reachable target is the state the refusal exists to catch. - 03
Set the USD ceilings that match how the agent actually fails
A per-request ceiling catches one absurd call. An hourly ceiling catches a retry loop. A daily or monthly ceiling catches slow drift. Set the subset that matches the failure you are guarding against rather than all four out of tidiness, and remember the windows differ in shape: the hour is a rolling sixty minutes, while the day and month start at 00:00 UTC so the monthly figure lines up with a vendor billing period. - 04
Set rate ceilings underneath the money ones
Requests, tool calls and tokens per minute are decided in the same deterministic pass as permissions and policy, and return a 429 naming the limit type, the configured value and the observed figure. They are the faster brake: a runaway loop hits a per-minute ceiling in seconds, where an hourly USD ceiling only bites once the money has accumulated. - 05
Put a warn rule below the wall
A spend-triggered policy with the warn action lets the request proceed and publishes an event, so somebody hears about an anomaly before work stops. Token Observe also publishes a budget warning once any window reaches 80 per cent of its ceiling and a budget-exceeded event at 100 per cent — that check reads the hour, day and month windows only, since a per-request ceiling has nothing to warn about. - 06
Decide in advance what happens when a ceiling bites
A refusal is a 429 with a typed reason and a recorded blocked trace, and nothing partially executes. Decide who is allowed to raise the ceiling, whether the agent should degrade or stop, and accept the availability trade: a call under a USD ceiling gets one potentially billable egress, so it will not be retried or failed over to a second provider.
Why the money verdict is taken last, and priced against everything reachable
The pipeline decides the cheap and certain things first. An engaged kill switch beats everything, then the agent’s lifecycle status, then deny-by-default permissions including every link of any delegation chain, then the rate ceilings, then the policies whose scope selects this subject. USD is deliberately skipped in that pass for any agent that has a ceiling configured, because money cannot be decided honestly until routing has fixed which upstream will actually serve the call.
Once the route is resolved, the primary target and every fallback behind it are priced against the resolved model and provider kind rather than the model name the caller typed. That distinction is the difference between a ceiling and a suggestion: route rules can send a requested model somewhere else, tier selection can substitute a cheaper one, compatibility filtering can drop a target, and failover can promote a fallback — all after the caller supplied a name. Pricing only the name they typed would leave a rerouted target or an unpriced fallback as a zero-dollar escape hatch.
The estimate is an upper bound rather than a guess, on both legs. The input bound is the UTF-8 byte length of the complete serialised outbound request — tool definitions, JSON-schema keys, tool arguments, passthrough configuration and every message boundary included — plus 256 tokens of framing overhead, on the reasoning that a tokenizer cannot emit more ordinary tokens than there are bytes. It replaced a four-characters-per-token estimate for one stated reason: a hard money ceiling may be conservative and may not be optimistic. The output leg is priced at the caller’s max_tokens, or an assumed 4,096 where none is named, and the outbound request is stamped with that same value so an upstream cannot quietly answer past the amount reserved. Each leg takes the highest rate in the reachable candidate set — the input leg the maximum of every candidate’s input, cache-read and cache-write columns, the output leg the maximum output rate.
Admission is then one transaction per agent. It reads the three USD windows excluding this trace, tests whether adding the estimate would cross a configured ceiling, and writes the reservation only if it would not, taking BEGIN IMMEDIATE on SQLite and a transaction advisory lock on the agent id on PostgreSQL. That is a correction rather than an original property: the spend window used to aggregate completed traces only, so several concurrent requests each saw zero in-flight spend and all passed a cap one of them would have breached. A trace that is running or parked awaiting a human now contributes the greater of its billed cost and its reservation.
- perRequestUsd
- Refuses one call whose conservative estimate exceeds the ceiling. It is inert on a subscription seat, because a subscription prices the seat rather than the request and the marginal cost of one call is not knowable at the moment of deciding.
- hourlyUsd, dailyUsd, monthlyUsd
- A rolling sixty minutes, the UTC calendar day and the UTC calendar month. Each blocks both when the window is already at the limit and when the projection would cross it, so one large call cannot step over a limit it was already close to.
- The unpriced refusal
- A 409 before egress naming the model, the provider that would have served it and how many further fallbacks are also unpriced. An agent with no USD ceiling is explicitly unbudgeted and keeps recording what it can.
- Long cache writes
- Anthropic and Bedrock bill a five-minute cache write at 1.25 times input and a one-hour write at 2 times. The price table has one cache-write column, so on those two provider kinds any explicit cache TTL other than five minutes is priced at no less than twice the input rate, and an unrecognised TTL gets the same treatment rather than an optimistic assumption.
What a parked request holds, and what is actually billed afterwards
A request stopped on a human approval has been priced and not spent, and both obvious treatments of that money are wrong. Forget the estimate and an agent can queue a thousand expensive calls past its ceiling while somebody deliberates. Hold it forever and a denied or abandoned approval counts against that agent’s budget until someone edits the database by hand — which is not hypothetical, because the release path did not exist at all in an earlier build, and what an operator met was two surfaces disagreeing by orders of magnitude with nothing to say which was right.
So the reservation is held for exactly as long as the decision is outstanding, and every terminal path gives it back in the same transaction as the state change that ended the wait. Denial releases it with the decision. Expiry releases it with the sweep. An approved but unredeemed approval deliberately keeps it, because the action has not happened and the money is still about to be spent, and its own expiry is what returns it. When the agent comes back with the identical payload, the atomic admission excludes the original approval trace while writing the new reservation, so the estimate is transferred rather than counted twice, and re-polling a still-pending approval takes no second reservation at all.
What is finally billed comes from the provider’s own reported usage, normalised into four buckets that are mutually exclusive by construction — uncached input, cache read, cache write and output — before any arithmetic happens. Provider usage is treated as untrusted wire data even where a type declaration calls it a number: anything that is not a non-negative safe integer is refused outright, because a negative count would subtract from a budget and a non-finite one would turn the ledger into NaN. An OpenAI response claiming more cached tokens than prompt tokens is rejected rather than clamped.
Pricing then uses the immutable row pinned before egress for the provider that actually served the call, rather than re-reading current prices at completion — a catalogue sync landing mid-call would otherwise reprice an admitted request, including down to zero. OpenRouter is the one upstream that reports an authoritative charge of its own, and where it is present it is preferred over the local arithmetic so the ledger matches the invoice; the field is absent rather than zero when unknown, so free stays distinguishable from not reported. Where a price row names no cache columns, cache reads and writes both bill at the full input rate, which is conservative by design and never under-bills.
What this still does not solve
Doing everything above leaves a residue. It is smaller than what you started with and it is not nothing, and knowing its shape in advance is the difference between a control you trust correctly and one you trust too much.
- There is no team-level or fleet-level budget pool. Ceilings are enforced per governed subject, and the team and fleet figures in the budget report are the sum of the per-agent ceilings that exist, published beside a count of the agents that have none.
- Nothing is reconciled against a vendor invoice. Every figure is metered from the provider’s reported usage against the price rows you hold, and a shipped price row is a list price captured on a date that will go stale — the catalogue loads additively, so a shipped row is never refreshed in place on upgrade.
- A hard ceiling costs you retry and failover. One potentially billable egress is the whole chain, because a timeout cannot prove the vendor did not complete and bill the call, and one reservation must not end up covering several independently billable attempts.
- Token Observe cannot bound spend it never sees. A vendor credential used directly, or a developer subscription outside a managed seat, is a discovery problem before it is a budget one — and the number of agents you do not know about is the denominator for every coverage claim you make about the ones you do.
If one of those residues is the thing that actually worries you, that is the conversation worth having rather than the one about the steps above it.
Talk it throughThe parts of the platform this uses
Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
One billable egress: no retry, no failover
Model routing
Six upstreams behind one set of policies, and a fallback chain that will not launder a refusal.
Nothing verifies a provider’s ZDR or training claim
Policy engine
One deterministic verdict on every governed request: allow, block, redact, or park it for a human.
Injection scoring is nine fixed patterns, not a model
Shadow AI radar
Five evidence sources for AI activity that never touched the gateway, and a coverage model that refuses to call a dead feed a clean estate.
No billing, network or IAM access — you send the export
The other jobs
What happens when an agent hits its monthly ceiling mid-conversation?
The next call is refused before it reaches a provider, with a 429 and a typed reason naming the limit type, the configured value and the actual figure. Nothing partially executes and nothing is billed, because the refusal happens at admission rather than after egress, and the attempt is still recorded as a blocked trace so the evidence of the refusal exists. Separately, a post-call check publishes a budget warning at 80 per cent of any window’s ceiling and a budget-exceeded event at 100 per cent, so somebody hears about the wall before an agent walks into it.
Can two concurrent requests both slip past the same cap?
No. The USD admission runs inside one per-agent transaction that reads the windows, tests the projection and writes the reservation before releasing the lock — BEGIN IMMEDIATE on SQLite, a transaction advisory lock on the agent id on PostgreSQL. Requests still in flight are counted: a running or awaiting-approval trace contributes the greater of its billed cost and its reservation. This is a fix rather than an original property, and a deployment whose trace store cannot offer that atomic admission has its USD-budgeted traffic refused outright rather than quietly downgraded to completed-spend accounting.
What happens if a model has no price row?
For an agent with any USD ceiling configured, the request is refused with a 409 before egress, naming the unpriced model, the provider that would have served it and how many further fallbacks are also unpriced. Adding a price row, or removing every USD ceiling from an agent you intend to leave unbudgeted, resolves it. The alternative was tried and was worse: an unpriced model produced a zero estimate, which silently disarmed every per-request ceiling on the documented production install while the console went on showing the ceiling.
Does this stop spending on developer subscriptions and direct API keys?
Not by itself. A ceiling binds a governed subject, so it reaches an agent whose traffic arrives at the gateway and a subscription seat whose enforcement Token Observe issues a bundle for; it does not reach a vendor credential somebody is using directly. Finding that spend is the discovery job rather than the budget one, and the source that finds it is the vendor bill: monthly vendor lines are reconciled against metered spend for the same month, with a gap having to exceed both one dollar and five per cent of the larger side before anything is reported.
Will Token Observe ever route a request to a more expensive model to save time?
No. The router never serves above the tier the caller asked for, and it does not upgrade on anyone’s behalf, because an upgrade would spend money nobody authorised and no ceiling in the data model would bound it. A downgrade requires the agent’s allowDowngrade flag and a classification confidence of at least 0.75, and the agent’s maxTier ceiling caps the result regardless of both. A model the price table does not name is treated as reasoning tier for ceiling purposes, so an unpriced model cannot walk through the bound that exists to cap it.
Describe the version of this you actually have.
The steps above are the general shape. Which of them matter, and in what order, depends on what your agents do and which of them worries you — say that and you will get a straight answer, including when the answer is that Token Observe is not what you need for it.
no form · no qualification step · no sales desk · the other three ways in