Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
A budget that only reports is not a budget
Most agent cost tooling is a dashboard. It tells you what you spent after you spent it: useful in a monthly review, useless at two in the morning when a retry loop is a third of the way through the month’s budget. The control a finance owner asks for refuses the call, and that needs a defensible price before it leaves the building.
This product once shipped its price catalogue in the demo seeder that the setup documentation tells operators not to run, so the documented install started with an empty price table: every trace recorded $0 and every per-request ceiling admitted every request. The control was off while appearing to be on. So the price table is part of the control. A ceiling may be conservative. It may not be optimistic.
How it works
Steps 6, 8 and 11 of the request path. The order is encoded in the evaluator, and where each step sits is the reason the guarantee holds.
- 01
Kill switch, before anything else
Step one of the pipeline finds any engaged switch reaching this subject and returns a 403, ahead of lifecycle status, permissions, budgets and policy. Team scope matches case-insensitively: a team name is typed by a human under pressure. - 02
Rate limits decided in the pure pass
Requests, tool calls and tokens per minute return a 429 naming the limit, the configured value and the observed figure, and a blocked attempt is recorded before the refusal, so it is evidence rather than a free retry. USD is skipped here for any agent with a ceiling: money cannot be decided until routing fixes which upstream serves the call. - 03
Resolve the route, then price everything it could reach
Route rules, tier selection and failover all change what may leave, so every fallback is priced alongside the primary against the resolved model: pricing the name the caller typed would leave a rerouted target as a zero-dollar escape hatch. Both legs are bounded upwards: the serialised request in bytes plus 256 tokens of framing, and the caller’s max_tokens or 4,096. - 04
Serve a cheaper tier, never a dearer one
A deterministic heuristic (no classifier model, no learned weights) routes a request down a tier and never up. It needs the agent’s allowDowngrade flag and a confidence of at least 0.75, the maxTier ceiling applies regardless, and an unpriced model counts as reasoning tier so it cannot walk through the bound that caps it. - 05
Admit atomically, then meter against the pinned price
One per-agent transaction reads the three USD windows, tests the projection and writes the reservation only if it clears, so two concurrent callers cannot both decide against the same window. Provider usage is then normalised into four mutually exclusive buckets (uncached input, cache read, cache write, output) and priced against the row pinned before egress. Providers disagree about whether cached tokens sit inside the prompt total, and getting that wrong misprices cache-heavy agent traffic badly.
Every check lives in one function, so you can read your rules and know what they do.
POST /v1/chat/completions
The key resolves to one registered agent with a named owner, and the trace is opened before anything can refuse the request.
Kill switches, then lifecycle, then deny-by-default permissions across every delegation hop, then rate limits, then your policies by priority.
returns: allow · block · require_approval — plus a redaction plan
A spend ceiling is the deliberate exception: it is deferred until routing has fixed which provider and model will serve the call, so it is tested against the price you will pay.
Routed to the provider you configured, on your key. Metering runs last and the trace closes with a terminal status.
The plan is applied to the outbound payload and a redaction event records kinds and counts, never values. A redaction that cannot be applied faithfully becomes a refusal instead.
An approval bound to a hash of this exact payload, single-use and expiring. The caller receives the approval id and the header to resume with.
429 ACP_BUDGET_EXCEEDED or ACP_RATE_LIMITED where a ceiling decided it. The trace closes as blocked and is kept as thoroughly as one that went through.
Verdict names and status codes are the product’s own. Every rule can run in shadow mode first, where the decision is recorded and the request is not changed. That is how you learn your false-positive rate before you refuse real work.
The ceilings, and the windows they run over
Four optional USD ceilings and three optional rate ceilings, in any subset. The windows are a rolling sixty minutes, the UTC day and the UTC month, all in UTC so the monthly figure lines up with a vendor billing period. Each blocks both when the window is already at the limit and when the projection would cross it, so one large request cannot step over a limit it was close to. Spend still in flight counts: a running or awaiting-approval trace contributes the greater of its billed cost and its reservation, so nothing is admitted against a window that ignores the calls already dispatched.
- perRequestUsd
- Refuses a call whose conservative estimate exceeds the ceiling. Inert on a subscription seat, which prices the seat rather than the request; the window ceilings still bite there.
- Warnings, after the fact
- A budget warning is published at 80 per cent of any window and a budget-exceeded event at 100. It tells a human; the pre-flight check has already refused anything that would breach.
serialised request + framing 41,464 input tokens
max_tokens named by caller 2,000 output tokens
dearest rate in the route $3.00 / $15.00 MTok
reserved before egress $0.154392
hourlyUsd ceiling $5.00
spend in the rolling hour $4.91 (2 still running)
projected $5.064392 -> refused, 429An unpriced route fails closed
Price rows are matched by specificity and versioned rather than overwritten, so a superseded row is filtered out by its effective window before any match. Where a candidate has no active row and the agent has any USD ceiling, the request is refused with a 409 before egress, naming the model, the provider and how many fallbacks are also unpriced.
That refusal is the fix for the empty price table, and it carries its own cost. The catalogue loads at boot additively: a model, provider and effective date already present is skipped, so a restart cannot overwrite a price an operator corrected by hand, and a shipped price is never refreshed in place. Stale-but-present beats absent: absent means zero, and zero means a disarmed ceiling.
- beat 01Name the caller
- 1Authenticate
- 2Resolve the agent
Nothing is decided for an anonymous caller. The key resolves to one registered agent with a named owner, and its roles, its ceilings and any stop switch in force are read live from the store on every request, not from a cached copy, so a suspension takes effect on the next call instead of after a redeploy.
- beat 02Start the record, then read what is being sent
- 3Open the trace
- 4Sanitise
- 5Scan
The trace is opened before anything can refuse the request, which is why a blocked call is recorded as thoroughly as one that went through. Only then is the text normalised and scanned, in that order, so the detectors see what the model will read rather than what a human reviewer can see.
- beat 03Decide once
- 6Govern
- 6bIntersect with the named human
- 7Enact
One function returns one verdict: allow, block, or hold it for a named person to approve. Every check lives in that one place rather than scattered along the path, which is what makes it possible to read your rules and know what they do.
- beat 04Send it, and only then spend the money
- 8Route
- 9Call upstream
Routing fixes which provider and model will serve the call, so where an agent has a USD ceiling it is tested against the price you will pay, not the one the caller typed. Routing is also the last step that can still refuse locally: after it, the request has left your network.
- beat 05Govern the answer, then count the cost
- 10Govern the response
- 11Meter and record
A tool call the model proposes is judged before it reaches your agent. Metering runs last and on every ending, including the refusals and the streams a client abandoned halfway, because the tokens were spent either way and a partial trace is still evidence.
Step names and their grouping are the product’s own, and the order is encoded in the evaluator rather than drawn for the page.
Each line closes off an assumption a reader would otherwise carry into a proof of concept.
What this does not doWhat this leans on
Model routing
Six upstreams behind one set of policies, and a fallback chain that will not launder a refusal.
Policy engine
One deterministic verdict on every governed request: allow, block, redact, or park it for a human.
Flight recorder
Every governed request in a timeline a compliance officer can read, and a search box that never writes SQL.
Where this comes up
Jobs that draw on it
What happens if a model has no price row?
It is refused with a 409 before egress, naming the unpriced model, the provider that would have served it and how many further fallbacks are also unpriced. Adding a price row resolves it, as does removing every USD ceiling from an intentionally unbudgeted agent. The alternative was tried and was worse: an unpriced model produced a $0 estimate, which silently disarmed every per-request ceiling.
How much should the retrospective savings number be trusted?
As a pointer, not a figure: read it to decide where to look, and do not quote it. The automatic prompt excerpt it was designed to read has been removed from the gateway, so it now judges a call by token counts and whether tools were offered and returns every result at low confidence. Unpriced calls are skipped rather than assumed free, the scan is bounded to a page of recent traces, and the report ships those assumptions in its own payload. The route to a defensible number is to run the router in shadow mode on your own traffic.
Prefer to ask a person? Write to James
Bring us the agent you are least comfortable with.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.