Spend controls

Most agent cost tooling is a dashboard. It tells you what you spent after you spent it, which is useful in a monthly review and useless at two in the morning when a retry loop is a third of the way through the month’s budget. The control a finance owner actually asks for is the one that refuses the call, and refusing a call means having a defensible price for it before it leaves the building.

That pricing is harder than it looks, and it is where most implementations are quietly wrong. Providers disagree about whether cached prompt tokens sit inside or outside the prompt total: Anthropic reports cache reads and cache writes alongside its input count, while OpenAI’s cached-token detail and Gemini’s cached-content count are already inside theirs. Treating one convention as the other misprices cache-heavy traffic by 50 to 90 per cent — the range the pricing module states in its own header — and agent traffic is typically cache-heavy, because the system prompt and the retrieved context repeat on every turn.

The failure mode that motivated the current design was worse than an inaccurate number. The shipped price catalogue was loaded only by the demo seeder, which the setup documentation explicitly tells production operators not to run, so the documented production path produced an install with an empty price table. An unknown model priced at zero meant every trace recorded $0 and every per-request ceiling admitted every request: the control was off while appearing to be on. An estate spending nothing and an estate spending unmetered emit identical bytes, and the customer finds out from the vendor invoice.

So Token Observe treats the price table as part of the control rather than as reporting furniture. The catalogue loads at boot on the documented production path, a budgeted agent whose resolved route cannot be priced is refused before egress, and every ceiling is checked against a conservative upper bound rather than a friendly estimate. A ceiling may be conservative. It may not be optimistic.

Every governed subject carries four optional USD ceilings and three optional rate ceilings, and any subset of them can be set. A per-request ceiling is compared against the pre-flight estimate for that one call. The hourly, daily and monthly ceilings block both when the window is already at or over the limit and when the projection — spend so far plus this call’s estimate — would cross it, so a single large request cannot step over a limit it was already close to.

The windows are not all the same shape, and the difference matters when you are reading a report. The hour is a rolling sixty minutes back from now. The day starts at 00:00 UTC and the month at the first of the month, UTC, which is what makes the monthly figure line up with the budget report and with a vendor’s billing period rather than with a local timezone that would drift against both.

Spend still in flight counts. A trace that is running or parked awaiting human approval contributes the greater of its billed cost and its reservation, so a request cannot be admitted against a window that ignores the four calls already dispatched. That is a correction rather than an original design: the spend window used to aggregate completed traces only, so N concurrent requests each saw zero in-flight spend and all passed a cap that one of them would have breached.

Approvals made that accounting subtle, and the subtlety is stated rather than hidden. A denied or expired approval closes its trace and clears the estimate. An approved-but-unredeemed approval deliberately keeps its reservation, because the action has not happened yet and the money is still about to be spent; its expiry is what releases it. When the exact payload is resumed, the atomic admission excludes the original approval trace while transferring the estimate to the execution trace, so the same action is never counted twice, and re-polling a still-pending approval takes no second reservation at all.

The kill switch stops a named agent, a whole team, or everything, and it is the first thing the governance pipeline checks — ahead of lifecycle status, permissions, budgets and every policy. Engaging one is an admin action, requires a stated reason, and cannot be done anonymously, because the audit entry is the only account anyone will ever have of why an entire fleet stopped. Engagement and release both append to the hash-chained audit log and publish an event; releasing is admin-only too.

Scope is deliberately three values rather than four. A global switch reaches every governed subject. A team switch compares team names case-insensitively, on the reasoning that a team name is being typed by a human under incident pressure and a capital letter should not be the difference between a fleet stopping and not. An agent switch matches the subject id exactly, and because a subscription seat is a governed subject with its own prefixed id, the same scope stops a seat.

The predicate that answers whether a switch reaches a subject is written once and exported, and that is load-bearing rather than tidy. The same function the gateway stops a request with is the filter that decides which switches are compiled into an endpoint seat’s signed policy bundle. A second copy that was stricter by one character would omit an engaged switch from the artifact, and the big red button would be pressed in the console and reach nothing at all on the device.

The gap worth naming is the one that was found and closed. Token counting and the model-catalogue routes deliberately skip the trace-opening path, because they execute nothing and opening a trace for a size check would pollute the evidence record — but in skipping it they also skipped the agent-status and kill-switch checks, so a frozen agent could still price prompts and enumerate models. Those checks now run on all three routes. The switch is still an admission control, though: it refuses new work rather than recalling a request already dispatched upstream.

Token Observe can serve a cheaper model than the caller asked for, and never a more expensive one. The classifier is a heuristic and the source says so first: no classifier model, no embedding, no learned weights, because a router that had to call a model to decide which model to call would spend a meaningful share of the saving it exists to produce and would add a second upstream dependency to the request path. It scores a handful of signals that genuinely separate cheap work from expensive work — what the latest turn asks for, prompt size, the caller’s output cap, conversation depth, whether tools were offered, whether the prompt carries code — publishes the reasoning, and lets a per-agent ceiling override the whole thing.

Three properties are load-bearing. It is pure and deterministic, with no I/O, no clock and no randomness, so a route decision recorded on a trace can be reproduced from the trace months later. It is always explained: every decision carries its reasons — matched keywords, character counts, tool counts, the score and the tier it lands in, and the confidence bar by name when that is what stopped a swap — and never prompt text, and the decision is written to the trace whether or not a swap happened, because otherwise the router doing nothing and the router being switched off look identical. And it is conservative by construction, because spending more of someone’s money than they asked for is not a cost-optimisation feature.

Two rules decide the served tier, in this order. A downgrade is opt-in: it needs the agent’s allowDowngrade flag and a classification confidence of at least 0.75, and either alone is not enough. In practice that threshold means two agreeing cheap signals — a request that asks to extract will not on its own move a tier, but one that asks to extract and is a single short turn with no tools will. The agent’s maxTier ceiling then applies regardless of the opt-in, regardless of the confidence and regardless of what the caller asked for, because an opt-out ceiling would not stop anything. A model the price table does not name is treated as reasoning tier for ceiling purposes, so an unpriced model cannot walk straight through the bound that exists to cap it.

The confidence figure is not a probability and is not presented as one; nothing in the classifier was calibrated against labelled data. Its only job is to distinguish several signals agreeing from one weak signal tipping a borderline score, and the residual standard verdict starts low enough that it can never on its own clear the downgrade bar — a tier is never moved on the strength of the signals cancelling out. Only an explicit task-tier declaration from the caller returns full confidence, because that is a statement of fact rather than an inference. The per-request saving figure is an estimate from pre-flight token counts, clamped at zero, and measured from the cheapest equally-specific row for the requested model, which is the opposite bias to the ceiling lookup and for a stated reason: an overstated saving is a number the business would go on to quote.

Deciding whether to switch downgrade routing on at all is a separate, retrospective question, answered from the flight recorder rather than from a vendor claim. The savings analysis runs over completed calls only, so it costs nothing and changes nothing, and it splits its total by confidence — quote the high band, treat the total as a ceiling. It is a report rather than a screen: the figure is served by an API endpoint over a bounded scan, at most 500 traces and by default the last thirty days, and the payload says so by returning a truncated flag when the scan hit its cap. The conservatism is layered: an excerpt that reads as diagnosis, design or formal reasoning, or that contains code, puts the call back at the reasoning tier and it can never register as a saving at all, while a call that was offered tools or that ran on 8,000 or more input tokens is never suggested below standard whatever the prompt looks like. A call whose model has no price row is reported as skipped rather than assumed free. The report carries its own assumptions rather than a footnote: complexity is inferred from a redacted prompt excerpt and token counts and is not a judgement that the cheaper model would have answered acceptably; the modelled cost reuses the actual token counts, and a different model would tokenise differently and may be more or less verbose; and savings are priced at today’s rates, not the rates in force when the call was made.