Token Observe vs building it yourself
For a small estate, a few hundred lines of proxy is usually the right answer. The cost arrives later, and it arrives in specific places.
You own the code and the failure surface, and for a small estate that is the whole argument.
A proxy you wrote fits your deployment, your language, your on-call rota and your review process. There is no licence, no vendor conversation, no second supplier in the request path, and no component whose failure modes you have to learn from someone else’s documentation. If your governance requirement today is only a model allow-list and a per-day spend cap, that is a small amount of code that will be correct.
A sharper version applies to buying this. Token Observe sits in the request path and fails closed, so its availability becomes a governance property of your environment, and at this scale it is a single-writer process on one host with no replica, no clustering, no point-in-time recovery and no uptime SLA. One thing before the table: the products named above are the components a team assembles, and every row describes a first-cut proxy written in-house — not a claim about any of them, and not about your code either.
Token Observe and building it yourself, row by row
Where these two differ, one row at a time.
The first cut adds the provider’s usage numbers to a running total.
Normalise into mutually exclusive buckets first. Anthropic reports cache reads and writes outside the input total; OpenAI and Gemini report cached tokens inside it.
The first cut redacts the response once it has been assembled.
A hold-back buffer with a 64-character floor and a separate channel per tool-call argument, with the cut pulled back off any match it would split.
The first cut retries the next provider on error.
Seven typed classes: a 429, a timeout and a 5xx fail over; a content-policy refusal, an authentication failure, an over-long context and a malformed request do not.
The first cut is a queue, a boolean, and a retry that trusts the caller.
Bound to the SHA-256 of the canonical action plus its execution context, single-use through a compare-and-set so two concurrent retries cannot both execute, and expiring.
The first cut counts spend from the response, after the money is gone.
Reserved before egress in one per-agent transaction, priced at the most expensive reachable candidate, and refused with ACP_BUDGET_UNPRICED if any target has no price row.
- beat 01Name the caller
- 1Authenticate
- 2Resolve the agent
Nothing is decided for an anonymous caller. The key resolves to one registered agent with a named owner, and its roles, its ceilings and any stop switch in force are read live from the store on every request, not from a cached copy, so a suspension takes effect on the next call instead of after a redeploy.
- beat 02Start the record, then read what is being sent
- 3Open the trace
- 4Sanitise
- 5Scan
The trace is opened before anything can refuse the request, which is why a blocked call is recorded as thoroughly as one that went through. Only then is the text normalised and scanned, in that order, so the detectors see what the model will read rather than what a human reviewer can see.
- beat 03Decide once
- 6Govern
- 6bIntersect with the named human
- 7Enact
One function returns one verdict: allow, block, or hold it for a named person to approve. Every check lives in that one place rather than scattered along the path, which is what makes it possible to read your rules and know what they do.
- beat 04Send it, and only then spend the money
- 8Route
- 9Call upstream
Routing fixes which provider and model will serve the call, so where an agent has a USD ceiling it is tested against the price you will pay, not the one the caller typed. Routing is also the last step that can still refuse locally: after it, the request has left your network.
- beat 05Govern the answer, then count the cost
- 10Govern the response
- 11Meter and record
A tool call the model proposes is judged before it reaches your agent. Metering runs last and on every ending, including the refusals and the streams a client abandoned halfway, because the tokens were spent either way and a partial trace is still evidence.
Step names and their grouping are the product’s own, and the order is encoded in the evaluator rather than drawn for the page.
Start by scoping the build.
A proxy that holds the provider keys, checks a requested model against an allow-list, increments a counter and writes a log line solves the three problems most teams have in their first year: keys are no longer scattered across repositories, an agent cannot silently switch to a model nobody priced, and there is a record you can grep. Nothing here suggests replacing that before it starts failing.
It stops at the first question that is not about the request. Which agent was this, and who owns it. What was it allowed to do, as distinct from what it happened to do. Who approved the one action that needed approving, and against exactly which payload. Whether the log you are reading is the log that was written. Adding fields to the log line does not answer those, because the answer has to be trustworthy to somebody who was not there.
The four expensive parts, by name.
Each is a place where the naive implementation is not incomplete but silently wrong. A missing feature gets noticed; a control that reports success while doing nothing does not.
- Provider-specific cache-token accounting
- Anthropic reports cache reads and creation outside the input token count; OpenAI’s and Gemini’s cached tokens are already inside the prompt total. Add them the wrong way round and cache-heavy traffic is mispriced badly, which is why every adapter normalises into mutually exclusive buckets first.
- Streaming egress masking across chunk boundaries
- A card number split across two server-sent-event chunks escapes any redactor reading one chunk at a time, so outbound streams pass through a hold-back buffer with a 64-character floor. A JWT states no maximum length and matches nothing until its third segment arrives, so a fixed window once emitted the head of a 256-character token and reported it had masked nothing.
- Typed failover that does not launder a refusal
- A chain that retries on any error will eventually take a content-policy refusal and return the next provider’s compliant answer as a success, with nothing in the record saying a refusal happened. Classify the failure first.
- Payload-bound approvals
- An approval that authorises an action type authorises every future instance of it, so whoever approved a £200 refund has also approved a £20,000 one. The limit is that approving pushes nothing to the agent, which redeems it by repeating the request with its id.
Which of the two to put in.
Read the left one first. If two of those describe you, buy the other thing. If the right column describes you, the next step is thirty days with the source on your own hardware.
When to choose building it yourself
- You have a handful of agents, one provider, one team, no regulated data, and nobody outside engineering has asked you a question about them.
- Your governance requirement is only a model allow-list and a spend cap, and the actions your agents take are reversible.
- You cannot accept a component you did not write in the request path, which is defensible given that Token Observe fails closed and has no availability SLA.
- You have the appetite to own streaming redaction, provider cache accounting and typed failover as products with tests rather than as tickets that get closed once.
When to choose Token Observe
- The estate has passed one provider and the same rule now has to fire identically on all of them, with something better than a promise that it does.
- You are about to write the approval gate. It goes wrong quietly, because binding to an action type looks like binding to a payload until the day it does not.
- Cost reporting has started disagreeing with the invoice, and the disagreement is concentrated in cache-heavy traffic.
- Somebody outside engineering is going to read the record, which changes it from a log into evidence and changes who may read what.
If the left-hand column describes you, that is still worth an email: a short reply costs both of us less than an evaluation that ends in the same place.
See the thirty-day evaluationThe other comparisons
The other 6 categories, each opening with the case for the alternative.
How long does it take to build the equivalent?
No estimate is offered, because no measured one exists. What can be said is what the parts are: an inline decision point with a fixed evaluation order, provider adapters that normalise usage into mutually exclusive buckets, a streaming hold-back buffer with per-channel state, typed failure classification, payload-bound single-use approvals, budgets reserved before egress in one transaction, and a hash chain with an off-box anchor that is tamper-evident rather than tamper-proof.
Can we read the code before we decide?
Yes, and that is the intended way to evaluate it. The licence is commercial source-available: you may read, compile and modify the source and run it on infrastructure you control, including air-gapped environments, and there is a thirty-day evaluation so your security team, or a third party you commission, can read, run and attack it on your own deployment. Publication is drafted more narrowly than testing — benchmark results with no pre-approval provided the version and configuration tested are named, security findings after coordinated disclosure. Until counsel in England and Wales has reviewed it, it is intended terms rather than an executed grant.
Is buying lower risk than building?
Not automatically, and the product’s own documents are the reason to be careful. Token Observe is in the request path and fails closed; it is a single-writer process on one host with no replica, no clustering and no point-in-time recovery; there is no uptime SLA; and there is no SOC 2, no ISO certification and no independent penetration test. Buying trades a set of risks you know for a set you would have to learn. The trade is worth making when the questions you must answer have moved from traffic to authority and evidence, and not obviously before.
Tell us which way you are leaning, and why.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.