Token Observe vs building it yourself
For a small estate, a few hundred lines of proxy is usually the right answer. The cost arrives later, and it arrives in specific places.
LiteLLM, Envoy, NGINX, Open Policy Agent, OpenTelemetry Collector, PostgreSQL. Every claim about them on this page is that vendor's own published material, untested by us.
the other comparisonsOn this page
You own the code, you own the failure surface, and for a small estate that is the whole argument
A proxy you wrote fits your deployment, your language, your on-call rota and your review process. There is no licence, no vendor conversation, no second supplier in the request path, and no component whose failure modes you have to learn from someone else’s documentation. If your governance requirement today is genuinely a model allow-list and a per-day spend cap read from a counter, that is a small amount of code that will be correct, and buying a governance product to satisfy it is the more expensive and more fragile option.
There is a sharper version of this argument that applies specifically to buying Token Observe. It sits in the request path and fails closed, which means its availability becomes a governance property of your environment: if it stops, governed agents cannot call models. At its current target scale it is a single-writer process on one host with no replica, no clustering, no point-in-time recovery and no vendor-operated uptime SLA. If your estate is small enough that you can hold its entire failure surface in your head, introducing a fail-closed dependency you did not write is adding risk in exchange for evidence nobody has asked you for.
The named products above are the components a team usually assembles rather than a competing product, and nothing on this page tests, describes or asserts the behaviour of any of them. Every row in the table below, and every pattern in the sections after it, describes a first-cut proxy written in-house — drawn from the failure patterns the product’s own engineering notes record having hit while building this one. None of it is a claim about LiteLLM, Envoy, NGINX, Open Policy Agent, the OpenTelemetry Collector or PostgreSQL, and none of it is a claim about your code either.
Token Observe and building it yourself, row by row
One card per dimension rather than a three-column table, because the two sides are rarely the same length and a table of them is a horizontal scroller on a phone.
The first cut adds the provider’s usage numbers to a running total.
Normalise into mutually exclusive buckets first. Anthropic reports cache reads and writes outside the input total; OpenAI and Gemini report cached tokens inside it. The product’s own cost module puts the error from getting that wrong at 50 to 90 per cent on cache-heavy agent traffic — its own figure, stated beside the code that avoids it, not a measured benchmark.
The first cut redacts the response once it has been assembled.
A hold-back buffer with a 64-character floor and a separate channel per tool-call argument, with the cut pulled back off any match it would split and out of the middle of an unbroken run of value characters.
The first cut retries the next provider on error.
Seven typed classes: a 429, a timeout and a 5xx fail over; a content-policy refusal, an authentication failure, an over-long context and a malformed request do not.
The first cut is a queue, a boolean, and a retry that trusts the caller.
Bound to the SHA-256 of the canonical action plus its execution context, single-use through a compare-and-set so two concurrent retries cannot both execute, and expiring.
The first cut counts spend from the response, after the money is gone.
Reserved before egress in one per-agent transaction, priced at the most expensive reachable candidate on the route, and refused with ACP_BUDGET_UNPRICED if any reachable target has no price row.
The first cut treats configuration that parses as configuration that works.
Typed configuration errors at boot for half-configured couplings — one to three of the four identity values, an audit key below the minimum length — because the alternative is a control that looks configured and enforces nothing.
What a day’s work genuinely buys, and where it stops
A proxy that holds the provider keys, checks a requested model against an allow-list, increments a counter and writes a log line solves the three problems most teams actually have in their first year: keys are no longer scattered across repositories, an agent cannot silently switch to a model nobody priced, and there is a record you can grep. That is a genuine control and it is proportionate to a small estate. Nothing on this page suggests replacing it before it starts failing.
It stops at the first question that is not about the request. Which agent was this, and who owns it. What was it allowed to do, as distinct from what it happened to do. Who approved the one action that needed approving, and against exactly which payload. Whether the log you are reading is the log that was written. Those are questions about authority and evidence rather than about traffic, and they are not answered by adding fields to the log line, because the answer has to be trustworthy to somebody who was not there.
The point at which teams typically discover this is not a design review. It is an invoice that does not reconcile, an auditor’s question with a date attached, or an agent that did something a person has to explain. The product’s own framing of the buying trigger is a consequential, externally observable action — a refund, a deployment, an email, a ticket transition, a database update — where the sentence the API returned success is not acceptable as proof.
The four expensive parts, named precisely
Each of these is a place where the naive implementation is not merely incomplete but silently wrong, which is the property that makes them expensive. A missing feature gets noticed; a control that reports success while doing nothing does not.
- Provider-specific cache-token accounting
- Anthropic reports cache reads and cache creation exclusive of the input token count; OpenAI’s cached tokens are already inside the prompt total, as are Gemini’s. Add them the wrong way round and cache-heavy agent traffic is mispriced badly — the product’s own cost module states the error at 50 to 90 per cent on that kind of traffic, which is the reason every adapter normalises into buckets that are mutually exclusive by construction before any arithmetic runs. Provider usage is also treated as untrusted wire data even where a type says number, because a negative or non-integer value would subtract from a budget or turn cost arithmetic into a value SQLite cannot store.
- Streaming egress masking across chunk boundaries
- A card number split across two server-sent-event chunks escapes any redactor that looks at one chunk at a time, so outbound streams pass through a hold-back buffer with a floor of 64 characters and a separate buffer for each tool-call argument channel. Two rules are needed rather than one: the cut is pulled off any detected match it would split, and out of the middle of an unbroken run of value characters. Sixty-four characters exceeds every kind that states a maximum length — the longest is a spaced 34-character IBAN — but a JWT states none and matches nothing at all until its third segment arrives, so a fixed window alone once emitted the head of a 256-character token and then reported that it had masked nothing. A run beyond 4,096 characters is replaced with an irreversible marker and suppressed through its delimiter, so memory stays bounded without releasing half of an ambiguous value.
- Typed failover that does not launder a refusal
- A fallback chain that retries on any error will, sooner or later, take a provider’s content-policy refusal and return the next provider’s compliant answer as a success. Nothing in the record then says a refusal happened. Token Observe classifies the failure first: a 429, a timeout and a 5xx move to the next candidate; a content-policy refusal, an authentication failure, an over-long context and a malformed request stop where they are. Every fallback candidate is also re-checked against the agent’s data policy, so failing over cannot route around a no-training or region requirement, and each provider carries its own circuit breaker.
- Payload-bound approvals
- An approval that authorises an action type authorises every future instance of it, which means the human who approved a £200 refund has also approved a £20,000 one. The binding here is the SHA-256 of the canonical action plus the execution context it was proposed in, so changing one argument makes the retry a mismatch rather than a near-enough match; consumption is a compare-and-set, so two concurrent retries cannot both execute; and it expires, at 60 minutes by default. The honest limit travels with it: approving pushes nothing to the agent, because there is no way to call an agent back — the agent redeems the approval by repeating the identical request with its id.
- The one-egress budget rule
- A hard spend ceiling and a retry policy are in direct conflict, and most home-grown proxies never notice the conflict because they count spend afterwards. A hard-budgeted call here permits exactly one potentially billable provider egress and therefore has no ambiguous-failure retry or failover, and the admitted charge is retained after an ambiguous failure or a partial stream. That is a deliberate trade of availability for a boundary you can defend, and it is stated as one.
The parts nobody puts in the estimate
Evidence is the largest of them. A hash-chained log is easy; a hash chain that means something to a party who does not trust your database administrator is not. Entry digests become HMAC under a key held outside the database only if that key exists, and initialising it needs a two-boot ceremony because a chain that silently changed protection mode mid-life would be worse than one that never had a key. Above it, an Ed25519 anchor signs a statement of the head on a schedule and publishes it to a sink the database administrator does not control, and it refuses to sign a chain that does not verify, a head that has moved backwards, or a rewritten anchored entry — signing over a rewrite would launder it under a key auditors trust. What that buys is one narrow thing, and overclaiming it is the usual mistake: any copy you kept off-box beats any rewrite made after you took it. Key theft, pre-anchor history, collusion among every sink, and time itself all remain open, and only a timestamping authority or a public ledger proves when.
Fail-closed configuration is the second. A control that half-starts is worse than one that does not start, so half-configured couplings are refused at boot with a typed error rather than degraded: one to three of the four identity values instead of all four or none, a local-login switch turned off without complete identity configuration, an audit key below its minimum length. Two traps in particular are worth knowing because they are the shape of the whole problem — an anchor key generated in the wrong format passes configuration parsing and then dies later at container construction with an encoding error, and an identity issuer with a trailing slash fails at first sign-in rather than at boot, so it passes the readiness check and looks fine.
The rest is a long tail that never appears in a build estimate: an unpriced route refusing before egress rather than pricing at zero, because an empty price table is exactly how a ceiling gets silently disarmed; keys stored as digests with a display prefix so a lost token can only be replaced rather than recovered; auth rejections rolled up hourly for the radar without ever storing the presented token or a digest of it; a search box that emits a validated filter object rather than SQL because trace content is attacker-influenced; and read paths that are themselves attributable because evidence that anyone can read anonymously is not evidence about who looked.
Which of the two you should actually put in.
Both columns are real answers and both are the same length on the page. Read the left one first: if it describes your estate, it is the cheaper decision and this page has done its job.
When to choose building it yourself
- You have a handful of agents, one provider, one team, no regulated data, and nobody outside engineering has asked you a question about them.
- Your governance requirement is genuinely a model allow-list and a spend cap, and the actions your agents take are reversible.
- You cannot accept a component you did not write in the request path, which is a defensible position given that Token Observe fails closed and has no availability SLA.
- You have the appetite to own streaming redaction, provider cache accounting and typed failover as products with tests rather than as tickets that get closed once.
When to choose Token Observe
- The estate has passed one provider and the same rule now has to fire identically on all of them, with something better than a promise that it does.
- You are about to write the approval gate. That is the piece that goes wrong quietly, because an approval bound to an action type looks exactly like one bound to a payload until the day it does not.
- Cost reporting has started disagreeing with the invoice, and the disagreement is concentrated in cache-heavy traffic.
- Somebody outside engineering is going to read the record, which changes it from a log into evidence and changes who may read what.
If the left-hand column is the one that describes you, that is still worth an email: a straight answer about which of these to buy costs both of us less than an evaluation that ends in the same place.
Ask which one fitsThe other comparisons
Same template, same order, same concession first. Claims about every named product on all of them are that vendor's own and have not been independently tested.
Token Observe vs LLM gateways
Token Observe is a gateway in delivery. The gateway is how it arrives, not what it is for.
Token Observe vs LLM observability
One refuses the call inline. The other scores it afterwards. Most estates need both, and they are not substitutes.
Token Observe vs AI security platforms
Token Observe is not a complete AI security suite, and the product’s own strategy document forbids selling it as one.
Token Observe vs cloud-native controls
If every agent, model and tool lives in one cloud, use that cloud’s controls. The argument here is for the estate that does not.
Token Observe vs doing nothing
With three agents, no regulated data and no incident, doing nothing is often the correct decision. This page is about what changes it.
How long does it take to build the equivalent?
No estimate is offered, because no measured one exists and inventing a number to win an argument is exactly the kind of claim this product refuses to make. What can be said is what the parts are: an inline decision point with a fixed evaluation order, provider adapters that normalise usage into mutually exclusive buckets, a streaming hold-back buffer with per-channel state, typed failure classification, payload-bound single-use approvals, budgets reserved before egress inside one transaction, a hash chain with an off-box anchor, and a search path that never builds SQL from model output. Price those against your own team’s rates and your own appetite for owning them for the next three years.
Can we read the code before we decide?
Yes, and that is the intended way to evaluate it. The licence is commercial source-available and its intended terms are these: you may read, compile and modify the source and run it on infrastructure you control, including air-gapped environments, and there is a 30-day evaluation written so that a prospective customer’s security team can read, run and attack the software before a purchase order is raised — no gag clause and no pre-approval of results. Redistribution and offering it as a competing hosted service are not permitted. The caveat is on the licence’s own first page rather than in ours: it was drafted by the engineering team, and until qualified counsel in England and Wales has reviewed it and every square-bracketed placeholder has been completed it is a statement of intended commercial terms, not an executed grant of rights. The governance domain itself is a package with zero runtime dependencies, so what allow, block, approval and delegation actually mean can be read as pure functions without standing up a database, a network or a clock.
What if we build now and buy later?
That is a reasonable sequence and the migration is not dramatic, because the integration point is the same one your own proxy uses: the agent’s base URL and key. What does not transfer is history — a record produced by a different system is not evidence in this one’s chain, and there is no import path that would make it so without pretending. The practical version is to run both for a period, route the agents that take consequential actions through Token Observe first, and keep your proxy for the rest, which is also the shape of the product’s own pilot boundary of roughly five to fifty agents owned by one platform team.
Is buying actually lower risk than building?
Not automatically, and the product’s own documents are the reason to be careful. Token Observe is in the request path and fails closed; it is a single-writer process on one host at this scale with no replica, no clustering and no point-in-time recovery; there is no vendor-operated uptime SLA and the reasons for that are published; and there is no SOC 2, no ISO certification and no independent penetration test. Against a home-grown proxy that you operate and understand, buying trades a set of risks you know for a set you would have to learn. The trade is worth making when the questions you must answer have moved from traffic to authority and evidence, and not obviously before.
Which parts of this would a small team realistically get wrong?
Judging by where the product’s own engineering notes record the effort going: cache-token accounting, because the providers genuinely disagree by construction and the error compounds silently in a cost report; streaming redaction, because a value split across chunks defeats a per-chunk redactor and a fixed window defeats itself on a token with no maximum length; failover, because retrying on any error eventually returns a compliant answer in place of a refusal and nothing in the record says so; and approvals, because binding to an action type is indistinguishable from binding to a payload until the payload changes. All four fail without an error, which is the property that makes them worth buying rather than writing.
Tell us which way you are leaning, and why.
Write to hello@tenhaw.com with what your agents do, which providers they call and what would have to be true for you to put something in front of them. James Rooney replies. You will get a straight answer about whether Token Observe fits, including when it does not.
no form · no qualification step · no sales desk · the other three ways in