--- title: "Token Observe versus building it yourself" canonical_url: "https://tokenobserve.com/compare/build-your-own" published: 2026-09-02 last_updated: 2026-10-21 --- # Token Observe versus building it yourself For a small estate, a few hundred lines of proxy is usually the right answer. The cost arrives later, and it arrives in specific places. ## The comparison For three agents, one provider and no auditor, writing the proxy yourself is usually correct: a model allow-list, a spend counter and a log is a day’s work, your team owns it, and it will be right because the surface it covers is small. This page is about four places a home-grown proxy becomes expensive, all drawn from the parts of Token Observe that took longest to get right: provider-specific cache-token accounting, streaming egress masking across chunk boundaries, failover that does not launder a refusal into a success, and approvals bound to an exact payload rather than an action type. None is hard in principle. They are hard in aggregate, they fail quietly rather than loudly, and they are not what your platform team was hired to do. ## Named products in this category - LiteLLM - Envoy - NGINX - Open Policy Agent - OpenTelemetry Collector - PostgreSQL ## Facts - Where build wins: One provider, few agents, no auditor, reversible actions - First hard part: Cache-token accounting differs per provider by construction - Second: A secret split across two stream chunks escapes redaction - Third: A fallback chain that turns a refusal into a success ## The limit Who needs it: A small estate rarely needs any of this ## Where they win: You own the code and the failure surface, and for a small estate that is the whole argument. A proxy you wrote fits your deployment, your language, your on-call rota and your review process. There is no licence, no vendor conversation, no second supplier in the request path, and no component whose failure modes you have to learn from someone else’s documentation. If your governance requirement today is only a model allow-list and a per-day spend cap, that is a small amount of code that will be correct. A sharper version applies to buying this. Token Observe sits in the request path and fails closed, so its availability becomes a governance property of your environment, and at this scale it is a single-writer process on one host with no replica, no clustering, no point-in-time recovery and no uptime SLA. One thing before the table: the products named above are the components a team assembles, and every row describes a first-cut proxy written in-house — not a claim about any of them, and not about your code either. ## The difference, dimension by dimension ### Cache-token accounting Them: The first cut adds the provider’s usage numbers to a running total. Token Observe: Normalise into mutually exclusive buckets first. Anthropic reports cache reads and writes outside the input total; OpenAI and Gemini report cached tokens inside it. ### Streaming egress Them: The first cut redacts the response once it has been assembled. Token Observe: A hold-back buffer with a 64-character floor and a separate channel per tool-call argument, with the cut pulled back off any match it would split. ### Failover Them: The first cut retries the next provider on error. Token Observe: Seven typed classes: a 429, a timeout and a 5xx fail over; a content-policy refusal, an authentication failure, an over-long context and a malformed request do not. ### Approvals Them: The first cut is a queue, a boolean, and a retry that trusts the caller. Token Observe: Bound to the SHA-256 of the canonical action plus its execution context, single-use through a compare-and-set so two concurrent retries cannot both execute, and expiring. ### Budgets Them: The first cut counts spend from the response, after the money is gone. Token Observe: Reserved before egress in one per-agent transaction, priced at the most expensive reachable candidate, and refused with ACP_BUDGET_UNPRICED if any target has no price row. ### Start by scoping the build. A proxy that holds the provider keys, checks a requested model against an allow-list, increments a counter and writes a log line solves the three problems most teams have in their first year: keys are no longer scattered across repositories, an agent cannot silently switch to a model nobody priced, and there is a record you can grep. Nothing here suggests replacing that before it starts failing. It stops at the first question that is not about the request. Which agent was this, and who owns it. What was it allowed to do, as distinct from what it happened to do. Who approved the one action that needed approving, and against exactly which payload. Whether the log you are reading is the log that was written. Adding fields to the log line does not answer those, because the answer has to be trustworthy to somebody who was not there. ### The four expensive parts, by name. Each is a place where the naive implementation is not incomplete but silently wrong. A missing feature gets noticed; a control that reports success while doing nothing does not. - Provider-specific cache-token accounting: Anthropic reports cache reads and creation outside the input token count; OpenAI’s and Gemini’s cached tokens are already inside the prompt total. Add them the wrong way round and cache-heavy traffic is mispriced badly, which is why every adapter normalises into mutually exclusive buckets first. - Streaming egress masking across chunk boundaries: A card number split across two server-sent-event chunks escapes any redactor reading one chunk at a time, so outbound streams pass through a hold-back buffer with a 64-character floor. A JWT states no maximum length and matches nothing until its third segment arrives, so a fixed window once emitted the head of a 256-character token and reported it had masked nothing. - Typed failover that does not launder a refusal: A chain that retries on any error will eventually take a content-policy refusal and return the next provider’s compliant answer as a success, with nothing in the record saying a refusal happened. Classify the failure first. - Payload-bound approvals: An approval that authorises an action type authorises every future instance of it, so whoever approved a £200 refund has also approved a £20,000 one. The limit is that approving pushes nothing to the agent, which redeems it by repeating the request with its id. ## Choose the alternative when - You have a handful of agents, one provider, one team, no regulated data, and nobody outside engineering has asked you a question about them. - Your governance requirement is only a model allow-list and a spend cap, and the actions your agents take are reversible. - You cannot accept a component you did not write in the request path, which is defensible given that Token Observe fails closed and has no availability SLA. - You have the appetite to own streaming redaction, provider cache accounting and typed failover as products with tests rather than as tickets that get closed once. ## Choose Token Observe when - The estate has passed one provider and the same rule now has to fire identically on all of them, with something better than a promise that it does. - You are about to write the approval gate. It goes wrong quietly, because binding to an action type looks like binding to a payload until the day it does not. - Cost reporting has started disagreeing with the invoice, and the disagreement is concentrated in cache-heavy traffic. - Somebody outside engineering is going to read the record, which changes it from a log into evidence and changes who may read what. ## Questions and answers Q: How long does it take to build the equivalent? A: No estimate is offered, because no measured one exists. What can be said is what the parts are: an inline decision point with a fixed evaluation order, provider adapters that normalise usage into mutually exclusive buckets, a streaming hold-back buffer with per-channel state, typed failure classification, payload-bound single-use approvals, budgets reserved before egress in one transaction, and a hash chain with an off-box anchor that is tamper-evident rather than tamper-proof. Q: Can we read the code before we decide? A: Yes, and that is the intended way to evaluate it. The licence is commercial source-available: you may read, compile and modify the source and run it on infrastructure you control, including air-gapped environments, and there is a thirty-day evaluation so your security team, or a third party you commission, can read, run and attack it on your own deployment. Publication is drafted more narrowly than testing — benchmark results with no pre-approval provided the version and configuration tested are named, security findings after coordinated disclosure. Until counsel in England and Wales has reviewed it, it is intended terms rather than an executed grant. Q: Is buying lower risk than building? A: Not automatically, and the product’s own documents are the reason to be careful. Token Observe is in the request path and fails closed; it is a single-writer process on one host with no replica, no clustering and no point-in-time recovery; there is no uptime SLA; and there is no SOC 2, no ISO certification and no independent penetration test. Buying trades a set of risks you know for a set you would have to learn. The trade is worth making when the questions you must answer have moved from traffic to authority and evidence, and not obviously before.