Token Observe vs Agent Router

Their usage-based rate limiting page is direct about its own mechanism. Envoy “checks the token budget using usage that has already been charged”, and “token usage is charged after the response completes”; the quota page repeats it in the same register — for each completed request, the token cost is computed using the configured cost expression and charged against the bucket. Streaming is called out explicitly: AI Gateway “does not interrupt an already admitted stream if that stream pushes the token count over the configured limit”, and later requests get 429 until enough usage expires from the window. It is the design a data plane wants, because it needs no price catalogue, no estimate and no lock, only a check on the way in against usage already charged.

What it buys you is fair-share throughput: a tenant that has been consuming heavily gets throttled, promptly, at the next request. What it does not attempt is a hard ceiling in currency on the request that crosses the line, as the vocabulary shows: the limit is written in tokens, weighted by a CEL expression, not in dollars against a per-agent budget. If your requirement is that no single team may exceed a token allowance on a shared cluster, that is the control, and it is free.

Token Observe makes the opposite trade because it was built for a different failure. The USD verdict is deliberately the last one taken: permissions, rate limits and policy are decided first, then the route is resolved, then every provider and fallback that route could execute is priced, and the highest rate in the reachable candidate set is reserved against the agent’s hour, day and month windows inside a single per-agent transaction — BEGIN IMMEDIATE on SQLite, an advisory lock on PostgreSQL — so two concurrent callers cannot both decide against the same pre-reservation window. The estimate is conservative on both legs: the input bound is the UTF-8 byte length of the serialised outbound request plus 256 tokens of framing, and the output leg is priced at the caller’s max_tokens or 4,096 when none is named. A ceiling may be conservative; it may not be optimistic.

The reason that design exists is a defect in Token Observe’s own history rather than an argument against anybody else. The shipped price rows once loaded only under the demo seeder, which the setup documentation tells production operators not to run, so the documented production install had an empty price table: every trace recorded $0 and every per-request ceiling admitted every request. The control was off while appearing to be on, and an estate spending nothing and an estate spending unmetered emit identical bytes. That is why a budgeted agent whose resolved route has an unpriced reachable target is now refused with a 409 before egress, and why the price catalogue is treated as part of the control rather than as reporting furniture. The cost is that a hard-budgeted call gets one potentially billable egress and no failover, because a timeout cannot prove the vendor did not complete and bill the call.

This is the capability the two products are furthest apart on. Agent Router’s published documentation as of 5 October 2026 describes access control through Envoy Gateway SecurityPolicy, quota and usage-based rate limiting, upstream authentication, MCP tool authorisation through JWT scopes, claims and CEL rules, and observability through Prometheus, OpenTelemetry and access logs. A mechanism that parks one exact proposed action on a named person and holds the request until they decide is not described in that material. That is a statement about their documentation on that date. Put it to the project in writing, because an undocumented capability and an absent one look identical from outside, and their reference architecture explicitly invites you to add your own authorisation logic through Envoy’s extension filters, which is a place such a thing could plausibly be built.

What Token Observe does here is specific enough to be checkable. A policy whose action is require_approval refuses the request with a 403, mints an approval record bound to the SHA-256 of the canonicalised action plus the execution context it was proposed in — this subject, with these grants, through this delegation chain, making this call with these arguments — and holds the trace open until somebody decides. Change one argument and the hash no longer matches, and the retry is refused as a mismatch rather than allowed as near enough. Consumption is a compare-and-set, so two concurrent retries cannot both execute, and the approval expires: 60 minutes by default, and anywhere from one minute to seven days by policy.

The limit is the part operators get wrong. Approving does not push anything to the agent. There is no callback; an approval is a permission the agent redeems by repeating the identical request with its id, and the console says so in those words because it once did not, and an operator who believes the work is unblocked goes back to their day while it is not. Reach has a limit too: response-side evaluation of proposed tool calls extends the gate to agents that execute tools in their own process, and the architecture calls that defence in depth rather than a guarantee. Token Observe can only refuse a proposal it is shown.

Their reference-architecture post describes the shape of the product: you “implement safety checks and output validation rules that enable your team to control quality and compliance centrally, rather than embedding these checks individually within applications”, you “set usage-based guardrails directly in Envoy AI Gateway to prevent cost overruns”, and you “add your own authorization logic and/or custom functionality via Envoy’s extension filters”. The tier-one gateway centralises coarse-grained policies — authentication, top-level routing, global rate limiting — and provides what they call a single control point for platform-wide governance. That is a foundation you build the specific control on, which is the right architecture for a general data plane serving seventeen providers and an unknown set of requirements, and it is why the project can be as widely adopted as it is.

Token Observe arrives with the specific control already decided, and with the corresponding loss of flexibility. One pure function evaluates every governed request — the model gateway, the tool path, a backtest replay and a bundle compiled for a developer laptop all call the same evaluator, so a rule means the same thing wherever it is evaluated. A policy is a trigger, an action and a scope. Triggers match on the tool and its argument values, the model and its estimated input size, accumulated spend, request and token rate, the data classes detected in the payload, the prompt-injection score and the source it came from, or the hour of day in UTC. Actions are block, require approval, redact, warn and suspend the agent, resolved to a single verdict in which a block beats an approval and an approval beats a redaction. You cannot write an arbitrary filter; you can write those.

Which of the two you want follows from who is going to maintain it. A platform team with Envoy expertise and a clear internal specification will build a better-fitting control on extension filters than any vendor’s policy language would give them, and will keep it running on infrastructure they already operate. A team whose requirement arrived from a risk committee rather than from an architecture review usually wants the opinionated version, because the argument they need to win is about evidence rather than about topology, and “here is the verdict, the rule that produced it, the human who approved it and the chain entry that proves neither was edited afterwards” is a different artefact from a metrics dashboard, however good the dashboard.

Both products put a gateway in front of Model Context Protocol servers, and the overlap is real. Theirs is documented to “aggregate multiple MCP servers into a single unified endpoint”, prefixes tool names with the backend name so routing is unambiguous, filters the exposed set with a toolSelector taking either exact names or regular expressions, implements the MCP Authorization specification with issuer configuration and audience validation, and evaluates access against JWT scopes and claims with CEL expressions and a default fallback action, plus OpenTelemetry tracing and Prometheus metrics on every MCP request, using the same stack as their LLM traffic.

Token Observe’s gateway asks a narrower question and a later one. Tools reach an agent namespaced server.tool and filtered to that agent’s grants, but the filtering is treated as usability rather than as the control: every tools/call is authorised again at execution, because a client can guess a tool name, and a tool the caller may not use is answered as unknown rather than forbidden, because confirming that it exists is an inventory disclosure. Each tool’s name, description and input schema is hashed when an operator approves it and re-checked on every catalogue refresh, so an upstream that quietly rewrites a descriptor — which is a rewrite of the model’s instruction surface, not of its data — is quarantined and refused until a human approves it again. An unpinned tool stays usable, because a gateway that refused every unreviewed tool would not be adopted.

The third difference is on the way back. A tool result is scanned as its own source and weighted 1.25 times the same words typed by a person, on the reasoning that the author of a tool result is data rather than a principal, and it is re-evaluated against data-class and injection rules before the model sees it. That is the channel through which agents get hijacked, and it is the part of the tool path that neither authorisation nor telemetry addresses. Their material describes authorisation and observability on MCP requests; result inspection of that kind is not described in their published documentation as of 5 October 2026, so ask about it rather than assume either way.