MCP gateway
The half of an agent that moves money is a tool call, not a sentence. This is the checkpoint that call goes through.
Governing the model call governs the half that only produces text.
The half of an agent that moves money, writes to a system of record or emails a customer is a tool call, and in most deployments it goes straight from the agent’s process to the upstream server: no permission check outside the agent’s own code, no record an auditor can read, and no way for a rule about refunds to bind when one is issued.
The tools themselves are the second problem, because they are an instruction surface. A tool’s name, description and input schema are text the model reads and obeys, so an upstream that changes them rewrites what your agent believes it is doing without touching a line of your code. The version that matters is patient: a tool that behaved for six weeks, then acquired a new sentence.
How it works
The whole request path, run on a tool call. The order is encoded in the evaluator, and where each step sits is the reason the guarantee holds.
- 01
Answer the list with the pinned, granted, inspected subset
Authority is re-derived from the bearer token on every request; sessions carry no permissions. The catalogue is every enabled upstream server’s tools, namespaced server.tool and filtered to the caller’s grants, and each descriptor is scanned in its own right, so one carrying a credential or matching the injection heuristics is withheld rather than shipped into the model’s context. - 02
Re-authorise the call, and answer an invisible tool as unknown
The call path checks permissions again from scratch. Existence and visibility collapse into one answer: a tool that does not exist and a tool this agent was never granted both return unknown tool, because confirming a tool exists is an inventory disclosure. The probe is still traced. - 03
Inspect the arguments, then decide with the same evaluator
The whole argument tree is walked under one budget, and crossing any bound refuses the call rather than forwarding an uninspected remainder. Then kill switch, lifecycle, permissions intersected across every hop, budget and rate ceilings, and policies in priority order. - 04
Scan what comes back, then deliver it
The result is inspected under the same bounds and re-evaluated against data-class and injection rules only, so a rate rule or an approval just satisfied does not fire twice. A refusal on the way back is terminal and says so, because the tool has already run.
- beat 01Name the caller
- 1Authenticate
- 2Resolve the agent
Nothing is decided for an anonymous caller. The key resolves to one registered agent with a named owner, and its roles, its ceilings and any stop switch in force are read live from the store on every request, not from a cached copy, so a suspension takes effect on the next call instead of after a redeploy.
- beat 02Start the record, then read what is being sent
- 3Open the trace
- 4Sanitise
- 5Scan
The trace is opened before anything can refuse the request, which is why a blocked call is recorded as thoroughly as one that went through. Only then is the text normalised and scanned, in that order, so the detectors see what the model will read rather than what a human reviewer can see.
- beat 03Decide once
- 6Govern
- 6bIntersect with the named human
- 7Enact
One function returns one verdict: allow, block, or hold it for a named person to approve. Every check lives in that one place rather than scattered along the path, which is what makes it possible to read your rules and know what they do.
- beat 04Send it, and only then spend the money
- 8Route
- 9Call upstream
Routing fixes which provider and model will serve the call, so where an agent has a USD ceiling it is tested against the price you will pay, not the one the caller typed. Routing is also the last step that can still refuse locally: after it, the request has left your network.
- beat 05Govern the answer, then count the cost
- 10Govern the response
- 11Meter and record
A tool call the model proposes is judged before it reaches your agent. Metering runs last and on every ending, including the refusals and the streams a client abandoned halfway, because the tokens were spent either way and a partial trace is still evidence.
Step names and their grouping are the product’s own, and the order is encoded in the evaluator rather than drawn for the page.
A descriptor that changes is quarantined.
Name, description and input schema are hashed into one SHA-256 over a canonical serialisation, so descriptors differing only in key order hash identically. An operator approves that hash; every refresh recomputes and compares it, so the administrative view of a server and the integrity check on it are one operation.
- Unpinned
- Nobody has approved it. Still listed and callable, surfaced to operators, described to the model as unverified. A gateway refusing everything unreviewed would not be adopted.
- Quarantined
- The description or input schema changed after approval. Hidden from the catalogue, refused on call, published as an event and raised as a high-severity radar finding.
- Missing upstream
- Approved here, and the server no longer offers it. Reported rather than dropped: an approved capability disappearing without a change request is itself evidence.
tool orderdb.issue_refund
approved hash 4f1c9a… over { name, description, inputSchema }
hash now b70e23…
state quarantined · detected 29 Aug 2026 04:31
approved by operator@example.com · 14 Aug 2026 11:02
the approved TEXT is not stored — only its hash, so no diff existsIt can only refuse a call it is shown.
A rule saying refunds over £200 need approval ought to mean the same thing whichever way the refund is issued. It binds here because this endpoint is the execution path, but an agent holding the tool itself never reaches it. So the model gateway evaluates tool-call rules against the calls a model proposes, before the response returns.
That is defence in depth, not a guarantee. An agent routing nothing through Token Observe is caught by the shadow-AI radar instead. An enforced control and a detective one are not the same purchase: the difference is between a refund that did not happen and a refund you found out about.
agent -> Token Observe /mcp -> upstream tool
enforced: the call is refused, redacted or parked for a human
agent -> Token Observe /v1 -> model provider
enforced on the proposal: the response is refused before it
returns, but the agent still holds the tool
agent -> vendor API directly
not governed: found by the radar, refused by nothingAnything that speaks an OpenAI-compatible, Anthropic or Gemini dialect. One base URL and one key change.
Authorise, redact, budget, decide. Then forward, refuse, or hold for a human.
Fails closed. A decision it cannot make is a request that does not go.
OpenAI, Anthropic, Google Gemini, OpenRouter, Amazon Bedrock, Azure OpenAI, MCP tool servers, and any OpenAI-compatible endpoint you host.
Registry, policy decisions, the hash-chained audit log and the flight recorder. Held in your database, on your disk. Nothing goes to the vendor.
Each line closes off an assumption a reader would otherwise carry into a proof of concept.
What this does not doWhat this leans on
Agent permissions
Deny by default, explicit deny wins, and delegation intersects, so an agent cannot borrow authority it was never granted.
Effect contracts
The action leaves once, and success is what a second pinned tool observed.
Shadow AI radar
Five evidence sources for AI activity that never touched the gateway, and a coverage model that refuses to call a dead feed a clean estate.
Where this comes up
Jobs that draw on it
Guides that lean on it
The term, defined
An AI gateway is a policy-bearing proxy for AI traffic: it terminates the calls an application or agent makes to models, tool servers and other agents, and applies authorisation, inspection, quota and logging rules to them before they reach the upstream. The term is used for at least three different products — an API gateway extended to model traffic, a security-inspection layer sold as an AI firewall, and a cloud runtime that bundles gateway, identity and observability — so the label describes a position in the architecture rather than a set of guarantees.
Why does the gateway check permissions twice?
Because filtering a tool list is a usability feature, not access control: a client can call a name it was never shown, so the call path re-evaluates from scratch. A tool the agent cannot see returns the protocol error unknown tool, without distinguishing “does not exist” from “not yours”, because confirming existence is an inventory disclosure. A tool it can see but may not use returns an error-flagged result the model can act on.
What happens when an upstream tool’s description changes?
It is quarantined on the next catalogue refresh and refused until a human approves it again: the mismatch hides the tool, records the drift with both hashes, publishes an event and raises a high-severity radar finding. Unpinned tools are treated differently on purpose: they stay usable, and their description tells the model the descriptor is unverified.
Does the gateway govern tool calls an agent executes itself?
Not directly, and Token Observe is explicit about it. When a model proposes a tool call, the model gateway evaluates that proposal against the same rules before returning the response, so a refunds rule binds whether or not execution is routed here. That is defence in depth rather than a guarantee: it can only refuse a proposal it is shown.
Prefer to ask a person? Write to James
Bring us the agent you are least comfortable with.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.