AI agent governance
Any agent that never routes through it — that is a discovery problem first
every questionOn this page
What the phrase means, once you take the marketing out of it
Governance is the set of decisions that are made about an agent by somebody other than the agent, and that the agent cannot decline. That definition is narrow on purpose, because it excludes most of what gets sold under the name. A system prompt asking a model to be careful is not governance; the model can be talked out of it. A framework callback that checks an amount before calling a tool is not governance either; it is enforced by the thing being governed and it changes whenever somebody redeploys. A dashboard that shows you last month’s spend is reporting, and reporting is not a control — it tells you what happened after the money left.
The distinguishing property is that the decision point is outside the agent and in the path. If an agent can reach a model or a tool without the decision being taken, the decision is advisory. This is why almost every serious implementation converges on the same architecture regardless of vendor: something sits between the agent fleet and the providers, resolves who is calling, decides whether the call may proceed, and records what happened either way. What differs between implementations is not the shape but the order of the checks and the honesty of the record.
Three questions organise the whole field, and they are worth holding onto because they sort tools into what they can and cannot answer. Before an agent acts: can you show that the authority it is exercising traces to a delegation somebody made deliberately, and that nothing has widened it since? After it acts: can you show what it actually did — not what it was asked to do, and not what it said it did? And when something fails: can you show that the control was engaged at the time, rather than asserting it was?
The third question is the one that gets skipped, and it is the one an incident turns on. An estate that spent nothing and an estate that spent nothing it could measure emit identical bytes. A policy in shadow mode and a policy nobody wrote look the same from the outside. A radar feed that died in July and a genuinely clean estate both render as an empty findings list. Governance that cannot distinguish those pairs is a control surface with the same failure mode as no control at all, and the failure is silent, which is worse.
The systems you already own, and the specific gap between them
Most organisations arrive at agent governance already holding four things that each cover part of it. An identity platform issues and governs the agent’s identity, ownership, sponsorship and lifecycle. A security stack inspects traffic, applies data-loss rules and hunts anomalies. An API gateway or LLM proxy carries the request, applies quotas and caches responses. An observability tool draws the trace tree and scores the outputs. Each of those is real, and the sensible position is to federate them rather than to rebuild them.
The gap they leave is specific. Identity proves which agent is calling; it does not establish that this call is inside the authority that agent was delegated for this task, on behalf of this person, through this chain of other agents. Traffic inspection reads the payload; it does not know that the refund in that payload is above the threshold a human was supposed to see. The gateway carries the request; carrying it is not deciding it. Observability records the trace; a trace tree is a description of what happened, and a description written by the same process that did the thing is not evidence against the process.
So the work that is genuinely left over is narrow and it is the part that binds: the exact delegated authority at the moment of the call, the business effect that resulted, and a record of both that survives the person who could edit it. Everything else on a governance feature list — the registry, provider routing, quotas, cost dashboards, prompt data-loss controls, trace trees, tool access lists — is table stakes. You need them, they are not hard to find, and a vendor whose lead story is one of them is describing a category rather than a position.
This matters practically when you are choosing what to build first, because it changes the acceptance criterion. A registry that nothing enforces against is a spreadsheet with better formatting; a registry the gateway resolves on every call cannot drift from what is running, because there is only one list. A permission model that grants systems rather than actions hands over the refund endpoint along with the order lookup. A tool access list that filters what the agent can see, without re-checking what it may call, is a usability feature being asked to do access control.
- Identity and registry systems
- Answer which agent, whose agent, and is it still meant to exist. They are the authoritative upstream for ownership and lifecycle, and governance should read from them rather than maintain a competing inventory.
- Security and DLP
- Answer what is in the payload. Necessary, and blind to authority: the same prompt is legitimate from one agent and an incident from another, and the difference is not in the bytes.
- Gateways and proxies
- Answer where the request goes and how often. Quotas bound volume; they do not bound money reliably, because money depends on which model actually served the call and on how the provider counts cached tokens.
- Observability and evals
- Answer what the system did and how good the answer was. Indispensable for quality, and structurally weak as evidence, because the record is produced by the party being examined.
Six controls, and the property each one has to have to count
The list below is not a maturity model and there is no scoring. It is the minimum set that lets you answer the three questions, with the property that separates a working version of each control from a decorative one. Every property here is a hard-won distinction rather than a preference: each is the difference between a control that refuses something and a control that is quietly inert.
Notice how many of the properties are about failure. That is not pessimism, it is where the value is. Controls are easy to build for the case where everything is configured correctly; the engineering is in what happens when the price table is empty, when the delegation header is forged, when the evidence feed stopped arriving six weeks ago, and when the person who can rewrite the database is the person you are collecting evidence about.
- An inventory the enforcement point reads
- One record per agent with a named human owner, a declared purpose and a lifecycle status — and it has to be the same record the gateway resolves on every call. An inventory maintained beside the runtime is updated by whoever remembers, while the runtime is updated by whoever ships, and the two diverge from the first week.
- Action-level permissions that deny by default
- The unit of authorisation is the action, not the system: an agent that needs one read against the order database must not receive the refund endpoint sitting next to it. Delegation between agents must intersect rather than accumulate, or a low-privileged agent escalates simply by asking a higher-privileged one to do the work.
- A policy layer that can do more than block
- Block, redact, park on a human, warn, and stop the agent. And it has to be stageable: a rule you can only learn the false-positive rate of by switching it on will teach you that rate by stopping somebody’s work. Shadow mode is the difference between a control and an outage with a policy id.
- Hard spend and rate ceilings
- Decided before egress, priced against what could actually leave rather than what the caller typed, and refused rather than reported. A budget that only reports is a dashboard, and a price table that cannot price the request has to fail closed — an unpriced model estimated at zero disarms every ceiling above it while the console still shows the ceiling.
- A tamper-evident record of administrative acts
- Hash-chained, so an edit or a deletion breaks verification at a named sequence number. Keyed under a key the database administrator cannot read, if you want it to survive an insider. Anchored off the box with a signature, if you want an auditor to check it without trusting you.
- A way of finding what is not covered
- None of the five controls above sees an agent that calls a provider directly. Discovery is a separate discipline with a separate failure mode, and its cardinal rule is that a dead evidence feed must never be indistinguishable from a clean estate.
The order that survives contact, and why it is not the obvious one
The obvious order is to write the policy first, because policy is the part everyone has an opinion about. It is the wrong order, and the reason is that a policy you cannot measure the effect of is a policy nobody will let you enforce. Every rule that reaches production has to answer the question whose work does this stop, and the only way to answer it is to have been recording traffic for long enough to replay the rule against it.
So the sequence that works starts with the boring parts. Get the traffic through one point and record it. Register the agents that are producing it, with a named owner each, because every later control is scoped to an agent and an unowned agent is a decision nobody can make. Then write permissions, because deny-by-default is the control that keeps working when everything cleverer fails, and because the exercise of writing them tells you what your agents are actually doing. Only then write policy, and stage every rule in shadow before it enforces.
Two things run in parallel with all of that rather than after it. Evidence integrity should be switched on early, not because you need it early but because the guarantee is retrospective: a keyed audit chain covers entries written after the key was configured, and entries written before it are covered only by the checkpoint over the head they reached. The same is true of anchoring — the first anchor attests the head as it stood when the first anchor was made, and says nothing about whether the history beneath it was honest. Every month you wait is a month of history that will never be covered by more than a plain hash.
And discovery should start before you believe you need it, because its output is the denominator for everything else. Governance coverage claims are meaningless without it: an estate where 40 agents route through the gateway and an unknown number do not is not 100 per cent governed, and the honest form of that sentence needs a number on both sides of it.
What nothing in this field can evidence, and why saying so is load-bearing
A governance layer that sits between agents and providers can evidence a great deal about runtime behaviour and almost nothing about the model. It does not evidence training-data provenance, model cards, or bias and fairness testing; those are provider obligations and separate work. It does not record hidden model reasoning, because the gateway sees the request and the response and not the deliberation between them. It does not govern retrieval-layer or vector-store weaknesses when there is no retrieval layer in scope, and it governs training pipelines not at all.
It also cannot control what the calling application does with the text it receives. Output redaction masks values on the way back; the moment the application renders that text into an HTML page or passes it to a shell, the failure is in the application and not in the gateway. This is the standard improper-output-handling risk, and a governance product that claims to close it is describing a control it does not have.
The most important limit is the one about evidence and insiders, and it is the one to press a vendor on. A hash chain catches any alteration that does not also recompute every downstream digest — which is exactly the alteration that a person with write access to the database will not make. Keying the digests moves the target from whoever can write the database to whoever holds the key. Signing a statement of the head with an asymmetric key and publishing it somewhere the database administrator cannot reach moves it again. None of those steps reaches tamper-proof, and a product that uses that word about a database it also writes to has not thought about it.
Finally: no tool makes you compliant. The EU AI Act’s deployer obligations are duties of the deploying organisation. A control can produce the inventory, the logs, the oversight records and the export; deciding the risk classification, running the fundamental-rights assessment where one is required, choosing a retention period that satisfies both the minimum-retention and the storage-limitation duties, and notifying an authority all remain yours. Anything that says otherwise is selling a certificate it does not hold.
How to put AI agent governance into practice
- 01
Route the traffic through one point
Change the base URL and the credential on each agent so its model calls and its tool calls arrive at a single governed endpoint. Until traffic is in one place, every later control is a partial control and every coverage claim is unmeasurable. - 02
Register every agent against a named human
One record each, carrying an owner email, a team and a declared purpose written as a sentence somebody would defend. Make it the record the enforcement point reads, so the inventory cannot drift from what is running. - 03
Write permissions as an allowlist of actions
Grant the specific tools and models each agent needs, deny by default everywhere else, and check that delegation between agents intersects rather than accumulates. Expect the exercise to surface grants nobody could justify; that is the exercise working. - 04
Set hard ceilings before you set policy
Per-request, hourly, daily and monthly spend limits plus per-minute rate limits, decided before the request leaves your network. Confirm that an unpriced model fails closed rather than being estimated at zero, because a zero estimate disarms every ceiling above it. - 05
Stage every policy in shadow, then promote it deliberately
Run each rule in observation mode over real traffic, read what it would have blocked, and replay it against recorded history where you can. Promote it to enforcing as a separate, attributable act rather than as part of authoring it. - 06
Switch on evidence integrity while the history is short
Configure the audit key from a secret manager your database administrators cannot read, then start anchoring and retain the anchors off the box. Both guarantees are forward-looking, so the cost of waiting is a permanently weaker prefix. - 07
Stand up discovery and read its coverage first
Feed the evidence sources you can — the vendor bill, egress logs, provider key listings, endpoint telemetry — and treat the coverage report as the headline. A clean findings list from a source that has never delivered is not a result.
Where this argument meets an implementation
Agent registry
One record per agent, and it is the record the gateway enforces against.
Policy engine
One deterministic verdict on every governed request: allow, block, redact, or park it for a human.
Audit chain
Every administrative act hash-chained; seal it under a key held off the box, and anchor it with a signature your auditor can check alone.
Shadow AI radar
Five evidence sources for AI activity that never touched the gateway, and a coverage model that refuses to call a dead feed a clean estate.
Those pages are one product's implementation of what this guide argues for; describe what your agents actually do and you will get a straight answer about whether you need any of it, including when the answer is no.
Talk it throughHow is AI agent governance different from AI governance?
AI governance is mostly about systems: which models you use, what they were trained on, how they are assessed, and who signed off on deploying them. Agent governance is about actions: an agent holds a credential, calls tools, spends money and changes records in other systems, and the questions become which authority it exercised, on whose behalf, and what effect resulted. The two overlap at the inventory and the risk assessment, and they diverge at the enforcement point — a model card is a document, while an agent permission is something that has to refuse a call at two in the morning.
Do we need agent governance if we already have an LLM gateway?
A gateway gets you the hard part of the plumbing, which is that all traffic arrives at one place. What it usually does not get you is authority: quotas bound volume rather than money, API keys identify an application rather than an agent acting for a person, and a proxy that carries a tool call is not deciding it. The practical question to ask of an existing gateway is whether it can refuse one specific action for one specific agent, whether it can park that action on a named human and bind the approval to the exact payload, and whether its own administrative log survives someone with database access.
Where should a programme start if the estate is already running?
With inventory and recording, in that order, and with discovery running alongside both. Route what you can through one point, register each agent against a named owner with a declared purpose, and start recording — you cannot write a defensible policy until you can replay it against your own traffic. Resist starting with policy: the first thing an unstaged rule teaches you is which team’s work it stopped. In parallel, feed whatever discovery evidence you already have, because the number of agents you do not know about is the denominator for every coverage claim you will make later.
What does a governance layer genuinely evidence for an auditor?
Runtime facts, and only for traffic that passed through it: which agent made which call, under which permissions and delegation chain, which policies matched and what they did, which human approved what and when, what it cost, and every administrative change to the governing configuration in a chain where an edit breaks verification at a known point. What it does not evidence is training-data provenance, model cards, bias and fairness testing, hidden model reasoning, or anything at all about an agent that never routed through it. The last of those is the reason discovery belongs in the same programme rather than in a later phase.
Is a governance product a substitute for certification?
No, and the distinction is worth being blunt about because it is where mapping tables overclaim. A control helps you evidence a clause; it does not make you compliant with it, and installing software does not confer a certificate on the installer. Deployer obligations under the EU AI Act — deciding risk classification, running a fundamental-rights assessment where required, choosing and defending a retention period, informing workers, notifying authorities — remain duties of the deploying organisation. Token Observe itself holds no SOC 2 report, no ISO 27001 certificate and no independent penetration test, and says so rather than letting a mapping table imply otherwise.
Prefer to ask a person? Write to us →
Bring us the question this guide did not answer.
Write to hello@tenhaw.com with what your agents do, which providers they call and what would have to be true for you to put something in front of them. James Rooney replies. You will get a straight answer about whether Token Observe fits, including when it does not.
no form · no qualification step · no sales desk · the other three ways in