AI agent governance
also called agent governance · agentic AI governance · AI agent governance framework · autonomous agent governance
The definition is narrow on purpose, because most of what is sold under the name fails it. A system prompt asking a model to be careful is not governance; the model can be argued out of it, in text it was asked to read. A callback inside the agent’s own framework that checks an amount before calling a tool is enforced by the thing being governed, and it changes whenever somebody redeploys. A dashboard showing last month’s spend is reporting, not a control — it describes what happened after the money left. What distinguishes a control from all three is that the decision point sits outside the agent, in the path, and the agent cannot decline it.
The field is organised by when the decision falls. Before the action: whether the authority in play traces to a deliberate delegation and has not widened since. After it: what the agent did, taken from the record rather than from the agent’s own account of it. And across both: whether the control was running at the moment it was needed, which is a property of the record rather than of the policy.
Four adjacent systems each cover part of this. Identity answers who is calling, security tooling answers what is in the payload, a gateway answers where the request goes, and observability answers what the run looked like from inside; none of them settles the tie between the authority delegated for one particular call and the effect that call had on the business. The remainder decomposes into six controls: an agent inventory, action-level permissions that deny by default, a policy layer, spend and rate ceilings, a tamper-evident record of administrative acts, and a way of finding traffic that routes past the other five. The guide on this site that asks what AI agent governance actually requires takes all six at length, with the order to build them in and the acceptance test for each.
The regulatory anchors — EU AI Act Articles 12, 14 and 26, with the six-month log-retention floor in Article 26(6); ISO/IEC 42001; NIST AI RMF 1.0; and the OWASP LLM and agentic risk lists — set the evidence an operator must produce rather than the architecture that produces it, and a layer between agents and providers evidences nothing about training-data provenance, model cards or bias testing, records no hidden model reasoning, does not control what the calling application does with the text it receives, and reaches only what routes through it.
A support agent that can issue refunds
A support agent is given a read on the order database and a refund tool. Governance is what makes four things true at the moment it acts, none of which the agent controls: the refund tool is in its granted action set and the account-deletion tool sitting next to it in the same system is not; a refund above the threshold somebody chose is held for a named human rather than issued; that hold is bound to the exact refund amount and account, so approving £40 does not authorise £40,000; and the whole sequence — the request, the verdict, who approved it, what the refund system was afterwards observed to have done — is recorded where the person who ran the agent cannot quietly amend it. Remove any one of the four and you still have a support agent that issues refunds. You no longer have an answer to the question a regulator, an auditor or a customer will actually ask, which is who decided this.
What ai agent governance is routinely confused with
- AI governance
- AI governance is the organisation-wide programme — risk classification, model procurement, impact assessment, policy, training, committee structure. Agent governance is the runtime subset that acts on individual actions while they are happening. One produces documents and decisions; the other produces refusals and records.
- Model safety and alignment
- Safety work changes what the model is disposed to do, and it is done by whoever trained it. Governance changes what the deployment permits, and it holds when the model’s disposition fails — which is the case worth designing for, since a model talked into ignoring its instructions has not stopped being able to call the tools it was given.
- LLM observability
- Observability answers what the system did and how good the answer was, and it is produced by the same process that did the thing. Governance answers whether it was allowed to and can refuse before the fact. A trace is a description; evidence is a description somebody other than the describer can check.
Related terms
Agent action assurance
Agent action assurance is the practice of proving that the exact authority delegated for one agent action was the authority actually used, and that the action produced the business effect it reported producing. Its subject is the consequential, externally observable action — a refund, a deployment, an outbound email, a ticket transition, a row written to a system of record — where a provider returning a success status is evidence that a request was transported, and not evidence that anything happened.
AI control plane
An AI control plane is the layer that holds the authoritative configuration for an organisation’s AI agents — which agents exist, who owns each one, what each may call, what each may spend, and what happens when a rule is broken — and that makes the configuration binding by sitting in, or being consulted by, the path those agents’ requests take. The name is borrowed from networking, where the control plane decides what should happen and the data plane carries the traffic that does it; the distinction matters here because a product can implement either half and still be sold under the same label.
Inline enforcement
Inline enforcement means the decision to allow, refuse, alter or hold an action is taken in the path the action must travel, before it takes effect, by a component the acting system cannot bypass or overrule. The alternatives — a rule the agent is asked to follow, a check inside the agent’s own framework, an alert raised afterwards — are advisory rather than enforcing, because in each case the party being governed is also the party enforcing.
Fail-closed
Fail-closed describes a control that denies the action it governs whenever it cannot complete its own check — because a dependency is unavailable, a required piece of evidence is missing or stale, a value cannot be evaluated, or the control itself is down. The opposite arrangement, fail-open, permits the action in those same circumstances, which turns every outage in the control into a temporary and silent absence of the control.
Where Token Observe does this
The definition above is the field's, not the product's. This is the part of the product that implements it, for a reader who wants to see one.
Agent registry
One record per agent, and it is the record the gateway enforces against.
An overdue review never suspends the agent itself
Agent permissions
Deny by default, explicit deny wins, and delegation intersects — so an agent cannot borrow authority it was never granted.
Off by default; only enforce refuses anything
Policy engine
One deterministic verdict on every governed request: allow, block, redact, or park it for a human.
Injection scoring is nine fixed patterns, not a model
Audit chain
Every administrative act hash-chained; seal it under a key held off the box, and anchor it with a signature your auditor can check alone.
Unkeyed, a rewrite that re-hashes everything verifies clean
The terms next to this one
What the field is called, what its members have in common, and the one property that separates them from each other.
Definitions are the easy part.
The glossary is written to be useful whether or not you ever buy anything. If you have got to the point of deciding how to implement one of these in your own estate, say what your agents do and you will get a straight answer about what it would actually take.
no form · no qualification step · no sales desk · the other three ways in