Measuring AI agent risk

None of these needs a new tool. They need somebody to look at rows you already have and write down what they found.

The grant set, counted as actions rather than as roles. How many distinct actions can this agent take, and how many of them does its declared purpose actually require? The exercise of comparing the two is the single most productive hour in this whole area, because grants accumulate — an agent gets a role that was convenient at the time, the role gets widened for somebody else, and nobody removes anything. Note also whether the agent could obtain, through a delegation chain, an action it does not hold directly. Where the chain intersects rather than accumulates it cannot, but that is a property worth confirming rather than assuming.

Irreversibility, marked per action rather than per agent. Token Observe supports pinning a tool as requiring an effect contract, which is a durable classification independent of whether a contract is currently active: with the pin set and no active valid contract, raw execution is refused. That flag is useful as a risk input as well as a control, because it forces somebody to make and record the judgement that this particular call has an external effect worth constraining.

Untrusted-text exposure, established from what the agent reads rather than from what it is called. Which of its tools return content written outside your organisation? Does it consume retrieved documents? Does it read tool results at all, or only produce them? A directive inside a tool result is weighted higher than the same words from a user for exactly this reason, and an agent whose entire input is internal and structured is exposed differently from one triaging a public queue.

Money, measured as headroom rather than as spend. What ceilings does this agent have — per request, rolling hour, day, month — and does it have any at all? The number worth putting in front of a finance owner is not the total spend but the count of agents with no ceiling configured, because those are the ones outside the control entirely. Rate limits belong here too, on requests, tool calls and tokens per minute, because they bound the loop rather than the invoice and catch a runaway in minutes rather than at the daily boundary.

Token Observe records a risk tier per agent — minimal, limited or high, defaulting to limited — and it is worth being precise about what that field is and is not.

What it is: a place to record a decision a person made, displayed in the console, carried verbatim into every recertification snapshot so a named reviewer attests it, and sealed by that snapshot’s digest. That makes the tier reviewable rather than merely present, which is most of the value: the failure mode of a risk register is not that the tiers are wrong but that nobody can say when they were last looked at or what configuration they were looking at.

What it is not: a policy-scope dimension. Policy scope matches on agent id, on team case-insensitively, and on tag case-sensitively — not on tier. So a rule intended to bite on your high-risk agents is written against a tag you maintain, and the tag and the tier can drift apart unless somebody keeps them together. That asymmetry in case handling is worth knowing before you name things, because to a kill switch two spellings of a team are the same team, while to a policy two spellings of a tag are two different tags.

There is also no scoring model, no weighting and no computed rating anywhere in the product, and that absence is deliberate rather than a gap waiting to be filled. A number carries an authority its inputs do not support: an agent scored 7.4 invites a conversation about whether it should be 7.1, when the useful conversation is about which two grants to remove. What replaces the score is the recertification record — a named human attesting an exact configuration, with a validity period, so the question becomes when was this last reviewed and by whom rather than what does the dashboard say.

Four surfaces produce figures worth putting in a quarterly review, and each has a caveat that belongs beside it.

A governance report over a window: blocks, redactions and approvals. Read a rising block rate as a question rather than as a success — it can mean a control is working or that a rule is mis-scoped and somebody is about to route around it. Read a zero block rate as a question too.

A budget report: utilisation against ceilings with headroom, aggregated by team and by fleet. The caveat is structural and worth stating: ceilings bind per agent rather than per pool, so team and fleet figures are the sum of the per-agent ceilings that exist, published beside a count of the agents that have none. That count is the number to look at, because an agent with no ceiling is not covered by any of this.

A policy replay against recorded traffic, which is the only way to know what a candidate rule would have done to last month. Three properties decide whether the report means anything. Coverage is full, partial or none, and none returns null counters rather than zeroes — null means the recorded traffic never carried the input this rule triggers on, while zero means it did and the rule would have changed nothing. The denominator is the count evaluated, not the count scanned: requests refused before policy evaluation, by a bad credential, permissions, a kill switch or a delegation denial, are excluded because a rule cannot be credited with stopping a request that never reached it. And the earliest trace actually seen is reported, so a window trimmed by retention is visible rather than implied.

Discovery coverage and findings, read in that order. The findings are the interesting part and the coverage is the part that tells you whether the findings mean anything, which is why the coverage belongs above them on the page rather than in a tab.

One figure to treat with more suspicion than the rest: any retrospective savings estimate. It can model what a cheaper model would have cost at the same token counts and it cannot tell you the cheaper model would have answered acceptably. Quote the high-confidence band, carry the assumptions with the number, and treat it as a ceiling rather than a forecast.