Stop one agent, a whole team, or everything, now

The first instinct is to revoke the credential, and it works — on the next authentication attempt, for the calls that use that credential, in the systems where you can find it. What it does not do is answer quickly enough when the question is which credential, because an agent that has been running for six months may hold one key at the gateway and a second one directly against a vendor, and revoking the first tells you nothing about the second. Revocation is also indiscriminate in the wrong direction: it stops everything that agent does, which is right in an incident and wrong when what you needed was to stop one tool.

The second is to redeploy with the agent disabled, which is the honest answer in most estates and takes as long as a deploy takes. That is minutes at best, and it depends on the pipeline being healthy at the moment you need it least. It also requires the team that owns the agent to be awake, which is usually the actual bottleneck: the person who can stop the agent and the person who noticed the problem are different people in different time zones.

The third is to suspend the agent in a console and assume that covered it. That is closer, and the gap is specific and was found in this product rather than theorised about: token counting and the model-catalogue routes deliberately skip the trace-opening path, because they execute nothing and opening a trace for a size check would pollute the evidence record — and in skipping it they also skipped the agent-status and kill-switch checks, so a frozen agent could still price prompts and enumerate models. Those checks now run on all three routes. An emergency stop an agent can still work around is not a stop.

The fourth is the one nobody plans for, which is what a stop means for a developer subscription running on a laptop. There is no network path to intercept, the vendor’s credential is not yours, and the enforcement point is the vendor’s own administrator hook deciding locally against a signed bundle. A stop there has to reach the artefact the device already holds, which is why the switch predicate is one exported function rather than two implementations that could differ by one character.

Scope is deliberately three values rather than four. A global switch reaches every governed subject in the estate, agents and seats alike. A team switch compares team names case-insensitively, on the reasoning that a team name is being typed by a human under incident pressure and a capital letter should not be the difference between a fleet stopping and not. An agent switch matches the subject id exactly, and because a subscription seat is a governed subject with its own prefixed id, the same scope value stops a seat — a fourth scope would have to be threaded through the store, the API and the console to make a distinction the predicate already makes from the id.

That asymmetry is worth knowing before you name things, because policy scope does not share it. Kill switches and policy scope both match on team case-insensitively, and both match on tags case-sensitively: Finance and finance are the same team to a kill switch, while pci and PCI are two different tags to a policy.

The predicate that answers whether a switch reaches a subject is written once and exported, and that is load-bearing rather than tidy. The same function the gateway stops a request with is the filter that decides which switches are compiled into an endpoint seat’s signed policy bundle. A second copy that was stricter by one character would omit an engaged switch from the artefact, and the big red button would be pressed in the console and reach nothing at all on the device.

A released switch reaches nobody, immediately, on the next request. There is no propagation step on the gateway path because the resolver loads the agent, its roles, the switches currently in force and its recent spend window live from the store on every request. The one deliberate exception is the seat bundle, which is a snapshot by design and is bounded by a freshness window rather than read live — a hook that phoned home would convert every outage, slow network and DNS failure into a silent policy bypass, because every vendor fails open when a hook times out.

A kill switch is the blunt instrument, and reaching for it when something narrower would do is how an incident becomes an outage. Underneath it sit four controls that stop less and are audited just as well, and knowing which one to reach for is most of the skill in operating this.

The narrowest is a policy. A rule can block one action for one agent rather than stopping the agent, and its suspend_agent action does both — it blocks the request and takes the agent out of service, writing the status change into the audit log. If that status write fails the block still stands, because losing the suspension is serious and is not a reason to let the request through. A policy is also the only one of these you can rehearse: run it in shadow mode and read what it would have stopped before it stops anything.

Next is the lifecycle state. Suspension flips the status and the evaluator refuses that agent on its lifecycle check, taken at the single decision point rather than at the door, so the attempt is still recorded against the trace opened earlier instead of vanishing. Suspending and retiring both free licensed capacity and neither is ever refused on licence grounds — a limit that blocked its own remedy would be an outage wearing a licence’s clothes.

Then the credential, which is the right tool when the problem is who is holding the key rather than what the agent is doing. And finally the budget, which is the slowest brake and the only one that acts without a person: a per-request, hourly, daily or monthly ceiling refuses the call before egress once the money crosses the line. It is worth having configured before an incident precisely because it does not need somebody to notice.