Threats and defences

Shadow AI

Shadow AI is any use of AI models, assistants or agents inside an organisation that does not pass through the controls the organisation built for them — a service calling a provider API directly with a key issued from the vendor console, a coding assistant left pointed at its default endpoint, an agent nobody registered, or a model whose usage no internal system meters. It is a coverage problem before it is a security problem: it is what makes the denominator of every governance claim unknown.

also called ungoverned AI · unsanctioned AI use · BYOAI · shadow LLM usage

The version of shadow AI that gets discussed is an employee pasting a customer record into a consumer chatbot. That is real, and mostly an acceptable-use question with well-understood answers. The version that breaks governance is quieter and lives inside your own systems. It arrives in four recognisable shapes: a team that stood up an agent with a provider key from the vendor console because that was faster than requesting a gateway credential; a service account created for a proof of concept two quarters ago and still in production because the proof of concept worked; a coding assistant on a developer laptop pointed at the vendor default, which is what every one of those tools does out of the box; and, strangest of the four, a governed agent calling a model your price table does not know, so the traffic passes through your controls metered at zero while the vendor bills for it in full.

The reason to care is arithmetic rather than tidiness. An estate where forty agents route through a gateway and an unknown number do not is not a governed estate; it is one governed by an unmeasured fraction, and every statement you make about policy coverage, spend or audit completeness carries that unknown inside it. Discovery findings belong in a risk register for that reason, not in a backlog. It also inverts the usual severity ordering: an ungoverned agent is not merely unmonitored, it is the agent with no permission model, no approval gate, no redaction on egress and no injection defence, so it is simultaneously the least visible and the most exposed thing in the estate.

Discovery is reconciliation rather than detection: independent evidence, compared against what your controls actually saw. Five sources, ordered by signal. The vendor’s master bill is ground truth for what was spent and the only source that can find usage which left no network, identity or endpoint trace at all — reconcile it against what you metered, with a tolerance, since rounding and mid-month proration make small gaps meaningless. Network egress logs are next: a workstation or service talking straight to a model API is bypassing the gateway by construction. Then the provider’s own listing of its API keys, because long-lived unattributed keys are the mechanism that makes bypass possible. Then endpoint telemetry about coding assistants, because an unset base URL is less a misconfiguration than the default state of the world. And last your own tables, which need no export and are authoritative for the estate you do control.

One rule decides whether any of this is trustworthy: a dead feed must never be indistinguishable from a clean estate. An export that stopped arriving in July and an estate where nobody is calling a model directly both render as an empty findings list, which a console renders as a green all-clear. So coverage has to travel with every surface that can report a clean result, and it needs two facts per source rather than one — a delivery proves the connector is alive, and a delivery carrying rows proves the estate was observed. Collapsing those produces the commonest real failure: an exporter delivering an empty page every hour holds its source at 'fresh' indefinitely, which is worse than no feed at all because coverage is now affirmatively asserting freshness over nothing. Token Observe records five coverage states for this reason — fresh, empty, feed failing, stale and never connected — of which exactly one entitles a surface to an unqualified all-clear.

Two limits are structural and should be stated whenever the word 'discovery' is used. A finding is only as current as the export it was computed from, so between deliveries the picture is the estate as it was rather than as it is, and nothing inside the discovery system can tell the difference. And the evidence itself is sensitive: reconciliation data holds workstation hostnames, staff usernames and service-account identifiers — personal data about named employees, typically retained outside whatever erasure rules cover ordinary traffic records. A programme that produces an inventory of who is using what has produced a monitoring dataset, and it needs a retention answer and an access scope of its own before its first report.

in practice

An all-clear that means nothing

A monthly report shows zero shadow-AI findings across the estate. Read alongside coverage, it says something different: billing was reconciled nine days ago and matched within tolerance; the provider key listing was exported last week; endpoint telemetry arrived this morning; and the network egress feed has delivered nothing since 14 July, because a credential rotation broke the exporter and the failure was logged where nobody looks. Egress is the only source that catches an open channel while it is still open. The report is not evidence that nobody is calling a model API directly — it is evidence that for seven weeks nothing has been looking.

not the same as

What shadow ai is routinely confused with

Shadow IT
Shadow AI is a subset, and two things make it behave differently. It is metered and billed per unit of use, so the finance system is a discovery channel that shadow IT rarely offers — an unexplained line on a provider bill is proof of usage that left no other trace. And what is unsanctioned here acts rather than merely stores: an unregistered SaaS tool holds data, while an unregistered agent takes actions in systems that trust it.
Data leakage to AI vendors
Leakage is about what a payload contains and where it goes; shadow AI is about whether the payload passed a control point at all. A perfectly governed prompt containing a customer record is a redaction and routing question. The same prompt sent from an unregistered service is invisible to every redaction and routing rule you own, which is why coverage is the prior question.
Model inventory
An inventory is the list you maintain; shadow AI is everything absent from it. The distinction matters because a registry that nothing enforces against will always look complete: it can only ever contain what somebody remembered to add, so its completeness has to be measured against outside evidence rather than asserted from within.
next

Related terms

Data exfiltration

Data exfiltration in an AI system is the movement of sensitive data out of the boundary that held it by way of the model’s own context — carried in a prompt sent to a provider, in a link or image the receiving client renders automatically, in the arguments of a tool call, or in a response the model was persuaded to produce. Its defining property is that the channel is usually legitimate: no malware runs and no unusual connection is made, because the agent is doing exactly what it was built to do and the data is riding along.

PII redaction

PII redaction is the removal or replacement of personal and sensitive data in a payload before it crosses a boundary — in an AI system, typically before a prompt reaches a model provider, before a response reaches a user, and before either is written to a log or a trace. It comes in two forms with different consequences: masking, which destroys the value irreversibly, and tokenising, which replaces it with a stable placeholder so the text still makes sense to whatever reads it next.

Indirect prompt injection

Indirect prompt injection is prompt injection delivered through content an AI system retrieves rather than through anything its user typed — a web page, a document, an email, a support ticket, a database row, a code comment or a tool’s response — so the attacker never needs an account, a session or any access to the application itself, only write access to something the agent will read. It is the form of injection that hijacks autonomous agents, because the attacker’s text arrives inside a request that is otherwise entirely legitimate.

Threats and defences

The terms next to this one

How agents actually get hijacked, and what the available defences are worth. Every entry here states its own false-negative rate honestly.

get in touch

Definitions are the easy part.

The glossary is written to be useful whether or not you ever buy anything. If you have got to the point of deciding how to implement one of these in your own estate, say what your agents do and you will get a straight answer about what it would actually take.

no form · no qualification step · no sales desk · the other three ways in