Shadow AI radar
Five evidence sources for AI activity that never touched the gateway, and a coverage model that refuses to call a dead feed a clean estate.
Nothing found and nothing looked at are the same empty list.
Any tool produces a list of findings. What matters is the sentence underneath it when the list is empty. A console that renders no findings as a green banner has made a claim about your estate, and usually has no idea whether it is entitled to. An export that stopped in July looks exactly like a clean estate.
Freshness alone is always late. A master bill is monthly, so a credential revoked this morning leaves its source reading fresh until September, while the connector’s own verdict is available today. The later of two true answers is the same lie, slower.
How it works
- 01
Your systems deliver; nothing here reaches into them
Your exporter posts evidence rows on a credential Token Observe issued and you can revoke, scoped to those streams and nothing else. Two scheduled pull connectors are the exception (GitHub Copilot, Cisco Umbrella), each on a narrow read-scoped credential you supply. - 02
Two clocks per stream, never one
One clock for when a delivery last arrived, empty or not; one for when a delivery carrying a row last arrived. The first is liveness, the second freshness. An exporter delivering an empty page hourly once held its source at fresh indefinitely. - 03
Clearing a finding is harder than raising one
A finding closes by absence only when the run completed, the source is one where absence is evidence, the snapshot was whole and the whole population was seen. A wrongly resolved finding is worse than a stale one: still true, now filed as handled.
Sample data in the real Token Observe interface, recorded from a local evaluation with a temporary database, a mock provider and one request. No live provider key was used.
A dead feed is not a clean estate.
Coverage records two facts per source (when evidence last arrived, and how long it may plausibly go quiet) and computes one of five states. Only the first permits an unqualified all-clear.
It turns on the difference between a delivery and a row: one proves the connector is alive, the other proves the estate was observed. Connector health can take a verdict away and never hand one back.
- connected_fresh
- Evidence arrived inside the bound. The only state a clean result can stand on.
- connected_empty
- Deliveries arriving, none carrying a row. Fresh is the lie; stale sends somebody to fix a working connector.
- feed_failing
- Evidence inside its bound, and the thing delivering it has stopped. Freshness alone would call this covered for another forty-five days.
- connected_stale
- Evidence arrived before, and not recently enough. The dangerous state: everything still looks configured.
- never_connected
- Nothing has ever arrived. A clean result from this source is not a result.
stream bound last delivery last row state
egress 6 hours 4 min ago 4 min ago connected_fresh
egress 6 hours 9 min ago 31 hours ago connected_empty
egress 6 hours 3 days ago 3 days ago connected_stale
billing 45 days 6 hours ago 6 hours ago feed_failing *
service_accounts 10 days never never never_connected
* inside its bound, but its ingest credential was revoked today.Five sources, and what each one can prove.
The vendor bill runs first: ground truth for spend, and the only source that finds usage leaving no network, IAM or endpoint trace. Egress matches a fixed hostname list. The key audit reports keys past rotation, never used, or idle. IDE telemetry recognises six coding agents and flags any pointed anywhere but this gateway. The fifth reads the gateway’s own tables, so it cannot go stale between exports.
billing gap > $1 AND > 5% of the larger side
seats largest single month: $50 / $250 / $1,000
egress 1 MiB / 50 MiB / 500 MiB, or 100 / 1,000 connections
service keys > 90d medium, > 365d high, > 60d idle low
ide vendor default or a vendor host: high
self unpriced model: 50k / 1M / 10M tokens; cache hits
excluded by requiring input + output tokens > 0Each line closes off an assumption a reader would otherwise carry into a proof of concept.
What this does not doWhat this leans on
Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
Agent registry
One record per agent, and it is the record the gateway enforces against.
Endpoint seats
Policy enforced inside each vendor’s own administrator hook, decided offline against a signed bundle, because a hook that phones home fails open.
Where this comes up
Jobs that draw on it
How do you tell a dead feed from a clean estate?
By recording two facts per source and reporting both beside every clean result: when a delivery last arrived, which proves the connector is alive, and when a delivery carrying a row last arrived, which is what freshness is computed from. Those give five states, and only connected_fresh permits an all-clear.
Does Token Observe need access to our finance system or our firewall?
No. Your own exporter pushes on a credential Token Observe issued and scoped to those evidence streams: it cannot read findings, run a scan or change a policy. Two optional pull connectors are the exception and do hold a vendor credential you supply (Cisco Umbrella, GitHub Copilot), both off until you configure them.
Can a finding be closed automatically when the problem is fixed?
Only for the source that reads the gateway’s own tables, and only when the run completed, the snapshot was not truncated and the whole population was seen. The four operator-fed sources never clear by absence: an export omitting last month’s shadow key has not remediated it.
Prefer to ask a person? Write to James
Bring us the agent you are least comfortable with.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.