Token Observe vs AI security platforms
Token Observe is not a complete AI security suite, and the product’s own strategy document forbids selling it as one.
Palo Alto Prisma AIRS, Cisco AI Defense, Zenity, Noma, WitnessAI, Lakera. Every claim about them on this page is that vendor's own published material, untested by us.
the other comparisonsOn this page
On detection, coverage and the parts of the estate that never touch a gateway
Detection quality is these vendors’ product and it is not Token Observe’s. Every description in this paragraph is the vendor’s own published material, quoted rather than tested. Palo Alto publishes Prisma AIRS as a runtime firewall and API with discovery and posture, model and skill scanning, red teaming, MCP inspection and DLP. Cisco publishes AI Defense as AI asset and MCP discovery, model, repository and MCP supply-chain scanning, red teaming and runtime protection. Noma publishes automatic discovery, a live registry, identity and tool policies, runtime blocking and behaviour-chain detection. Zenity, WitnessAI and Lakera each publish their own material on discovery, prompt-injection defence and intent-aware enforcement; read it from them rather than from here, and check what each product actually covers today. Against a nine-pattern heuristic, a maintained detection product ought to win, and the product’s own documentation concedes the point rather than arguing it: heuristic detection has false negatives, that residual risk requires a named CISO’s written acceptance, and injection findings are treated as one input to policy rather than as the control.
Coverage is the second and larger advantage. Token Observe sees what routes through the gateway and nothing else. A security platform on the network path, in the browser, or on an unmanaged device sees traffic that never presents a Token Observe credential — which is most of the shadow AI in a real organisation. Token Observe’s own answer to that is a radar that reconciles evidence you feed it, and its support boundary says the rest plainly: findings tell you those agents exist, and chasing them down inside your organisation is your work. Token Observe’s endpoint hook for managed developer subscriptions is explicitly preview, reports itself as not production-eligible, and should not be bought as an equivalent to an inline network control on an unmanaged device.
There is a procurement advantage too, and it is worth stating because it decides real deals. Established security vendors are generally able to hand a buyer an audit report and a test result when procurement asks — ask each one which certifications it holds, for which scope and to which date, because that is the only version of the answer worth having and this page does not hold it for them. Token Observe holds none: no SOC 2, no ISO 27001, no ISO 42001, no independent penetration-test result. It says so first rather than under questioning, and if that is your gate then the comparison ends here in the other product’s favour. All the vendor descriptions above are taken from public, vendor-authored material and have not been independently tested; the product’s own benchmark carries that caveat verbatim, and adds that the named list is a comparison with the most relevant category leaders rather than a claim that the category contains only these vendors.
Token Observe and AI security platforms, row by row
One card per dimension rather than a three-column table, because the two sides are rarely the same length and a table of them is a horizontal scroller on a phone.
Detection is the product: trained classifiers, red teaming, model and MCP supply-chain scanning.
Detection is one policy input among seven trigger kinds, published with its confidence scores so a policy can set its own threshold.
Runtime filtering of the traffic that matched.
Deny-by-default action-level permissions, an approval bound to the exact payload, a hard USD ceiling and a kill switch — the damage bounded rather than the string caught.
The whole AI estate, including browsers, unmanaged devices and agents no gateway sees.
Only what presents a credential to the gateway. Everything else is a radar finding, and chasing it is your work.
You grant the platform access to inspect.
You configure an outbound feed in a console you already administer; Token Observe holds no credential into your security stack.
Alerts, posture and incident response.
Evidence: a hash-chained audit log, an off-box Ed25519 anchor, an export carrying the chain verdict and a named approver against each decision.
Ask each vendor which certifications and independent test results it holds, for what scope and to what date.
None held. A source-available licence — itself a template pending counsel, not an executed grant — a published residual-risk register, a published defect list, and a 30-day evaluation written so your security team may attack it before you buy.
The detection limits, stated before the capability
Prompt-injection scoring is nine weighted heuristics over sanitised text, with the score raised by a factor of 1.25 when the text arrived as a tool result — because indirect injection arrives through tool output far more often than through the user turn, and the product treats tool results as the channel that actually gets agents hijacked. It is not a model, it has false negatives, and the residual risk is recorded as one requiring the CISO’s dated written acceptance. The reason it does not gate traffic on a classifier is recorded too: blocking on a classifier with a meaningful false-positive rate would break legitimate traffic on the hot path.
Sensitive-data detection is patterns too, and the same caveats apply to it. Eleven classes, three of them checksum-validated — Luhn for cards, mod-97 for IBANs, mod-11 for NHS numbers — and the remaining eight are pattern matches with no arithmetic behind them, which is why they score lower. The confidence scores are published per kind rather than averaged into a marketing number: 0.90 to 0.98 for the checksum-validated kinds, 0.99 for a PEM private key, 0.95 for a JWT, an AWS access key or a prefixed API key, 0.90 for an email address, 0.85 for a US Social Security number or a UK National Insurance number, and 0.70 for a phone number. Free-text personal data — a name, an address, a medical detail written in a sentence — is not detected at all. Those scores are exposed so a policy can set its own threshold rather than inherit one. The Data Protection Officer is the named acceptor of that residual risk, and the product’s own phrasing is that it is a compensating control and not the customer’s only DLP.
Where the engineering effort actually went is the egress boundary rather than the classifier. A streamed response passes through a hold-back buffer with a floor of 64 characters and a separate buffer per tool-call argument channel, and the cut is pulled back off anything it would split — off any match that straddles it, and out of the middle of an unbroken run of value characters. Both rules are needed, and the reason is instructive: 64 characters exceeds every kind that states a maximum length, but a JWT states none and matches nothing at all until its third segment arrives, so a fixed window alone once shipped the head of a 256-character token and then reported that it had masked nothing.
What Token Observe consumes from the security stack you already run
The shadow-AI radar is built around reconciliation rather than inspection, and four of its five evidence sources come from outside. Receivers exist for Cloudflare Logpush from a Zero Trust Gateway, Palo Alto Strata Logging Service, Zscaler Internet Access web logs, Netskope transaction events, Elastic through Filebeat, Logstash or a watcher, and Splunk through a forwarder or a webhook alert action, plus a generic shape that reads Token Observe’s own field names for any tool that can export on a timer. That is six vendor-shaped mappings and the generic one, and a mapping exists only for the streams a vendor genuinely emits — Cloudflare publishes no invoice, so there is no Cloudflare billing mapping to be had. Where the vendor field is omitted it is inferred from fields only one product emits, refusing rather than guessing, because the wrong mapping produces rows that parse, land, and describe flows that never happened.
The security-review argument for that shape is the one worth putting in front of a CISO. Token Observe holds no credential into your security stack. You configure an outbound feed in a console you already administer and it delivers on a credential Token Observe issued, so the connection points inwards. That is both the shorter security review and the smaller blast radius: the worst a compromised Token Observe can do to your SIEM is stop receiving from it.
The other half of the design is the refusal to report silence as safety. Every delivery is bounded — 10,000 records and 8 MB, refused whole rather than truncated, with rate ceilings applied before the body is read — and a batch may lose up to 5 per cent of its records to unmappable rows, counted and reported, before the whole delivery is refused, because half not mapping is what a wrong mapping looks like rather than a dirty feed. Coverage travels with every surface that can report a clean result, so no caller can read zero findings from the API without also being handed which sources were silent when it said so. A dead feed must never be indistinguishable from a clean estate.
- Five radar evidence sources
- Vendor bill reconciliation, network egress analysis, service-account key audit, IDE and CLI telemetry, and Token Observe’s own caller and price consistency checks over its own tables.
- Findings do not clear themselves
- A source may only clear a finding when its run actually completed, so a failed, timed-out or truncated pass leaves every existing finding standing rather than silently resolving the estate.
- Credential probing at the door
- Every authentication rejection is folded into an hourly roll-up the radar reads, recording the reason and, where it resolved, the key id — never the token and never a digest of it, because a digest of a live secret is an offline oracle against that secret.
What actually bounds an injection that gets through
Assume the detection missed it, because sometimes it will. What remains is the deterministic layer, and this is the part Token Observe is actually arguing for. Permissions are action-level and deny-by-default: an action no role names is refused, an explicit deny beats every allow wherever it is written, and a delegation chain intersects rather than unions, so a low-privileged agent gains nothing by asking a higher-privileged one to act for it. A hijacked agent inherits the authority it already had, and nothing more.
Above that sit the gates that do not depend on recognising anything. A policy can require a human approval, and that approval is bound to the SHA-256 of the canonical action plus the execution context it was proposed in, is single-use through a compare-and-set, and expires — 60 minutes by default, one minute to seven days by policy — so a retry with one argument changed is refused as a mismatch rather than allowed as near enough. Hard USD ceilings per request, per rolling hour, per UTC day and per UTC month are reserved before egress in one per-agent transaction. A kill switch scoped to one agent, one team or everything is checked first in the pipeline. Tool descriptors are hashed at approval and drift quarantines the tool, which is the answer to a rug-pull rather than to a prompt.
The honest boundary on all of it is that Token Observe can only refuse a proposal it is shown. Tool calls the model proposes are evaluated against policy on the way back, which is why a rule about refunds over a threshold binds even when the agent executes the tool itself — but an agent that never routes any of its traffic through the gateway is caught by the radar, if you have fed the radar, or not at all.
Which of the two you should actually put in.
Both columns are real answers and both are the same length on the page. Read the left one first: if it describes your estate, it is the cheaper decision and this page has done its job.
When to choose AI security platforms
- You need coverage of the whole AI estate — browsers, unmanaged devices, SaaS copilots — rather than of the agents that hold a credential you issued.
- Detection quality is the requirement: model, repository and MCP supply-chain scanning, red teaming, or a maintained injection classifier rather than nine published heuristics.
- Your procurement process requires SOC 2, ISO certification or an independent penetration test from the vendor. Token Observe has none of those and will tell you so on the first call.
- You are consolidating security vendors, and a second contract for a control that overlaps at the edges is a harder sell than a wider deployment of one you already have.
When to choose Token Observe
- You have one of these platforms already and the gap you have found is the action rather than the content — what the agent was allowed to do, who approved it, and whether it actually happened.
- The refusal has to be deterministic and explainable: a named permission, a named policy, a named approver, not a score you have to defend to an auditor.
- You want your existing security stack’s verdicts to become policy inputs without granting a new vendor a credential into that stack.
- Self-hosted with zero vendor egress is a hard requirement, including air-gapped environments, which the licence is drafted to permit — though that licence is still a template pending counsel rather than an executed grant.
If the left-hand column is the one that describes you, that is still worth an email: a straight answer about which of these to buy costs both of us less than an evaluation that ends in the same place.
Ask which one fitsThe 7 named products in this category
This page argues about the category. If you have already shortlisted a product, these go head to head with it — every claim about the other product paraphrased from that vendor's own published material on a stated date, and none of it independently tested.
Keycard
Keycard decides whether the agent gets a credential. Token Observe decides the call and then proves what the call actually did.
Palo Alto Prisma AIRS
Prisma AIRS decides whether the content is malicious. Token Observe decides whether the agent that sent it was allowed to.
Cisco AI Defense
AI Defense attaches a policy to an application’s connection and inspects what crosses it. Token Observe attaches permissions to an agent and decides what it may do.
Zenity
Zenity gets into the path the agents are already on. Token Observe is the path the agents are pointed at.
Noma Security
Noma decides from what the agent appears to be doing. Token Observe decides from what the agent was allowed to do, before anything is scored.
Lakera
Lakera tells your application the content is an attack. Token Observe is the thing that refuses to send it.
WitnessAI
WitnessAI stands in front of the interaction and classifies the intent. Token Observe stands in front of the API call and decides the action, its cost and its evidence.
The other comparisons
Same template, same order, same concession first. Claims about every named product on all of them are that vendor's own and have not been independently tested.
Token Observe vs LLM gateways
Token Observe is a gateway in delivery. The gateway is how it arrives, not what it is for.
Token Observe vs LLM observability
One refuses the call inline. The other scores it afterwards. Most estates need both, and they are not substitutes.
Token Observe vs cloud-native controls
If every agent, model and tool lives in one cloud, use that cloud’s controls. The argument here is for the estate that does not.
Token Observe vs building it yourself
For a small estate, a few hundred lines of proxy is usually the right answer. The cost arrives later, and it arrives in specific places.
Token Observe vs doing nothing
With three agents, no regulated data and no incident, doing nothing is often the correct decision. This page is about what changes it.
Is Token Observe an AI firewall?
No, and the product’s own roadmap names a generic AI firewall, prompt scanner or red-team platform as one of seven strategic non-goals. The stated consequence for the security-platform category is to consume threat and identity signals from those systems rather than lead with a complete AI security platform, generic runtime filtering or another red-team dashboard. Token Observe does inline PII and secret detection and injection heuristics, because a decision point that cannot see the payload cannot decide about it, but those are inputs to a policy rather than the product.
Your injection detection is regex. Why should we take it seriously?
Take it as what it is described as. It is nine weighted patterns over Unicode-sanitised text, scored 1.25 times higher when the text is a tool result, it has false negatives, and the residual risk is on the register with the CISO named as the required acceptor. Two reasons are given for not doing more: blocking on a classifier with a meaningful false-positive rate would break legitimate traffic on the hot path, and injection findings are one input to policy rather than the only control — action-level permissions, tool scoping, payload-bound approvals and hard budgets bound what a successful injection can do. If detection quality is your requirement, buy it from a vendor whose product it is.
How do we feed our security tooling into Token Observe?
You configure an outbound feed in a console you already administer, pointing at a receiver on a credential Token Observe issued. Vendor-shaped receivers exist for six: Cloudflare Logpush, Palo Alto Strata Logging Service, Zscaler, Netskope, Elastic and Splunk, plus a generic shape reading Token Observe’s own field names for anything that can export on a timer. Mappings exist only where a vendor genuinely publishes that stream, and an unrecognised shape is refused rather than guessed at. The direction is the point: Token Observe holds no credential into your security stack, so the worst a compromised deployment can do to your SIEM is stop receiving from it. Every delivery is bounded and every refusal is visible on a connectors surface, because a connector delivering into a void must be reported as never accepted rather than mistaken for a source nobody set up.
What about agents that never route through the gateway?
Token Observe does not see them, and the documentation says so in several places rather than one. The compliance mapping lists anything about agents that never route through the gateway under what the product does not evidence; the support boundary excludes them explicitly and says that chasing them down inside your organisation is your work; and the reconciliation model states that observation stays bounded by the telemetry and inventory you supply. What exists is the radar, which reconciles bills, egress, service-account keys, IDE and CLI telemetry and Token Observe’s own caller and price consistency into findings that belong in a risk register. Absence of evidence is not evidence of absence.
Have you tested Prisma AIRS, Cisco AI Defense or Lakera against this?
No. Every strength attributed to those products on this page is taken from their vendors’ own public material, has not been independently tested, and the product’s internal benchmark states that caveat about itself in the same terms. There is no witnessed bake-off, and one is named in the product’s own launch gates as evidence that does not yet exist. Nor should the absence of a capability from this page be read as its absence from a vendor’s product: these are fast-moving products, and the accurate account of any of them is the one you get from the vendor. What is available instead is a 30-day evaluation written so that a prospective customer’s security team can read, run and attack the software before a purchase order is raised, with no gag clause and no pre-approval of results — under a licence that is still a template pending review by counsel in England and Wales rather than an executed grant of rights.
Tell us which way you are leaning, and why.
Write to hello@tenhaw.com with what your agents do, which providers they call and what would have to be true for you to put something in front of them. James Rooney replies. You will get a straight answer about whether Token Observe fits, including when it does not.
no form · no qualification step · no sales desk · the other three ways in