Security and threat model, answered in full.
- Questions in this subject
- 10
- Questions across the whole set
- 49
- Subject, in the reading order
- 3/7
Each answer stands on its own
Every answer here is written to be read without its question, without the paragraph before it and without the rest of the site, because the form it will most often be read in is somebody else's summary. Where there is a limit, it is in the same sentence as the claim rather than in a note underneath it.
What is Token Observe’s threat model?
Token Observe publishes a STRIDE model organised by trust boundary, carrying a residual risk on every row and the role whose dated acceptance is still required. The headline positions: governance fails closed, so missing, unresolvable or erroring state denies rather than allows, and the kill switch is evaluated first and beats everything else. Agent permissions deny by default and delegation intersects at every hop, so a forged chain can only narrow authority, never widen it. The database writer is deliberately inside the model — a new path that evades or weakens the audit MAC epochs, checkpoints, signed anchors or boot verification is expressly in scope for a vulnerability report. Nine risks are listed as needing a named human to accept them, among them heuristic injection detection, regex personal-data detection, the unauthenticated on-behalf-of header, single-writer SQLite and long-lived agent bearer tokens. None counts as accepted until that role records a dated acceptance.
Where do provider API keys live, and what stops Token Observe leaking them?
Token Observe reads provider API keys from the process environment only: they are never written to the database, and never to logs, error responses or traces. A provider row references a key by environment-variable name rather than storing its value, which creates the real risk: an unconstrained registry write would be equivalent to reading every secret in the one process that deliberately concentrates every provider key in the estate. Two allowlists close it and neither may be empty. ACP_ALLOWED_PROVIDER_HOSTS limits which hosts a provider base URL may name; ACP_ALLOWED_KEY_ENV_PREFIXES limits which environment variables a provider may reference, so the session secret and unrelated cloud credentials stay unnameable. Narrowing both to your own endpoints and credential names is the single highest-value hardening step for a deployment whose model endpoints are not the public vendor ones. An upstream provider’s error body is never forwarded downstream, because it can echo prompt content.
How does Token Observe handle prompt injection, and how good is the detection?
Token Observe sanitises unicode first, then scores prompts and tool results heuristically, weighting tool results higher because that is the channel where indirect injection actually arrives. Findings feed an injection policy trigger rather than acting as the only control. The detection is heuristic and has false negatives; that is stated in the threat model and named as a residual risk a CISO has to accept in writing. Two reasons the decision went that way are recorded. Blocking on a classifier with a meaningful false-positive rate would break legitimate traffic on the hot path. And injection findings are one input among several, because deny-by-default permissions, tool scoping and payload-bound approvals bound the damage a successful injection can do. Every policy can run in shadow mode first, so you learn your false-positive rate on your own traffic before you start blocking real work.
What are the limits of Token Observe’s redaction?
Token Observe matches on regular expressions plus checksums, so free-text personal data and non-UK or non-US identifier formats are not detected at all. That limit is published beside the confidence scores so policies can set thresholds rather than trust one verdict: checksum-validated card, IBAN and NHS numbers score 0.9 to 0.98, US social security and UK national insurance numbers 0.85, phone numbers 0.7. Detection runs inline and in-process before the request leaves, in both directions, so a response-side rule fires on an identifier the model produced even though nobody sent one. Secret kinds — JWTs, cloud access keys, prefixed vendor API keys, private-key headers — are always masked irreversibly whatever a policy’s mode says, because a reversible placeholder for a credential is a credential. Token Observe is a compensating control rather than your only DLP, and the licence says the same thing in harder language.
Can an administrator rewrite the audit log?
Against the unkeyed audit chain Token Observe ships by default, yes: an administrator with write access to the database can rewrite it, and a test in the repository proves the forged chain passes verification, so neither half of that claim can drift. Unkeyed hash chaining is the default: it detects an edited row, a deleted middle row and a deleted tail, but someone with write access to the database who recomputes every downstream hash is not detected. Configure ACP_AUDIT_HMAC_KEY and entry digests become MACs sealed at every boot by a checkpoint, so a rewrite needs the key as well as the database — which only helps if the key is injected from a secret manager the database administrator cannot read. Configure an anchor key and the head is Ed25519-signed on a schedule and published off-box, supporting exactly one sentence: any copy of the anchors you kept off-box beats any rewrite made after you took it. The chain is tamper-evident, not tamper-proof.
What does off-box anchoring not prove?
Off-box anchoring — Ed25519-signing the head of Token Observe’s audit chain on a schedule and publishing it outside the database, which stays off until you configure a signing key — does not prove four things, and Token Observe states all four rather than rounding them off. Key theft: whoever holds the signing key can sign anything, so a stolen current key produces anchors that verify. Pre-anchor history: entries written before the first anchor are covered by no anchor, and a checkpoint can only vouch for history from the moment the key was introduced, not for whether that history was already honest. Sink collusion: a sink administered by the same party that runs the database is not independent, because deleting the anchors and the rows becomes one action rather than two — put one somewhere they cannot rewrite. And time: the signing timestamp sits inside the signed bytes so it cannot be edited afterwards, but a signer can write whatever instant it likes at the moment of signing, and only an RFC 3161 timestamp authority or a public ledger proves when. There is no Merkle tree, so partial-log proofs are unavailable.
Has Token Observe had an independent penetration test?
No. Token Observe has had no third-party security assessment, red-team engagement or external code audit, and it holds no SOC 2 or ISO 27001 certification. What has been done is internal: a five-dimension adversarial review with a three-verifier refutation panel per finding, a STRIDE threat model, targeted testing of specific attacks including a forged audit chain and adversarial regular-expression inputs, and automated release gates covering lint, typecheck, the unit and integration suites, an artifact smoke test, a dependency audit, a secret scan over the whole history and an image scan that refuses every high or critical finding. That is a genuine amount of work and it is not the same thing as an external test, so it is not presented as one. A pre-purchase test by your team or your assessor is expressly permitted by the licence: no pre-approval of benchmark results, and findings publishable after coordinated disclosure, provided a benchmark names the version and configuration tested.
How do we report a vulnerability, and what happens next?
Report a vulnerability in Token Observe through a private security advisory on the repository, which gives a CVE path, or to security@tenhaw.com — never a public issue, pull request or discussion. The published figures are targets for a small team rather than a contractual SLA, and where an order form or design-partner agreement states different ones that agreement governs: a human acknowledgement within two business days, a triage verdict within five, and a fix plan within ten; patch targets, measured from the triage verdict rather than the report, are seven calendar days out of band for critical, thirty for high, ninety for medium and the next release for low. Severity starts from CVSS v3.1 and is then adjusted for the context the product runs in, so a flaw that defeats a governance decision or corrupts the evidence record is rated above what its base score alone would suggest. Good-faith research gets safe harbour under the Computer Misuse Act 1990 and equivalent laws, with a default ninety-day disclosure deadline from the triage verdict. There is no bug bounty, which is a resourcing statement rather than a judgement of the work.
Does the vendor see our prompts?
No. Token Observe is self-hosted and bring-your-own-key: there is no telemetry, no phone-home, no licence callback and no vendor-operated component in the path, and the licence carries that as a contractual undertaking rather than only a claim. Governed payloads leave your network only for the model and tool providers you configure, after policy and redaction. You do not have to take it on trust, and the check takes about five minutes: list every outbound call site and every hard-coded URL in the server and core packages, confirm each destination comes from operator-supplied configuration bar the admin-invoked price catalogue, check the dependency list, then run the process with egress allowed only to your providers and watch nothing break. Two qualifications belong beside that. Anything you voluntarily send in a support ticket is governed by your support agreement, not by the architecture. And whether the vendor is legally never a processor is counsel’s analysis, not an engineering fact.
Who inside our organisation can read prompts and traces?
Token Observe gates every human read of prompts and traces on four control-plane roles — admin, operator, auditor, viewer — plus a separate per-user team scope, and the two are deliberately independent: a viewer scoped to Finance must not gain Engineering’s evidence merely because both are viewers. The scope predicate is derived from the signed-in user and applied inside the store queries on detail, search, export, compliance, spend, approvals and seat paths, so a caller cannot widen it with a query parameter, and demoting someone without an explicit scope drops any inherited wildcard to nothing rather than preserving full-corpus access. Estate-wide surfaces — the audit chain, the anchors, retention controls, the shadow-AI radar — return 403 to a team-scoped user rather than showing a misleading slice. Sensitive trace reads append attributable audit events, because reading one person’s prompt corpus is itself an act worth recording. The gap to know: local accounts have no MFA unless you put your identity provider in front of them.
If the answer above is close but not quite the case you have, the specific version of it is a better question than the general one, and it gets a specific reply.
Ask the specific version49 questions, in 7 subjects
What Token Observe is
The category, what the product does, what it deliberately is not, who it is for, and what stage it is actually at.
8 questionsHow it works
The request path step by step: how an agent is onboarded, what happens inline, what streaming and failover do, and what happens when something is unavailable.
7 questionsCompliance and evidence
EU AI Act, ISO/IEC 42001, NIST AI RMF and OWASP mappings, what an evidence export proves, retention and erasure — and which certifications do not exist.
6 questionsDeployment and operations
What it runs on, how long onboarding takes, backup and restore, upgrades and rollback, air-gapped installs, and why there is no high-availability topology.
7 questionsLicence, pricing and support
The source-available licence, how the product is metered, the thirty-day evaluation, the support model, and what procurement has to accept in writing.
6 questionsAlternatives and adjacent tools
Where Token Observe sits against gateways, observability, security platforms and identity products — and the case for and against building it yourself.
5 questionsBring the question this page did not answer.
Write to hello@tenhaw.com with what your agents do, which providers they call and what would have to be true for you to put something in front of them. James Rooney replies. You will get a straight answer about whether Token Observe fits, including when it does not.
no form · no qualification step · no sales desk · the other three ways in