Security and threat model, answered in full.
- Questions in this subject
- 10
- Questions across the whole set
- 54
- Subject, in the reading order
- 4/7
Each answer stands on its own
Open a question for the answer, its limits and links to the detail.
What is Token Observe’s threat model?
Token Observe publishes a STRIDE model organised by trust boundary, carrying a residual risk on every row and the role whose dated acceptance is still required. The headline positions: governance fails closed, so missing, unresolvable or erroring state denies rather than allows, and the kill switch is evaluated first and beats everything else. Agent permissions deny by default and delegation intersects at every hop, so a forged chain can only narrow authority, never widen it. The database writer is deliberately inside the model: a new path that evades or weakens the audit MAC epochs, checkpoints, signed anchors or boot verification is expressly in scope for a vulnerability report. Nine risks are listed as needing a named human to accept them, among them heuristic injection detection, regex personal-data detection, the unauthenticated on-behalf-of header, single-writer SQLite and long-lived agent bearer tokens. None counts as accepted until that role records a dated acceptance.
Where do provider API keys live, and what stops Token Observe leaking them?
Token Observe reads provider API keys from the process environment only: they are never written to the database, and never to logs, error responses or traces. A provider row references a key by environment-variable name rather than storing its value, which creates the real risk: an unconstrained registry write would be equivalent to reading every secret in the one process that deliberately concentrates every provider key in the estate. Two allowlists close it and neither may be empty. ACP_ALLOWED_PROVIDER_HOSTS limits which hosts a provider base URL may name; ACP_ALLOWED_KEY_ENV_PREFIXES limits which environment variables a provider may reference, so the session secret and unrelated cloud credentials stay unnameable. Narrowing both to your own endpoints and credential names is the single highest-value hardening step for a deployment whose model endpoints are not the public vendor ones. An upstream provider’s error body is never forwarded downstream, because it can echo prompt content.
How does Token Observe handle prompt injection, and how good is the detection?
Token Observe sanitises unicode first, then scores prompts and tool results heuristically, weighting tool results higher because that is the channel where indirect injection arrives. Findings feed an injection policy trigger rather than acting as the only control. The detection is heuristic and has false negatives; that is stated in the threat model and named as a residual risk a CISO has to accept in writing. Two reasons the decision went that way are recorded. Blocking on a classifier with a meaningful false-positive rate would break legitimate traffic on the hot path. And injection findings are one input among several, because deny-by-default permissions, tool scoping and payload-bound approvals bound the damage a successful injection can do. Every policy can run in shadow mode first, so you learn your false-positive rate on your own traffic before you start blocking real work.
What are the limits of Token Observe’s redaction?
Token Observe matches on regular expressions plus checksums, so free-text personal data and non-UK or non-US identifier formats are not detected at all. That limit is published beside the confidence scores so policies can set thresholds rather than trust one verdict: checksum-validated card, IBAN and NHS numbers score 0.9 to 0.98, US social security and UK national insurance numbers 0.85, phone numbers 0.7. Detection runs inline and in-process before the request leaves, in both directions, so a response-side rule fires on an identifier the model produced even though nobody sent one. Secret kinds (JWTs, cloud access keys, prefixed vendor API keys, private-key headers) are always masked irreversibly whatever a policy’s mode says, because a reversible placeholder for a credential is a credential. Token Observe is a compensating control rather than your only DLP, and the licence says the same thing in harder language.
Can an administrator rewrite the audit log?
Against the unkeyed audit chain Token Observe ships by default, yes: an administrator with write access to the database can rewrite it, and a test in the repository proves the forged chain passes verification, so neither half of that claim can drift. Unkeyed hash chaining is the default: it detects an edited row, a deleted middle row and a deleted tail, but someone with write access to the database who recomputes every downstream hash is not detected. Configure ACP_AUDIT_HMAC_KEY and entry digests become MACs sealed at every boot by a checkpoint, so a rewrite needs the key as well as the database, which only helps if the key is injected from a secret manager the database administrator cannot read. Configure an anchor key and the head is Ed25519-signed on a schedule and published off-box, supporting exactly one sentence: any copy of the anchors you kept off-box beats any rewrite made after you took it. The chain is tamper-evident, not tamper-proof.
What does off-box anchoring not prove?
Off-box anchoring — Ed25519-signing the head of Token Observe’s audit chain on a schedule and publishing it outside the database, which stays off until you configure a signing key — does not prove four things, and Token Observe states all four rather than rounding them off. Key theft: whoever holds the signing key can sign anything, so a stolen current key produces anchors that verify. Pre-anchor history: entries written before the first anchor are covered by no anchor, and a checkpoint can only vouch for history from the moment the key was introduced, not for whether that history was already honest. Sink collusion: a sink administered by the same party that runs the database is not independent, because deleting the anchors and the rows becomes one action rather than two, so put one somewhere they cannot rewrite. And time: the signing timestamp sits inside the signed bytes so it cannot be edited afterwards, but a signer can write whatever instant it likes at the moment of signing, and only an RFC 3161 timestamp authority or a public ledger proves when. There is no Merkle tree, so partial-log proofs are unavailable.
Has Token Observe had an independent penetration test?
No. Token Observe has had no third-party security assessment, no red-team engagement, no external code audit and no independent penetration test. There is no certification either, and the full position on SOC 2, ISO 27001 and ISO/IEC 42001 is in the certification answer under compliance. What has been done is internal: a five-dimension adversarial review with a three-verifier refutation panel per finding, a STRIDE threat model, targeted testing of specific attacks including a forged audit chain and adversarial regular-expression inputs, and automated release gates covering lint, typecheck, the unit and integration suites, an artifact smoke test, a dependency audit, a secret scan over the whole history and an image scan that refuses every high or critical finding. None of that is an external test. A pre-purchase test by your team or your assessor is expressly permitted by the licence: no pre-approval of benchmark results, and findings publishable after coordinated disclosure, provided a benchmark names the version and configuration tested. That licence is a template pending review by counsel rather than legal advice, so the terms you would rely on are the ones in your signed agreement.
How do we report a vulnerability, and what happens next?
Report a vulnerability in Token Observe by email to security@tenhaw.com, never a public issue, pull request or discussion. The product’s SECURITY.md also names a private security advisory on the repository, but GitHub offers private reporting only on public repositories and the repository is private, so email is the route that works. The published figures are targets for a small team rather than a contractual SLA, and where an order form or design-partner agreement states different ones that agreement governs: a human acknowledgement within two business days, a triage verdict within five, and a fix plan within ten; patch targets, measured from the triage verdict rather than the report, are seven calendar days out of band for critical, thirty for high, ninety for medium and the next release for low. Severity starts from CVSS v3.1 and is then adjusted for the context the product runs in, so a flaw that defeats a governance decision or corrupts the evidence record is rated above what its base score alone would suggest. Good-faith research gets safe harbour under the Computer Misuse Act 1990 and equivalent laws, with a default ninety-day disclosure deadline from the triage verdict. There is no bug bounty, which is a resourcing statement rather than a judgement of the work.
Does the vendor see our prompts?
In cloud mode, the hosted service processes requests you route through it and holds workspace evidence and supplied credentials. Conversation capture defaults to metadata, which retains no message text; bounded redacted text capture is opt-in. That does not mean prompts never reach the service: it processes routed requests before forwarding them to the configured provider. Customer-hosted software has a separate no-phone-home architecture, with its evidence and keys on infrastructure you control. The customer-hosted statement that Tenhaw receives no runtime data does not describe cloud hosting.
Who inside our organisation can read prompts and traces?
Token Observe gates every human read of prompts and traces on four control-plane roles (admin, operator, auditor, viewer) plus a separate per-user team scope, and the two are deliberately independent: a viewer scoped to Finance must not gain Engineering’s evidence merely because both are viewers. The scope predicate is derived from the signed-in user and applied inside the store queries on detail, search, export, compliance, spend, approvals and seat paths, so a caller cannot widen it with a query parameter, and demoting someone without an explicit scope drops any inherited wildcard to nothing rather than preserving full-corpus access. Estate-wide surfaces (the audit chain, the anchors, retention controls, the shadow-AI radar) return 403 to a team-scoped user rather than showing a misleading slice. Sensitive trace reads append attributable audit events, because reading one person’s prompt corpus is itself an act worth recording. The gap to know: local accounts have no MFA unless you put your identity provider in front of them.
Use the published source checks to verify what your deployment sends and where it goes.
Verify it yourself- Workspace configuration, registry, policies and decisions
- Billing evidence from the connections and imports you supply
- Supplied connection credentials, encrypted at rest
- Request metadata for traffic routed through the workspace
- Bounded, redacted prompt and reply text when capture is enabled
- Billing queries using the reporting permissions you supply
- Routed model payloads after the applicable policy checks
- Model-assisted search using a separately configured inference key
- Authorized evidence reads and CSV exports to your client
The cloud service processes and stores workspace evidence and supplied credentials. Supabase handles account authentication; Railway runs the cloud pilot. Cloud hosting does not keep all data inside your network.
Billing cannot recover historical conversations. Activity covers routed traffic or supported evidence supplied to the workspace. Text capture is off by default, and redaction is heuristic.
Customer hosting is an optional, separate deployment choice. Its infrastructure and data boundary differ from the cloud pilot.
54 questions, in 7 subjects
What Token Observe is
What the product does, who it suits, its current stage, that it has no reference customers yet, and the limits of a pilot.
12 questionsLicence, pricing and support
The source-available licence, how the product is metered, the free 30-day trial, the support model, what procurement needs on data processing, and how your records leave if you stop.
6 questionsDeployment and operations
What it runs on, how long onboarding takes, backup and restore, upgrades and rollback, air-gapped installs, and why there is no high-availability topology.
7 questionsCompliance and evidence
EU AI Act, ISO/IEC 42001, NIST AI RMF and OWASP mappings, what an evidence export proves, retention and erasure, and which certifications do not exist.
6 questionsHow it works
The request path step by step: how an agent is onboarded, what happens inline, what streaming and failover do, what happens when something is unavailable, and what you do when a policy blocks work it should not have.
8 questionsAlternatives and adjacent tools
Where Token Observe sits against gateways, observability, security platforms and identity products, and the case for and against building it yourself.
5 questionsTry it free for 30 days.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.