Cloud custody, evidence and security limits.
The scope and limits travel with every claim.
On this page
The answers a questionnaire asks for.
Each one links to the section that proves it. If your question is not here, it is almost certainly under “what is not claimed”.
- What does the vendor receive from a running deployment?
- Cloud workspaces store account records, billing evidence, configured activity and encrypted connection credentials on Token Observe infrastructure. Optional customer-hosted runtime operation has no vendor telemetry or licence callback.
- Where does our data go?
- Cloud processing uses Token Observe’s Railway-hosted runtime, Supabase Auth and the supported provider sources you connect. Optional customer hosting has a separate configurable egress inventory below.
- Do you store our prompts and the model's replies?
- Not by default. Conversation capture ships in metadata mode and keeps no prompt or reply text; switching it to redacted stores both, mask-redacted before truncation and bounded. Tool arguments and results are stored either way, capped at 16,000 characters.
- Can we trust the spend numbers?
- As far as they claim to go. Billed and metered are never added, an import finds what the invoice lists, and the estate report states its own completeness as false.
- Is the audit trail tamper-proof?
- No. It is tamper-evident, and the customer-hosted default is unkeyed. A test forges a chain on a default install deliberately, so the limit is a tested fact.
- Are you certified, and has it been penetration-tested?
- No SOC 2, no ISO 27001, no ISO/IEC 42001, and no independent penetration test. The internal work that has been done is published rather than summarised.
- Who has to sign for the risks that remain?
- Nine risks, each with the role the product's own threat model names. At a business without a CISO or a DPO, all nine land on the head of IT.
- What is our half of this?
- For cloud, choose connected accounts, credentials, capture mode and routed clients. The host/TLS/backup controls below apply to optional customer-hosted installation.
- What does each individual capability not do?
- Every capability's own limits are published here rather than on its product page, grouped under the three pillars, each at its own anchor. The list above is the product-wide summary; that section is the per-feature detail an assessment works from.
- Is Token Observe wrong for us?
- Possibly. The short version: no availability service level, no certificate, one node, no discovery of subscriptions nobody exported, no single blended AI total, and a kill switch that stops agents rather than the vendor's billing.
What crosses the boundary, and what does not.
Cloud traffic reaches Token Observe infrastructure; configured policy and redaction run before provider egress
- Workspace configuration, registry, policies and decisions
- Billing evidence from the connections and imports you supply
- Supplied connection credentials, encrypted at rest
- Request metadata for traffic routed through the workspace
- Bounded, redacted prompt and reply text when capture is enabled
- Billing queries using the reporting permissions you supply
- Routed model payloads after the applicable policy checks
- Model-assisted search using a separately configured inference key
- Authorized evidence reads and CSV exports to your client
The cloud service processes and stores workspace evidence and supplied credentials. Supabase handles account authentication; Railway runs the cloud pilot. Cloud hosting does not keep all data inside your network.
Billing cannot recover historical conversations. Activity covers routed traffic or supported evidence supplied to the workspace. Text capture is off by default, and redaction is heuristic.
Customer hosting is an optional, separate deployment choice. Its infrastructure and data boundary differ from the cloud pilot.
Verify an optional customer-hosted runtime’s egress.
This procedure applies to the customer-hosted server entrypoint. It does not prove that the separate hosted cloud service receives no data: cloud accounts, evidence and encrypted credentials are described above.
- 01List every outbound call site. Run grep -rn "fetch(" packages/server/src and read each hit. What matters is not the count but where each destination comes from. The OpenAI, Anthropic, Google, Bedrock and Azure provider clients, the embeddings path, the MCP upstream client, the webhook publisher, the OIDC client in auth/oidc.ts and the anchor sink publisher in audit/anchoring.ts all take their URL from operator-supplied configuration — the issuer you named, the sink you named, the provider you registered. Two do not: the model-price catalogue, whose URL is openrouter.ai and which runs only when an admin invokes a price sync, and the vendor pull connectors under connectors/, whose hosts are fixed on the connector definition at api.github.com, *.githubusercontent.com and api.umbrella.com and which open no socket at all until the connector scheduler is enabled and the credential variables are set. Neither destination is Tenhaw, which is the claim this step exists to check.
- 02Account for the one network client that is not a fetch. The event publisher contains an SMTP module for emailing approval notifications, and the composition root never configures it, so it opens no socket in this release. A reviewer auditing the source for network clients will find it, and presence of a primitive is not presence of a feature.
- 03List every hard-coded URL. Run grep -rn "https\?://" packages/server/src packages/core/src. The non-example, non-comment hits are the provider base URLs used as seed and onboarding defaults, the Bedrock endpoint derived from a region, the price-catalogue URL, the two connector API bases in connectors/githubCopilot.ts and connectors/umbrella.ts, a local demo MCP URL, two schema identifiers embedded as literal strings in the Microsoft Teams card payload which are sent to your Teams receiver and never fetched, and the OpenRouter attribution header naming the project’s public repository. That header is the only Tenhaw-owned string in the list: it is a header value rather than a destination, it goes to OpenRouter rather than to Tenhaw, and it is disclosed in the data-flow document. No Tenhaw endpoint is contacted, because none exists.
- 04Watch it. Run Token Observe with egress allowed only to your model providers, your tool servers and your webhook receivers, and confirm nothing is blocked. That covers a default install, because the connector scheduler ships disabled. If you intend to enable a pull connector, add that vendor's API host deliberately and watch that nothing else appears. A delivered image can run fully air-gapped and the dashboard bundles no CDN assets; building that image from source still requires access to the configured base image, the Debian archives and the npm registry.
- 05Check the dependency list. packages/core, which contains the entire governance domain, has zero runtime dependencies, so the code that decides allow or block can be read end to end without standing up any infrastructure. The server’s runtime dependencies are deliberately few, and the CycloneDX SBOM attached to each release lists the complete transitive set, alongside a SHA256SUMS file covering every release artifact.
Section 9 of the licence goes further than permission. You, and anyone operating an evaluation, may inspect, test, fuzz, penetration-test and reverse-engineer the software as deployed on infrastructure you control, and may commission a third party to do it. Publication is narrower than that testing right: benchmark results may be published with no pre-approval, provided the publication names the version and the configuration tested, and security assessment findings may be published, naming the software, after coordinated disclosure. The licence records that the maintainers already publish their own outstanding defects, and says the section is meant to be consistent with that practice rather than to suppress yours. A pre-purchase penetration test is welcome, and the published defect list and threat model are the recommended starting point so the test is not spent re-finding what is already known.
Release and CI evidence include dependency inventories, checksums and source-bound test/scan receipts. The current cloud pilot passed exact-source gates and an actual runtime-filesystem scan with no high/critical findings, while medium, low and unknown findings remain. Actual Railway OCI layer bytes have not been retrieved or scanned; CI layer checks are not a substitute. A passing scan is not an independent penetration test or a guarantee of security.
The dependency position is a security position, not a taste. Runtime dependencies are kept deliberately few because every addition is supply-chain surface in a product that gets deployed air-gapped. The governance domain (the code that decides allow or block) lives in packages/core, which has zero runtime dependencies, so it can be read end to end without standing up any infrastructure at all.
- The evaluation licence
- For optional customer-hosted evaluation, thirty days from first installation, for internal evaluation, security review and proof of concept, with no order form, no signature and no fee. Section 3.3 says in terms that the clause exists so a prospective customer’s security team can read, run and attack the software before a purchase order is raised. It must not be used to process live production traffic or regulated personal data, and it carries no support commitment, no warranty and no service level of any kind.
- What you may publish
- Benchmark results, naming the version and configuration, with no pre-approval. Security assessment findings, naming the software, after coordinated disclosure. Note the drafting: section 9.1 widens the right to test to any person operating an evaluation, while the publication rights at 9.2 and 9.3 are drafted as the Licensee’s, so an evaluator publishing before an order form exists should agree that in writing first.
- Where to start a review
- The published known-issues list and the STRIDE threat model. The first names the file, states the concrete cost and describes the attack that works; the second names the residual risk on every trust boundary and the role that has to accept it.
- The status of the licence itself
- Section 6.1 undertakes the no-telemetry position in terms rather than merely describing it — but the published licence carries its own banner: it was drafted by the engineering team rather than by qualified counsel, placeholders remain in it, and it asks to be treated as a statement of intended commercial terms rather than as an executed grant of rights until that review is complete. The clauses quoted on this page are quoted as that stated position and not as an executed one, and the agreement you sign is what ultimately binds. The product documentation is likewise expressly not a warranty or a contractual specification unless an order form makes it one.
- Licence-compliance verification
- Self-certification only, and by construction: because the software reports nothing to the licensor, there is no visibility of your usage. One written certification per twelve months on thirty days’ notice is permitted; there is no right to inspect your systems, to require a metering component, or to receive your records.
The artefacts, and where to get them
This page cites five documents as its evidence, so here is where each one lives. They are in the product repository; where your organisation does not have access yet, it is granted with the evaluation, and asking for it is the fastest thing on this page.
- The published defect listdocs/KNOWN-ISSUES.md
- Names the file, states the concrete cost, describes the attack that works, and records which findings were refuted on verification as well as which were confirmed. There is no warranty that it is exhaustive.
- The STRIDE threat modeldocs/THREAT-MODEL.md
- The residual risk on every trust boundary and the role whose written acceptance it needs. The nine risks lower down this page are taken from it unchanged, roles included.
- The data-flow documentdocs/DATA-FLOW.md
- Table by table: what is stored, why, how sensitive it is and what the retention window reaches. It is the source for the stored-data section above.
- The licenceLICENSE
- Section 3 for the evaluation grant, section 6.1 for the no-telemetry undertaking, section 9 for testing and publication — read with the template banner at the top of the file, which is not decoration.
- The release assetsSBOM (CycloneDX) and SHA256SUMS, on every release
- The complete transitive dependency set, and a digest covering every artifact, so what you ran can be checked against what was published.
Source, build and verification materials are available through an assisted customer-hosted review. Access is arranged separately from the cloud trial. Request security review materials
What the hosted cloud pilot receives and keeps
The default product path is a hosted cloud workspace. Token Observe runs its dashboard, API, MCP and evidence storage and holds encrypted connection credentials. Supabase Auth handles account passwords and email confirmation. Customer hosting remains optional and has a separate data boundary.
Billing connections use the required organization/reporting credentials for OpenAI, Anthropic, xAI or a configured Google Cloud BigQuery export. They supply reported costs, not historical prompts, replies or consumer-subscription discovery. Billing and inference keys are separate; billing credentials stay in the parent registry rather than entering gateway children. Model traffic and model-assisted search use your own inference account and can incur provider charges; Google billing queries may also incur charges.
Each workspace has a separate process, database and application secrets on a shared managed host. The processes share one OS user: this is application separation, not a tenant VM or container security boundary. A process compromise or host administrator remains inside that boundary. Arbitrary network MCP and webhook destinations are disabled in the initial cloud offering.
The cloud evaluation lasts 30 days from workspace creation and collects no card. There is no automatic paid conversion. Expiry stops new model traffic, changes and scheduled billing collection, while authorized retained reads/exports remain subject to the shared monthly traffic quota. Expiry does not delete the account or workspace. Cloud activity retention is configured to 30 days; other records and backups have different scopes.
The current deployment is a pilot. Exact source tests, public readiness/guards and a bounded synthetic read workload have passed. Successful live signup/email confirmation, authenticated customer workflows, four live provider integrations, actual backup/off-box/key recovery, customer capacity and independent security review remain unqualified. No availability commitment, certification or production outcome follows from the tests.
The deployment handoff records one Railway replica and its persistent volume in Amsterdam, using Supabase Auth. This does not establish that all Auth, email, backups or supplier access remains in Europe. Cloud commercial terms, privacy roles, data-processing and transfer arrangements remain under review; this page invents no approved agreement.
Optional customer-hosted runtime egress
For optional customer-hosted installations, this inventory covers server-runtime egress, not cloud custody: configured model providers, MCP tool servers, webhook receivers, the model-price catalogue, an OIDC issuer, an anchor sink and optional vendor pull connectors. Cloud hosting additionally uses Railway infrastructure, Supabase Auth and customer-supplied provider billing credentials, as described above. The cloud pilot disables arbitrary network MCP and webhook destinations.
You can test this. If your network policy allows outbound connections only to your model providers, your tool servers and your webhook receivers, Token Observe functions completely — adding your identity provider and your sink host only if you enabled them, and the two vendor APIs, api.github.com with *.githubusercontent.com and api.umbrella.com, only if you enabled a pull connector. Write the allowlist for the deployment you run, not the feature list: those three hosts have no business in a default install's rule, and turning a connector on behind a rule that omits them produces failing connector runs carrying a typed error on the run record rather than a silent gap. That is the right way to fail, and still one to avoid: edit the rule before you set the credential. A delivered image can run fully air-gapped and the dashboard bundles no CDN assets; an air-gapped install never calls the price catalogue at all and loads rates through POST /api/prices instead, and it can run no connector at all, because every connector destination is a vendor on the internet.
Egress is allowlisted, not merely configurable, and the reason is specific to what this process holds. A provider row names both a URL and an environment variable, and Token Observe resolves that variable and sends its value to that URL as a bearer credential — so an unconstrained registry write would be equivalent to reading every secret in the process, in the one process that deliberately concentrates every provider key in the estate. The MCP case is worse than the provider case and is guarded harder for that reason: an MCP server row names a variable whose value is sent raw as an Authorization header to a URL in the same row, it is writable at operator rather than admin rank, and the outbound call is triggered by any viewer listing that server’s tools. Unguarded, it would read any variable in the process and post it anywhere.
So there are four allowlist pairs rather than one — providers, MCP servers, webhook receivers, and vendor organisation-admin credentials — each pair naming which hosts may be registered and which environment-variable prefixes may be referenced. MCP hosts default to loopback only, because a sidecar is the one topology safe to assume and “somewhere on your network” and “somewhere on the internet” are the same string to this process. Webhook hosts have no safe default at all, because a webhook destination is always third-party, so an unstated policy resolves to the existing rows alone. The credential prefixes default to purpose-specific names such as AZURE_OPENAI_ and AWS_BEDROCK_, with bare AZURE_ and AWS_ deliberately absent so that a host’s ambient cloud credentials, AWS_SECRET_ACCESS_KEY included, stay unnameable from a provider row.
One primitive is disclosed here because it is not on the destination list and a reviewer auditing the source for network clients will find it: the event publisher contains an SMTP module for emailing approval notifications, and the composition root never configures it, so this release opens no mail socket. Presence of a primitive is not presence of a feature.
- Model providers
- The governed request after unicode sanitisation and after the redaction plan has been applied, plus your API key read from the process environment. OpenAI, Anthropic, Google, OpenRouter, Bedrock, Azure, or any OpenAI-compatible endpoint you register, including a self-hosted one.
- MCP tool servers
- The tool name and its arguments, after policy evaluation and after inbound redaction, with the per-server auth header read from the environment. On every allowed tools/call, and on catalogue refresh.
- Webhook receivers
- A JSON event carrying identifiers, counts, policy names and a one-line human summary, signed with x-acp-signature when a secret is configured; Slack receivers get the summary text only. Sent best-effort on a later tick, because a webhook can never delay or fail the governed request that produced it. Treat receivers as inside your data boundary all the same: an approval.requested event carries the action summary, which is the agent, tool or model name plus the policy’s own reason — your reason text, which can perfectly well name a customer, an amount or a destination. It carries no model-proposed arguments.
- The model-price catalogue
- A GET for the public OpenRouter price list, sending no prompt, no trace and no identifier — with one caveat: it does send your OpenRouter key if one is configured, for rate-limit purposes. It runs only when an admin calls POST /api/prices/sync. The call also carries HTTP-Referer and X-Title attribution headers, which identify the gateway software and name the project’s public repository; they do not identify the agent, the user or your organisation, and they reach OpenRouter rather than the vendor.
- Your identity provider
- The standard Authorization Code with PKCE exchange — the issuer’s discovery document and JWKS, then a POST to its token endpoint. No prompt, no trace, no agent, no payload; this flow is about one human signing in, and an install that never configures OIDC makes no such call.
- Your anchor sink
- One signed statement per anchoring interval about the shape of the audit chain: head sequence and hash, entries covered, the previous anchor’s hash, the key id and the signature. No payload, no identifier and no personal data. A file sink writes the same statement to local disk and makes no network call at all.
- Vendor pull connectors, off by default
- Two exist and no more. GitHub Copilot seat spend is read from api.github.com, using a GitHub App private key that buys nothing but a nine-minute JWT and a one-hour installation token; a usage report can come back as a signed link on a *.githubusercontent.com host, which is fetched without the credential attached. Cisco Umbrella DNS and proxy activity is read from api.umbrella.com under an API key scoped to reporting read. What goes out is the credential, your organisation login and a date window — no prompt, no trace, no payload and nothing out of your own evidence store — and what comes back is that vendor's own reporting data. The destination is the vendor's API, not Tenhaw's. Both connectors are gated twice over: ACP_CONNECTORS_ENABLED defaults to false, and a connector counts as configured only when the environment variables its definition declares are present, so withholding the credential is a supported position.
# The single highest-value hardening step for a deployment whose model
# endpoints are not the public vendor ones. No list may be empty: a
# variable that is set but lists nothing is refused at boot, because an
# empty allowlist would otherwise read as "any host" and "any variable".
ACP_ALLOWED_PROVIDER_HOSTS=llm.internal.example.com
ACP_ALLOWED_KEY_ENV_PREFIXES=ACP_PROVIDER_
ACP_ALLOWED_MCP_HOSTS=tools.internal.example.com # default: loopback only
ACP_ALLOWED_MCP_KEY_ENV_PREFIXES=ACP_MCP_
ACP_ALLOWED_WEBHOOK_HOSTS=siem.internal.example.com # no safe default exists
ACP_ALLOWED_WEBHOOK_KEY_ENV_PREFIXES=ACP_WEBHOOK_
# The fourth pair matters only if you enable a vendor pull connector.
# It defaults to the two vendor APIs the shipped connectors need, and
# ACP_CONNECTORS_ENABLED defaults to false, so leaving all three lines
# alone leaves both connectors off and neither host contacted.
ACP_ALLOWED_VENDOR_ADMIN_HOSTS=api.github.com,*.githubusercontent.com,api.umbrella.com
ACP_ALLOWED_VENDOR_ADMIN_KEY_ENV_PREFIXES=ACP_VENDOR_ADMIN_
ACP_CONNECTORS_ENABLED=false
# The credential buckets are disjoint by design: a name that a provider
# row may reference is not one an MCP, webhook or connector row may use.What is kept, where it is kept, and what is deliberately not kept
The cloud service has a directory database for accounts, workspaces, billing evidence and encrypted credentials, plus a separate evidence database for each workspace. Cloud account passwords and email verification belong to Supabase Auth. Optional customer-hosted server installations use their configured SQLite or PostgreSQL backend. A backup of one workspace database alone does not recover the cloud registry, Auth or custody keys.
The most sensitive table in the system is trace_events, because prompts and tool arguments are where unplanned personal data accumulates. What it holds by default is bounded, and it is smaller than an earlier release made it: the automatic post-redaction prompt excerpt has been removed, and a gateway test now asserts that the llm_call event carries no excerpt field at all. What remains without any further configuration is MCP tool arguments and results capped at 16,000 characters, the policy decisions, the redaction records as kinds and counts rather than values, and response metadata — stop reason, upstream request id, token counts and content-block types. Prompt and reply text is kept only if you switch conversation capture on. Read the next paragraph before quoting a retention position to anybody.
Conversation capture has two modes, and the default keeps no prompt or reply text at all. Out of the box ACP_CONVERSATION_CAPTURE is metadata: each request and each response carries a capture record describing the shape of the exchange — the mode, the source, how many messages and blocks were omitted, how many characters were captured, whether it was truncated, and whether the provider returned a reasoning summary — and no message content is even constructed in memory. Set ACP_CONVERSATION_CAPTURE=redacted and that changes in both directions at once: the prompt as sent and the assistant’s reply are both written as evidence, under sources named gateway_request, provider_response and response_cache. So “model response text is not stored” is true of the default mode only, and any version a data protection officer acts on should name the setting.
What redacted mode keeps is bounded, and the order of operations matters. Each text block is unicode-sanitised and passed through the mask redactor before it is truncated, so a credential split across the character cap cannot leave its prefix behind in evidence. That redaction runs whatever your outbound policy allows, on the product’s own stated reasoning: permission to send personal data to a provider is not permission to retain a second unredacted copy in the flight recorder. Images are recorded as omitted rather than captured. Tool calls and tool results are captured as text under the same treatment. The whole record stops at 32,000 characters and 100 messages by default and marks itself truncated when it hits either, and both bounds are configurable within limits the server enforces. Switching capture on is therefore a decision about what your evidence contains, and it should be made with the same people who set your retention window.
The approval record used to be the one place raw model output was persisted, and it is not any more. When a policy holds a tool call the model proposed, the approval’s action summary is now built from the tool name plus the policy’s own reason, passed through the mask redactor and cut at 240 characters; on the request path it is the model name plus the policy reason under the same cap. The code carries the reasoning in its own comment: the action payload belongs only in opt-in conversation evidence, and copying it into approvals and notifications would bypass metadata-only capture. No model-proposed arguments reach the approvals table or the approval.requested webhook. Keep treating both at trace sensitivity all the same, because a policy reason is written by your organisation and can perfectly well name a customer, an amount or a destination.
Credential storage depends on mode. The cloud registry encrypts saved billing and inference connection credentials; inference keys are supplied to the selected workspace runtime, while billing credentials stay in the parent registry. In optional customer-hosted server configuration, provider keys, MCP auth headers and webhook secrets are normally referenced by environment-variable name. Issued agent keys are held as SHA-256 with a display prefix rather than recoverable plaintext. None of this means that cloud processing receives no keys.
The least obvious of those refusals is the smallest. When a caller presents a credential Token Observe does not recognise, the rejection is counted in an hourly roll-up (the reason and the count), and the presented token is not stored, and neither is a digest of it. A digest would buy nothing defensively, because an unknown credential resolves to no key in the database by definition and there is nothing to compare it against. What it would create is an offline oracle against a secret that may well be live somewhere else in your estate: a mistyped production key from another system, a credential pasted into the wrong base URL. Anyone who could read that table could then confirm guesses at that secret at their leisure, against a copy that outlives the mistake. A governance product must not turn someone else’s typo into a durable cracking target.
The public cloud portal uses HTTPS with Secure/HttpOnly authentication cookies and encrypts saved connection secrets in its registry. Workspace evidence databases are not field-by-field encrypted by the application: captured text and ordinary tool arguments remain evidence data. Custody of the cloud master key, host storage and backups belongs to operating the service; an actual cloud snapshot/off-box/key recovery remains unqualified. For optional customer hosting, the server expects a terminating TLS proxy and host/volume encryption. Encryption of saved keys is not a claim that host administrators cannot access runtime data.
- seats
- The only table whose primary subject is a human other than an operator: a named employee and the subscription tool they hold a seat on. It supports a coverage census, so it is a list of who is and is not governed. Trace retention does not cover it and subject erasure does not reach it.
- user_idp_groups
- The directory groups your identity provider asserted about one named person at their last sign-in. That is an attribute of a person, held about a person, and a data protection officer should treat it as HR-adjacent data rather than as configuration. Capped at 500 keys per person, replaced wholesale at each sign-in, and aged out on its own window.
- radar_findings
- Detector output from the evidence the radar holds — what you exported into it, plus whatever a pull connector you enabled has read back from GitHub Copilot or Cisco Umbrella. It may include workstation hostnames, staff usernames, service-account and key identifiers, billing amounts, and — where per-user seat spend is present — the email addresses of named employees. Retained indefinitely; neither the retention window nor subject erasure reaches it.
- approvals
- The decision record: who approved, when, their written rationale, the payload hash, and the action summary — the tool or model name plus the policy’s own reason, capped at 240 characters, and mask-redacted on the proposed-tool-call path. Not purged by the retention window and not reachable by subject erasure, so it is the one place policy-reason text outlives the trace it came from.
- webhook_deliveries
- The exact bytes POSTed to each receiver, kept so a signature dispute can be settled, and not purged.
- audit_anchors
- No payload, no identifier and no personal data — a statement about the shape of the chain rather than about anything in it. This is the one table meant to be handed to an outside auditor.
What the spend numbers mean, and what they cannot mean
The estate report states its own completeness as false and prints the reasons. The register holds three approval states and coverage is measured on a separate axis; a billed charge and a metered estimate are two numbers that are never added and no currency is converted; an import finds what the invoice lists and discovers nothing on its own; and a network feed gives you a destination rather than a person or a transcript.
For the invoice-import path, map finance charge lines onto the ten-column template and submit CSV in the dashboard or the same fields as JSON. The parser validates columns and previews the rows before writing. That import needs no live vendor billing credential; the separate cloud organization-billing connections do. Optional customer-hosted GitHub Copilot and Cisco Umbrella connectors feed radar under their own read-scoped credentials. Imported lines carry an external id, vendor, tool, team, owner, kind, currency, amount, charge date and service period, bounded to 1,000 lines and 2 MiB. Identity is team plus source plus external id: an identical reimport adds nothing, while changed data under an existing identity conflicts and rolls back the import. None of these paths provides complete subscription discovery or removes your data/security review responsibilities.
Billed and metered are reported as two numbers, per currency, and never as one. A subscription invoice is a fact about money that has already moved; a metered token estimate is a calculation over observed traffic against a price table. They have different error bars and only one of them will ever appear on a bank statement, so subscription and API amounts are reported separately with their evidence counts, metered estimates are recorded apart from both, and no exchange rate is applied anywhere. A blended total would reconcile against neither the vendor invoice nor the gateway, and the first person who checked it would be right to stop believing the rest of the screen.
Classification is three states, and coverage is a different question. A register entry holds one of approved, prohibited or unknown — the database enforces that set with a CHECK constraint — and unknown is the default on creation, so nothing passes by silence. Whether anything has been observed at all is a separate axis with five states of its own, measured across six evidence streams, and it is reported separately rather than folded into the decision. Keeping them apart is what stops the two failures that matter: an approved, exactly attributed subscription is legitimate paid usage and is never reported as an unauthorised-use finding, and a source nobody has connected reports no traffic, which reads as clean and means unknown. That second one is why every radar pass records a run row whatever the outcome — a dead feed must never be indistinguishable from a clean estate.
Two scoping facts belong with the classification rather than after it. The suppression of approved subscriptions is one detector, not the whole radar: the register is read in exactly one place, inside the per-user seat-spend detector, and billing reconciliation, network egress, service-account key audit and IDE telemetry findings never consult it. And approval is the current decision for an exact vendor, tool, team and owner match. It is not a historical attestation, so the register does not prove that a tool was authorised at the moment somebody used it.
Identity is where this surface is most likely to be over-read. A network egress feed you supply gives you a source hostname and a destination hostname. A hostname is not a verified employee identity, network contact does not establish that anybody used AI, and it establishes no paid subscription, no prompt, no reply, no token count and no authorisation — the radar records that limitation on the finding itself rather than leaving it to be inferred. Browser AI destinations are classified as metadata observations with authorization fixed at unknown, and nobody should read this product as a way to see what staff typed into a chat window. The personal data that does accumulate here is named in the tables above: seats carries employees by name, radar_findings can carry workstation hostnames, staff usernames and named employee email addresses, and neither the retention window nor subject erasure reaches either of them.
Two further absences that readers ask about. Hidden chain of thought is not readable from any provider, so it is not recorded and it is not claimed; where a provider returns a reasoning summary, that is what is captured and it is labelled as a summary. And the kill switch stops every agent and seat authenticated to Token Observe, which is the whole of what it does: it cancels no vendor subscription, reclaims no seat, changes no payment plan and ends no browser AI session. The register records the decision and names the owner. Only the vendor can end the seat.
You map the charge lines out of your own finance export onto the import template, and the dashboard validates the file and shows you the parsed lines before anything is written.
- externalId
- vendor
- tool
- team
- ownerEmail optional
- subscriptionId optional
- kind
- currency
- amount
- chargedAt
- periodStart
- periodEnd
File totals before duplicate detection. Nothing is written yet.
| invoice line | tool / team | kind | amount | charged |
|---|---|---|---|---|
| INV-2026-09-001 | chat assistant Marketing | subscription | 1,240.00 USD | 2026-09-01 |
| INV-2026-09-002 | coding agent Engineering | api | 806.40 USD | 2026-09-01 |
| INV-2026-09-014 | meeting notetaker Sales | subscription | 276.50 GBP | 2026-09-01 |
Repeat imports are recognised by team, source and invoice-line identifier, so re-importing an export you already loaded skips the lines it already holds rather than doubling the bill.
- A billing credential
- A login to a vendor console
- A connector for somebody to review
Column names and bounds are the product’s own; the lines are illustrative. Timestamps are UTC, kind is subscription or api, currency is three uppercase letters, and an import is capped at 1,000 lines and 2 MB. Import charge lines once, never a bill total alongside the lines that make it up.
- Estate report: API activity
- “Only gateway model requests are priced here; imported telemetry can omit costs, so a zero recorded estimate does not prove free or fully accounted use.” Quoted from the report’s own completeness block, which is hard-coded to complete: false.
- Estate report: billing coverage
- The invoice ledger covers supplied imports. The cloud provider-account report separately covers connected organization billing sources and exposes collection freshness and coverage. Live customer account collection remains unqualified.
- Estate report: what registration means
- “Subscription registration expresses approval, not proof of traffic capture or endpoint enforcement.”
- Estate report: subscription traffic
- “A missing trace is unknown coverage, not proof of inactivity; browser visits and total vendor usage are not included.” Charge detail and the register are capped at 1,000 rows each, and any truncation is declared rather than left to be noticed.
- Reading it from an agent
- The MCP spend report returns the same figures under the same team scope as the dashboard, and the calling subject needs an explicit report grant and a report-team grant. Being assigned to a team does not by itself confer reporting access, which is what stops an Engineering-scoped agent reading Finance’s invoices.
An empty result, and the product refuses to read it as an all-clear: nothing found is not the same as nothing being there, and a zero here is not a clean estate. Coverage stays on its own axis, so silence is never mistaken for an approval.

Sample data in the real Token Observe interface, recorded from a local evaluation with a temporary database, a mock provider and one request. No live provider key was used.
What is enforced inline, and what that enforcement is worth
Token Observe fails closed for policy decisions: the governance evaluator returns allow only when every gate passes, and missing, unresolvable or erroring state denies the request. The kill switch is evaluated first and beats everything else, which is the literal implementation of the EU AI Act Article 14(4)(e) stop capability. The trade-off: because governance is inline, Token Observe is a single point of failure in the agent request path, and if it is unavailable the fleet stops. That is intentional: a control you can bypass by turning it off is not a control.
Redaction runs inline, in-process, before the request leaves — not asynchronously and not after the fact. Detection covers credit card with Luhn validation, IBAN with mod-97, UK NHS number with mod-11, US Social Security number, UK national insurance number, email and phone, plus the secret kinds: JWTs, AWS access keys, prefixed vendor API keys and PEM private-key headers. Secret kinds are always masked irreversibly and never tokenised, whatever a policy’s mode says, because a reversible placeholder for a credential is a credential. Other kinds are masked or tokenised per policy, and a tokenised value keeps a stable placeholder across one conversation so the model can still reason about “the same customer”.
It runs in both directions: responses are scanned too, so a data-class rule fires on an identifier the model produced even though nobody sent one. Streaming is covered by a hold-back buffer with a floor of 64 characters and a separate buffer for each tool-call argument channel, so a value split across two chunks is still caught, and the response-side data-class decision is taken before the first byte, because a stream has no later enforcement point.
The limit sits beside the claim. Matching is regex plus checksum, so free-text personal data and non-UK/US identifier formats are not detected at all, and the confidence scores are published so a policy can set a threshold: checksum-validated kinds score 0.9 to 0.98, national insurance and Social Security numbers 0.85, and phone 0.7. Token Observe is a compensating control, not a complete DLP, and it should not be bought as one.
Authority is deny-by-default and narrows at every hop. Agent RBAC is explicit-deny-wins with action-level tool scoping; /v1/models is filtered against the registry, and tools/call re-checks grants independently of tools/list. Delegation intersects permissions at each hop, so a forged delegation chain can only narrow authority, never widen it. Approvals are bound to a payload hash, are single-use, and expire. One residual remains: an approver who skims the action summary approves what they were shown.
Failover is typed, because the alternative launders a refusal into a success. A 429 or a timeout fails over; a content-policy refusal, an auth failure, an invalid request and a context-length error do not. Per-agent data policy (zero data retention, no training on payloads, a required serving region) is enforced on the fallback chain as well as on the primary route, and when no route satisfies it the request fails closed with ACP_ZDR_UNAVAILABLE rather than quietly downgrading. Those provider flags are operator-asserted and unverified: setting them records your assertion about your contract with that provider, which Token Observe cannot check, and the assertion itself is audited with the actor who set it.
Every policy can run in shadow mode first, so you learn your false-positive rate before you start blocking real work. A stricter option exists and is off by default: with backtest-before-enforce switched on, the transition into enforcement is refused until a backtest of that exact rule (digested over its trigger, action, scope and priority) has been run and acknowledged by a named person, whose name and the accepted figures are copied into the audit entry. It is a process control, not a technical one.
Every check lives in one function, so you can read your rules and know what they do.
POST /v1/chat/completions
The key resolves to one registered agent with a named owner, and the trace is opened before anything can refuse the request.
Kill switches, then lifecycle, then deny-by-default permissions across every delegation hop, then rate limits, then your policies by priority.
returns: allow · block · require_approval — plus a redaction plan
A spend ceiling is the deliberate exception: it is deferred until routing has fixed which provider and model will serve the call, so it is tested against the price you will pay.
Routed to the provider you configured, on your key. Metering runs last and the trace closes with a terminal status.
The plan is applied to the outbound payload and a redaction event records kinds and counts, never values. A redaction that cannot be applied faithfully becomes a refusal instead.
An approval bound to a hash of this exact payload, single-use and expiring. The caller receives the approval id and the header to resume with.
429 ACP_BUDGET_EXCEEDED or ACP_RATE_LIMITED where a ceiling decided it. The trace closes as blocked and is kept as thoroughly as one that went through.
Verdict names and status codes are the product’s own. Every rule can run in shadow mode first, where the decision is recorded and the request is not changed. That is how you learn your false-positive rate before you refuse real work.
- Injection scanning on both channels
- Unicode is sanitised to a fixpoint before anything reads the payload, then prompts and tool results are scored, with tool-result findings weighted 1.25× because that is where indirect injection arrives. The residual: these are regex heuristics, and paraphrase, translation or non-English phrasing defeats them. Sanitisation is a separate control with a separate limit — the unicode tag block, zero-width characters, bidirectional overrides and private-use ranges are stripped before storage and display, so ASCII smuggling cannot hide instructions from a human approver, but homoglyph substitution is not addressed at all.
- MCP tool integrity pinning
- A descriptor hash over the canonicalised name, description and input schema, re-verified at the 60-second catalogue TTL and on live-listener maintenance; drift quarantines the tool and emits an event. The residual is the window: a change may be served from the trusted snapshot until the next refresh, at most 60 seconds on access.
- The console is a trust boundary of its own
- Every response carries nosniff, X-Frame-Options DENY and Referrer-Policy no-referrer, and console responses add a same-origin content security policy. A test asserts that no http://, https:// or * can ever appear in it, because a well-meant font CDN is how a narrow policy stops being one. The directive that matters most is frame-ancestors none: without it, a page on any origin can frame this console and land a click on Engage kill switch.
- Budgets and rate limits
- Per-request, hourly, daily and monthly ceilings checked pre-flight, with USD admission atomically reading projected spend and reserving per agent. The residual is arithmetic rather than architectural: a provider-reported final cost can exceed the pre-flight estimate, and the token and request windows are not distributed admission counters.
- Trace search never generates SQL
- The model emits a schema-validated filter object, never SQL; queries are parameterised; and the signed-in user’s team predicate is applied after translation, so it cannot be supplied or removed by the model.
403 ACP_KILL_SWITCH_ENGAGED evaluated first; beats every other gate
403 ACP_AGENT_NOT_ACTIVE lifecycle state; step 6, right after the switch
403 ACP_RBAC_DENIED deny-by-default; explicit deny wins
403 ACP_POLICY_BLOCKED a policy matched and its action was block
403 ACP_DELEGATION_DENIED the chain could only narrow, and it did
403 ACP_ZDR_UNAVAILABLE no route satisfies this agent's data policy
429 ACP_BUDGET_EXCEEDED pre-flight admission, not post-hoc billing
429 ACP_RATE_LIMITED carries Retry-After
503 ACP_AUDIT_UNAVAILABLE verification failed; writes latchedWho can read the evidence, and how that boundary is held
In optional customer hosting, control-plane access is a named human with a server-side session, deny by default, at one of four ranks (admin, operator, auditor, viewer), and evidence reads are additionally bounded by per-user team scopes that a caller cannot widen with a query parameter. There is no admin bearer token: an earlier revision of the API contract documented one that was never implemented, and the claim was removed rather than the feature added, because every control-plane action being attributable to a named person is what makes the audit log evidence.
The cloud portal uses confirmed Supabase Auth identities and current workspace membership, revalidated before dispatch. Human API requests can use a verified managed-user Bearer token; machine access requires an active key in the target workspace. The proxy strips caller authority headers and supplies bounded signed identity assertions. Cloud self-service creates one owner/admin workspace and currently exposes no colleague invitation or organization membership management.
The local-login, independently configured OIDC and SCIM details below describe optional customer hosting. The cloud proxy disables child-local login and child SSO and blocks identity-management routes; you do not configure your own IdP as part of the default cloud signup.
The scope is derived from the signed-in user by the server and applied inside the trace and usage-store queries, then repeated on detail, search, export, compliance, spend, governance, approvals, savings and seat-registry paths — because scoping the list alone leaves the detail URL as a way around the boundary. Surfaces that cannot be projected onto one team return 403 to a scoped user rather than a partial answer: the audit chain, the anchors, the retention controls, the shadow-AI radar, the seat census and the fleet summary. Sensitive trace list, search and detail reads append attributable audit events of their own, and demoting an admin or operator to auditor or viewer without an explicit scope drops any inherited wildcard to the empty scope, so a role reduction cannot accidentally preserve full-corpus access.
Two smaller decisions matter to a procurement reviewer. Authorisation is checked before existence, so probing identifiers cannot distinguish a real account from an absent one. And accounts are disabled rather than deleted (there is no delete endpoint for a user), so the audit trail keeps resolving the actor, while a patch that would leave the install with no enabled admin is refused outright.
Single sign-on is Authorization Code with PKCE, verified against the issuer’s JWKS, with the ID token read and discarded. Setting ACP_LOCAL_LOGIN_ENABLED=false makes OIDC the only login path and refuses to boot unless the OIDC configuration is complete, which is the mode the local technical-readiness check requires. The directory group snapshot is capped at 500 keys per person and replaced wholesale at each sign-in — a group the provider stopped sending has been revoked, and merging would hide that — and it ages out on its own window, 24 hours by default. Provider-specific caveats are surfaced by a status endpoint rather than discovered in production: Entra’s group overage, Okta’s extra scope, and the fact that Google Workspace emits no group claim at all.
SCIM is a bounded Users lifecycle, not a directory mirror, and it is off until a bearer token is configured. Every SCIM-created account is a viewer with no evidence scopes, and SCIM cannot set or change roles, evidence scopes or passwords. It may disable a higher-authority row, but only a named admin can rename or reactivate one — which is what stops email-first joining from turning the provisioning credential into the role or scopes already held by an unbound account. Setting a user inactive revokes all of their sessions before the disabled row is written, and disabling the final active administrator is refused. Bodies are capped at 64 KiB and 32 nesting levels; the token is at least 32 bearer characters, is never logged, and is compared through fixed-length SHA-256 digests with a timing-safe comparison. There are no Groups, no live directory read and no hard delete.
Local passwords are scrypt with a per-user salt behind two independent throttles — a per-IP budget as the CPU shield, deliberately generous so a shared office address is not locked out and not reset by a successful sign-in, and a per-account budget as the credential-stuffing shield, cleared on success so it counts consecutive failures. Local accounts have no MFA. That is the residual risk recorded below, and it is why the recommended posture is single sign-on with local login disabled.
- Two role universes, kept apart
- A user’s control-plane rank is only ever set by an admin. Roles mapped from identity-provider groups are a permission mask used on the agent path alone. No directory group can promote anyone in the console.
- The on-behalf-of mask
- Two of the three enforcement settings refuse nothing: off is the default and reads no principal at all, and shadow resolves and records what it would have refused while letting the request through. Only enforce is a control; the other two are instrumentation, and an install that has not reached enforce should not describe the intersection as a mitigation it holds.
- Sessions
- Server-side rows with a 24-hour expiry that survive a restart, so revocation is real: an admin can drop every session for a user, and a password change drops that user’s other sessions. Expired rows are removed by an hourly pass even if the cookie is never presented again.
- /metrics
- Unauthenticated by Prometheus convention, and it exposes agent identifiers and month-to-date spend, so it must be reachable only from your monitoring network. Only the matched route pattern is ever used as a label, so a caller cannot write arbitrary text into the scrape output and cardinality is bounded by construction.
The evidence chain, and what each layer proves
The audit chain is a hash-chained record of governance-plane changes (actor, action, resource, detail), and it is tamper-evident, not tamper-proof. Which guarantee you hold depends on one setting, and every export says which: the protection field reads unkeyed or keyed, and the difference is not cosmetic.
Optional customer-hosted installations default to unkeyed protection unless an audit HMAC key is configured; cloud provisioning creates a workspace audit key. On an unkeyed installation, The chain is hashed with an unkeyed SHA-256 and anchored nowhere outside the database, so an operator with write access to the database file can rewrite an entry and recompute every downstream hash, and verification will report valid: true. That is not a theoretical concession — a test does exactly that on a default install, deliberately, so the limit is a tested fact rather than a caveat in prose, and the same test asserts that the forgery fails once a key is configured, so neither half of the claim can drift. What unkeyed offers is tamper-evidence against alteration that does not also recompute the chain, which is integrity verification and should not be presented to an auditor as more.
Keyed protection makes entry digests HMAC-SHA256 under an external key, and seals the chain head at every boot with a checkpoint MAC. Rewriting an entry now needs the key as well as database access. Entries written before the key was configured cannot be re-MAC’d (doing so is the very act being prevented), so they are covered instead by the checkpoint over the head they reached, and altering any of them moves that head. First enablement is a deliberate two-boot ceremony under an external one-shot flag, and rotation is a second one; every ordinary keyed boot refuses a missing checkpoint schedule even when the rows form a valid plain-SHA chain, which is what stops a database writer deleting all the checkpoints, rehashing plain, and asking the product to seal the downgrade as though it were first enablement. The key ring is finite by design: keep the whole ordered set for as long as the live database or any backup may need to verify, to a maximum of 64 predecessors, and understand that dropping the oldest key is not rollover — it makes retained checkpoints unverifiable.
Four qualifications belong in front of an auditor with the keyed claim. It holds only if the key is injected from a KMS or secret manager the database administrator cannot read; a key in a file beside the database buys nothing. An attacker with host access reads the key out of the process environment and is out of scope, because that is a host compromise and no in-database scheme survives one. The checkpoint vouches for pre-key history only from the moment the key was introduced, and cannot say whether that history was already honest. And rotation needs the complete ordered predecessor ring plus the ceremony, because during an intentionally authorised transition the database alone cannot distinguish a real hand-off from selective checkpoint deletion plus a tail forged with the previous key.
Anchoring exists because a MAC has a structural problem as evidence: the key that verifies the chain is the key that can forge it. Hand it to an auditor and you have handed them the power to fabricate the very record they were given it to check — so they cannot verify independently, and you cannot prove you did not. Ed25519 splits those powers. With a signing key configured, Token Observe periodically signs a statement of the chain head and appends it to a table chained anchor-to-anchor, publishing each one to a file sink, a URL sink, or both, because an anchor that never leaves the database it attests is only as durable as that database. The private half never leaves the signer port.
Anchoring supports one sentence and only one: any copy of the anchors you kept off-box beats any rewrite made after you took it. That is narrow and it is checkable. It does not say the history beneath the first anchor was honest, that the signer’s clock was truthful, or that someone holding the signing key could not have produced the same file. Anchoring is also off unless the key is set, so an install that has not configured it does not hold this property at all.
What anchoring refuses to do is the feature. It refuses to sign when the chain does not verify, when the head has moved backwards, and when an already-anchored entry no longer matches the hash that was attested — because signing over a forged head would launder the rewrite under a key the auditor was told to trust, whereas not signing leaves the previous anchor standing, and that anchor still contradicts the rewrite. Refusing is strictly better than proceeding. Signing-key rotation is handled the same way: the first anchor under a new key embeds a transition statement signed by both the retiring and the incoming key, binding the installation id, sequence, previous-anchor link, attested head, entries covered, protection state, asserted time, cadence and both key ids, so the hand-off is independently checkable without rewriting an old anchor or trusting a key carried beside the evidence.
Evidence exports are digest-sealed, not signed. A trace export and a compliance bundle carry a SHA-256 the recipient recomputes over the canonical JSON of the bundle, plus the embedded chain-verification verdict — valid, entries checked, the sequence any break was located at, the protection mode and the checkpoint it was verified against. The unkeyed export digest detects accidental or post-export edits but does not prove origin; provenance comes from authenticated delivery and, for the ledger head, from keyed audit plus an independently held Ed25519 anchor. Separately, an intrinsic verification failure at runtime latches readiness and audit and domain writes unavailable until restart, later governed requests receive ACP_AUDIT_UNAVAILABLE, and the detecting response does not append an audit row onto the chain it has just found unsafe. That latch is monotonic, restart does not clear it, and there is no online endpoint to clear it.
- Edit a row without re-hashing
- Detected either way, and located to an exact sequence number.
- Delete a middle row
- Detected either way.
- Delete the tail and stop logging
- Ordinary SQL deletion is detected, because SQLite does not lower the sequence high-water mark on DELETE — but a malicious database writer can lower that local row too, so only a previously recorded off-box head or anchor proves a stopped tail against that actor.
- Rewrite the entry and recompute the whole chain
- Undetected on the unkeyed default. Detected keyed, because the digests are MACs and cannot be recomputed without the key.
- Rewrite an entry written before the key existed
- Undetected unkeyed. Detected keyed, because altering it moves the head the boot checkpoint sealed.
- Read the key out of the host’s environment
- Undetected either way, and explicitly out of scope. That is a host compromise, and no in-database scheme survives one.
# Once, at key generation. The private half is what the deployment holds
# and belongs in your KMS; the public half is all an auditor ever needs.
openssl genpkey -algorithm ed25519 -out anchor.pem
openssl pkey -in anchor.pem -pubout -out anchor.pub
# Months later. No install, no database, no network. The verifier imports
# only the zero-dependency core and Node built-ins, loads nothing native
# and touches no database.
node scripts/verify-anchor.mjs --anchors anchors.jsonl --public-key @anchor.pub
# After one rotation, retired keys newest to oldest:
node scripts/verify-anchor.mjs --anchors anchors.jsonl \
--public-key @anchor-new.pub \
--previous-public-key @anchor-old.pub
# exit 0 trusted, 1 not trusted, 2 usage error. --json for a pipeline.
# --public-key is REQUIRED and must arrive out of band: a signature
# checked against a key read out of the artefact being verified proves
# nothing at all. --allow-partial is required before a file that does
# not start at anchor 1 is accepted, because a chain beginning at 4 is
# a deletion rather than a shorter file.Cloud responsibilities and optional customer-hosted hardening
For optional customer hosting, terminate TLS, protect the configured secrets and storage, narrow integration allowlists and test backup recovery. In cloud mode those host and runtime controls belong to the service operator; you still choose connected accounts, routed clients, capture mode, permissions and provider usage. The current cloud pilot’s actual backup/key recovery and customer-sized capacity remain unqualified.
For optional customer hosting, no automatic vendor-side runtime copy exists. Losing the configured database without a recoverable backup loses its evidence. The cloud service does retain workspace data and encrypted credentials; its backup, key custody and recovery need separate qualification. Neither architecture settles whether an agreement is required.
Cloud workspace activity is configured to a 30-day trace-retention window. Optional customer-hosted runtime leaves that window unset by default, meaning keep forever until the operator configures it. An hourly purge removes aged traces in small transactions. Neither default is a legal retention decision for your organisation, and the table/backup scope below still applies.
Read this before quoting a retention period to a data protection officer, because they will ask. The window and the erasure endpoint act on traces, trace events, the search index and trace scores, and nothing else. The audit log is never touched, by design, which is what lets the record of a deletion outlive the deleted data and why erasure is safe to offer at all. Approvals are not purged and cannot be erased by subject, which makes them the one place payload-adjacent text survives a purge that removed the trace it came from. Radar findings are not purged and cannot be erased by subject, and they may carry staff usernames and workstation hostnames. Webhook deliveries keep the exact bytes POSTed. Identity-provider group snapshots age out on their own authority-aligned window, and the rejected-caller roll-up purges on a 90-day window of its own, so quoting the trace window for it is wrong in both directions.
Erasure itself is audited on both halves. A dry run returns the count before anything is removed, and both the preview and the deletion write an audit entry — the preview because reading the prompt corpus for a named person is itself an act worth recording. The entry carries a SHA-256 digest of the subject identifier rather than the identifier, along with the trace ids removed: enough to tie the entry to the export that preceded it, without the erasure record becoming a fresh copy of the thing just erased.
Backups are outside that boundary. Those controls act on the live primary database only. A retained full-snapshot backup necessarily preserves whatever data existed when it was taken; restoring a pre-erasure snapshot can resurrect erased traces and can also lose the audit row that recorded the later erasure. There is no external deletion-tombstone ledger and no automatic reapplication of deletions after recovery. Until a separately approved backup-retention window, destruction evidence when that window closes, and a controlled isolated recovery procedure that reconciles every deletion in the lost interval are all in place, “erased from the live primary” is the precise claim, not “gone from every copy”.
- Terminate TLS in front of it
- And do not expose the control plane to the internet. HSTS is sent only when the request resolves as https, which behind a terminating proxy means telling Token Observe to trust that proxy — the same switch that decides whether the session cookie is marked Secure.
- Set the session secret from a secret manager
- The server refuses to start without it outside tests. Set the trust-proxy flag only when a trusted proxy is in front, because it is what makes a client-supplied forwarded header believable.
- Set the four allowlist pairs to your own endpoints
- The defaults are not one policy and should not be read as one: provider hosts default to the public vendor endpoints, MCP hosts to loopback only, webhook hosts to nothing at all — an unstated webhook policy resolves to the rows that already exist — and the vendor-admin pair to the two vendor APIs its connectors need. Each credential-prefix list defaults to purpose-specific names. Narrowing the first three to your own endpoints, and all four to your own variable names, is what stops an admin-level registry write from turning into a read of every secret in the process environment — the fourth pair cannot be narrowed to an endpoint of yours, because the only destinations it admits are the two vendor APIs, and the way to narrow that one is to leave the connector scheduler off.
- Keep /metrics on the monitoring network
- It is unauthenticated by Prometheus convention and exposes agent identifiers and month-to-date spend.
- Set NODE_ENV=production
- Which also makes the mock provider opt-in rather than opt-out.
- Put single sign-on in front, then disable local login
- This is the recommended posture, and it is the only one that closes the no-MFA residual risk: local control-plane accounts have no multi-factor authentication of their own, and setting ACP_LOCAL_LOGIN_ENABLED=false makes OIDC the only login path and refuses to boot unless the OIDC configuration is complete. It is also what the local technical-readiness check requires.
- Treat the database file as evidence
- Restrict access to it, back it up to immutable storage, and check that the audit verification endpoint returns valid: true after a restore. Backup scheduling, off-box placement, legal holds, freshness alerting and proven deletion are operator-owned controls, and live deletion does not reach data already in a backup.
ACP_TRACE_RETENTION_DAYS=180 # unset means keep forever
ACP_CALLER_SIGHTING_RETENTION_DAYS=90 # a separate window; 0 keeps it all
GET /api/retention/status
reports the window, the exact cutoff instant, how many traces are
eligible now and the outcome of the last pass -- with the caller
roll-up reported as a sibling block rather than folded in, because
the two windows disagree by default and a reader who inherits one
number for the other question has been misled by the shape of the
answer.
POST /api/retention/run admin; runs one pass on demand
DELETE /api/retention/subject?subject=...&match=any&dryRun=true
returns the count before anything is removed. Both the preview and
the deletion are audited, under a digest of the subject rather than
the subject itself.What is deliberately not claimed.
Taken from the product’s own published limitations. If something is not on this list and not on the published defect list, it is not known: nothing known was left off.
Token Observe has not had an independent penetration test.
No third-party security assessment, red-team engagement or code audit has been carried out on it. The security work done so far is internal: a five-dimension adversarial review with a three-verifier refutation panel per finding, a STRIDE threat model, targeted testing of specific attacks including a forged audit chain and adversarial ReDoS inputs, and the automated release gates. That is real work, and it is still not an external test, so it is not presented as one. A pre-purchase penetration test by a prospective customer is welcome and expressly permitted by the licence.
There is no SOC 2, ISO 27001 or ISO/IEC 42001 certification.
None is held, none is in progress, and certification is deliberately not on the near-term roadmap. What exists instead is a control-by-control mapping to EU AI Act deployer articles, ISO/IEC 42001 Annex A, NIST AI RMF and the OWASP LLM and agentic lists, each framed as “this feature helps evidence that clause” rather than as compliance. A design-partner agreement can carry a contractual commitment, with a named date, to any certification milestone your procurement process requires.
The audit chain is tamper-evident, not tamper-proof.
Unkeyed is the default, and on a default install an operator with write access to the database file can rewrite an entry, recompute every downstream hash, and get valid: true. A test does this deliberately, so the limit is a tested fact. Keyed protection and off-box Ed25519 anchoring close most of it and both are off until you configure them; even then, key theft signs anything, history before the first anchor is covered by no anchor, a sink administered by the party that runs the database is not independent, and only an RFC 3161 authority or a public ledger proves when.
Evidence exports are digest-sealed, not signed.
A trace export and a compliance bundle carry a SHA-256 over the canonical JSON of the bundle and the embedded chain-verification verdict. That detects accidental or post-export edits; it does not prove origin. Durable origin evidence comes from authenticated delivery and, for the ledger head, from keyed audit plus an Ed25519 anchor retained independently of the database. WORM export, RFC 3161 timestamping and a Merkle tree for partial-log proofs are not built.
There is no availability service level and there are no service credits.
No uptime percentage, availability service level or service credits are offered for the cloud pilot. Live customer capacity, source-outage handling and actual backup/key recovery remain open qualifications. For customer hosting, availability also depends on the operator’s infrastructure. Existing software-support templates do not create a cloud service-level commitment.
The cloud pilot uses one supervisor replica with bounded workspace child processes and separate SQLite databases. No high availability, point-in-time recovery or proven customer recovery objective is claimed.
The recorded cloud deployment has one supervisor replica. Its parent registry and workspace databases, custody key and managed Auth have separate recovery requirements; backing up one child is not recovery of the service. Actual cloud snapshot/off-box/key recovery and timed customer recovery remain unqualified. Optional customer-hosted runtime counters and sessions must not be assumed safe to multiply across replicas. Planning recovery objectives are not commitments.
There is no throughput or capacity commitment.
The how-it-works benchmark is a thirty-second mock-provider laptop baseline for the server runtime, with its conditions and explicit limits. The newer cloud source also passed a bounded finite-resource synthetic read gate; it does not qualify real Auth/provider latency, inference throughput, customer workloads or Railway durability. Neither result is a customer capacity commitment.
There is no claim that the vendor is legally never a processor.
Cloud hosting makes Token Observe custodian of workspace data and encrypted connection credentials. The former claim that ordinary supply has no vendor processing path applies only to optional customer-hosted runtime operation without support disclosure. Legal roles, data processing agreements, subprocessor arrangements and transfer assessments remain for counsel review; no approved cloud agreement is asserted here. Redact diagnostics before sending and agree their handling.
Token Observe does not make you compliant with any law, regulation or standard.
It is a compensating control. It does not discharge any obligation you owe to a regulator, a data subject or a customer, and the vendor does not sign off your DPIA, FRIA, record of processing or risk register. Deciding risk tiers, running a fundamental-rights impact assessment and notifying regulators remain the deploying organisation’s duties; authoring your policy set and mapping it to your control framework is a professional-services engagement rather than support.
The policy, redaction, injection-detection and routing controls are heuristic, and are not warranted to identify every instance of what they are designed to detect.
Personal-data matching is regex plus checksum, so free-text personal data and non-UK/US identifier formats are not detected at all. Injection scoring is regex-based, and paraphrase, translation or encoding defeats it; homoglyph substitution is not addressed; markdown-image exfiltration is detected but outbound URLs are not rewritten. There is also no warranty that the published defect list is exhaustive.
There is no MFA on local control-plane accounts, and SCIM is a bounded Users push rather than a live directory.
For optional customer hosting, single sign-on moves MFA to your identity provider, and local login can be disabled entirely once it is rolled out — an install that configures neither retains local-password and manual-leaver risk. SCIM has no Groups, no live directory read and no hard delete, and group claims are a snapshot taken at sign-in rather than a live read, so a revoked group keeps granting authority until the person next signs in or the capture ages out.
Endpoint-seat governance is a preview and is not equivalent to an inline gateway.
The seat hook is marked preview and not production-eligible: it is an unsigned Node command rather than a signed native binary, its replay protection is bounded by a device clock the governed party controls, and its local evidence spool is unsigned. On an unenrolled device the files it relies on are ordinary files the user owns — device management is a prerequisite rather than a mitigation. A per-request USD ceiling is also explicitly inert against a subscription seat, because a subscription prices the seat rather than the call, so the marginal cost of one request is not knowable at the moment of deciding. Pre-1.0, only the latest published minor receives security patches; there is no long-term support branch and no backporting, because with one engineering team an LTS promise that cannot be kept is worse than none.
An invoice import finds what the invoice lists; billing connections do not discover every subscription.
The mapped invoice-import path reads supplied CSV/JSON, not a live vendor account. Cloud organization-billing connections separately use explicit reporting/admin credentials with provider-specific scope and unknown/incomplete coverage. Optional customer-hosted pull connectors and other supplied evidence have their own boundaries. A subscription absent from all connected or supplied sources stays unknown; no complete discovery or live-customer qualification is claimed.
A billed charge and a metered estimate are never added together, and no currency is converted.
They are different measurements. An invoice line is money that has already moved; a metered estimate is a calculation over observed traffic against a price table, and a blended figure reconciles against neither the vendor invoice nor the gateway. So subscription and API amounts are reported per currency with their evidence counts, metered estimates are recorded separately, and no exchange rate is applied anywhere — a euro charge is reported in euros. If what you need is one number for a board slide, this product will not give you one, and that is a decision rather than a missing feature.
A source nobody has connected is reported as unknown, never as zero.
Zero usage and no visibility look identical on a chart and mean opposite things, so they are never rendered as the same value. Coverage is measured on its own five-state axis across six evidence streams, every radar pass records a run row whatever the outcome so that a dead feed is distinguishable from a clean estate, and the estate report hard-codes its completeness to false and prints the reasons. Imported telemetry can also carry no cost at all, so a zero recorded estimate does not prove free or fully accounted use. Nothing on this site should be read as “every AI tool in your business”.
There are three authorization states, not four, and approval is a current decision rather than a historical one.
The register stores approved, prohibited or unknown, enforced by a CHECK constraint in the schema, with unknown as the default on creation so nothing passes by silence. Coverage (whether anything has been observed at all) is a separate axis and is reported separately. Approval is the current decision for an exact vendor, tool, team and owner match, so the register does not prove that a tool was authorised at the moment somebody used it, and a match that is missing the tool or the team stays unknown rather than inheriting an approval from elsewhere.
A hostname is not a verified employee identity, and a network feed is not a transcript.
An egress export you supply gives a source host and a destination host. Network contact does not establish who used AI, whether a subscription was paid for, what was typed, what came back, how many tokens it cost or whether anybody authorised it — the radar writes that limitation onto the finding rather than leaving it to be inferred, and browser AI destinations are classified as metadata observations only. Nobody should buy this expecting to read what staff typed into a chat window. The personal data that does accumulate is named here: the seat census holds employees by name, and radar findings can carry workstation hostnames, staff usernames and named employee email addresses. Neither the retention window nor subject erasure reaches either table.
Hidden chain of thought is not available from any provider, so it is not recorded and not claimed.
A record of what an agent was asked and what it then did is not a record of why it decided. Where a provider returns a reasoning summary, that summary is captured and labelled as one; the model's hidden reasoning is not exposed by any provider Token Observe routes to, and no configuration produces it. Conversation capture is also off by default: the default mode stores no message text at all, and prompts and replies are kept only when it is switched to redacted.
The kill switch stops agents authenticated to Token Observe. It cancels nothing at the vendor.
It is evaluated first and beats every other gate, scoped global, team or agent, and it reaches every governed subject enrolled with the product — agents and seats. That is the whole of it. It does not cancel a vendor subscription, reclaim a seat, change a payment plan, stop the charge at the vendor or end somebody's browser AI session. What it does instead is record the decision and name the owner, which is the part a vendor console will not do for you.
Traffic that never comes through the gateway is not governed by it.
Every inline control (policy, redaction, budgets, approvals, the audit record) acts on requests that route through Token Observe. An agent nobody pointed at it is invisible to all of them, which is why the shadow-AI radar exists as a separate control with its own operator-supplied evidence rather than as a claim that the gateway sees everything. Four of the radar's five sources are operator-fed, so it sees what you give it — plus, on the two streams a shipped pull connector covers, whatever that connector reads back once you have enabled it and supplied the credential, and no more.
An approval queue nobody reads is a rubber stamp, and no product fixes that.
Approvals are bound to a payload hash, single-use and expiring, and every decision records the approver, the time and their written rationale — but an approver who skims the action summary has approved what they were shown. The mechanisms that keep the queue readable are process controls: gates are scoped by risk rather than universal, every policy can run in shadow mode first so the false-positive rate is known before real work is held, and an install can refuse the move into enforcement until a backtest of that exact rule has been acknowledged by a named person. None of that is a guarantee that the queue gets read.
There is no independently witnessed bake-off and no reference production deployment.
No independent comparative test or reference production customer can be cited. The cloud pilot’s source tests, public guards and signed-out checks do not establish successful customer signup, provider collection or production outcomes. A customer-hosted evaluation remains an optional separate path with licence-scoped testing rights on infrastructure you control.
Governance is inline, so Token Observe is a single point of failure in the agent request path. If it is unavailable, the fleet stops.
That is a design decision, the same one that makes it fail closed: the governance evaluator returns allow only when every gate passes, and missing, unresolvable or erroring state denies the request, because a control you can bypass by turning it off is not a control. The consequence is that Token Observe's own availability becomes a governance property rather than only an operations one — which is why timeouts, provider circuit breakers and restore-tested backups are treated here as security work, and why a deployment that is serving traffic happily but has stopped recording it is the highest severity in the support model rather than a middling one. Read it beside the two entries above, because they compound: there is no availability service level to hold anyone to when it happens, and there is no second node to fail over to.
This paragraph is the short version of that list. Token Observe is wrong for you today if you need an availability service level, a SOC 2 report or an ISO certificate in the file before you can sign, or more than one node — and it is wrong for you this month if what you need is a tool that discovers subscriptions nobody exported, one blended number for the whole AI bill, a per-request ceiling that bites on a subscription seat, a way to read what staff typed into a browser chat window, or a switch that cancels the vendor's billing rather than stopping the agents. Every one of those is above, at its own anchor, with the reason it is that way. We will say so on the first call.
What each capability does not do.
The list above bounds the product. This one bounds each part of it, capability by capability, in the words of the page that capability belongs to. These are the lists those pages used to publish inline, moved here in full and rendered from the same modules so the two cannot drift apart.
Read it as a scope statement, not a feature list. The product-wide limits decide whether Token Observe is the right kind of thing; these decide whether a particular control reaches a particular clause, which is the question that stalls a review. Each block names the capability, links to its page, and lists what that capability does not do — and every line of it is lifted live from the capability's own module, so a limit cannot be softened on a product page without changing here in the same commit.
The grouping is the product's three pillars, in the order the platform pages use. Some of these restate a product-wide limit in the words of the capability it bounds, on purpose: a reviewer assessing the shadow-AI radar should not have to already know that the entry they need is filed above under coverage.
Spend
Track connected provider-account costs and the AI subscriptions and API accounts you register.
AI spend and subscriptions
A subscription that is on no invoice you give it
- It discovers no subscriptions on its own. An import finds what the invoice you gave it lists; a service nobody has invoiced stays invisible to it.
- The cloud pilot connects OpenAI, Anthropic, xAI and configured Google Cloud Billing reporting sources. Reported costs preserve their period and source status and are separate from imported invoices and gateway estimates; the adapters do not cover every subscription plan. Real customer billing accounts and permissions remain unqualified in the product record.
- A finance system's own export is not loaded as it stands. The parser refuses an unrecognised column and a missing required one, so the charge lines are mapped onto the template first.
- It never converts currencies, so there is no consolidated cross-currency total to report to a board.
- It cannot recognise two differently labelled copies of the same real-world bill as one charge.
- It will not allocate one invoice across teams. That is a reviewed decision a person makes.
- Registering a subscription changes nothing at the vendor: no cancellation, no payment-plan change, no control on an endpoint.
- The invoice and register ledger sits outside subject erasure, which clears trace evidence rather than billing records. That ledger and its backups belong in your retention scope as a separate item.
- Estate reporting is not a complete claim about the business. An invoice, a network observation or a missing trace cannot establish comprehensive coverage, and the reports say so.
- Not equivalent to an inline network gateway on an unmanaged device. The seat reads, the provisioning payload and the census carry lifecycle preview and productionEligible false, and active seats keep the onboarding readiness gate red.
- Expiry and freshness are bounded by the device’s own clock, with no monotonic anchor and no high-water mark. On an unmanaged device the governed party can wind it back, resurrecting an expired bundle and reviving a lapsed approval. The product records this as the sharpest edge here.
- Approvals cannot be single-use offline. Whether one has been consumed is frozen at issuance, so an approval can be spent twice inside a bundle’s freshness window, and an older bundle that is still fresh still verifies.
- No device attestation and no signed endpoint artefact. The hostname is self-reported, the hook ships as a Node command rather than a code-signed native binary, and the unsigned spool lets a root user delete a record before collection. That append is synchronous, so pathological storage can delay an already-computed refusal.
- Token Observe cannot revoke a subscription. Retiring a seat stops it reaching Token Observe and stops bundles being issued; the vendor admin console is what ends the entitlement.
- The hook never redacts. It can refuse a call carrying PII but cannot rewrite a payload on its way out, so a redact policy in enforce mode becomes a refusal on the device.
- Managed-settings artifacts are generated for Claude Code, Codex CLI and Copilot CLI only, from published documentation, and none has been exercised against a running vendor client. Gemini CLI and Cursor register with nothing generated, a Copilot developer working in the IDE is not covered by a CLI mechanism, and Claude Code’s managed sources do not merge: it applies the first that delivers a non-empty configuration.
- A post-tool refusal cannot always withhold the result. On Claude Code it can only stop the loop: the original stays in the transcript and may reappear on resume, recorded as a terminal stop rather than as withheld. Codex exposes no dependable failure event.
- There is no team-level or fleet-level budget pool. Ceilings are enforced per governed subject; the team and fleet figures are the sum of the per-agent ceilings that exist, published beside a count of the agents that have none.
- Nothing here is reconciled against a vendor invoice. Every figure is metered from the provider’s own reported usage against the price rows you hold (OpenRouter, which reports an authoritative charge, is the exception), and a shipped row is a list price captured on a date that will go stale.
- A hard-budgeted call forgoes retry and failover after its one potentially billable egress. That is a deliberate availability cost in exchange for a spend boundary, and an agent with no USD ceiling does not pay it.
- The retrospective savings report is degraded and should not be quoted. The automatic prompt excerpt it was built to read has been removed from the gateway, so it judges a call by token counts and whether tools were offered, and every result comes back at low confidence until an excerpt path exists again.
- Even at full strength that report could not tell you a cheaper model would have answered acceptably. It models the cheaper model at the actual token counts, though a different model would tokenise differently and may be more or less verbose, and it prices savings at today’s rates rather than the rates in force when the call was made.
- The savings report has no dashboard screen. It is an API endpoint over a bounded trace scan, so reading it today means calling it or wiring it into your own reporting.
- The kill switch is an admission control. It refuses new work rather than recalling a request already dispatched upstream.
- A trace store that cannot make the USD admission atomic has its budgeted traffic refused outright rather than quietly downgraded to completed-spend accounting.
- Token Observe cannot bound spend it never sees. A vendor credential used directly, outside the gateway and outside a managed seat, is a discovery problem before it is a budget one.
Activity
Follow requests routed through your workspace. Bounded, redacted text capture is off by default.
- It does not record hidden chain of thought. No provider exposes it, and encrypted provider reasoning state is not readable evidence.
- It is not on by default. Metadata capture keeps no message text at all, and redacted is a decision somebody has to make.
- Redaction is heuristic. The detectors find known personal-data and credential shapes and cannot identify every confidential business fact in ordinary prose.
- Related-turn links are an exact agent and session correlation, not a reconstructed thread, because a reused transport session can span unrelated tasks.
- Captured evidence is bounded, so a long exchange is retained in part, with the omission recorded rather than hidden.
- Provider-native summary paths, signed-state continuation and approved client integrations are work in progress, not shipped behaviour.
- No conversation record is a complete claim about the estate. A missing trace cannot prove comprehensive coverage of what AI was asked.
- A trace is not a transcript. At the default capture mode no prompt or reply text is kept; under ACP_CONVERSATION_CAPTURE=redacted both are kept, mask-redacted before truncation and bounded.
- Hidden chain of thought is not available from any provider, so it is neither recorded nor claimed. A provider-returned reasoning summary is captured, and labelled.
- The search cannot aggregate, group or correlate. Which agents used the same card number twice is not expressible, and a new question shape is a code change.
- The interpreted filter is not guaranteed to mean what you asked, and the explanation shown beside the results is advisory rather than a proof.
- The keyword fallback is materially worse and degrades quietly. Phrasing it does not recognise becomes an over-broad full-text term rather than an error, so it returns too much.
- Evidence bundles are digest-sealed, not signed, and capped. The digest detects an edit after issue but says nothing about who issued the file, and the approvals cap sets no truncation flag.
- Erasure and retention act on the live primary only, and reach neither approvals, radar findings nor webhook deliveries. A restored backup can resurrect what was erased.
- No Merkle tree and no inclusion proofs. An anchor covers a head, so a verifiable subset cannot be handed over without the rest of the log.
- No trusted timestamp. An anchor's creation time is inside the signed bytes but asserted by the signer. Only a timestamp authority proves when.
- No defence against host compromise. Whoever reads the audit MAC key or the anchor key out of the host can produce a history that verifies.
- Compliance exports are digest-sealed, not signed. The digest detects an edit after issue and attributes nothing; the signature is on the anchors.
- The offline verifier checks anchors, not today's chain: once it has grown past its newest anchor, the live-chain check is reported as not performed.
- No certifications: no SOC 2 report, no ISO 27001 certificate, no independent penetration test. This is a control that evidences clauses.
- Detected corruption latches the install. A content mismatch or a sequence gap fails readiness and later governed requests with a 503 until restart, because nothing is appended onto a chain proved unsafe.
- No after-the-fact redaction of an audit entry. Editing one breaks verification from that sequence onward, so it is the wrong place for anything you may later have to erase. A trace purge leaves verification passing: traces sit in another table.
- The data-handling flags on a provider are operator-asserted. Nothing checks that a row marked zero-data-retention has an agreement behind it.
- Failure classification is lossy. A 5xx that is really a refusal is failed over (the laundering this design exists to prevent), so the control depends on upstream error hygiene.
- Region matching is string equality with no hierarchy: an agent requiring eu will not match a provider tagged eu-west-1, and EU or UK cannot be expressed.
- There is no mid-stream failover. It applies to opening the stream; once bytes reach the client a retry would emit the opening of the answer twice, so the stream ends.
- Embeddings are proxied to OpenAI and OpenRouter rows only. Anthropic, Google, Bedrock and Azure targets fail closed before the credential resolves.
- Routing is not one-directional. A rule an operator writes can point a request at a costlier target, and no check compares the prices. Only the optional tier optimiser goes strictly downward.
Access
Record tool approvals and govern permissions and budgets for agents you connect.
Approved and unauthorised AI
That a particular person's subscription is authorised
- It stops nobody using a tool. Marking a service prohibited records the decision and classifies later evidence; it enforces nothing at the vendor or on an endpoint.
- It cancels no subscriptions, changes no payment plans and reclaims no seats. Nothing here reaches into a vendor account.
- A hostname is not a verified employee identity, so observed traffic cannot establish that a particular person's use is authorised or unauthorised.
- Identity mapping to real people, wider vendor coverage and the network collection rollout are remaining work rather than shipped behaviour.
- Approval is the current decision only. It makes no statement that use before the decision was permitted.
- Being outside the gateway is a coverage fact, never on its own evidence of a policy violation.
- Suppression is one detector, not the radar. Only the per-user seat-spend detector reads the register (billing reconciliation, network egress, the service-account key audit and IDE telemetry do not), so approving a subscription does not quieten those.
- No report here is a complete claim about the estate. A network observation cannot prove comprehensive coverage, and an absence of findings is not an assurance.
- No standing broad credential into your stack: your exporters push inwards on a credential this deployment issued and can revoke, and a customer-hosted runtime has no vendor phone-home. Cloud hosting holds workspace data and supplied credentials. Two scheduled pull connectors are the exception (GitHub Copilot seat spend, Cisco Umbrella activity), each on a narrow read-scoped credential you supply and can withhold.
- Four of the five sources are pure functions of rows you export. A finding is only as current as that export.
- The radar detects and reports; it blocks nothing. It cannot stop a workstation talking to a vendor API, and cannot tell you where a workload the gateway refused went next.
- Egress detection matches a fixed list of vendor API hostnames plus regional Bedrock endpoints; IDE detection recognises six named coding agents. Anything outside those lists is not detected, and adding one is a code change.
- Per-provider billing attribution exists only where you have declared which invoice accounts belong to which provider. Nothing is name-matched or attributed by elimination.
- A silent endpoint seat cannot be told from a person on leave, and the finding says so.
- Neither shipped pull connector has been run against a live vendor tenant. Query parameters, pagination, paths and field names come from recorded fixtures and published vendor documentation; a renamed field lands as a typed schema error rather than as silence.
- Token Observe does not discover your agents. Registering one is a deliberate act by a named person; anything calling a model without a record here is the shadow-AI radar’s problem.
- There is no recertification notification scheduler, and an overdue agent is never suspended automatically. That needs a per-install grace period, an escalation owner and a dry-run path first.
- A recertification is not a signature. It is a hash record bound to a keyed audit entry, so on an install with no audit HMAC key configured it evidences internal consistency and nothing about a determined database writer.
- The licensed agent ceiling is not enforcement. Deleting the licence file returns the install to an unlimited fallback with every control still running; the ceiling produces an audited record of an install operating past its terms.
- Risk tier is recorded and reviewed but is not a policy-scope dimension. Policies match on agent id, team and tag, so a rule intended for high-risk agents is written against a tag you maintain.
- The audit diff names fourteen fields. A change to routing or to the on-behalf-of requirement is audited as an edit but is not named in that list; the recertification digest, which does cover both, is what catches it.
- There is no role inheritance. Roles are flat permission lists, and a superset role has to restate what it supersets.
- There are no time-bound or just-in-time grants. A permission stands until somebody edits the role, and nothing expires on its own.
- Permissions do not read arguments. Refunds over £200 need approval is a policy, not a permission: this layer knows the tool, not the amount.
- Only one verb is issued. Everything is evaluated as invoke; the action field is matched and the wildcard honoured, but a finer verb set is reserved.
- The on-behalf-of header carries no signed claim. It narrows and never grants, and an agent that omits it skips the mask unless configured to require a principal.
- Group claims are a snapshot from the person’s last sign-in, not a live directory read, and are refused once older than the window (24 hours by default). Entra ID stops emitting the claim past roughly 200 groups and the directory link is not followed, so those requests are refused. Okta needs the groups scope granted. Google Workspace emits no group claim at all, so the intersection is unavailable there.
- Injection detection is nine fixed weighted patterns, not a classifier. A novel phrasing matching none of them scores zero, and a rule with a minimum confidence never fires on it. Text past a 64 KB cap is not scanned.
- The eleven data classes are the detectors written into the code. A policy chooses which to match on; it cannot add a twelfth, and no free-text trigger blocks a prompt for a phrase.
- Redaction rewrites text. It does not rewrite a JSON object key or the pixels of an image: those requests are refused rather than changing an executable contract or claiming to have masked something unreachable.
- A backtest cannot replay tool-call arguments: the governance feature vector it reads carries no argument values, so a rule with argument conditions reports coverage none rather than zero impact. Spend and rate rules never score better than partial.
- Shadow mode records that a rule matched, not the payload it would have produced: a shadow redact rule is skipped before any plan is built. It also only sees traffic arriving while it runs, so a rule staged on a Tuesday says nothing about month-end.
- None of this governs an agent that does not route through Token Observe. A call made straight to a provider is not sanitised, scanned or policed; finding it is the radar’s job.
- Token Observe cannot call an agent back. Approving unblocks nothing on its own, and an agent that never retries leaves the action undone with an approval nobody spent.
- There is no approver routing. Any operator whose evidence scope covers the trace’s team can decide any approval in it: no per-policy approver list, no escalation path, no two-person rule. The mandatory written reason is a console rule: the decision endpoint takes it as optional, so an install driving decisions through the API enforces that itself.
- The reviewer reads a summary, not the payload. It is the tool or model name plus the policy’s own reason, mask-redacted and capped at 240 characters, beside sixteen characters of the payload fingerprint; no model-proposed arguments reach the approvals table or the approval.requested webhook on any path. The linked trace holds what was recorded of the request rather than the request: tool arguments through the tool gateway, and prompt text only where conversation capture has been switched from its metadata default to redacted.
- Offline approvals on a developer seat cannot be single-use. A signed bundle records consumption as of the moment it was issued and nothing on the device can change that, so within one bundle’s freshness window a granted approval can be spent twice.
- The gate binds only the calls Token Observe sees. Tool rules are enforced on the tool gateway and on tool calls a model proposes; an agent executing a tool without routing it here is a matter for the shadow-AI radar.
- Token Observe cannot refuse a tool call that never reaches it. Evaluating the calls a model proposes extends the reach to agents that execute tools themselves, but that is defence in depth rather than a guarantee.
- An unpinned tool is not blocked. Refusing every unreviewed tool would make the gateway unadoptable, so an unreviewed tool stays callable and is described to the model as unverified.
- A quarantine screen cannot show you a diff. Only the hash of the approved descriptor is stored, not its text, so what changed cannot be reconstructed; the console shows the descriptor as it reads now and says so.
- Non-text tool output is withheld, not inspected. Images, audio, blobs and fetched resources are refused after the tool has already run, because there is no bounded media decoding or OCR on this path. A result whose JSON keys carry something the redaction plan covers is withheld rather than repaired, for the same reason a key is never rewritten on the way out.
- Injection scoring on a tool result is nine fixed weighted patterns with a 1.25× multiplier, not a classifier. A novel phrasing that matches none of them scores zero.
- Trace event content is capped and summarised past roughly sixteen thousand characters. One enormous payload is recorded as evidence that it happened rather than kept whole.
- No distributed exactly-once. Token Observe claims at most one dispatch from its own side; whether the action happened exactly once also depends on the downstream honouring the key it is sent, and that half is outside the boundary.
- A different pinned verifier is not an independent authority; it is separation of contract role. Nothing proves a different organisation or an attested system controlled the observation, and putting verifier evidence behind separately controlled infrastructure remains follow-on work.
- Nothing is reversible unless you built the reverse. An external effect is reversible only where its contract carries a tested compensation and compensation verifier; without that pair a failed effect is a manual review and a forward fix.
- Receipt-key custody is unproved, and with no signing key configured there is no receipt at all; the run API reports that absence rather than promoting an HMAC to a trust root. Verification accepts a public key the caller already trusts; provisioning, rotation, retention and independent custody of that key are deployment duties this product does not evidence. The anchor block inside a receipt is a signed reference, not an offline inclusion proof.
- Multi-replica concurrency is not qualified. Locked migrations, fenced stage leases and a leader-elected recovery worker exist and carry test coverage, but the known-issues register is explicit that this is not retained evidence of real replicas racing, losing leadership and recovering against a live downstream system. That evidence remains a release gate.
Residual risks, and the role that has to accept each one.
Nine residual risks are recorded with the role whose written acceptance is required, and none of them is evidence of approval until that role records a dated acceptance in the evidence pack. If your business has no CISO and no data protection officer, all nine land on you. Anything not on that list and not mitigated elsewhere is an unrecorded gap, which is treated as a finding rather than as an absence.
The defect list is published for the same reason, and it is unusually specific: it names the file, states the concrete cost, describes the attack that works, and records which findings were refuted on verification as well as which were confirmed. Most vendors will not share that under NDA. Publishing it is a deliberate trade — it hands an attacker a starting point, and it also means a buyer’s security review is reading the same list the engineering team is working from, so the review can be about the substance rather than about what has been left out. The corollary: if you find something that is not on that list, the maintainers do not know about it, and they want it.
The same discipline is enforced against the documentation. A documentation claim that materially overstates a security property and cannot be substantiated in the code is treated as a security defect rather than a typo, and three such gaps have already been found and corrected. That rule is why the limits on this page sit beside the claims in the same register: a sentence here that the source cannot support is a defect against the same list. It is also why an earlier description of an internal multi-pass review as “independent” was withdrawn rather than left standing — only reproducible tests and linked implementation count as evidence, and independent assurance remains an external gate that has not yet been passed.
Four items are expected to be accepted in writing before a deployment starts, and only one of them is on the list below. That one is the single-node SQLite topology. The other three are absences rather than mechanisms — no vendor-operated service level, no independent certification, and the preview limitations on endpoint-seat governance — so they are recorded under what is not claimed instead, and a reviewer looking for them on the residual list will not find them there. The role beside each of the nine is the one the product's own threat model names as best placed to weigh it, and it is printed rather than translated: at a business where the head of IT is also the CISO, the data protection officer and the person who signs for product risk, the column is a checklist of the four hats you are wearing on the day you sign.
Heuristic injection detection has false negatives.
Injection scoring is regex-based, and paraphrase, translation or encoding defeats it. The proposed rationale for accepting it is that blocking on a classifier with a meaningful false-positive rate would break legitimate traffic on the hot path, and that injection findings are one input to policy rather than the only control — RBAC, tool scoping and approvals bound the damage a successful injection can do.
Regex personal-data detection has both false positives and false negatives.
Checksum-validated kinds (card, IBAN, NHS number) score 0.9 to 0.98; national insurance and Social Security numbers score 0.85 and phone scores 0.7; free-text personal data is not detected at all. The proposed rationale is that Token Observe is a compensating control rather than your only DLP, and that confidence scores are exposed so policies can set their own thresholds.
The audit chain is tamper-evident, not tamper-proof.
Off-box Ed25519 anchoring is built but off by default, and WORM export, RFC 3161 timestamping and a Merkle tree for partial-log proofs are not built. An anchor buys exactly one sentence and no more: any copy you kept off-box beats any rewrite made after you took it. Four things it does not close, and no signature can close them: key theft, because whoever holds the private half signs any history they like; pre-anchor history, because anchor one says nothing about whether the entries beneath it were already honest; sink collusion, because suppression is only detectable if a copy exists somewhere the install cannot reach; and time, because the creation instant is asserted by the signer and only a timestamp authority or a public ledger proves when. The proposed rationale is that detecting operator tampering already exceeds the append-only-table baseline, and that the remaining gap is closed by key custody, a control that belongs to your KMS, not to the product.
The on-behalf-of header is unauthenticated.
There is no signed actor or subject claim, and an agent that omits the header skips the mask entirely unless that agent is configured to require it. The proposed rationale is that the mask can only narrow authority, never grant it (a forged principal buys an attacker strictly less than sending no header at all), and that the agent’s own RBAC decision is computed before any principal field is read.
Identity-provider group claims are a snapshot, not a live directory read.
Token Observe holds no refresh token and requests no offline scope, so a revoked group keeps granting authority until the person next signs in or the capture ages past the claims window, 24 hours by default. An Entra group overage sends a Graph link instead of the groups, which is not followed, so those principals capture zero groups and are denied. The proposed rationale is that following a directory API would mean a new credential, a new egress host, a directory-wide read permission and a network dependency inside a login (all wrong for a product that ships air-gapped), and that both failure directions are closed rather than open.
Provider data-policy flags are unverified operator assertions.
Marking a provider as zero-data-retention, no-training or region-pinned records your assertion about your contract with that provider; Token Observe cannot verify it, and enforces routing on the assertion. The proposed rationale is that the assertion is itself auditable (it appears in the audit log with the actor who set it), and that automated verification of a contractual term is not achievable.
SQLite is a single writer on a single host.
Sustained write load degrades governance latency, and there is no replica. The proposed rationale for accepting it at the current target scale is that the swap remains mechanical behind the store ports.
There is no MFA on local control-plane accounts, and SCIM is an optional bounded Users push rather than a live directory.
For optional customer hosting, single sign-on supplies your identity provider’s MFA, and configured SCIM supplies username, name and active-state lifecycle with immediate session revocation. An install that configures neither retains local-password and manual-leaver risk, and Groups, hard delete and live directory reads remain absent. Acceptance is conditional on three things: TLS terminating in front of the deployment, local login being disabled after single-sign-on rollout, and the SCIM token staying in a secret manager when that realm is enabled.
Agent credentials are long-lived bearer tokens with manual rotation.
Holding one is being that agent, and a token is valid until it is revoked or expires; it is not workload-bound. The proposed rationale is that workload identity such as SPIFFE cannot be presented by the agent frameworks Token Observe must support today, and that revocation, optional expiry and last-used timestamps compensate.
Cloud availability and recovery remain pilot qualifications.
No cloud availability service level or credits are published. The software-support draft below is not an agreed hosted service commitment.
There is no availability SLA, no uptime percentage and no service credits. The cloud pilot has one replica, shared resource and traffic quotas, and no completed live customer capacity or recovery qualification. For customer-hosted installations, three additional limits explain this position. First, Tenhaw does not operate your deployment: you control the host, the network, the TLS terminator, the disk and the restart policy, and an uptime number from a party with access to none of those, receiving no telemetry from them, would be unmeasurable by either side. Second, no retained partner-shaped sustained-load result exists. A bounded mixed-journey soak harness exercises allow, block, approval and read paths, and the older laboratory throughput probe establishes an order of magnitude for a loopback mock-provider path, but the evidence a commitment would need is a retained multi-hour run against your corpus, your concurrency, your policies and your provider latency, with an agreed degradation curve. Third, recovery automation exists while partner RPO and RTO proof does not: the release workflow boots the packaged image, takes and restores a live backup and checks the restored chain against externally retained values, and recovery is SQLite full-snapshot restore rather than point-in-time recovery, with backup scheduling, off-box placement, retention and freshness alerting all operator-owned. Two consequences matter to anyone assessing this risk: Token Observe is deliberately in the request path and fails closed, so if it is down, governed agents cannot call models and it is a single-writer process on one node by design at this scale, with no replica, no clustering and no high-availability story. An availability or performance commitment becomes possible once the partner-shaped load and recovery results are retained and approved. Until then no number is offered, because nobody could stand behind one. Ask for the date; it is a fair thing to put in an agreement.
Security research is permitted, and so is publishing it.
Report a vulnerability by email to security@tenhaw.com, or through your named escalation contact if you are a design partner. Do not open a public issue, pull request or discussion for a security report. The product's SECURITY.md also names a GitHub private security advisory as a channel; GitHub offers private reporting only on public repositories, and the product repository is private, so email is the route that works. A human confirms receipt within two business days and names a point of contact; it is not an automated reply. An advisory is published within seven days of a fix being released, with wording and timing coordinated with you, and the default disclosure deadline is 90 days from the triage verdict, whether or not a fix has shipped. A CVE is requested for Critical and High issues. There is no bug bounty and no monetary reward, which is a resourcing statement rather than a judgement of the work. Severity starts from CVSS v3.1 and is then adjusted for the context this product runs in: a flaw that defeats the governance decision or corrupts the evidence record is rated above what its base score alone would suggest, because those are the two things the product exists to provide.
Test it, and publish what you find
If you make a good-faith effort to comply with the policy, Tenhaw will not pursue or support legal action against you in relation to your research, and will treat your activity as authorised under the Computer Misuse Act 1990 and equivalent laws. Good faith means you test only against a deployment you own or are permitted to test, you do not access, modify or retain anyone else’s data, you stop as soon as you have established that a vulnerability exists, you do not degrade a live service, and you give a reasonable opportunity to fix the issue before disclosing it. Section 9 of the licence explicitly permits security testing and the publication of results, on terms that differ by kind: benchmark results need no pre-approval provided the publication names the version and configuration tested, and assessment findings may name the software after the coordinated-disclosure process above. There is no clause that suppresses a finding. Two scope notes before you start. A new path that evades or weakens the keyed audit epochs, the boot checkpoints, the signed anchors or the off-box head comparison is expressly in scope, because the database writer is part of the evidence-integrity threat model. And algorithmic denial of service (a single small request that consumes disproportionate CPU) is in scope and has been found here before, while denial of service by volume is not.
Reports go to security@tenhaw.com, which is the address published in the product’s own security policy and is read by a person rather than a queue.
Commercial source-available
Source-available, not open source. A product that calls itself open source and then forbids redistribution has told its most technical reader something untrue in the first sentence.
- Use, modify and self-host under a licence
- Read the entire governance domain before buying: the core package is pure functions with zero runtime dependencies
- Inspect, test, fuzz, penetration-test and reverse-engineer your own deployment, or commission a third party to do it for you
- Publish benchmark results with no pre-approval, provided the version and configuration tested are named; publish security findings after coordinated disclosure
- Redistribution
- Offering Token Observe as a competing hosted service
The published customer-hosted licence is a draft requiring legal review. Your signed agreement sets the final terms.
The three published reasons there is no availability service level are above; the support severities and the response targets behind them are on the pricing page, and the framework mappings with a stated coverage level on every row are on the compliance page.
What does the Token Observe vendor receive from a running deployment?
Cloud hosting receives workspace and account data, encrypted connection credentials, provider-cost evidence and the activity actually routed or imported into the workspace. Optional customer-hosted runtime sends Tenhaw no product telemetry or licence callback. Voluntary sales/support information is another data flow. The cloud custody boundary is described above.
Do we need a data processing agreement or sub-processor terms?
Cloud hosting requires a different data-processing assessment from supplying customer-hosted software: Token Observe holds workspace data and encrypted provider connection credentials, with Supabase Auth and Railway infrastructure involved. Data-processing agreements, legal roles, supplier terms and transfers remain under review. Customer hosting does not automatically transfer runtime data to Tenhaw, but support access or voluntary disclosure can create another arrangement.
Is the audit chain tamper-proof?
No. It is tamper-evident. On the optional customer-hosted unkeyed default, an operator with write access to the database file can rewrite an entry, recompute every downstream hash, and have verification report valid: true. A test does this on a default install, deliberately, so the limit is a tested fact. Configuring an audit HMAC key from a secret manager the database administrator cannot read makes the digests MACs and seals the head at every boot, which defeats a full recompute. Configuring an Ed25519 anchor key adds one further sentence and only one: any copy of the anchors you kept off-box beats any rewrite made after you took it. It does not say the history beneath the first anchor was honest, that the signer’s clock was truthful, or that someone holding the key could not have produced the same file.
Are you SOC 2 or ISO 27001 certified, and has this been penetration-tested?
No to all three, and none of them is implied anywhere. There is no SOC 2, no ISO 27001 and no ISO/IEC 42001 certification, and there has been no independent penetration test, red-team engagement or third-party code audit. The security work that has been done is internal: a five-dimension adversarial review with a three-verifier refutation panel per finding, a STRIDE threat model, targeted testing of specific attacks including a forged audit chain and adversarial ReDoS inputs, and the automated release gates. None of it is an external test. A pre-purchase penetration test by a prospective customer is welcome and expressly permitted by section 9 of the licence. Publishing what you find is permitted too, on terms that differ by kind: benchmark results with no pre-approval, provided you name the version and configuration tested, and security findings after coordinated disclosure.
Can our security team review and attack it before we buy?
For software on infrastructure you control, the licence permits this; it does not authorize attacking the shared cloud service. Any person may install and operate Token Observe for internal evaluation, security review and proof of concept for thirty days from first installation, provided it is not used for live production traffic or regulated personal data. Section 9 permits you, or a third party you commission, to inspect, test, fuzz, penetration-test and reverse-engineer it as deployed on infrastructure you control, to publish performance results naming the version and configuration with no pre-approval, and to publish security assessment findings naming the software after coordinated disclosure. The published defect list and threat model are the recommended starting point, so a paid test is not spent re-finding what is already known.
What is the retention position, and can you erase one person’s data?
Cloud activity retention is configured to 30 days; optional customer-hosted runtime leaves trace retention unset, meaning keep forever until its operator sets it. Subject erasure supports an audited dry run and removes matching traces, events, search-index entries and trace scores from the live primary. It does not erase audit records, approvals, radar findings, all account records or backups. A restored pre-erasure snapshot can resurrect deleted records. Trial expiry does not automatically delete the workspace.
Can it run air-gapped?
For optional customer hosting, a delivered image can run fully air-gapped, the dashboard bundles no CDN assets, and an air-gapped install never calls the model-price catalogue at all — prices are loaded through the API instead. Set the provider host allowlist to your own internal model endpoints and the credential-prefix allowlist to your own variable names, and egress is confined to endpoints you named. The build is a different question: building that image from source still requires access to the configured base image, the Debian archives and the npm registry. Runtime dependencies are kept deliberately few for this reason (every addition is supply-chain surface in a product that gets deployed air-gapped), and the governance domain itself has zero runtime dependencies.
What happens to our agent fleet if Token Observe is unavailable?
Requests routed through an unavailable gateway stop; traffic that bypasses it is outside its control. The cloud pilot has no published availability service level or credits, and actual recovery/customer capacity remain unqualified. Optional customer-hosted availability also depends on the operator’s deployment and backups.
Send the questionnaire, and we will tell you what is still open.
Most of a standard vendor security questionnaire is answered by this page, the compliance mappings and the published defect list. Anything left goes to security@tenhaw.com and gets a written answer.