Produce audit evidence for AI agents that an auditor accepts
One bundle with a verification verdict inside it, and the protection level beside the verdict. The head of IT handed an evidence request with a date on it You assemble exports the week before the audit, and every one was produced by, or about, the party under examination. The auditor asks how you would know the file was not edited, and nothing in the folder answers it.
A folder of exports says nothing about itself.
Framework logs, a provider usage report and a spreadsheet of who approved what are all produced by, or about, the party under examination. An append-only table enforced by database permissions is better and still fails the threat model that matters, which has to include the operator. A restore drill that passes verification proves nothing, because lopping entries off the end leaves every remaining link valid.
4 steps, in this order.
Key the chain while it is short
Inject an audit MAC key from a secret manager your database administrators cannot read. The guarantee is forward-looking, so waiting costs a weaker prefix.
Anchor off the box
With an Ed25519 key set, the chain head is signed daily to a sink you do not administer. Any copy you retained beats any later rewrite.
Record the head out of band
Write the attested head sequence and entry count down outside the deployment, then pass them back to the verification endpoint after a restore.
Export, and read the truncation flags
Traces, events and audit entries each set a flag when their cap bites; the approvals cap sets none, so read that count against your period.
These steps produce a bundle for your auditor to review, including its verification result and export limits.
See what the chain guaranteesBuilt for
- An evidence request with a period on it, where somebody outside your organisation must accept the file.
- A deployment where you can inject an audit MAC key your database administrators cannot read.
- A reviewer who will read the protection level beside the verdict rather than stopping at the word valid.
Wrong for
- Anyone who needs a tamper-proof log. The word appears nowhere in the product, and unkeyed a rewrite that recomputes verifies clean.
- Anyone who needs a signed export. A bundle is digest-sealed, not signed, and whoever can rewrite it can recompute the digest.
- Anyone who needs to hand a regulator a verifiable subset. There is no Merkle tree and there are no inclusion proofs.
One thing on the record
Token Observe ships a forgery test asserting that a rewrite-and-recompute PASSES verification on a default install, and asserting that it fails once an audit MAC key is configured.
It is a test in the product repository rather than a sentence on a page, so neither half of the tamper-evidence claim can drift from the code.
Where this is publishedThe protection level is the field to read first.
A bare valid flag invites the reader to assume more than an unkeyed chain offers: it means nothing was altered without recomputing, not that nothing was altered. The seal is a SHA-256 over the canonical JSON of the body, not a signature. It confirms the file matches a digest given through another channel, and anyone who can rewrite the bundle can recompute it.
- Refusals are in it
- The trace opens at step 3, before sanitisation and before the verdict, so a request blocked a millisecond later is recorded rather than missing.
- Shadow decisions are in it
- Every rule that matched writes a decision event naming the policy, its mode and why it matched, so an observing rule leaves evidence it ran.
- The read is in it too
- Listing, searching, opening and exporting each append an audit entry, and the export entry carries the digest of the bundle it issued.
- Not in it
- Hidden model reasoning at every setting, training-data provenance, model cards, and anything about an agent that never routed through Token Observe. Prompt and reply text are absent only while capture is left at its metadata default; set ACP_CONVERSATION_CAPTURE=redacted and both are written as evidence, mask-redacted and bounded.
- beat 01Name the caller
- 1Authenticate
- 2Resolve the agent
Nothing is decided for an anonymous caller. The key resolves to one registered agent with a named owner, and its roles, its ceilings and any stop switch in force are read live from the store on every request, not from a cached copy, so a suspension takes effect on the next call instead of after a redeploy.
- beat 02Start the record, then read what is being sent
- 3Open the trace
- 4Sanitise
- 5Scan
The trace is opened before anything can refuse the request, which is why a blocked call is recorded as thoroughly as one that went through. Only then is the text normalised and scanned, in that order, so the detectors see what the model will read rather than what a human reviewer can see.
- beat 03Decide once
- 6Govern
- 6bIntersect with the named human
- 7Enact
One function returns one verdict: allow, block, or hold it for a named person to approve. Every check lives in that one place rather than scattered along the path, which is what makes it possible to read your rules and know what they do.
- beat 04Send it, and only then spend the money
- 8Route
- 9Call upstream
Routing fixes which provider and model will serve the call, so where an agent has a USD ceiling it is tested against the price you will pay, not the one the caller typed. Routing is also the last step that can still refuse locally: after it, the request has left your network.
- beat 05Govern the answer, then count the cost
- 10Govern the response
- 11Meter and record
A tool call the model proposes is judged before it reaches your agent. Metering runs last and on every ending, including the refusals and the streams a client abandoned halfway, because the tokens were spent either way and a partial trace is still evidence.
Step names and their grouping are the product’s own, and the order is encoded in the evaluator rather than drawn for the page.
Three layers of integrity, and what each moves the target to.
The bare hash chain is tamper-evidence and is the default, and a verification walk names which failure it found: a link mismatch, a content mismatch, or a sequence gap, including a missing prefix, because deleting entry 1 and recomputing from genesis leaves a chain that checks out link by link. Keying moves the target to whoever holds the key; anchoring moves it again, and refuses to sign when the chain does not verify.
4 things this still does not solve.
- There is no Merkle tree and no inclusion proofs, so a verifiable subset of the log means handing over the rest of it.
- There is no trusted timestamp. An anchor's creation time is inside the signed bytes, but only a timestamp authority proves when.
- Nothing survives host compromise. Anyone who reads the audit MAC key or the anchor private key can produce a history that verifies.
- An audit entry cannot be removed without breaking verification onward, so the governance-plane log is the wrong place for anything you may have to erase.
Check the product limits alongside the job you want to do.
What we do not claimThe parts that do the work.
Audit chain
Every administrative act hash-chained, and anchored with a signature your auditor can check alone.
Unkeyed, a rewrite that re-hashes everything verifies clean
Flight recorder
Every governed request in a timeline a compliance officer can read, and a search box that never writes SQL.
The filter cannot group, count or correlate across traces
Human approvals
One human decision, bound to one exact payload, spendable once.
An approval takes effect only when the agent retries
Guides that cover the same ground.
Questions
Is the audit log tamper-proof?
No, and Token Observe will not use the word. Unkeyed, which is the default, the chain is tamper-evident: any edit that does not also recompute every downstream hash breaks verification at a named sequence. An operator with write access can recompute it, and the repository's own forgery test asserts the forged chain passes.
Are compliance exports signed?
No. A bundle is digest-sealed with a SHA-256 over the canonical JSON of its body, and it carries the verification verdict, the entries checked and the protection level. That digest is not a signature. The Ed25519 signature lives on the anchors.
Is the prompt and reply text in the bundle?
That depends on one setting, and it belongs in your covering note. Under the default metadata capture mode no message text is stored at all. Set ACP_CONVERSATION_CAPTURE=redacted and both the prompt as sent and the assistant's reply are written as evidence, mask-redacted and bounded at 32,000 characters.
Describe the version of this you have.
Say what your estate contains and which part of it worries you, and the reply will say where to start.