AI conversation evidence
What the AI was asked and what it returned, redacted before it is stored and bounded before it is kept.
Nobody can say what it was asked.
Something goes wrong with an AI-assisted process (a customer receives a reply that should never have gone out, a tool action fires against the wrong record), and the first question is always the same. What was it asked, and what did it say back? A trace showing a request at a timestamp with a token count establishes that something happened and nothing about what.
The content that would answer it is the most sensitive material the system touches. So capture is a per-deployment decision rather than a default, what is kept is redacted before storage and bounded before it is kept, and both the bounds and the redactions are visible in the record, so a reader knows what they are not seeing.
How it works
- 01
Default to metadata, opt in to text
Capture ships as metadata and stores no message text. Changing it to redacted governs gateway model conversation evidence and not all audit logging (MCP and OTLP ingestion are governed separately), so it widens nothing else. - 02
Redact, then truncate, in that order
Redaction runs first, independently of outbound policy: content is redacted for storage whether or not a policy fired on it. Truncate first and a credential in the last paragraph of a long prompt survives right up to the cut. - 03
Bound every event, and label its source
32,000 characters and 100 messages per event, both configurable, and what was omitted is recorded rather than dropped. Every message carries its source — gateway request, provider response or response cache — so a cached reply is distinguishable from a live one. - 04
Keep evidence for refused requests too
Requests blocked by policy or held for approval retain bounded evidence, because the stopped ones are the ones somebody reviews. Blocked model output is withheld: the record shows what was asked and that it was refused.
- Workspace configuration, registry, policies and decisions
- Billing evidence from the connections and imports you supply
- Supplied connection credentials, encrypted at rest
- Request metadata for traffic routed through the workspace
- Bounded, redacted prompt and reply text when capture is enabled
- Billing queries using the reporting permissions you supply
- Routed model payloads after the applicable policy checks
- Model-assisted search using a separately configured inference key
- Authorized evidence reads and CSV exports to your client
The cloud service processes and stores workspace evidence and supplied credentials. Supabase handles account authentication; Railway runs the cloud pilot. Cloud hosting does not keep all data inside your network.
Billing cannot recover historical conversations. Activity covers routed traffic or supported evidence supplied to the workspace. Text capture is off by default, and redaction is heuristic.
Customer hosting is an optional, separate deployment choice. Its infrastructure and data boundary differ from the cloud pilot.
No provider exposes hidden reasoning.
Reasoning models produce internal reasoning that is never returned to the caller. Some providers return a summary of it, some an encrypted state token only they can interpret, and none the reasoning itself.
Token Observe records what a provider exposes as text, and retains an exposed summary where one is supplied. Encrypted reasoning state is not readable content and is not evidence.
- Recorded
- Request and response text, model-proposed tool activity and provider-exposed summaries, under redaction and the configured bounds, once capture is set to redacted.
- Not recorded
- Hidden chain of thought, which no provider exposes, and encrypted provider reasoning state, which is not readable content.
Related turns are a correlation, not a thread.
Related turns are linked through an exact agent and session correlation. The conversation list filters directly on text, team, person, tool and outcome.
That correlation is exact, and it is deliberately not a reconstructed conversation. A transport session can be reused across unrelated tasks, so two turns sharing one are not necessarily two turns of the same work, and a trace with no session stands alone. Read a related-turn link as a strong lead, not an assembled transcript.
Sample data in the real Token Observe interface, recorded from a local evaluation with a temporary database, a mock provider and one request. No live provider key was used.
Redaction is not classification.
The detectors find personal data and credentials by pattern and checksum. They catch known shapes and cannot identify every confidential business fact in free text: a prompt describing an unannounced acquisition in ordinary prose contains nothing a detector recognises, and it will be stored. Redaction reduces the sensitivity of this store considerably; it does not make it a store of non-sensitive material.
Captured conversation inherits the trace retention, erasure and team access controls you already configured. Dashboard viewers read content only within their assigned evidence teams, and MCP access needs its own grant for trace content, separate from the spend and governance grants. Trace erasure reaches trace evidence and not the invoice and subscription ledger, which keeps its own retention scope.
Each line closes off an assumption a reader would otherwise carry into a proof of concept.
What this does not doWhat this leans on
Flight recorder
Every governed request in a timeline a compliance officer can read, and a search box that never writes SQL.
Approved and unauthorised AI
Approved, prohibited or unknown, so the subscriptions you already pay for stop arriving in the seat-spend report.
AI spend and subscriptions
What you were billed and what you were metered, recorded as two numbers and never added into one.
Where this comes up
Does Token Observe store our prompts by default?
No. ACP_CONVERSATION_CAPTURE ships as metadata, which keeps no message text at all; retaining request and response text is an explicit per-deployment choice. That setting governs gateway model conversation evidence rather than all audit logging, so enabling it does not widen the MCP or OTLP ingestion records. Leave it alone and the product records that a request happened without keeping what it said.
Can you see what the model was thinking?
No, and nobody can. Reasoning models do not return their internal reasoning: some providers return an exposed summary, some an encrypted state token only they can interpret, and none the reasoning itself. Token Observe retains text a provider exposes, including those summaries. Hidden chain of thought is not available, is not recorded, and is not claimed anywhere on this site.
If redaction runs, is the stored content safe to keep anywhere?
Treat it as sensitive regardless. Redaction runs before truncation on the personal-data and credential detectors, which removes the shapes they know. It is heuristic: a prompt describing an unannounced deal in ordinary prose matches no pattern and will be stored, inside the same retention and access controls as the rest of your evidence.
Prefer to ask a person? Write to James
Bring us the agent you are least comfortable with.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.