Stop agents leaking personal data and secrets to model providers

The first attempt is almost always a redaction step inside the agent: a regular expression in a try block, run over the prompt before the SDK call. It works, in the sense that it catches card numbers in the demo, and it fails as a control for a structural reason rather than a technical one. It is enforced by the thing being governed, it changes whenever somebody redeploys, it exists in one of your seven agents because one team wrote it, and nobody outside that team can say whether it ran. When the compliance question arrives, the honest answer is that a check happened somewhere in a codebase, which is not an answer.

The second attempt is the data-loss tooling you already own, pointed at the egress traffic. That gets you inspection outside the agent, which is real progress, and it leaves two specific gaps. It reads the request body as one document, so it does not distinguish the prompt a person typed from a tool result somebody outside your organisation wrote into a ticket — and the second of those is where the interesting data arrives. And it usually has no answer for the response leg at all, which is where a credential comes back out of a model that was shown one earlier in the conversation.

The third attempt is the one that gets switched off again. Somebody turns on masking across the estate on a Tuesday, and by Thursday an agent is producing subtly wrong output because the rewrite changed what the prompt said. The failure that motivated the current detector ordering is exactly this: an IBAN pattern whose match ended on the trailing separator swallowed the space after the number, and the prompt reached the model reading “<IBAN_1>for the payout”. Nothing errored. The model simply read a different sentence from the one the agent composed, and the trace recorded a successful call.

The fourth is subtler and is the one that survives longest undetected. A payload is scanned before it is normalised, so the scanner is reading a different document from the one the model will read. The Unicode tag block at U+E0000 to U+E007F encodes a complete invisible ASCII alphabet, and a value written in it is invisible to a reviewer, invisible to a naive pattern and perfectly legible to the model. Ordering the normalisation before the detection is not tidiness; it is the whole of that attack.

Detection is regular expressions plus checksums, run over text that has already been normalised. Eleven kinds ship, and they are ordered so that more specific detectors claim their span first and later ones skip anything overlapping — which is why a card number is not also reported as a phone number. Each kind carries a confidence, and the checksum-validated kinds carry the high ones, because a twelve-to-nineteen digit run that passes Luhn is a card in a way that a nine-digit run matching a shape is not a national insurance number.

Four of the eleven are credential shapes rather than personal data, and Token Observe treats them differently at every point in the pipeline. They are masked irreversibly even under a rule that asked for tokenising, because a reversible placeholder for a live credential is a credential leak with extra steps, and they are in the response-leg mask set whether or not a policy names them. A defect found and fixed during this build was precisely that boundary: enabling card-number redaction had replaced the secret kinds in the egress plan rather than adding to them, so switching on one control switched off another.

The scan itself is inline, synchronous and running on text an attacker may control, which makes its cost a security property rather than a performance note. A JavaScript runtime cannot interrupt a running regular expression, so one pattern that backtracks super-linearly stalls every other request sharing the process. Two such patterns were found in Token Observe’s own injection scanner and are published in its defect list rather than left out of it — the reported one at 121 milliseconds on a 200 KB input, and a worse one found by auditing the rest, the markdown exfiltration pattern, at 51 seconds on the same input. Both were rewritten to be linear, a scan cap of 65,536 characters was added, and adversarial 200 to 400 KB inputs were re-measured at under 4 milliseconds each. Those figures come from one measurement exercise on one machine rather than a published benchmark; read them as the shape of the problem.

That cap is a stated trade rather than a subtlety. Text past 65,536 characters is not scanned, on the reasoning that anything worth detecting has to be read by the model to have an effect and is therefore near the start. A payload that hides a value at character 70,000 is not detected, and the layers below — grants, approvals, and what the application does with the answer — are what stand between it and a consequence.

The return leg is governed as well as the outbound one, and only the data-shaped rules run there — data class and injection. Re-running the whole set would fire a rate rule or an approval requirement a second time for one logical call, including an approval that had just been satisfied. On a buffered response the evaluation happens over the complete text. On a stream the plan has to be resolved before the first byte, because bytes already written cannot be recalled, and the hold-back buffer has to be deep enough that a value split across two chunks cannot escape half-masked.

The bound that makes streaming honest is a declared maximum for an unbroken value: 4,096 characters. Two of the detectors — the JSON web token and the prefixed key shape — state no maximum length in their patterns, so a scanner asked never to split a run would otherwise buffer a base64 blob without limit. A run longer than the bound is replaced wholesale and suppressed to its delimiter: Token Observe stops claiming to identify the exact kind past that point, and it never emits either half of the ambiguous value. A buffered scan sees the whole text at once and needs no such fallback.

What the flight recorder keeps afterwards is the part most reviews miss, and it follows the same rule in the same order. The prompt is stored as a post-redaction excerpt capped at 4,000 characters, read back off the outbound payload rather than off the original, so the search index cannot contain what the redactor has just removed. The model’s answer text is not stored at all — the response event records the stop reason, the upstream request id, the token counts split by cache bucket, the number of placeholders issued and the types of the content blocks. A trace is therefore evidence of what was decided and what it cost, and not a transcript you could replay.

One payload-adjacent exception is worth knowing before somebody finds it. Where a policy requires approval for a tool call the model proposed, the approval record’s summary is the tool name plus up to 160 characters of that proposal’s arguments, built before egress redaction has run on the response. So an approval record and an approval webhook can carry that much unredacted model-generated text, and both should be treated at the sensitivity of trace content. Approvals raised on the request path and on the tool gateway carry no arguments at all.