--- title: "Use case: Stop agents leaking personal data and secrets to model providers" canonical_url: "https://tokenobserve.com/use-cases/stop-agents-leaking-pii" published: 2026-09-02 last_updated: 2026-10-21 --- # Use case: Stop agents leaking personal data and secrets to model providers Decide it at the Token Observe gateway rather than inside the agent. Model-visible text is normalised to a fixpoint, eleven pattern-and-checksum detectors run over it, and a data-class policy blocks the call or rewrites the payload before it is routed. Where a rewrite would change meaning rather than sensitivity, the call is refused. ## The stated limit What the detector cannot see: Free-text personal data: a name, an address, a described condition ## A regular expression inside the agent is not a control. The first attempt fails structurally rather than technically: it is enforced by the thing being governed, it changes whenever somebody redeploys, and it exists in one of your seven agents because one team wrote it. Pointing your existing data-loss tooling at egress traffic leaves the response leg, which is where a credential comes back out. ## The steps, in this order 1. Route both legs through one endpoint. Change the base URL and credential on each agent. Until then a rule polices whichever half somebody remembered. 2. Stage the rule in shadow. It is evaluated exactly as an enforcing rule and then skipped, so you learn the false-positive rate before it stops anyone's work. 3. Choose mask or tokenise. Mask when the model has no use for the entity, tokenise when it must keep customers distinct. Credentials are always masked. 4. Cover the way back. Rules run again on the response leg: after the full response when buffered, before the first byte on a stream. ### Eleven kinds, and the confidence that comes with each. Detection is regular expressions plus checksums over text that has already been normalised, and normalising first is the whole of one attack. The Unicode tag block at U+E0000 to U+E007F encodes an invisible ASCII alphabet: legible to the model, invisible to a reviewer. The scan caps at 65,536 characters, so a value hidden past that is not detected. - Checksum-validated: credit_card by Luhn, iban by mod-97, uk_nhs_number by mod-11. A shape failing its checksum is not reported at all. - Pattern-only personal data: us_ssn, uk_nino, email and phone. Lower confidence by design, so a policy sets its own threshold. - Secret kinds: jwt, aws_access_key, api_key and private_key. Prefixed vendor shapes are enumerated rather than caught by entropy. - Not detected: Names, addresses, free-text descriptions, and any identifier outside the shipped UK and US shapes. ### Two rewrites are refused. A sensitive value in a JSON object key is refused, because renaming a key changes which argument the tool receives. Text found by OCR inside an image is refused, because a text redactor cannot reach pixels. The redaction event carries direction, mode, kinds and counts, and never the matched value, since recording it would move the leak into the flight recorder. ## What this still does not solve - Free-text personal data is not detected. A name, an address or an account narrative in prose matches no pattern, and neither does any identifier outside the shipped set. - What your application does with the returned text is outside the boundary. The moment your code renders it into HTML, no gateway control reaches it. - An agent that calls a provider directly is not redacted at all. Nothing inspects a payload that never arrives, which makes coverage a discovery question first. - Switching capture to redacted puts a second copy of the prompt and reply in your own evidence store, and cloud activity retention is configured to 30 days; customer-hosted trace retention is unset by default, which means keep forever. ## Questions and answers Q: Which personal data classes does Token Observe detect? A: Eleven kinds: credit card by Luhn, IBAN by mod-97, NHS number by mod-11, plus US social security numbers, UK national insurance numbers, emails and phone numbers by pattern, and four credential shapes: JSON web tokens, AWS access keys, prefixed vendor API keys and PEM private-key headers. Free-text personal data is not detected. Q: What is the difference between masking and tokenising? A: Masking replaces the value with an irreversible marker naming the kind it was. Tokenising replaces it with a placeholder stable for that value across one governed call, so a model can treat two customers as distinct without receiving either. Credentials are always masked irreversibly. Q: Is this enough to satisfy a data protection officer? A: It is a compensating control and should be described as one: an enforcement point outside the agent, a record of kinds and counts with the values never stored, and a refusal rather than a silent rewrite. It is not complete coverage of personal data. State the detected set, state the gap beside it, and set a retention window deliberately.