Prompt injection defence

The version everybody has seen is a user typing ignore your previous instructions into a chat box. It is a good demonstration and it is not the thing that hijacks production agents, because the person typing it is a principal you can identify, rate-limit and hold accountable, and because the agent that reads it usually cannot do much anyway.

The version that causes incidents arrives in a tool result. An agent triaging support tickets reads the ticket body. An agent summarising a page reads whatever the page says. An agent querying a table reads rows that some other system wrote, and some of those rows came from a form on the internet. In every one of those cases the model receives attacker-authored text through a channel your architecture treats as data, and models do not reliably maintain the distinction between data and instruction. The attacker does not need access to your prompt. They need access to something your agent will read.

This reframes the defence problem in a useful way. The question is not how to stop a model from being persuaded — that is an open research problem and anyone who tells you they have solved it is selling something. The question is what the model can do once it has been persuaded, and how you would find out. An injected instruction that can only cause the agent to call a read-only tool it already had is an annoyance. The same instruction reaching an agent that holds a refund tool, or an agent whose output is rendered as HTML into somebody’s browser, is an incident.

Two less obvious surfaces belong in scope for the same reason. The first is the tool definition: a tool’s name, description and input schema are part of the model’s instruction surface, so an upstream tool server that quietly rewrites a description can steer an agent without changing a line of your code. The second is any passthrough field a gateway forwards without reading — a prompt smuggled into an unrecognised top-level field is read by the model exactly like one in the messages array, and a scanner that inspects only the fields it recognises has left a hole shaped like the fields it does not.

Token Observe scores injection with nine weighted patterns run over the sanitised text. The weights are summed, multiplied by 1.25 when the fragment came from a tool result, and capped at 1. That multiplier is the entire point of the design: a directive inside a tool result is more suspicious than the same words typed by a user, because a tool result is data rather than a principal. A tool-result directive weighs 0.4 on its own and 0.5 once the multiplier applies, so a rule set at a minimum confidence of 0.5 fires on the ticket body and not on the person typing into your support console.

The highest score across fragments is taken rather than the average, and each fragment is scanned under its own source. Averaging would dilute exactly the signal the multiplier exists to raise: one hostile paragraph inside a large legitimate payload is the case that matters, and it is the case an average erases.

The first way this fails is the obvious one and it should be stated before the feature list rather than after it. Nine fixed patterns are not a classifier and not a model. A phrasing nobody wrote a pattern for scores zero, and a rule with a minimum confidence never fires on it. An attacker iterating against a deployed detector will find a phrasing that scores zero, because they get unlimited attempts and the patterns do not change between attempts. Treat the score as one signal among several, stage injection rules in observation mode first, and read the heuristic names recorded on the trace when one does fire.

The second way is less obvious and it is a real availability risk in any implementation. This scan runs inline, synchronously, on attacker-controlled text, and a JavaScript runtime cannot interrupt a running regular expression: one pattern that backtracks super-linearly stalls every other request sharing the process until it finishes. Two such patterns were found in Token Observe’s own scanner and are written up in its published defect list rather than left out of it. The reported one, the large-base64 heuristic, took 121 milliseconds on a 200 KB input; auditing the rest found a worse one — the markdown exfiltration pattern — at 51 seconds on the same input, which is a single-request denial of service. Both were rewritten to be linear, the remaining patterns were measured and judged safe, a scan cap of 65,536 characters was added, and adversarial 200 to 400 KB inputs were independently re-measured at under 4 milliseconds each. Those figures come from that one measurement exercise on that hardware, not from a published benchmark, so read them as the shape of the problem rather than as numbers to quote back. If you write your own patterns, the constraint to enforce is that each must be linear in the length of its input: no variable-width span that can run past its own delimiter to end of input, and no two variable-width spans separated only by an optional literal.

The cap — 65,536 characters per scan, about 64 KB of plain ASCII — is a deliberate trade with a stated cost. An injection payload has to be read by the model to work, so anything worth detecting is already near the start; past the cap, a caller is only buying scan time charged to every other request on the process. Text beyond it is not scanned at all, and that is a limit rather than a subtlety: a payload that hides its directive at character 70,000 is not detected, and the containment layers below are what stand between it and an effect.

An injection rule is a policy, and the way policies fail in production is by being switched on blind. Most engines let you write a rule and enable it, and the first thing you learn about its false-positive rate is which team’s work stopped. That is why every rule in Token Observe can run in shadow mode first: it is evaluated exactly as an enforcing rule, its match is recorded on the trace with the policy, its action and why it matched, and then it is skipped and the request proceeds untouched. You learn the rate before you pay for it.

Scope the rule to the source rather than to the score alone. A rule filtered to findings from tool results polices indirect injection without blocking the person typing into your support console, which is the single most useful configuration in this whole area and the one most implementations cannot express because they score the payload as a whole.

Set the threshold from your own traffic rather than from a default. The composition is additive, so two mild patterns together can clear a threshold that neither reaches alone; whether that is a true positive depends entirely on what your agents read. Shadow mode answers this for the traffic that arrives while it runs, and a retrospective replay against recorded traffic answers it for the traffic that has already been. They are complements: one is forward-looking, the other is the only way to know what month-end looks like when you staged the rule on a Tuesday.

Finally, record the refusals. A blocked call is evidence: the trace opens before the refusal, so an agent probing for grants it does not hold, or repeatedly proposing an action a policy keeps stopping, is visible afterwards. An injection defence whose successful blocks leave no trace has thrown away the detection data that would have told you an attack was in progress.