Platform engineering
One base URL changes. The fail-closed trade is the thing to decide before you change it.
This is already on your desk.
- 01
The agents are already in production and you did not write them
Governance that requires every team to refactor is governance that does not happen. What you can realistically change is one environment variable in someone else's service, which is why the ingress dialects and the error shapes matter more than the console does. - 02
You are the one paged when a control blocks real work
A rule promoted straight to enforce is an outage with a compliance justification. Shadow mode exists so the first day produces findings rather than incidents, and an optional gate refuses the move into enforcement until a named person acknowledges a backtest. - 03
Spend is a reliability problem before it is a finance problem
Hard per-agent circuit breakers by request, hour, day and month are why a looping agent stops at a ceiling rather than at an invoice. The sharp edge is metering: a model with no price row meters at zero, so an unpriced target is refused before egress.
- 01
If this is down, my agents stop. Why would I put it in the path?
Because a control you can bypass by turning it off is not a control, so Token Observe's availability becomes a governance property of your environment. There is no fail-open setting, deliberately. What replaces it is a decision made in advance: run it close to the agents, watch the readiness endpoint, and write down what you do if it stops. The escape hatch is real but visible: a base URL pointed back at the provider produces ungoverned traffic the radar later reconciles.
- 02
What does it add to request latency?
One published number, and it is a laboratory baseline rather than a capacity commitment: 206.2 requests per second at concurrency 16 over 30.1 seconds, p50 71.2 ms, p95 163.8 ms, p99 223.4 ms, against a mock upstream on a single laptop. It measures the inline governance work and excludes provider latency, streaming, retries, failover and a partner-sized database. The figure for your estate comes from your own traffic.
- 03
Will it break my agents?
A block returns in the shape your client already parses. On OpenAI-compatible ingress that is the OpenAI error envelope with a typed code such as ACP_POLICY_BLOCKED, and where the policy asked for a human it carries the trace id, the approval id and the header to resume with, which is single-use and bound to the same payload. Anthropic and Gemini get their native error shapes with the same detail. Beyond that: shadow mode first, and typed failover, so a 429 fails over while a content-policy refusal does not.
Cloud custody and optional customer-hosted controls are described separately. The 24 limits we publish, and the 9 risks a named person has to accept in writing, are on one page.
What we do not claimSet up one agent. Decide its first policy.
The mechanical part is bounded by npm, the build and SQLite: install, configure, boot, one quickstart call, mint a key, redirect one base URL, watch the first trace appear. The two steps that take the rest of the hour are judgements rather than commands: who owns this agent and what it is for, and which rule you are willing to have block production traffic at three in the morning.
Anything that speaks an OpenAI-compatible, Anthropic or Gemini dialect. One base URL and one key change.
Authorise, redact, budget, decide. Then forward, refuse, or hold for a human.
Fails closed. A decision it cannot make is a request that does not go.
OpenAI, Anthropic, Google Gemini, OpenRouter, Amazon Bedrock, Azure OpenAI, MCP tool servers, and any OpenAI-compatible endpoint you host.
Registry, policy decisions, the hash-chained audit log and the flight recorder. Held in your database, on your disk. Nothing goes to the vendor.
Missing, unresolvable or erroring state denies the request.
The kill switch is evaluated first and beats every other gate. After it, the gateway resolves the agent record itself rather than an exported copy, so an edit or a suspension applies on that agent's next request with no redeploy. Policy failures fail closed, and data-policy requirements (zero retention, no training on payloads, a serving region) are enforced on the fallback chain as well as the primary route, so a request with no qualifying route is refused rather than quietly downgraded.
- Shadow first, then a backtest gate
- Every policy can run in shadow so you learn your false-positive rate before blocking. An optional setting refuses the move into enforcement until a backtest of that rule digest is acknowledged by a named person.
- Typed failover
- A 429 or a timeout fails over. An authentication error, an invalid request and a content-policy refusal do not; otherwise the fallback chain launders a refusal into a success.
- Your reverse proxy has five requirements
- Terminate TLS, do not buffer responses, preserve the authorisation and API-key headers, set the forwarded headers, and keep the read timeout above your longest model response.
- Rollback is not ordinary once effect pins are active
- If an effect-required pin has ever been activated, an N-1 rollback needs drained traffic, every affected pin quarantined through an effect-aware binary, and every replica stopped. Two releases back is not tested.
This is what does the work.
Policy engine
One deterministic verdict on every governed request: allow, block, redact, or park it for a human.
Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
Model routing
Six upstreams behind one set of policies, and a fallback chain that will not launder a refusal.
Questions
Is there an uptime SLA?
No. The vendor does not operate your deployment, cannot observe it and cannot restart it, so an uptime number from that party would be unmeasurable by either side. No partner-shaped sustained-load result exists yet, and a capacity figure published before one would be a guess wearing a number. There are no service credits, because there is no availability commitment to credit against.
SQLite in production?
It is a single-writer process on one host by design at this scale, and the engineering lead is named as the role that has to accept it: sustained write load degrades governance latency and there is no replica. A PostgreSQL backend exists but is evaluation-only in the current release. If your write profile is heavier than a bounded agent fleet, that is a sizing conversation to have before the pilot.
Can we keep using our existing gateway, tracing and identity stack?
Yes, and that is the intended shape. Routing, retries, caching and quotas are table stakes, so Token Observe integrates above or beside a gateway and does not compete on connectivity. Telemetry arrives over OTLP and becomes flight-recorder evidence alongside your tracing product. Identity comes from your own OIDC provider, and no group claim can promote anyone in the console.
Bring the objection that is not on this page.
Create a cloud account and workspace. No card or server installation. Model calls and Google BigQuery scans can incur separate provider charges.