Token Observe vs Braintrust
Braintrust is in the request path too. What it does there is deliver the call and record it; what it blocks is the release that would have made the call worse.
2 September 2026
- Braintrust home
- Braintrust documentation home
- Gateway
- Use the proxy (deprecated)
- Security
- Self-hosting Braintrust
- Self-hosting: security and access
- Self-hosting: networking and connectivity
- Deployment options
- Access control
- Online scoring
- Set up human review
- Evaluate
- Pricing
Their claims, not our testing. Verify anything that decides it for you.
On this page
On evaluation, on assurance and on deployment maturity, Braintrust is ahead, and for most readers it is the right purchase
The loop Braintrust is built for is the loop a team improving an agent actually works in, and Token Observe has none of it. Their evaluate page describes comparing prompts, models and strategies with side-by-side diffs, datasets built from “production logs, user feedback, or manual curation”, and three kinds of scoring composed together — deterministic code checks on “format, structure, accuracy”, LLM judges assessing “tone, helpfulness, reasoning quality”, and human review where subject-matter experts evaluate and build labelled datasets — with a no-code playground carrying tool calls and MCP, GitHub Actions integration, and the same scorers running offline in experiments and online in production. Online scoring runs those scorers against production logs as they arrive with automation rules that set the scorer, the span or trace scope and optional filters. Loop, on their home page, is described as generating better prompts, scorers and datasets from a description of what you want to optimise. Token Observe has one hook for trace scores and nothing that fills it; semantic caching, evaluation harnesses and session replay were designed for in the data model and deliberately not implemented, and another standalone tracing and evaluation product is a named strategic non-goal rather than a queued item.
Braintrust is also further ahead on everything a security review and a procurement team ask about, and the gap is not narrow. Their security page names SOC 2 Type II, describes HIPAA support through Business Associate Agreements and GDPR through Data Processing Agreements, and states that organisations wanting full EU data residency “should use a BYOC or self-hosted deployment”. Three deployment shapes are published — SaaS, BYOC in which Braintrust still operates the data plane but it “runs inside your dedicated cloud account or project”, and self-hosted in which “you deploy and operate the data plane in your own cloud” — and the self-hosting documentation gives official Terraform modules for AWS, GCP and Azure, with a control plane that “provides the web UI, authentication, user management, and metadata storage” but “does not store or process your sensitive data”, and the plain statement that “Braintrust’s servers and employees do not require access to your data plane for it to operate”. Enterprise also closes a gap this page might otherwise have claimed for itself: a self-hosted data plane can record row-level query.read audit logs covering “both SQL queries run manually and ones the Braintrust UI runs implicitly when users browse logs, experiments, and traces”, with a strict mode that writes the audit row before the results reach the caller. Token Observe has no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, states all four in its own security policy rather than leaving them to be discovered, and runs as a single Node process over one SQLite file with no replica, no clustering and no vendor-operated availability commitment.
The third advantage is commercial and it decides more purchases than the first two. Braintrust publishes prices you can act on today — a $0 Starter tier with $10 of model credits, 1 GB of processed data, 10k scores a month and 14-day retention, a $249 per month Pro tier with $100 of credits, 5 GB, 50k scores, 30-day retention, custom charts, environments, priority support and RBAC, and Enterprise at custom pricing for “custom data retention and export, RBAC, and premium support with on-prem or hosted deployment for high volume or privacy-sensitive data”. Token Observe publishes no price at all, because the licence a figure would be quoted under is a template that has not yet been reviewed by counsel. Everything in the Braintrust column on this page comes from Braintrust’s own published material read on 2 September 2026 and has not been independently tested; where a cell reads as an absence, treat it as a question to put to Braintrust in writing rather than as a finding, because these products change quickly and this page is not a test report.
Token Observe and Braintrust, capability by capability
The Braintrust column paraphrases Braintrust’s own published material as it stood on 2 September 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Check anything that decides it for you against their own current documentation.
Where each one sits
In it, for traffic you route through the Gateway. Their documentation says to “Point your SDKs to the Gateway URL” at https://gateway.braintrust.dev, changing the base URL and passing a Braintrust API key: “That’s it — no other code changes needed.” The supported paths it lists are /auto, /embeddings, /chat/completions, /completions and /moderations.
In it. You change OPENAI_BASE_URL or ANTHROPIC_BASE_URL and one key, and every governed request runs the same eleven-step path with one policy verdict at step 6, before anything leaves your network.
This is the row a reader arriving from the category page will expect to go the other way. Braintrust is not a beside-the-call SDK only; the Gateway is a documented, OpenAI-compatible hop, which makes the interesting question what each hop is permitted to do rather than where it sits.
The Gateway. The older AI proxy page carries the notice that “The AI proxy is deprecated and will no longer be regularly maintained. Use the gateway instead for production-grade reliability.”
One path since the beginning, and the same eleven steps for the model gateway, the MCP tool path and a replayed backtest, because one pure function decides all of them.
Braintrust, for Gateway traffic. The instruction on their Gateway page is to “Add your provider API key in Braintrust so the Gateway can call it on your behalf — you won’t need to set your provider key locally”, at organisation or project level. On the cache the same page states that “Braintrust cannot see your data and does not store or log API keys”, the cache key being derived from your Braintrust API key.
You do. Self-hosted and bring-your-own-key, with the vendor receiving no product telemetry, phone-home data, prompts, keys or trace database, and the runtime data flow documented so you can verify that rather than take it on trust.
“A unified API to access LLM models from OpenAI, Anthropic, Google, AWS, and other providers”, described as “a large and fast-moving set of models across OpenAI-compatible, Anthropic, Google, and AWS Bedrock APIs”, plus custom models or endpoints through custom provider configuration. The deprecated proxy page cites “over 100 models including GPT-5, Claude 4, Gemini 2.5, and Llama models”.
OpenAI, Anthropic, Gemini, OpenRouter, Amazon Bedrock and Azure OpenAI as first-class upstreams, plus OpenAI-compatible endpoints of your own, with identical policy, redaction, budget and trace behaviour across all of them enforced by a table-driven test over every provider kind.
For the hosted Gateway they describe a global endpoint using “DNS latency routing and health checks across Braintrust-hosted Gateway regions”, retries “against configured fallback providers in order” on a retryable provider error, and a service “designed for production workloads” whose “uptime is tracked on the Braintrust status page under AI Gateway”. What client traffic should do when the Gateway itself is unreachable is not described in their published documentation as of 2026-09-02.
It stops. A boot-time streamed walk of the audit chain that finds corruption latches readiness and audit writes unavailable, and governed requests receive a 503 carrying ACP_AUDIT_UNAVAILABLE. The latch survives a restart deliberately; there is no online clear, and recovery means restoring a database whose chain and off-box head both verify.
Fail-closed is the strongest argument against Token Observe on this page, and it is deliberate: a control you can bypass by turning it off is not a control. Ask Braintrust in writing what their Gateway does on its own bad day, because the answer belongs in the same conversation.
What each one enforces
The release. Their evaluate page describes “CI/CD quality gates block bad changes”, added to CI through GitHub Actions, so a regression is caught before it ships.
The request. One verdict of allow, block, redact or require approval at a single decision point, enacted before the payload reaches a provider, with the caller receiving ACP_POLICY_BLOCKED and the trace closing as blocked.
Both are enforcement and they operate on different clocks. A quality gate stops the next thousand calls from being worse; a policy verdict stops this one from happening. An estate that ships carefully and still issues refunds needs both.
An automation rule for online scoring, defining “which scorer to run, the scope (span-level or trace-level), and optional filters”, so scorers run against production logs “as they arrive” for continuous quality monitoring.
A policy: a trigger, an action and a scope. Seven trigger kinds — tool call and argument values, model and estimated input size, accumulated spend, request and token rate, detected data class, injection score and its source, and hour of day in UTC — resolved to one verdict in which a block beats an approval and an approval beats a redaction.
Not described in their published documentation as of 2026-09-02. The Gateway page covers routing, failover, caching, credential handling and logging, and the words policy, guardrail and redaction do not appear on it; online scoring is documented as producing scores on logs rather than as altering or withholding them. The Gateway does serve a /moderations path, but as one of the OpenAI-compatible endpoints it passes through to a provider rather than as a rule it applies of its own.
Yes, and the streaming case is specified rather than assumed. Response-side data-class policy is resolved before the first byte from the policies that could apply, a blocking class ends the stream with an in-band ACP_POLICY_BLOCKED frame the instant it is seen, and a hold-back buffer with a 64-character floor stops a card number split across two chunks escaping masking.
Human review: structured judgement on production traces “to build ground truth, validate automated scores, and surface edge cases your scorers miss”, with queues to “assign, filter, and track review across your team” and categorical, continuous or free-form score types. A human decision gating a response before it is served is not described in their published documentation as of 2026-09-02.
An approval bound to the SHA-256 of the canonical action plus its execution context, single-use, expiring at 60 minutes by default and configurable from 1 minute to 7 days per policy, returned to the caller as a 403 carrying the approval id, a status URL and a resume contract.
Both put a person in front of agent output, at opposite ends of the same call. Braintrust’s reviewer improves the next answer; Token Observe’s approver is the reason this one has not happened yet. The published limit on Token Observe’s side is that nothing tells the agent — an approval takes effect only when the agent retries.
Braintrust API keys and service accounts, which “can be members of permission groups, just like users”, with “The hierarchy and cascade rules apply identically” and an additive model in which “Permissions granted at a higher scope cannot be removed at a lower scope”. The objects those permissions are granted over are Braintrust’s own — organisations, projects, experiments, datasets, logs, prompts, playgrounds. A permission set over what an agent may do downstream, at the level of an individual action, is not described in their published documentation as of 2026-09-02.
Deny-by-default and action-level: a support agent may Read: Customer Account and Update: Shipping Address while Delete: Account is simply absent and therefore denied. Explicit denies win, and delegation chains intersect permissions across every hop so one agent cannot escalate by asking a higher-privileged one.
Typed, and narrower than the shape of this page would lead you to guess. The Gateway “retries the same request against configured fallback providers in order” on a retryable provider error, and their documentation enumerates that set rather than gesturing at it: “Provider unavailability. Rate limits (429). Provider server errors (5xx).” It then excludes the rest — “Failover does not retry Gateway authentication errors, Gateway validation errors, or provider client errors other than rate limits” — and if every usable provider fails, “the Gateway returns the last provider error”.
Seven typed failure classes. A 429, a timeout or a 5xx moves on; a content-policy refusal, an authentication failure, an over-long context and a malformed request stop where they are, because a fallback chain that retried a refusal would launder it into a success and nothing in the record would say a refusal happened.
This row moved on re-check, and it moved towards Braintrust. Their published failover set is enumerated and already excludes provider client errors other than 429, which is the same instinct behind Token Observe’s typed classes. The two products agree more here than the rest of this page suggests; the remaining difference is that Token Observe treats the classification as a governance property, so a refusal is recorded as a refusal rather than being an error the chain declined to retry.
Rate limits yes, on their own surfaces; a spend ceiling on model traffic is not described in their published documentation as of 2026-09-02. A self-hosted data plane carries configurable rate limits on log ingestion, on SQL queries and on function invocation, per organisation and per project, over fixed windows, with an enforcement mode that either warns or rejects with a 429. Those meter calls into Braintrust rather than money leaving for a provider. For cost their Gateway page points at observation — create a project for Gateway logs, then “Create dashboards to track usage, costs, and errors” — and the pricing page meters model credits and processed data against your Braintrust bill.
USD ceilings per request, hour, UTC day and UTC month, and rate ceilings on requests, tool calls and tokens per minute, projected and reserved in one per-agent transaction before egress, with an unpriced resolved target refused outright for any budgeted agent. A kill switch is scoped to one agent, one team, or everything.
The word “rate limit” means different objects on the two sides and the row would mislead without saying so. Braintrust’s protect Braintrust — ingestion, queries and function invocation on their own API. Token Observe’s protect your provider account, and are reserved before the call rather than counted after it. Whether anything in Braintrust can cap what an agent spends at a provider is a question to put to them in writing.
What each one records, and who may read it
Traces and spans in the Logs interface: “Inspect every agent trace and tool call, search across millions of logs, and track latency, cost, and quality in real time”, with Gateway calls writing to your organisation’s configured data plane and returning an x-bt-span-id response header when x-bt-parent is set.
Governed requests, their tool calls and arguments, the policy decisions, the approvals with approver identity and rationale, and normalised usage and cost, in a timeline that explains each step in a plain sentence a compliance officer can read without a query language.
Volume and query speed. The self-hosting documentation puts “Brainstore (a high-performance query engine for real-time trace ingestion)” in the data plane alongside the Braintrust API, PostgreSQL, Redis and object storage, and says plainly that PostgreSQL “is not the primary store for your AI data — traces, spans, and logs live in Brainstore and object storage”. The home page describes searching across millions of logs in real time.
Evidence rather than volume. One SQLite file in WAL mode with an FTS5 index over redacted content only, and a hash-chained audit log whose append takes the chain tip inside the same transaction so concurrent writers cannot fork it.
These are honest answers to different questions. If your corpus is millions of spans a day, Braintrust’s store is engineered for that and Token Observe’s is not.
Not described in their published documentation as of 2026-09-02. The security page covers encryption in transit and at rest, AES-256 with unique 256-bit keys and nonces for sensitive credentials, and API keys “stored as one-way cryptographic hashes, never in plaintext”.
Each audit row’s hash covers its canonical content plus the previous row’s, SHA-256 by default and HMAC-SHA256 under an audit MAC key when one is configured, with an optional Ed25519 anchor over a canonical statement of the head published daily to a sink outside the database administrator’s control. Tamper-evident, not tamper-proof — and the published limit is that the default is unkeyed, where a rewrite that re-hashes everything verifies clean.
Enterprise is described on the pricing page as including “custom data retention and export”, and the security page describes configurable automated retention policies.
A compliance bundle carrying the traces, their events, the approvals that gated them, the audit entries that account for them and a chain verification naming the sequence number of any break, sealed with a SHA-256 digest over its canonical JSON. Digest-sealed, not signed: recomputing the digest detects an edit after issue but does not establish who issued the file.
Tiered and configurable: 14 days on Starter, 30 days on Pro, custom on Enterprise, with organisations able to “configure automated retention policies to delete logs, experiments, or datasets after specified periods”.
Unset by default, and unset means keep forever, because an upgrade that silently began deleting evidence would be the worse failure. Set a window and an hourly pass ages traces out in batches of 250, each its own short transaction, with a status endpoint reporting the cutoff and the last pass.
Search across millions of logs, and Topics, which their home page describes as continuously clustering every trace against dimensions you define — use case, customer segment, compliance, tone.
A plain-English question translated into a validated filter object over fourteen allow-listed fields, never into SQL, shown back beside the results as editable chips, degrading to a deterministic keyword parser when no translation model is configured. It cannot group, count or correlate across traces, so which agents used the same card number twice is not a question you can ask.
The constraint is deliberate rather than unfinished: trace content holds prompts and tool results written by external parties, so anything derived from it that reached an interpreter would be an injection surface.
Yes, on Enterprise, and this row moved on re-check. Their self-hosting security page states that “Braintrust logs administrative actions automatically” and that a data plane can also record row-level query.read audit logs “that capture both SQL queries run manually and ones the Braintrust UI runs implicitly when users browse logs, experiments, and traces”. These “are high volume, so they are disabled by default and require an Enterprise plan”, are enabled per organisation, and in strict mode each audit row is written “before returning query results, guaranteeing the read is recorded before results reach the caller”. An x-bt-enable-audit request header returns the requesting user’s id and email, a normalised endpoint path and the resources touched.
Listing, searching, opening and exporting each append an audit entry naming the actor, the interpreted filter, the teams the account was effectively authorised for and the row count, and the export entry carries the digest of the bundle it issued.
A page written from the marketing pages would have got this wrong, and an earlier draft of this one did. The honest remaining differences are narrow: Braintrust’s read auditing is an Enterprise feature that is off by default and configured on a self-hosted data plane, where Token Observe’s is on in every deployment and its entries land in the same hash-chained log as everything else. On the substance — a named reader attached to a read, written before the results are handed over — they are doing the same thing.
Identity, deployment and assurance
The security page describes UI support for “enterprise identity providers (Google, Okta, Microsoft)” and “SSO/SAML integration with major providers”, with API keys displayed only once on creation and stored as one-way hashes.
OIDC single sign-on with bounded SCIM Users provisioning for viewer accounts holding no evidence scopes, permitting disable but not rename or reactivation once an account holds more authority. SAML, SCIM Groups and live-directory reads are named as deliberately not built rather than pending.
Role-based access control with built-in groups — Owners on all plans, and Engineers, Viewers and All AI Provider Access on Pro and Enterprise — across three scopes of organisation, project and object, where “A user can belong to multiple groups. Their effective permissions are the union of every group they’re in.” Custom permission groups and object-level ACLs are Enterprise only.
Roles plus an explicit list of team scopes on each human account, with the query predicate derived server-side and not widenable by a query parameter, and the same check repeated on list, search, detail, single-trace export and compliance bundle.
Object-level ACLs on experiments, datasets, logs, prompts and playgrounds, on Enterprise, so access can be narrowed to a designated group; permissions are additive, and “Permissions granted at a higher scope cannot be removed at a lower scope.”
Organisation-wide surfaces — the audit ledger, retention controls, subject erasure, radar findings, the executive dashboard — return 403 to a team-scoped account rather than a narrower answer, because projecting those joins onto one team would produce a misleading result rather than a smaller one.
Three, and who operates the data plane differs across them. On SaaS “Braintrust operates both the control plane and the data plane”. On BYOC “Braintrust operates the control plane and the data plane, but the data plane runs inside your dedicated cloud account or project”. On self-hosted “Braintrust operates the control plane, and you deploy and operate the data plane in your own cloud”. Official Terraform modules cover AWS with ECS and EC2 and GCP and Azure with Kubernetes and Helm, and Braintrust “strongly recommends using these Terraform modules”.
Self-hosted only, on your infrastructure and your provider keys. One Node process and one SQLite file in WAL mode; PostgreSQL is implemented behind the store ports as an evaluation alternative with dual-backend CI, and is explicitly not a supported high-availability topology, multi-replica claim or point-in-time-recovery result.
The data plane “stores all sensitive data, including experiment records, logs, traces, spans, datasets, and prompt completions”, while the hosted control plane “provides the web UI, authentication, user management, and metadata storage” and “does not store or process your sensitive data”. The two “communicate only for authentication and metadata synchronization”, and “Braintrust’s servers and employees do not require access to your data plane for it to operate”. A self-hosted data plane serves its own /v1/proxy endpoint; running the Gateway inside that data plane is a separate step, gated on Terraform module v6.5.0 or Helm chart 6.6.0 and above and explicitly enabled before traffic is routed through it.
Nothing to the vendor. No product telemetry, no phone-home, no prompts, no keys, no trace database. Governed payloads leave your network only for the model and tool providers you configure, after policy and redaction, and the data flow is documented so you can check it rather than accept it.
SOC 2 Type II, described as confirming that “our controls related to security are operating effectively over time”, with HIPAA through Business Associate Agreements and GDPR through Data Processing Agreements, and the note that full EU data residency points you at BYOC or self-hosted.
None yet: no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, each stated in the product’s own security policy rather than left to be discovered. What is offered instead is a licence clause expressly permitting you to inspect, fuzz and penetration-test your own deployment before a purchase order, with no gag clause and no pre-approval of results.
What each one costs
Model credits, processed data, scores and retention. Starter is $0 with $10 of model credits, 1 GB processed data, 10k scores a month and 14-day retention; Pro is $249 a month with $100 of credits, 5 GB, 50k scores and 30-day retention; Enterprise is custom.
A subscription to one self-hosted deployment, metered on the number of deployments you run and the number of agents licensed to be active at once. There is no per-token or per-request component and there cannot be one, because governed traffic goes to your own provider accounts and the software reports nothing back that could be metered.
Yes for Starter and Pro, with the included allowances beside them; Enterprise is quoted.
No. The licence a figure would be quoted under says on its own first page that it must be reviewed and approved by qualified counsel in England and Wales before it is offered to or relied upon by any customer, and that has not happened, so every priced line is on application.
The Gateway is described as “in public preview and free to use” at the hosted endpoint, and caching is on by default in auto mode, where requests are cached when they carry temperature=0 or a seed, with results held for a week unless request headers say otherwise.
No separate charge and no cache. Semantic and response caching were designed for in the data model — there are cache namespaces — and deliberately not implemented, on the principle that speculative generality is worse than an absent feature.
Enterprise, at custom pricing, described on the pricing page as “premium support with on-prem or hosted deployment for high volume or privacy-sensitive data”, with the Terraform modules and cloud infrastructure your own cost.
Self-hosted is the only mode there is. Backup, disaster recovery, development, testing, staging and training copies count towards nothing as long as they serve no production traffic.
The Starter tier is free with “Unlimited users, projects, datasets, playgrounds, and experiments” within its allowances, and the Pro tier carries a startup discount of “6–12 months free for qualifying startups”.
A thirty-day evaluation grant in the licence for internal evaluation, security review and proof-of-concept purposes, with no licence state gating any control — the gateway, permissions, redaction, injection heuristics, approvals, budgets, rate limits, the flight recorder, the audit chain and the kill switch all enforce exactly as in a paid deployment.
Both products are in the request path, so the comparison is about the verb
The category assumption is that evaluation vendors watch and gateways enforce, and Braintrust’s own documentation breaks it on the first page. The Gateway takes OpenAI-compatible traffic at gateway.braintrust.dev with “no other code changes needed”, serves /chat/completions, /embeddings, /completions and /moderations, holds provider credentials “so the Gateway can call it on your behalf”, retries against configured fallback providers in order on an enumerated set of retryable provider errors, caches under AES-GCM with a key derived from your API key, and writes the resulting span to your organisation’s data plane. A self-hosted data plane serves its own /v1/proxy endpoint, and deploying the Gateway inside that data plane — a separate step, gated on Terraform module v6.5.0 or Helm chart 6.6.0 and above — changes what serves it. The older AI proxy page now carries a deprecation notice pointing at it. Anybody writing this comparison from the category rather than from the documentation would get the first row wrong.
What differs is authority. The Braintrust Gateway documentation describes routing, fallbacks, caching, credential handling and logging, and does not address policy enforcement or guardrails; where their material does describe blocking, the object being blocked is a change rather than a call — “CI/CD quality gates block bad changes”. Token Observe’s hop exists for one purpose the other does not claim: to refuse. Eleven steps run in an order that is load-bearing — authenticate, resolve the agent, open the trace so even a blocked request is recorded, sanitise Unicode so smuggled invisible characters are stripped before any detector reads a different string from the one the model will read, scan for sensitive data and injection, take one verdict, enact it, route honouring the agent’s data policy, call upstream, govern any tool call the model proposes on the way back, then meter and record.
Treating both as gateways also makes the operational question concrete rather than theoretical. Two proxies in series is a second failure domain and a second hop of latency, and it is a decision to take deliberately. Token Observe fails closed, and the consequence is published rather than implied: a boot-time streamed walk of the audit chain that finds corruption latches readiness and audit writes unavailable, governed requests receive a 503 carrying ACP_AUDIT_UNAVAILABLE, and restarting does not clear it. That is the price of a hop that is allowed to say no. Braintrust’s hosted Gateway is described with DNS latency routing and health checks across their regions; what it does when it is itself unreachable is a question for Braintrust, and it is a fair one to ask in the same conversation.
- What Braintrust publishes about its own hop
- An OpenAI-compatible Gateway, in public preview and free to use at the hosted endpoint, with unified provider access, ordered fallbacks over an enumerated set of retryable provider errors, three caching modes and span logging to your data plane. Load balancing across several keys for one model is documented on the deprecated proxy page rather than on the Gateway page.
- What Token Observe publishes about its own hop
- Eleven steps in a fixed order, one decision point at step 6 returning allow, block, redact or require approval, and a fail-closed posture whose recovery procedure is a database restore rather than a toggle.
- The question that actually separates them
- Not where the code runs, but what it is permitted to stop. One is engineered to deliver the call reliably and record it; the other is engineered to refuse it and prove why.
A quality gate stops the next thousand calls; a policy verdict stops this one
Braintrust’s enforcement story is coherent and it operates on the release. Scorers are written once and run in both places — offline against datasets in experiments, and online against production logs as they arrive, driven by automation rules that name the scorer, the span or trace scope and optional filters. Regressions show up as score movements, quality gates in CI block a change that scored worse, and human review turns the interesting failures into ground truth that improves the next evaluation. That loop is the reason a team’s agent gets better week over week, and Token Observe contributes nothing to it.
The loop has one structural property worth naming plainly, and it is not a criticism: a score is produced from a call that has already happened. Online scoring is documented as continuous quality monitoring over logs, and human review as structured judgement on production traces to build ground truth and validate automated scores. If the incident you are worried about is that answer quality drifted, that is exactly the right shape and the earliest useful signal you can get. If the incident you are worried about is that a customer’s card number reached a provider, or that a refund of £4,000 was issued against a policy ceiling of £200, a score afterwards is a finding rather than a prevention, and the difference between those two words is the entire reason Token Observe exists.
Token Observe’s verdict is deterministic for the same reason. Detection is heuristic and its numbers are published rather than implied — eleven sensitive-data classes of which three are checksum-validated by Luhn, IBAN mod-97 and NHS mod-11, and nine weighted injection patterns scored 1.25 times higher when the text arrives as a tool result, because injection in a ticket body is the channel that actually hijacks agents. Blocking is not left to a classifier, because a classifier with a meaningful false-positive rate on the hot path breaks legitimate work. What does the blocking is the deterministic half: action-level permissions that deny by default, an approval bound to one exact payload, a hard spend ceiling and a kill switch. Every rule can run in shadow mode first, recording what it would have done without stopping anything, and where the deployment turns the gate on no rule may begin enforcing until a backtest of that exact rule has been replayed against recorded traffic and acknowledged by a named person.
- Braintrust’s unit of enforcement
- The change. Quality gates in CI block a release that scored worse, and online scorers keep watching production so the next gate has better evidence.
- Token Observe’s unit of enforcement
- The request. One verdict before egress, a typed refusal to the caller, and a trace that closes as blocked with the rule that fired named on it.
- Why neither absorbs the other
- A gate on the release cannot stop a call that a passing build makes badly, and a verdict on the request cannot tell you your answers got worse. They are answers to different questions asked by different people.
What running both looks like, and what it will not buy you
The arrangement is straightforward because the two products want different things from the same traffic. Your engineers keep Braintrust exactly as it is — the SDKs, the datasets, the experiments, the playground, the scorers running online against production logs, the human review queues, the CI gates — and nothing about the improvement loop changes. Agents that take consequential actions have their base URL changed to Token Observe, so those calls acquire an identity, a deny-by-default permission set, a budget, a policy verdict taken before egress and a hash-chained record of that verdict. Where you want the Braintrust Gateway to remain the hop that reaches providers, it is an OpenAI-compatible endpoint and Token Observe registers OpenAI-compatible endpoints of your own as routable upstreams; where you would rather Token Observe reach providers directly, six upstreams are first-class and the same policies apply identically across all of them.
Two proxies in series is a real cost and should be chosen rather than inherited: a second failure domain, a second hop of latency, and two places holding credentials. The honest version of the advice is the same one the product’s own material gives about gateways generally — a chat assistant that drafts text a human reads does not need both, and a refund agent might. Start with the agents whose worst day involves money moving or data leaving, leave everything else pointing where it points now, and remember that Token Observe’s pilot boundary is roughly five to fifty agents owned by one platform team rather than an estate-wide rollout.
What running both will not buy you is one pane of glass, and pretending otherwise sets up a disappointment on day three. The two records have different shapes, different retention and different readers, and Token Observe has no trace viewer for application spans, no dataset management, no experiment runs and no evaluation harness to display Braintrust’s work in. Token Observe does run an OTLP over HTTP receiver taking bounded JSON and protobuf across logs, traces and metrics, attributing every write to the credential that presented it, but that is an input feeding the observed view of a reconciliation between what was declared, locked, deployed and seen — not a competing destination, and the published limits are that protobuf interoperability has repository tests rather than a live collector matrix and that the receiver does not attest the exporter that produced the telemetry. The integration is deliberately shallow, and the shallowness is the point.
- The clean division
- Braintrust answers whether the agent is any good and getting better. Token Observe answers whether it was allowed to do that, and who can prove it a year later.
- The overlap you will actually notice
- Both hold a per-request record and both can be the hop that reaches the provider. Decide which one holds the provider credential before you deploy, not after.
- The thing to decide first
- Whether a fail-closed dependency is acceptable for the agents you route through it, and who owns the decision about what happens when it is unavailable.
Where each one is the right answer
Choose Braintrust when
- The work in front of you is making an agent better: datasets, experiments, side-by-side prompt and model comparison, LLM-judge and code scorers, a playground and CI quality gates. Token Observe has none of that and names another standalone evaluation product as a strategic non-goal rather than a backlog item.
- You want continuous quality monitoring on production traffic — scorers running against logs as they arrive, with automation rules and human review queues turning failures into ground truth.
- A third-party attestation gates the purchase. Braintrust publishes SOC 2 Type II, HIPAA through BAAs and GDPR through DPAs; Token Observe holds none of the three and has had no independent penetration test.
- You cannot accept a fail-closed component in the request path, which is an entirely legitimate position for an estate whose agents draft text a person reads before anything happens.
- You need a published price you can put in a business case today, or a free tier you can start on this afternoon.
Choose Token Observe when
- The agents take actions somebody has to answer for — a refund, a deployment, an email, a ticket transition, a database write — and a score on the trace afterwards is a finding rather than a prevention.
- A named human has to approve a specific action before it happens, bound to that exact payload, single-use and expiring, rather than reviewing it afterwards in a queue.
- Spend has to stop rather than be reported: per-agent ceilings for the request, hour, day and month, reserved before egress, with an unpriced resolved target refused outright and a kill switch scoped to an agent, a team or everything.
- Nothing about the governed traffic may leave your network to a vendor: self-hosted only, on your keys, with no telemetry, prompts or trace database going anywhere.
- The reader of the record is an auditor or a compliance officer, reads of it need to be attributable to a named person, and the export needs to carry a chain verdict naming where any break occurred.
When you would run both
Running both is the normal answer, and it is the one Token Observe’s own roadmap points at, because another standalone tracing and evaluation product is a named strategic non-goal rather than something in the queue. Keep Braintrust doing what it is built for: the datasets, the experiments, the scorers running offline and online, the playground, the human review queues and the CI quality gates that stop a worse version shipping. Put Token Observe at the base URL for the agents whose worst day involves money moving or data leaving, so those calls acquire an identity, a deny-by-default permission set, a spend ceiling, a policy verdict taken before the payload reaches a provider, and a hash-chained record of that verdict. Decide deliberately which of the two holds the provider credential and reaches upstream, because both can — the Braintrust Gateway is an OpenAI-compatible endpoint Token Observe can register as a routable upstream, and Token Observe carries six first-class upstreams of its own — and two proxies in series is a second failure domain worth choosing rather than inheriting. The division of labour is clean because the two products were built for different readers: Braintrust tells your engineers whether the agent is getting better, and Token Observe tells your auditor whether it was allowed to do what it did.
The category argument sits above this one: Token Observe and llm observability covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
LangSmith
LangSmith’s callback handler watches the call from beside it, and their newer LLM Gateway now stands in it too. The remaining differences are narrower than they were, and worth stating precisely.
Langfuse
Langfuse says in its own documentation that its SDKs are asynchronous and that blocking is a guardrail library’s job. That sentence is the whole comparison, and it is not a criticism.
Arize
Arize is instrumented into your application and reads what it did. Token Observe is a hop your agents call through and decides what they may do.
Datadog LLM Observability
Datadog puts LLM spans beside the rest of your telemetry. Token Observe puts a verdict in front of the call. Datadog also sells a verdict — and where it is made is the whole comparison.
Braintrust is an evaluation platform. Is it really in the request path?
Yes, for traffic you route through it, and this is the fact most likely to be got wrong about them. Their Gateway documentation as of 2 September 2026 describes pointing your SDKs at https://gateway.braintrust.dev with no other code changes needed, serving OpenAI-compatible /chat/completions, /embeddings, /completions and /moderations paths, holding provider credentials so the Gateway can call on your behalf, retrying against configured fallback providers on an enumerated set of retryable provider errors, caching under AES-GCM, and writing the span to your organisation’s data plane. A self-hosted data plane serves its own /v1/proxy endpoint, and the Gateway can be deployed inside that data plane to serve it. So the comparison on this page is not about who is inline; it is about what each inline hop is permitted to stop.
Can Braintrust block a request the way Token Observe does?
Their Gateway documentation covers routing, fallbacks, caching, credential handling and logging, and does not address policy enforcement or guardrails; the blocking their material describes acts on releases rather than on calls, with the evaluate page stating that CI/CD quality gates block bad changes, and online scoring documented as running scorers against production logs as they arrive. Whether anything else in their platform can refuse or rewrite a call in flight is a question for Braintrust rather than for this page, and it should be asked in writing. What Token Observe does is a specific mechanism: one verdict at step 6 of an eleven-step path returning allow, block, redact or require approval, enacted before the payload leaves your network, with the caller receiving ACP_POLICY_BLOCKED and the trace closing as blocked — and on a stream, a blocking data class ending it with an in-band frame the instant that class is seen.
Do we have to replace Braintrust to use Token Observe?
No, and you should not. Token Observe has no datasets, no experiment runs, no scorers, no LLM-judge harness, no playground and no annotation queues, and those are absent by decision rather than by schedule — the roadmap names another standalone tracing and evaluation product as a strategic non-goal. The overlap is genuine, in that both products keep a per-request record and both can be the hop that reaches your provider, but Braintrust’s record exists so your engineers can make the agent better and Token Observe’s exists so a compliance officer can prove a decision. The intended arrangement is both, with the base URL of consequential agents pointed at Token Observe and Braintrust’s loop left exactly as it is.
How do the deployment and assurance stories compare?
Braintrust is ahead on both and it is worth being direct about it. Their published material describes three deployment shapes — SaaS, BYOC with the data plane in your cloud, and self-hosted operated by your team — with official Terraform modules for AWS, GCP and Azure, a control plane that holds the UI, authentication, user management and metadata while not storing or processing sensitive data, and the statement that their servers and employees do not require access to your data plane for it to operate. They name SOC 2 Type II, HIPAA through Business Associate Agreements and GDPR through Data Processing Agreements. Token Observe is self-hosted only, runs as one Node process over one SQLite file in WAL mode with PostgreSQL available behind the store ports as an evaluation alternative rather than a supported high-availability topology, and holds no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test. What it offers instead of a certificate is verifiability: no telemetry, prompts, keys or trace database reach the vendor, the data flow is documented so you can check that, the defect list is published with the attacks that still work, and the licence expressly permits you to penetration-test your own deployment before you buy.
We already send traffic through the Braintrust Gateway. Where would Token Observe go?
Either side, and the choice is about which hop holds the provider credential. Put Token Observe in front and it takes the policy verdict, the permission check and the budget reservation first, then routes to the Braintrust Gateway as one of your own OpenAI-compatible upstreams, so their caching, fallbacks and span logging are unchanged and the governance decision happens before that hop. Put it behind and Braintrust remains the entry point while Token Observe governs the leg that reaches the provider. Two proxies in series is a second failure domain and a second hop of latency either way, so route only the agents that take consequential actions through both and leave the rest pointing where they point now — Token Observe’s own pilot boundary is roughly five to fifty agents owned by one platform team.
Are the Braintrust claims on this page tested?
No. Everything in the Braintrust column paraphrases Braintrust’s own published material read on 2 September 2026 — the home page, the documentation home, the Gateway page, the deprecated AI proxy page, the security page, the self-hosting overview and its security-and-access and networking configuration pages, the deployment-options and access-control pages, online scoring, human review, the evaluate product page and the pricing page — and none of it has been independently verified. These products move quickly, and the deprecation notice on their proxy page is a reminder of how quickly, so treat any cell that reads as an absence as a question to put to Braintrust rather than as a finding, and check the pricing figures against their own page before they reach a business case.
Prefer to ask a person? Write to us →
Tell us which one you are already running.
If Braintrust is already in your stack, the useful question is not which to buy but what each is for, and where the seam between them sits. Say what you have and you will get a straight answer — including when the answer is that you do not need a second thing.
no form · no qualification step · no sales desk · the other three ways in