Token Observe vs LangSmith
LangSmith’s callback handler watches the call from beside it, and their newer LLM Gateway now stands in it too. The remaining differences are narrower than they were, and worth stating precisely.
2 September 2026
- LangSmith product page
- LangSmith documentation home
- LangSmith plans and pricing
- LLM Gateway
- LLM Gateway — spend policies
- LLM Gateway — rate limit policies
- LLM Gateway — data protection
- Audit logs
- Billing and spend limits
- LangSmith for Enterprise
- Self-hosted LangSmith
- Role-based access control
- Set up automation rules
- Usage and billing
- Mask inputs and outputs
- Observability concepts
- Bulk export trace data
Their claims, not our testing. Verify anything that decides it for you.
On this page
For the loop a developer actually works in, LangSmith is the right tool and Token Observe is not competing for it
The daily work of improving an agent is iterative and offline, and LangSmith is built for it end to end. Its documentation describes tracing that gives “full visibility into your LLM application: from individual traces to production-wide performance metrics”, automatic instrumentation for popular providers and agent frameworks alongside decorators, context managers and a low-level RunTree API when you want the detail yourself, dashboards and alerts over the result, annotation queues and user feedback for the human half of the loop, datasets you evaluate against, and an Engine for automated issue detection and root-cause analysis. Token Observe does none of that. Semantic caching, evaluation harnesses and session replay were designed for in the data model — there is a scores table and there are cache namespaces — and deliberately not implemented, on the principle that speculative generality is worse than an absent feature. Another standalone tracing and evaluation product is a named strategic non-goal rather than a backlog item, so this is not a gap that closes later.
They have also moved onto Token Observe’s ground, and the useful version of this page says so plainly. Their LLM Gateway — documented as in beta — is a real inline control plane, not a roadmap slide. Spend policies cap cost at organisation, workspace, API key or user scope over monthly, weekly, daily or hourly windows; the docs state the gateway “tracks spend in real time and blocks any request that would push spend past the cap, returning a 402 response”, that the most restrictive policy wins, and that evaluation happens “on every incoming request with sub-second enforcement latency”. Rate limit policies cap requests or tokens per minute or hour and return a 429 with Retry-After. Data-protection policies scan outbound requests “before they reach the LLM provider” for names, locations, US SSN and phone patterns and a long list of provider API keys and private keys, redact them behind SAFE_TO_USE placeholders, redact the same values in the trace, and de-redact on the way back. If what you wanted from Token Observe was a per-agent spend cap and outbound PII redaction on an OpenAI-compatible endpoint, LangChain now sells that too, and buying it from the vendor whose SDK you already run is a reasonable decision.
The architectural advantage on the tracing side matters too, and it is a genuine argument against buying Token Observe rather than a courtesy. LangSmith’s product page makes two promises Token Observe cannot make: that tracing is asynchronous so “your application performance is never impacted”, and that “if LangSmith experiences an incident, your agent keeps running normally”. An out-of-band exporter degrades by losing visibility. Token Observe is in band and fails closed, so a corrupt audit chain latches readiness and governed requests receive a 503 carrying ACP_AUDIT_UNAVAILABLE until somebody restores a database whose chain verifies. That trade is deliberate — a control you can bypass by turning it off is not a control — but it means LangSmith tracing can be adopted by one team on a Tuesday afternoon and Token Observe cannot be adopted without an operational conversation. If the appetite in your organisation is for visibility without a new dependency in the request path, LangSmith is the correct choice.
LangChain is also further ahead on the things a security review asks about. Their enterprise documentation names SOC 2 Type II, HIPAA and GDPR, publishes a shared responsibility model, and offers three deployment shapes — managed cloud with US or EU data residency, a hybrid mode running “the control plane in LangSmith’s cloud and your data plane in your own VPC”, and fully self-hosted on Kubernetes. Token Observe has no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, states all four plainly in its own security policy and README, and offers a licence that expressly permits a pre-purchase test instead of a certificate. Everything said about LangSmith on this page comes from LangChain’s own published material read on 2 September 2026 and has not been independently tested; where a row reads as an absence, treat it as a question to put to LangChain in writing rather than as a finding.
Token Observe and LangSmith, capability by capability
The LangSmith column paraphrases LangChain’s own published material as it stood on 2 September 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Check anything that decides it for you against their own current documentation.
Where each one sits
Both, depending on the component. Observability sits beside it — the product page describes an SDK that “uses an async callback handler that sends traces to a distributed collector”, plus decorators, context managers and a RunTree API. Their LLM Gateway sits in it: requests go to gateway.smith.langchain.com with a LangSmith API key, and “every gateway call appears as a LangSmith trace”. The gateway is documented as in beta.
In it, only. You change OPENAI_BASE_URL or ANTHROPIC_BASE_URL and one key, and every governed request runs the same eleven-step path with a single policy verdict at step 6, before anything leaves your network.
This row used to be the whole comparison and no longer is. The gateway narrows the gap to scope — what each one inspects and what it can do about it — rather than position.
For tracing, nothing: the product page states that “your application performance is never impacted” and that “if LangSmith experiences an incident, your agent keeps running normally”. For the gateway the posture is different by design — their data-protection page states that “scanner failures are fail-close”, so if a PII or secrets scanner “is unreachable, slow or errors, that stage blocks the request from proceeding”.
It stops. A boot-time walk of the audit chain that finds corruption latches readiness and audit writes unavailable, and governed requests receive a 503 carrying ACP_AUDIT_UNAVAILABLE. The latch survives a restart deliberately; there is no online clear.
Worth reading carefully rather than as a win. LangChain fails closed on a scanner too, which is the same reasoning; the difference is that Token Observe extends it to the whole evidence layer, and that a self-hosted deployment makes the outage yours to fix rather than a vendor’s.
SDKs for Python, TypeScript, Go and Java, automatic tracing for popular LLM providers and agent frameworks that “captures inputs, outputs, and metadata without requiring manual code changes”, and OpenTelemetry.
A base-URL and key change for supported OpenAI-compatible, Anthropic and Gemini ingress, which is normally the whole integration rather than an application refactor.
Docs describe working “with many frameworks and providers”, naming OpenAI, Anthropic, CrewAI and Pydantic AI among them.
OpenAI, Anthropic, Gemini, OpenRouter, Amazon Bedrock and Azure OpenAI as first-class upstreams plus your own OpenAI-compatible endpoints, with identical policy, redaction, budget and trace behaviour across them enforced by a table-driven test over every provider kind.
A run is “a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents”, and a trace is a collection of runs for a single operation; each trace is capped at 25,000 runs.
A trace is one governed request. Its id is minted at step 3 — before unicode sanitisation, before the detectors and before the verdict — so a request blocked a millisecond later is recorded rather than missing, and the id returns on x-acp-trace-id on every response including refusals.
What each one enforces
Two of them. An automation rule over trace data: an item type of runs or threads, filters over trace properties, and a sampling rate, which can “trigger certain actions on your trace data”. And a gateway policy applied to traffic passing through the gateway, in three kinds: a spend policy scoped to an organisation, workspace, API key or user; a rate limit policy scoped to users, workspaces or API keys; and a PII and secrets redaction policy, which their page describes as applying “to all requests that pass through the gateway in the scope where they’re configured” without enumerating the scopes.
A policy evaluated inline that returns one verdict — allow, block or require approval — plus a redaction plan, at a single decision point rather than a chain of independent middlewares that can each decide something different.
Depends which. Automation rules operate on independent polling schedules, and the docs warn that “a webhook rule may process a run before an evaluator rule has scored it, or vice versa”. Gateway policies run inline — the spend page says the gateway “evaluates them on every incoming request with sub-second enforcement latency”, and redaction scans outbound requests “before they reach the LLM provider”.
Before egress, synchronously. Response-side data-class policy is resolved before the first byte of a stream rather than from the classes that turn out to be present, because bytes already written cannot be recalled.
An automation rule can add to an annotation queue, add to a dataset, trigger a webhook, run an online or custom-code evaluator, extend data retention, or trigger an alert — in that fixed order, after the fact. A gateway policy can refuse, in real time: the spend page says the gateway “blocks any request that would push spend past the cap, returning a 402 response”, a rate limit returns “a 429 response with a Retry-After header”, and a redaction policy rewrites the outbound request behind SAFE_TO_USE placeholders, then restores the caller’s original values in the provider’s response.
A typed refusal. The trace closes as blocked, the caller receives ACP_POLICY_BLOCKED, and nothing reached the provider. On a stream, a blocking class ends it with an in-band ACP_POLICY_BLOCKED frame the instant that class is seen.
Their gateway blocks on cost and throughput and redacts on content. A verdict that blocks a request because of what is in it, or that routes a request to a named human for approval, is not described in their published documentation as of 2026-09-02.
Two boundaries. Client-side, before anything reaches LangChain: LANGSMITH_HIDE_INPUTS and LANGSMITH_HIDE_OUTPUTS, hide_inputs and hide_outputs callables on the Client, and a create_anonymizer for regex or callable rules traversing nested structures, with a noted “performance hit with complex regular expressions”. And gateway-side, before egress to the provider: Presidio for named entities — “names, locations, and NRP” — plus regex patterns for US SSNs and US phone numbers, and a secrets list covering OpenAI, Anthropic, GitHub, AWS, GCP, Slack and private keys among others, with matches replaced by placeholders in a documented SAFE_TO_USE format. Their page states the limits itself — provider responses are not redacted and “streaming response redaction is in progress”, and “system prompts, developer prompts, and tool-call arguments are not scanned”.
Server-side, before egress to the provider: eleven sensitive-data classes of which three are checksum-validated, applied as a redaction plan to the outbound payload, covering tool-call arguments, with a hold-back buffer on streamed responses so a card number split across two chunks cannot escape masking.
This is now a narrow difference rather than a categorical one, and it sits where LangChain’s own page draws the line: the response side and tool-call arguments. Both products are heuristic, and Token Observe’s licence disclaims any warranty that its detectors catch every instance.
Annotation queues, which a rule can route matching traces into for human review, plus user feedback attached to runs.
An approval bound to the SHA-256 of the canonical action plus its execution context, single-use, expiring at 60 minutes by default, returned to the caller as a 403 carrying the approval id.
Both put a person in front of agent output. One reviews a call that already happened; the other is the reason a call has not happened yet.
Two meters, both real. For their own bill, a workspace spend limit from which “LangSmith will determine an appropriate number of base and extended trace limits”, with the organisation-level limit described as absolute. For your provider bill, gateway spend policies at organisation, workspace, API key or user scope over monthly, weekly, daily or hourly windows, tracked in real time, where the most restrictive policy wins and an over-cap request is blocked with a 402. Rate limit policies cap requests or tokens per minute or hour and return a 429 with Retry-After.
Per-agent budgets and rate limits for the request, hour, day and month, projected and reserved in one per-agent transaction before egress, with an unpriced resolved target or fallback refused outright for any budgeted agent.
The closest row on the page. Their API-key scope maps onto an agent much as Token Observe’s does; the differences left are the pre-flight reservation, the refusal of an unpriced target, and that theirs is in beta and not yet in the self-hosted stable release.
What each one records, and who may read it
Traces and runs giving “full visibility into your LLM application: from individual traces to production-wide performance metrics”, with dashboards, alerts, filtering, sharing and comparison over them. Gateway traffic lands in the same place: “every gateway call appears as a LangSmith trace”.
Governed requests, their tool calls and arguments, the policy decisions, the approvals with approver identity and rationale, and normalised usage and cost, in a timeline that explains each step in a plain sentence.
Two tiers: base traces at 14 days and extended at 400 days, customisable at workspace level on Enterprise. After expiry, “traces are no longer accessible in the tracing project UI or via the API”, while “some metadata associated with each trace may be retained indefinitely for analytics and billing purposes”.
Unset by default, and unset means keep forever, because an upgrade that silently began deleting evidence would be the worse failure. Set a window and an hourly pass ages traces out in batches of 250, each its own short transaction, with a status endpoint reporting the cutoff and the last pass.
Their audit logs page describes a “tamper-resistant record of administrative and configuration actions” on Enterprise, returned as OCSF v1.7.0 API Activity events. Equivalent tamper evidence over the traces themselves — a chain, a signature, an anchor — is not described in their published documentation as of 2026-09-02; the trace material read covers retention, purging and deletion by metadata.
A hash-chained audit log in which each row’s hash covers its canonical content plus the previous row’s, an optional Ed25519 anchor published off-box on a schedule, and a chain verification that names the sequence number of any break. Tamper-evident, not tamper-proof.
A real bulk export, and a substantial one. Their data-export documentation describes writing a project’s traces over a date range to an S3-compatible bucket in Parquet matching the run data format, for offline analysis in BigQuery, Snowflake, Redshift or a notebook, against destinations configured once and referenced by id; their page states that for customers who signed up after 3 August 2026 bulk export is an Enterprise-plan feature, with a transition window for earlier customers. Traces can also be filtered, shared and compared in the UI.
A compliance bundle carrying the traces, their events, the approvals that gated them, the audit entries that account for them and a full chain verification, sealed with a SHA-256 digest over its canonical JSON. Digest-sealed, not signed: recomputing the digest detects an edit after issue but does not establish who issued the file.
Both products export, and the two exports are built for different readers rather than one being larger. Theirs is a warehouse feed sized for analysis; the Token Observe bundle is smaller and carries its own integrity check and the approvals and audit entries that account for the traces in it. A digest is not a signature, and Parquet in your own bucket is evidence in the ordinary sense too.
Filter traces by properties, build dashboards, configure alerts, and compare runs.
A plain-English question translated into a validated filter object over fourteen allow-listed fields, never into SQL, shown back beside the results as editable chips. It cannot group, count or correlate across traces, so which agents used the same card number twice is not a question you can ask.
Audit logs on Enterprise record administrative and configuration actions — API key creation and deletion, role and membership changes, SSO, workspace, dataset and retention configuration — viewable with the organisation:manage permission and available via API in OCSF format. Their page says these are “currently primarily focused on write operations”, and reading, viewing or exporting trace content is not among the operations it lists.
Listing, searching, opening and exporting each append an audit entry naming the actor, the interpreted filter, the teams the account was effectively authorised for and the row count, and the export entry carries the digest of the bundle it issued.
Identity, deployment and assurance
“SAML or OIDC single sign-on and just-in-time user provisioning” plus “SCIM for automated provisioning and deprovisioning”, with roles assignable automatically “via SCIM groups or SSO Groups Sync”.
OIDC single sign-on with bounded SCIM Users provisioning for viewer accounts holding no evidence scopes, permitting disable but not rename or reactivation once an account holds more authority. SAML, SCIM Groups and live-directory reads are named as deliberately not built.
Organization Admin, Operator, User and Viewer at the organisation level; Workspace Admin, Editor and Viewer inside a workspace, with custom workspace roles composed from named permissions. RBAC is “an Enterprise feature for managing workspace-level permissions”.
Roles plus an explicit list of team scopes on each human account, with the query predicate derived server-side and not widenable by a query parameter, and the same check repeated on list, search, detail, single-trace export and compliance bundle.
Attribute-based access control on Enterprise: “fine-grained, tag-based access policies to restrict resource access—including blocking PII data from specific users”.
Organisation-wide surfaces — the audit ledger, retention controls, subject erasure, radar findings, the executive dashboard — return 403 to a team-scoped account rather than a narrower answer, because projecting those joins onto one team would produce a misleading result rather than a smaller one.
Personal access tokens and workspace-scoped service keys, gated by the organisation and workspace roles above. The gateway adds per-credential governance: spend and rate limit policies can be scoped to a single API key or a group of them, and their policy-scope table gives “the customer support agent keys cannot spend more than $500/month cumulatively” as the illustration of an API-key-level policy. A permission set that permits one named action on one agent and denies another is not described in their published documentation as of 2026-09-02.
Deny-by-default and action-level: a support agent may Read: Customer Account and Update: Shipping Address while Delete: Account is simply absent and therefore denied. Explicit denies win, and delegation chains intersect permissions across every hop so one agent cannot escalate by asking a higher-privileged one.
Three. Managed cloud with US or EU data residency; hybrid, running “the control plane in LangSmith’s cloud and your data plane in your own VPC for full data isolation”; and self-hosted “entirely within your own infrastructure using Kubernetes”, which is “an add-on to the Enterprise plan” requiring a licence key from their sales team. The gateway runs on cloud and inside a BYOC data plane, but their docs state it “is not included in the LangSmith v0.16.0 self-hosted stable release”.
Self-hosted only, on your infrastructure and your provider keys. The vendor receives no product telemetry, phone-home data, prompts, keys or trace database, and the runtime data flow is documented so you can verify that rather than take it.
Kubernetes via Helm: frontend, backend, platform backend, playground, queue and an arbitrary-code-execution backend, over ClickHouse, PostgreSQL and Redis or Valkey with optional blob storage, of which “the only component that must be exposed to users” is the frontend.
One Node process and one SQLite file in WAL mode. PostgreSQL is implemented behind the store ports as an evaluation alternative with dual-backend CI, and is explicitly not a supported high-availability topology, multi-replica claim or point-in-time-recovery result.
Their enterprise documentation names SOC 2 Type II, HIPAA and GDPR, and publishes a shared responsibility model between LangChain and the customer.
None yet: no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, each stated in the product’s own security policy rather than left to be discovered. What is offered instead is a licence clause expressly permitting you to inspect, fuzz and penetration-test your own deployment before a purchase order, with no gag clause.
What each one costs
Seats plus traces plus consumption units. Developer is free with “Maximum of 1 seat (free)” and “5k base traces / mo included. Pay as you go thereafter”; Plus is “$39 / seat per month” with 10k base traces included; Enterprise is “Custom pricing”. Beside those the pricing page meters consumption in LangChain Compute Units at “$1.50 / LCU”, listed as covering Engine, Fleet and deployment and sandbox compute, and LangChain Storage Units at “$1.00 / LSU”, listed as covering traces alongside deployment and sandbox storage. Gateway model usage is billed either to your own provider account under bring-your-own-key or, on LangChain-managed credentials, through Gateway Credits.
A subscription to one self-hosted deployment, metered on the number of deployments you run and the number of agents licensed to be active at once. There is no per-token or per-request component and there cannot be one, because governed traffic goes to your own provider accounts and the software reports nothing that could be metered.
Yes for Developer and Plus, with the LCU and LSU rates on the same page and the per-trace charges published in the docs beside them; Enterprise is “Custom pricing” and quoted.
No. The licence a figure would be quoted under says on its own first page that it must be reviewed and approved by qualified counsel in England and Wales before it is offered to or relied upon by any customer, and that has not happened, so every priced line is on application.
Extended retention is charged as an upgrade. Their usage documentation gives the base charge as “.05¢ per trace” and prices an extended-retention trace at “10x the price of a base tier trace (.50¢ per trace)”, so “each upgrade costs .45¢”, quoted in cents on their own page. Certain actions, including online evaluators and automation rules with retention extension enabled, move a base trace to the longer, dearer tier.
Retention is a setting rather than a meter, and the storage is your disk. What it costs you instead is the discipline of deciding a window, because the default of keep-forever will otherwise grow without anybody choosing it.
Self-hosted and hybrid deployment, along with custom SSO, ABAC and RBAC, are Enterprise features at custom pricing, with a licence key obtained from their sales team.
Self-hosted is the only mode there is. Backup, disaster recovery, development, testing, staging and training copies count towards nothing as long as they serve no production traffic.
The Developer tier is free with one seat and 5k base traces a month, and a startup programme offers “up to $10,000 in credits” to VC-backed startups.
A thirty-day evaluation grant in the licence for internal evaluation, security review and proof-of-concept purposes, with no licence state gating any control — the gateway, permissions, redaction, injection heuristics, approvals, budgets, rate limits, the flight recorder, the audit chain and the kill switch all enforce exactly as in a paid deployment.
A callback handler and a chokepoint are answering different questions
LangSmith’s own description of its tracing position is precise and worth taking at face value: the SDK uses an async callback handler that sends traces to a distributed collector, so your application performance is never impacted, and an incident on their side leaves your agent running normally. That is the correct design for the job it does. Telemetry that can stall the thing it is watching is a liability, and a tracing tool that took an estate down would be a worse tool. Everything LangSmith Observability is good at follows from being out of band — you can instrument one service without a change-advisory board, you can sample, you can turn it off.
LangChain evidently agrees that some jobs need the other position, because they now sell one. The LLM Gateway is a beta component that takes the request itself: a Chat Completions, Messages or Responses call to gateway.smith.langchain.com authenticated with a LangSmith API key, routed to a provider by a prefixed model ID, with spend and rate limit policies evaluated on every request and redaction applied to the outbound body. It blocks — a 402 for spend, a 429 with Retry-After for throughput — and it fails closed when a scanner is unavailable. Anyone reading this page for a decision should read their gateway documentation directly, because it is the part of their estate that most nearly does what Token Observe does, and it is moving quickly enough that a page checked on 2 September 2026 is a snapshot rather than a standing description.
Token Observe takes the opposite trade for the opposite reason. It occupies the base URL, and the eleven-step path is ordered deliberately: authenticate, resolve the agent, open the trace, sanitise unicode so smuggled invisible characters cannot slip past a detector reading a different string from the one the model will read, run the personal-data and injection detectors, then take one verdict, then enact it, then route, then call upstream, then govern any tool call the model proposes, then meter and record. The verdict is a single point rather than a set of middlewares, and it happens before the payload leaves your network. A tracing tool tells you afterwards that an agent sent a customer’s card number to a provider; a chokepoint refuses the payload so it never arrives.
The cost of that position is not hidden anywhere in the product’s documentation and should not be hidden here. Being in band means the failures of the evidence layer are the failures of the agents. The boot sequence performs a full streamed walk of the audit chain before the process listens, and a walk that finds intrinsic corruption latches readiness and audit writes unavailable; later governed requests receive a 503 carrying ACP_AUDIT_UNAVAILABLE, restarting does not clear it, and recovery means restoring a database whose chain and independently retained head both verify. A product that exists to produce the record cannot serve traffic it has stopped recording. LangSmith has no equivalent problem because it is not holding anything back, and a reader who wants visibility without a new dependency in the request path should choose accordingly.
- What LangSmith publishes about its own path
- An async callback handler to a distributed collector, native tracing for popular agent frameworks and OpenTelemetry, and SDKs in Python, TypeScript, Go and Java, with the explicit claim that application performance is never impacted.
- What Token Observe publishes about its own path
- Eleven steps, load-bearing order, one decision point at step 6, and a fail-closed posture whose recovery procedure is a database restore rather than a toggle.
- Why this is not a feature race
- Neither position can be adopted by the other without becoming the other. An inline tracing tool would be a liability, and a bypassable control would not be a control.
Both products redact, and they are protecting different people from different things
LangChain documents redaction in two places, and conflating them makes the comparison useless. The first runs in your process before anything is transmitted to them: LANGSMITH_HIDE_INPUTS and LANGSMITH_HIDE_OUTPUTS as environment variables, hide_inputs and hide_outputs callables on the Client, and a create_anonymizer helper taking regex patterns or callables, with the explicit note that the anonymizer “might incur a performance hit with complex regular expressions or large payloads, as the anonymizer serializes the payload to JSON before processing”. That is redaction whose beneficiary is the boundary between you and LangChain.
The second is the gateway’s, and it defends the same boundary Token Observe does. Their data-protection page states that when a policy is active “the gateway scans outbound requests before they reach the LLM provider”, using Presidio for named entities — “names, locations, and NRP” — and regular expressions for US SSNs, US phone numbers and a list of provider credentials and private keys; matches are replaced with SAFE_TO_USE placeholders in both the request and the trace, since “redacted content is also redacted in the LangSmith trace”, and “as the upstream provider is returning a response, the gateway will replace the redaction placeholders with caller’s original values”. It is worth reading their limits in their own words rather than ours: model responses are not redacted and “streaming response redaction is in progress”; “system prompts, developer prompts, and tool-call arguments are not scanned”; traces written directly to the API bypass it entirely; and scanner failures fail closed.
Token Observe’s redaction covers that same egress boundary and extends to the surfaces their page excludes. The detectors cover eleven sensitive-data classes, of which the card number, the IBAN and the NHS number are checksum-validated, with confidence published per kind rather than implied, and the redaction plan produced by the verdict is applied to the payload on its way to the model provider. The response side is where the engineering sits, because a streamed answer cannot be re-decided once bytes are on the wire: outbound streams pass a hold-back buffer with a 64-character floor and a separate buffer per tool-call argument channel, and the cut is pulled back off anything it would split, so a value that has not finished arriving cannot have its head emitted before the detectors have seen it whole.
Neither product is a data-loss-prevention system and neither should be sold as one. Token Observe’s own material calls its detection a compensating control rather than a customer’s only such system, and its licence disclaims any warranty that the policy, redaction, injection-detection and routing controls identify every instance of what they are designed to detect. The differences that survive are specific and checkable — the response and streaming path, tool-call arguments, system prompts, and whether the component is in a stable self-hosted release — and those, rather than pattern-list length, are the questions to put to both vendors.
Telemetry is kept for the team that owns the service; evidence is kept for somebody else
The difference shows up first in retention, and LangChain publishes theirs clearly: base traces at 14 days, extended at 400, customisable at workspace level on Enterprise, with traces no longer accessible in the UI or the API after expiry and some metadata retained indefinitely for analytics and billing. Those are sensible defaults for a debugging corpus, and the auto-upgrade path — an online evaluator or an automation rule can move a base trace to the extended tier for a fee — is documented rather than surprising. Token Observe’s default is the opposite and for a different reason: retention is unset, unset means keep forever, and the argument for that default is that retention should be decided rather than inherited and an upgrade that silently began deleting a customer’s evidence would be the worse failure.
It shows up second in what the record is made of. The audit log is hash-chained, so each row’s hash covers its canonical content plus the previous row’s and any edit or deletion breaks verification at a named sequence number; appends take the chain tip inside the same transaction so concurrent writers cannot fork it; an optional Ed25519 anchor seals the head on a schedule and publishes it to a sink you site outside the database administrator’s control. What an anchor buys is exactly one thing, and overclaiming it would be the easiest mistake on this page: any copy you kept off-box beats any rewrite made after you took it. The chain is tamper-evident, not tamper-proof, and the compliance export is sealed with a SHA-256 digest rather than signed, so recomputing the digest detects an edit after issue but does not establish who issued the file.
It shows up third in who is allowed to read it and whether that read is itself an event. Pulling up one named person’s prompt history is a privileged read of a personal-data store the customer did not have before they deployed agents, so listing, searching, opening and exporting each append an audit entry naming the actor, the filter as interpreted, the teams the account was effectively authorised for and the number of rows returned. Search is constrained for the same reason it is useful: the translation model’s only permitted output is a JSON filter object validated against fourteen allow-listed keys before it reaches a parameterised query builder, never SQL, because trace content holds prompts, tool arguments and tool results some of which were written by an external party who wanted them read. The cost is stated beside the benefit — the filter cannot group, count or correlate, and misinterpretation replaces injection as the main failure mode, with the explanation shown beside the results being advisory rather than a proof.
- What LangChain publishes about retention
- 14-day base and 400-day extended tiers, workspace-level customisation on Enterprise, deletion of user data from internal systems within a day of expiry, and metadata possibly retained indefinitely for analytics and billing.
- What Token Observe publishes about retention
- Unset by default and unset means keep forever; an hourly pass in batches of 250 once a window is set; and the stated limit that erasure and retention act on the live primary database only and reach neither approvals, radar findings nor webhook deliveries.
- The wording that matters to a reviewer
- Tamper-evident rather than tamper-proof, and digest-sealed rather than signed. Both distinctions are the product’s own and both survive contact with a security questionnaire precisely because they were not inflated.
What running both actually looks like, and what it does not buy you
The shape is straightforward because the two products consume different things. Your agents keep the LangSmith SDK and keep sending runs and traces to whichever LangSmith deployment you have chosen — cloud with US or EU residency, hybrid with the data plane in your VPC, or self-hosted on Kubernetes — and the developer loop is unchanged. Their base URL changes to point at Token Observe, so the same calls now pass a policy verdict, a redaction plan, a budget check and a recorded decision on their way out. Two records exist afterwards, and they are not redundant: LangSmith holds the reasoning of your application in the detail an engineer needs, and Token Observe holds the decisions taken about it in the form a compliance officer can export.
Token Observe also ingests OpenTelemetry rather than competing for it. The receiver takes OTLP over HTTP for logs, traces and metrics in bounded JSON and protobuf, answers in the request encoding, and attributes every write to the seat or agent credential that presented it. What that feeds is the observed view of a four-view reconciliation — what was declared in the registry, what a manifest locked, what is deployed, and what has actually been seen — where the difference between the views is the finding. The published limits belong in the same paragraph: protobuf interoperability has repository tests rather than a live collector and vendor compatibility matrix, and the receiver does not attest the device or exporter that produced the telemetry, so collection trust is an open gap and is named as one.
What running both does not buy you is a single pane of glass, and pretending otherwise would set up a disappointment on day three. The two records have different shapes, different retention and different readers, and Token Observe has no trace viewer for application spans, no dataset management, no experiment runs and no evaluation harness to display them in. The integration is deliberately shallow — a base-URL change on one side, an optional OTLP feed on the other — and that shallowness is the point, because the alternative is a coupling that makes your tracing tool a dependency of your control plane.
Where each one is the right answer
Choose LangSmith when
- The question you need answered is why a chain is slow, why answer quality dropped, or which prompt version regressed. Token Observe records governed requests and their decisions, not the reasoning of your application.
- You need datasets, experiment runs, online evaluators, annotation queues or a judge harness. Those were designed for in Token Observe’s data model and deliberately not built, and building another one is a stated non-goal rather than a backlog item.
- You cannot accept a fail-closed component in the request path, which is an entirely legitimate position for an estate whose agents draft text a human reads before anything happens.
- A third-party attestation is a gate on the purchase. LangChain’s enterprise documentation names SOC 2 Type II, HIPAA and GDPR; Token Observe holds no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, and says so itself.
- You want a managed service with a published price you can put on a card today, rather than a self-hosted deployment quoted against a written scope.
Choose Token Observe when
- You need the payload stopped rather than annotated, because the sensitive value leaving your network is itself the incident rather than something to investigate afterwards.
- A named human has to approve a specific action before it happens, and the approval needs to be bound to that exact payload, single-use and expiring rather than a general permission.
- The reader of the record is an auditor or a compliance officer, reads of it need to be attributable to a named person, and the export needs to carry a chain verdict naming where any break occurred.
- Spend has to stop rather than be reported: per-agent budgets for the request, hour, day and month, reserved before egress, with an unpriced target refused outright.
- You already run LangSmith and the gap you have found is enforcement rather than visibility.
When you would run both
Running both is the normal answer and the one Token Observe’s own roadmap points at, because another standalone tracing and evaluation product is a named strategic non-goal rather than something in the queue. Keep LangSmith where it already is: your engineers keep the SDK, the annotation queues, the datasets, the online evaluators and the dashboards, and nothing about the developer loop changes. Put Token Observe at the base URL in front of it, so the same calls acquire an identity, a deny-by-default permission set, a budget, a policy verdict taken before egress and a hash-chained record of that verdict. Point your OTLP feed at Token Observe as well if you want the observed view of the reconciliation, remembering that it is an input rather than a competing destination and that the receiver does not attest the exporter that produced the telemetry. The division of labour is clean because the two products were built for different readers: LangSmith answers what the agent did and how well, and Token Observe answers whether it was allowed to and who can prove it.
The category argument sits above this one: Token Observe and llm observability covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
Langfuse
Langfuse says in its own documentation that its SDKs are asynchronous and that blocking is a guardrail library’s job. That sentence is the whole comparison, and it is not a criticism.
Arize
Arize is instrumented into your application and reads what it did. Token Observe is a hop your agents call through and decides what they may do.
Datadog LLM Observability
Datadog puts LLM spans beside the rest of your telemetry. Token Observe puts a verdict in front of the call. Datadog also sells a verdict — and where it is made is the whole comparison.
Braintrust
Braintrust is in the request path too. What it does there is deliver the call and record it; what it blocks is the release that would have made the call worse.
Does Token Observe replace LangSmith?
No, and it is not trying to. Token Observe has no prompt playground, no dataset management, no experiment runs and no evaluation harness, and the product’s roadmap names another standalone tracing and evaluation product as a strategic non-goal, so those are absent by decision rather than by schedule. The overlap is genuine — both products keep a per-request record — but LangSmith’s exists so an engineer can debug a chain and Token Observe’s exists so a compliance officer can prove a decision, and the two records have different shapes, different retention and different readers. The intended arrangement is both.
LangSmith says tracing never affects application performance. What does Token Observe cost the request?
It costs whatever the governance pipeline takes, plus the risk that the pipeline is unavailable, and those are different problems. LangSmith’s callback handler is asynchronous and out of band, so its worst failure is lost visibility; Token Observe is in band, so its worst failure is that governed agents cannot call models at all — a corrupt audit chain latches readiness and returns a 503 carrying ACP_AUDIT_UNAVAILABLE until a database whose chain verifies is restored, and there is deliberately no online clear. The product publishes a reproducible governed-path latency baseline with its methodology and its explicit non-claims rather than a marketing figure, and it publishes no availability percentage at all, on the stated grounds that the vendor does not operate your deployment and has no telemetry from it.
Can LangSmith block a request the way Token Observe does?
Yes, on cost and throughput, and this is the answer that changed most recently. Their LLM Gateway — documented as in beta — evaluates spend policies “on every incoming request with sub-second enforcement latency” and “blocks any request that would push spend past the cap, returning a 402 response”, and rate limit policies return “a 429 response with a Retry-After header”. Their gateway also redacts on content before egress, and fails closed if a scanner is unavailable. What their published documentation as of 2 September 2026 does not describe is a verdict that refuses a request because of what is in it rather than what it costs, or one that routes a request to a named human for approval before it proceeds; their automation rules, by contrast, act after the fact on independent polling schedules. Token Observe’s mechanism is a single verdict at step 6 of an eleven-step path returning allow, block, redact or require approval, enacted before the payload leaves your network, with the caller receiving ACP_POLICY_BLOCKED and the trace closing as blocked. If cost and throughput are the ceilings you need, their gateway already does it; put the content and approval questions to LangChain in writing rather than treating this page as settling them.
Both products redact. Do we need both?
LangChain redacts on two boundaries and Token Observe on one of them, so the honest answer is that this overlaps more than it used to. Their client-side masking — environment variables, hide_inputs and hide_outputs callables, and a create_anonymizer taking regex or callable rules — runs in your process before anything is sent to their backend, and protects the boundary between you and LangChain; Token Observe does not compete with that at all. Their gateway redaction defends the same boundary Token Observe does, scanning outbound requests “before they reach the LLM provider” and restoring the caller’s values in the response. The differences left are the ones their own page names: model responses are not redacted and “streaming response redaction is in progress”, “system prompts, developer prompts, and tool-call arguments are not scanned”, and traces written directly to their API bypass the gateway entirely. Token Observe covers those surfaces and holds back streamed output so a value split across chunks cannot escape masking. Neither is a data-loss-prevention product: Token Observe’s detection is heuristic, its own material calls it a compensating control rather than your only such system, and its licence disclaims any warranty that the detectors catch every instance.
How do the deployment and assurance stories compare?
LangChain is ahead on both and it is worth being direct about it. Their enterprise documentation publishes three deployment shapes — managed cloud with US or EU residency, a hybrid mode with the control plane in their cloud and the data plane in your VPC, and fully self-hosted on Kubernetes as an Enterprise add-on — and names SOC 2 Type II, HIPAA and GDPR alongside a shared responsibility model. Token Observe is self-hosted only, runs as one Node process over one SQLite file with PostgreSQL available behind the store ports as an evaluation alternative rather than a supported high-availability topology, and holds no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test. What it offers in place of a certificate is verifiability: the vendor receives no telemetry, prompts, keys or trace database, the data flow is documented so you can check that, the defect list is published with the attacks that still work, and the licence expressly permits you to penetration-test your own deployment before you buy.
Are the LangSmith claims on this page tested?
No. Everything in the LangSmith column paraphrases LangChain’s own published material read on 2 September 2026 — the product page, the documentation home, the pricing page, the four LLM Gateway pages, and the audit-logs, billing, enterprise, self-hosted, RBAC, rules, usage-and-billing, data-export, masking and observability-concepts pages — and none of it has been independently verified. The gateway in particular is documented as in beta and is moving quickly, so those rows date faster than the rest. These products change quickly, so treat any cell that reads as an absence as a question to put to LangChain rather than as a finding, and check the pricing figures against their own page rather than this one before they reach a business case.
Prefer to ask a person? Write to us →
Tell us which one you are already running.
If LangSmith is already in your stack, the useful question is not which to buy but what each is for, and where the seam between them sits. Say what you have and you will get a straight answer — including when the answer is that you do not need a second thing.
no form · no qualification step · no sales desk · the other three ways in