Token Observe vs Datadog LLM Observability
Datadog puts LLM spans beside the rest of your telemetry. Token Observe puts a verdict in front of the call. Datadog also sells a verdict — and where it is made is the whole comparison.
2 September 2026
- Datadog Agent Observability product page (formerly LLM Observability)
- Agent Observability documentation home
- Agent Observability SDK setup
- Agent Observability instrumentation
- Automatic instrumentation
- Agent Observability HTTP API
- Agent Observability terms and concepts
- Agent Observability evaluations
- Agent Observability cost monitoring
- Agent Observability metrics
- AI Guard
- Set up AI Guard
- AI Guard SDK
- Get started with AI Guard
- Datadog RBAC permissions
- Datadog Audit Trail
- Datadog data security
- Datadog pricing
Their claims, not our testing. Verify anything that decides it for you.
On this page
If you already run Datadog, Datadog wins the observability half and it is not close
The advantage is correlation, and it is structural rather than a feature Token Observe could add. Datadog’s pitch for this product is that LLM spans live in the same platform as everything else you already send it, and their documentation describes the machinery for that plainly: SDKs for Python, Node.js and Java whose auto-instrumentation captures, in their words, “Input prompts and output completions”, “Token usage and costs”, “Latency and error information” and “Model parameters”, so that with a supported framework “no manual span creation is required for LLM calls”; seven span kinds — LLM, workflow, agent, tool, task, embedding and retrieval — that model an agent run rather than a single completion; `ml_obs.*` metrics derived from those spans that are “100%-sampled”, follow “standard Datadog metric retention (15 months at full granularity)” and are “queryable from dashboards, monitors, and notebooks like any other Datadog metric”. When the answer to why is this agent slow turns out to be a database that is slow, Datadog is the only product on this shortlist that will show you both in the same trace. Token Observe cannot, does not try, and has no roadmap item that would let it.
The second advantage is the breadth of the instrumentation list, and it is worth reading before assuming coverage is comparable. Their auto-instrumentation page names, for Python alone, Amazon Bedrock and Bedrock Agents, Anthropic, the Claude Agent SDK, CrewAI, Google ADK, Google GenAI, LangChain, LangGraph, LiteLLM, MCP, MistralAI, OpenAI and Azure OpenAI, OpenAI Agents, Pydantic AI, Strands Agents, Vertex AI and vLLM; for Node.js, Amazon Bedrock, Anthropic, LangChain, MCP, OpenAI and Azure OpenAI, the Vercel AI SDK, Vertex AI and Google GenAI; for Java, OpenAI and Azure OpenAI; and an HTTP API for everything else. Evaluations sit on top of that: managed evaluations Datadog “builds and supports ... to support common use cases”, custom LLM-as-a-judge evaluations that “allow you to define your own evaluation logic using natural language prompts”, checks that in their words “automatically scan and redact any sensitive data in your AI applications and identify prompt injections”, topic clustering, anomaly detection over span names and workflow types, and human annotation and grading of outputs. Token Observe has none of it. Evaluation harnesses, semantic caching and session replay were designed for in the data model and deliberately not implemented, and another standalone tracing and evaluation product is a named strategic non-goal rather than a queue item.
The third is the one a security review reaches first, and the direction of the concession should be obvious. Datadog is a public company running a managed platform with a published compliance posture; its product page offers to let you “Confidently run AI in production with precise alerting, role-based access control, sensitive data protection, HIPAA compliance, and the governance teams need to reduce risk”, and its data-security documentation describes TLS and HSTS in motion, encryption and access controls at rest, Sensitive Data Scanner as a “stream-based, pattern matching service”, and a secure credential datastore for integration credentials. Token Observe holds no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, states all four in its own security policy, and offers in their place a published defect list naming the attacks that still work and a licence clause expressly permitting you to penetration-test your own deployment before a purchase order. Everything said about Datadog on this page comes from Datadog’s own published material read on 2 September 2026 and has not been independently tested; where a row reads as an absence, treat it as a question to put to Datadog in writing rather than as a finding.
Token Observe and Datadog LLM Observability, capability by capability
The Datadog LLM Observability column paraphrases Datadog’s own published material as it stood on 2 September 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Check anything that decides it for you against their own current documentation.
Where each one sits
Beside it. Their SDK documentation describes `ddtrace` for Python 3.7+, `dd-trace` for Node.js 16+ and a `dd-trace-java` JAR for Java 8+, with auto-instrumentation capturing input prompts and output completions, token usage and costs, latency and error information and model parameters, plus an HTTP API that submits spans to `/api/intake/llm-obs/v1/trace/spans` on your Datadog site for other languages.
In it. You change `OPENAI_BASE_URL` or `ANTHROPIC_BASE_URL` and one key, and every governed request runs the same eleven-step path with a single policy verdict at step 6, before anything leaves your network.
Through the Datadog Agent by default; their setup page documents `DD_LLMOBS_AGENTLESS_ENABLED=1` for sending directly when you are not running the Agent, with `DD_API_KEY` for authentication and `DD_SITE` naming the destination.
It does not travel. The deployment is self-hosted and bring-your-own-key, and the vendor receives no product telemetry, phone-home data, prompts, keys or trace database — a claim the documented runtime data flow is written to let you verify rather than accept.
A span, in seven kinds. An LLM span is “a call to an LLM”; workflow, agent, tool, task, embedding and retrieval spans nest inside a trace that “represents the work involved in processing a request”.
A trace is one governed request. Its id is minted at step 3 — before unicode sanitisation, before the detectors and before the verdict — so a request blocked a millisecond later is recorded rather than missing, and it returns on `x-acp-trace-id` on every response including refusals.
Named per language on their auto-instrumentation page. Python covers Amazon Bedrock and Bedrock Agents, Anthropic, the Claude Agent SDK, CrewAI, Google ADK, Google GenAI, LangChain, LangGraph, LiteLLM, MCP, MistralAI, OpenAI and Azure OpenAI, OpenAI Agents, Pydantic AI, Strands Agents, Vertex AI and vLLM; Node.js covers Amazon Bedrock, Anthropic, LangChain, MCP, OpenAI and Azure OpenAI, the Vercel AI SDK, Vertex AI and Google GenAI; Java covers OpenAI and Azure OpenAI.
Six first-class upstreams — OpenAI, Anthropic, Gemini, OpenRouter, Amazon Bedrock and Azure OpenAI — plus OpenAI-compatible endpoints of your own, with identical policy, redaction, budget and trace behaviour across all of them enforced by a table-driven test over every provider kind.
The two lists count different things and should not be read as a score. Datadog’s names the libraries it can instrument inside your process; Token Observe’s names the providers it can route to and refuse on. A framework Datadog instruments and Token Observe has never heard of is still governed when its calls go through the base URL, and an agent that is neither instrumented nor pointed at the base URL is invisible to both.
Add the SDK and set environment variables — `DD_LLMOBS_ENABLED` and `DD_SITE` are documented as required, `DD_API_KEY` only when you are not using the Datadog Agent, and `DD_LLMOBS_ML_APP` as optional on current SDK versions and required on earlier ones — after which supported frameworks need no manual span creation for LLM calls. Integrations are enabled by default and can be narrowed with `integrations_enabled=False` in Python or `plugins: false` in Node.js.
One environment variable and one key. For supported OpenAI-compatible, Anthropic and Gemini ingress that is normally the whole integration rather than an application refactor, which is also how it reaches a vendor binary whose source you cannot change.
The integration is a telemetry exporter with sampling as a supported control: `DD_LLMOBS_SAMPLE_RATE`, default `1.0`, which their SDK documentation says is “available in the Python SDK (ddtrace 4.12.0 or later) and the Node.js SDK (dd-trace 5.110.0 or later)” while “The Java SDK does not support trace sampling.” What their SDK does to a request when the destination is unreachable is not described in their published documentation as of 2026-09-02; ask Datadog directly rather than reading this cell as a finding.
They stop. A boot-time streamed walk of the audit chain that finds corruption latches readiness and audit writes unavailable, and governed requests then receive a 503 carrying `ACP_AUDIT_UNAVAILABLE`. The latch survives a restart deliberately — there is no online clear — so recovery means restoring a database whose chain and independently retained head both verify.
This is the strongest single argument against Token Observe on the page. An out-of-band exporter degrades by losing visibility; an in-band chokepoint degrades by refusing traffic. That trade is deliberate — a control you can bypass by turning it off is not a control — but it is why an SDK can be adopted by one team on a Tuesday and this cannot.
What each one enforces
A different one. The LLM Observability material describes monitoring, troubleshooting and evaluating; inline enforcement is published separately under Security as AI Guard, which their documentation describes as sitting “inline with your AI app/agent” and layering “on top of existing prompt templates, guardrails, and policy checks, to secure your LLM workflows in the critical path”.
One product and one decision point. The verdict at step 6 returns allow, block, redact or require approval, and it is the same evaluation for the prompt, for the response and for any tool call the model proposes on the way back.
This row is why the page exists in this shape. Datadog can refuse a call; the comparison is not enforce-versus-observe, it is where the refusal is decided and what is holding the payload while it is decided.
AI Guard’s setup documentation describes evaluations returning an action of `ALLOW`, `DENY` or `ABORT`, and states that they return an action “but does not block requests” by default; blocking must be explicitly configured, after which `DENY` and `ABORT` “actively prevent unsafe interactions from proceeding”.
allow, block, redact or require approval, enacted before the payload leaves your network. Refusal is what the verdict does rather than a setting layered over it; what is opt-in here is the reverse — shadow mode, so a policy runs and records for a while before it starts refusing real work and you learn your false-positive rate first.
Hierarchically, by service and environment: an organisation default, then per-environment, per-service and per-service-and-environment overrides, with the most specific winning. Evaluation sensitivity is “a value between 0.0 and 1.0, with a default of 0.5”, where a lower value increases sensitivity, and a Tool Blocklist blocks requests “for specific tools, for specific services and environments”.
By agent, and underneath the policy by permission. Permissions are action-level and deny-by-default: a support agent may `Read: Customer Account` and `Update: Shipping Address` while `Delete: Account` is simply absent and therefore denied. Explicit denies win, and delegation chains intersect permissions across every hop so one agent cannot escalate by asking a higher-privileged one.
Human review of recorded outputs is part of the evaluation loop: their product page offers to “Annotate and review outputs” and to “Label and grade outputs with annotations and human review”, bringing “expert judgment into evaluation where automated checks fall short”. A human decision that gates one specific call before it is made is not described in their published documentation as of 2026-09-02; ask Datadog rather than reading this cell as a finding.
A 403 carrying an approval id, bound to the SHA-256 of the canonical action plus its execution context, single-use and expiring at 60 minutes by default, with the approver’s identity and rationale recorded against the trace. A retry with one argument changed does not match the hash and is not approved.
Measured in detail. Their cost documentation describes an estimated cost per LLM request computed from providers’ public pricing across “800+ models”, with provider-specific cache read and cache write rates applied, surfaced as `ml_obs.*` cost metrics — `ml_obs.span.llm.total.cost` and its cache-read and cache-write siblings, in nanodollars — that you can dashboard, monitor and alert on. A ceiling that stops a call is not described in their published documentation as of 2026-09-02.
Budgets and rate limits per request, hour, day and month, projected and reserved in one per-agent transaction before egress, with an unpriced resolved target or fallback refused outright for any budgeted agent rather than billed at zero.
An alert and a ceiling are different instruments and most estates want both. Datadog’s monitor tells a named human that spend is climbing; Token Observe’s budget is the reason the next call did not happen at three in the morning.
Two mechanisms in their material. Sensitive Data Scanner, described on their data-security page as a “stream-based, pattern matching service” for identifying and redacting sensitive information, and out-of-the-box evaluations that “automatically scan and redact any sensitive data in your AI applications and identify prompt injections”. AI Guard’s own Sensitive Data Protection, which their overview page says “detects sensitive data such as personally identifiable information (PII) and secrets in LLM inputs and outputs”, is described in their setup documentation as “detection-only; findings do not independently trigger blocking”.
Eleven sensitive-data classes, three of them checksum-validated, redacted server-side before egress to the provider, with a hold-back buffer on streamed responses whose floor is 64 characters and whose cut is pulled back off anything it would split, so a card number arriving across two chunks cannot escape masking.
Neither product is a data-loss-prevention system. Token Observe’s detection is heuristic, its own material calls it a compensating control rather than your only such system, and its licence disclaims any warranty that the detectors identify every instance.
What each one records
Per-span visibility their documentation describes as input prompts and output completions, token usage and costs, latency and error information and model parameters, with traces that “can include input and output, latency, privacy issues, errors, and more”, rolled up into `ml_obs.*` span, token, cost, trace and embedding metrics.
The post-redaction prompt excerpt, the tool calls and their arguments, the policy decisions including shadow-mode ones, the human approvals with approver and rationale, the tokens and the cost, in a timeline that explains each step in a plain sentence rather than a log line.
The platform’s own query surfaces. `ml_obs.*` metrics are “queryable from dashboards, monitors, and notebooks like any other Datadog metric”, alongside the trace views and evaluation results in the product.
A question in English translated into a validated filter object over fourteen allow-listed fields, never into SQL, returned beside the results as editable chips so you can see how it was read; a deterministic keyword parser answers when no translation model is configured or the call fails.
The constraint is deliberate and it costs something. Trace content holds prompts, tool arguments and tool results, some written by an external party who wanted them read, so anything derived from it that reached an interpreter would be an injection surface. The price is that the filter cannot group, count or correlate across traces — which agents used the same card number twice is not a question you can ask.
Managed evaluations Datadog “builds and supports ... to support common use cases” and custom LLM-as-a-judge evaluations that “allow you to define your own evaluation logic using natural language prompts”, out-of-the-box checks that “automatically scan and redact any sensitive data in your AI applications and identify prompt injections”, Patterns to “identify coverage gaps, and monitor the quality of responses over time”, outlier detection across span name and workflow type, and human annotation and grading.
None. There is a scores table in the data model and nothing that fills it; evaluation harnesses, semantic caching and session replay were designed for and deliberately not implemented, on the principle that speculative generality is worse than an absent feature.
Datadog Audit Trail records requests to Datadog’s API as customer records plus product-specific events, covers configuration changes with an Inspect Changes diff over dashboard, notebook and monitor configuration, exports “up to 100K audit events as a CSV file locally” and archives to “Amazon S3, Google Cloud Storage, or Azure Storage”. A statement about tamper-evidence or immutability of that trail is not described in their published documentation as of 2026-09-02.
A hash-chained audit log in which each row’s hash covers its canonical content plus the previous row’s, so an edit or deletion that does not also recompute every downstream hash breaks verification at a named sequence number, with appends taking the chain tip inside the same transaction so concurrent writers cannot fork it, an optional audit MAC key that makes the digests HMAC-SHA256 and seals the head at every boot, and an optional Ed25519 anchor published on a schedule to a sink you site outside the database administrator’s control.
The wording matters more than the mechanism and is worth keeping precise: the chain is tamper-evident, not tamper-proof, and a compliance export is sealed with a SHA-256 digest rather than signed. The default is unkeyed, where an operator with write access to the database file can rewrite an entry, recompute the rest and have verification report valid — the repository ships a forgery test asserting exactly that, and asserting that it fails once a key is configured. What an anchor buys is exactly one thing — any copy you kept off-box beats any rewrite made after you took it.
Set per surface. Audit Trail defaults to 90 days with settable values of 3, 7, 15, 30 or 90; `ml_obs.*` metrics “follow standard Datadog metric retention (15 months at full granularity)”; span retention is configured with retention filters, which AI Guard’s onboarding also asks you to set up.
Unset by default, and unset means keep forever, on the argument that retention should be decided rather than inherited and that an upgrade which silently began deleting a customer’s evidence would be the worse failure. Once a window is set an hourly pass deletes in batches of 250, and erasure and retention act on the live primary database only.
Role-based access control with three managed roles — Admin, Standard and Read Only — and custom roles combining permissions. `llm_observability_read` sits on Read Only and `llm_observability_write` on Standard, with `ai_guard_view`, `ai_guard_evaluate` and `ai_guard_write` for the Security product.
Roles plus an explicit list of team scopes on each human account, with the query predicate derived server-side and not widenable by a query parameter, and every list, search, detail and export read appended to the audit log naming the actor, the filter as interpreted, the teams the account was effectively authorised for and the number of rows returned.
Pulling up one named person’s prompt history is a privileged read of a personal-data store the customer did not have before they deployed agents, which is why the read is an event here rather than a page view.
How each one deploys
Datadog’s platform. Setup names a Datadog site with `DD_SITE` and sends spans through the Datadog Agent or agentless with an API key; the same documentation states the product “is not supported for your selected Datadog site” for `app.ddog-gov.com` and `us2.ddog-gov.com`.
Self-hosted only, in your network, on your provider keys. One Node process and one SQLite file in WAL mode, with PostgreSQL implemented behind the store ports as an evaluation alternative under dual-backend CI and explicitly not a supported high-availability topology, multi-replica claim or point-in-time-recovery result.
A managed service Datadog operates, with the region chosen by `DD_SITE`. Their commercial availability commitments are a contract question for Datadog rather than something these documentation pages set out.
A single-writer process on one host at this scale: no replica, no clustering and no vendor-operated uptime SLA. The stated reason is that the vendor does not operate your deployment and receives no telemetry from it, so an uptime number from that party would be unmeasurable by either side.
Spans carrying input prompts and output completions go to the Datadog site you name, which is what makes their client-side controls relevant: Sensitive Data Scanner for identification and redaction, and the Agent- and tracer-level “custom obfuscating, scrubbing, excluding, and modifying of trace-related elements” their data-security page points to.
Governed payloads leave only for the model and tool providers you configure, after policy and redaction. Whether your support arrangement needs a data processing agreement is a contractual question for your counsel; the runtime data flow itself is documented so the claim can be checked rather than believed.
A second product with its own onboarding, and their setup page walks it in order: create API and application keys, instrument your application with an SDK — Python, JavaScript, Java or Ruby — or call the AI Guard REST API, create a custom retention filter for `resource_name:ai_guard` at 100% span and trace rate, and configure AI Guard policies including blocking, evaluation sensitivity and sensitive data scanning. “When adding scopes for the application key, add the `ai_guard_evaluate` scope.”
Nothing to add. Permissions, redaction, injection heuristics, approvals, budgets, rate limits, the flight recorder, the audit chain and the kill switch are the same deployment and the same request path, and every one of them enforces identically under the licence’s thirty-day evaluation grant and in a paid deployment.
Datadog publishes its compliance posture on its own security and trust material, which was not read for this page; the product page offers “precise alerting, role-based access control, sensitive data protection, HIPAA compliance, and the governance teams need to reduce risk”, and the data-security documentation describes TLS and HSTS in motion, encryption and access controls at rest, and a secure credential datastore for integration credentials.
None yet: no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test, each stated in the product’s own security policy rather than left to be discovered. What is offered instead is a published defect list naming the attacks that still work, a STRIDE threat model with residual risk on every row, and a licence clause expressly permitting you to inspect, fuzz and penetration-test your own deployment before a purchase order, with no gag clause.
What each one costs
Span volume. Their setup documentation states that billing is based on the volume of spans you send, and names trace sampling as one way to control the cost, configured with `DD_LLMOBS_SAMPLE_RATE` (default `1.0`) in the Python and Node.js SDKs and not supported in Java.
A subscription to one self-hosted deployment, metered on the number of deployments you run and the number of agents licensed to be active at once. There is no per-token or per-request component and there cannot be one, because governed traffic goes to your own provider accounts and the software reports nothing that could be metered.
Datadog publishes list pricing per product on its pricing page and lists Agent Observability among its AI products, with a note that it is currently unavailable on the US-FED site. A per-span rate for this product did not render on that page when it was read on 2026-09-02, so take the current figure from Datadog rather than from here.
None. The licence a figure would be quoted under says on its own first page that it must be reviewed and approved by qualified counsel in England and Wales before it is offered to or relied upon by any customer, and that has not happened, so every priced line is on application.
More spans, which is why sampling is documented as a cost control. Sampling below `1.0` sends fewer of them.
Nothing about traffic volume. What grows instead is your disk, because trace retention is unset by default and unset means keep forever.
The trade runs in opposite directions and both directions cost something. Sampling makes a telemetry bill predictable and makes the record incomplete; keeping everything makes the evidence usable and makes the storage yours to plan for. Neither default is wrong for the job the other product is doing.
Estimated per request from providers’ publicly available pricing across “800+ models”, with provider-specific cache read and cache write rates applied to cached input, and an explicit warning that “providing only a partial cache breakdown may result in inaccurate token counts or cost discrepancies”.
Normalised into mutually exclusive buckets before any arithmetic, then priced, reserved against the agent’s budget in the same per-agent transaction and written to a ledger, with USD stored to eight decimal places at write.
Both products handle cache tokens deliberately and both produce an estimate against a provider invoice rather than the invoice. The difference is what the number is for: Datadog’s is a figure you read afterwards, and Token Observe’s is a figure the request has to pass before it is sent.
Trial and plan terms are on Datadog’s own pricing page and are the right place to check them; the technical prerequisite in the documentation is a Datadog API key, plus an application key scoped `ai_guard_evaluate` if you are also evaluating AI Guard.
A thirty-day evaluation grant in the licence for internal evaluation, security review and proof-of-concept purposes, with no licence state gating any control — every enforcement path behaves exactly as it would in a paid deployment, which is the only way an evaluation tells you anything.
Both products can refuse a call. The comparison is what is holding the payload when they do
Take Datadog’s claim at face value, because it is accurate and it is the interesting part of this page: AI Guard sits “inline with your AI app/agent”, operates “in the critical path”, evaluates user input, assistant output and tool calls, and returns `ALLOW`, `DENY` or `ABORT`. Their setup documentation is equally clear about the default — an evaluation returns an action “but does not block requests” until blocking is configured, at which point `DENY` and `ABORT` “actively prevent unsafe interactions from proceeding”. That is a real enforcement product with a real verdict, and anybody writing that Datadog cannot block is describing a version of Datadog that stopped existing.
The distinction that survives is narrower. AI Guard’s evaluation is a call your application makes — through one of their SDKs in Python, JavaScript, Java or Ruby, through one of the automatic integrations their setup page names — LangChain, OpenAI and Anthropic for Python, the AI SDK, OpenAI and Anthropic for Node.js, RubyLLM for Ruby — or by calling the AI Guard REST API directly — so the set of actions it can refuse is the set of actions that were routed through it. Token Observe’s verdict is taken by the process that is holding the payload on its way to a provider, because the agent’s base URL points at it, so an agent that omits the check does not proceed unchecked; it fails to reach the model. Neither position is strictly better. An in-process evaluator sees the surrounding steps of an agent run in a detail a base-URL proxy never has, and it can be added to one service without a change-advisory board. A chokepoint sees less context and cannot be forgotten.
Token Observe’s own documentation names the limit of its position rather than letting a reader over-read it, and the same sentence belongs here. Tool calls are governed at step 10 when the model proposes them in a response and at the MCP gateway when execution routes through it — and that is defence in depth rather than a guarantee, because a proposal it is never shown is a proposal it cannot refuse. An agent that executes its own tools without passing them through anything is caught, if at all, by the shadow-AI radar’s five evidence sources — vendor bill reconciliation, network egress analysis, service-account key audit, IDE and CLI telemetry, and the product’s own caller and price consistency checks — rather than by the request path. That is the honest boundary of a chokepoint, and it is the reason the radar exists.
- What Datadog publishes about AI Guard’s position
- Inline with your AI app or agent, in the critical path, evaluating user input, assistant output and tool calls, returning `ALLOW`, `DENY` or `ABORT`, and blocking only once blocking is configured — with a Tool Blocklist that can block a named tool for a chosen service and environment.
- What Token Observe publishes about its own position
- Eleven steps in a load-bearing order, one decision point at step 6 returning allow, block, redact or require approval, a typed refusal carrying `ACP_POLICY_BLOCKED` when it blocks, and a fail-closed posture whose recovery procedure is a database restore rather than a toggle.
- The question to put to both vendors in writing
- Not whether the product can block, but what happens to an action that never reaches the check — and, on a streamed response, whether a blocking decision can still be taken after the first byte has been written.
A telemetry platform and a governance chokepoint want opposite things from the same data
Datadog’s design pressure is towards completeness across systems and economy per record, which is exactly right for what it does. Spans from an agent sit alongside the infrastructure, APM and log data you already send, `ml_obs.*` metrics are derived from those spans at full sampling and kept at full granularity for fifteen months, and span volume is the meter — which is why sampling is documented as a cost control and why `DD_LLMOBS_SAMPLE_RATE` exists at all. A telemetry corpus is a statistical object: a well-chosen sample answers why is quality dropping perfectly well, and paying to keep every span of a chatty agent forever would be a poor use of a budget.
Token Observe’s design pressure is the opposite, and it comes from who reads the record. Evidence is not a statistical object. A compliance officer asking whether any agent moved more than £500 without a human looking at it, in the last quarter, is not helped by a representative sample; a missing trace is not a gap in a chart, it is the one you will be asked about. So sampling does not exist, retention is unset by default and unset means keep forever, the trace id is minted before the detectors run so a blocked request is recorded rather than absent, and the whole thing is hash-chained so an edit after the fact breaks verification at a named sequence number rather than passing quietly.
The consequence is that neither product’s defaults would suit the other’s job, and an estate running both should expect to plan for both. Datadog’s cost management is a sampling and retention conversation. Token Observe’s cost management is a disk conversation, and the product’s published capacity work gives you storage growth and a reproducible governed-path latency baseline with its methodology and its explicit non-claims rather than a marketing figure. What Token Observe does not offer, and will not, is the correlation: when the slow step turns out to be a database rather than a model, it has nothing to show you, and Datadog has everything.
Telemetry is kept for the team that owns the service; evidence is kept for somebody else entirely
The difference shows up first in what the record has to withstand. Datadog Audit Trail, on their documentation, translates requests made to Datadog’s API into customer records alongside product-specific events, diffs configuration changes to dashboards, notebooks and monitors, exports up to 100,000 events as CSV and archives to S3, Google Cloud Storage or Azure Storage, with retention defaulting to 90 days and settable to 3, 7, 15, 30 or 90. That is a capable audit surface for a platform. Any claim about tamper-evidence or immutability of that trail is not described in their published documentation as of 2 September 2026, which is a statement about those pages on that date rather than a finding about the product — it is a fair question to put to Datadog in writing if the record has to survive an adversarial reading.
Token Observe’s answer to that question is the mechanism rather than an adjective, and the mechanism has published limits that matter more than the mechanism does. Each audit row’s hash covers its canonical content plus the previous row’s, so an edit or deletion that does not also recompute every downstream hash breaks verification at a known sequence number; appends take the chain tip inside the same transaction, so concurrent writers cannot fork it. The default is unkeyed SHA-256, and on that default an operator with write access to the database file can rewrite an entry, recompute every hash after it and have verification report valid — the repository ships a forgery test that performs exactly that on a default install and asserts it succeeds, and asserts it fails once a key is set. Configure an audit MAC key from a secret manager the database administrator cannot read and the digests become HMAC-SHA256 with a checkpoint MAC sealing the head at every boot, so a full recompute needs the key too. Configure an optional Ed25519 anchor and the head is signed on a schedule and published to a sink you site outside that administrator’s control, refusing to sign a chain that does not verify, a head that has moved backwards or a rewritten anchored entry. The claim that follows is exactly one sentence long: any copy you kept off-box beats any rewrite made after you took it. The chain is tamper-evident, not tamper-proof, exports are sealed with a SHA-256 digest rather than signed, and there is no RFC 3161 timestamp and no per-entry inclusion proof — anchoring fixes the head, it does not prove when.
It shows up second in who is allowed to read it and whether reading is itself an event. Datadog’s RBAC is role-and-permission shaped in the way a platform’s has to be: three managed roles, custom roles combining permissions, and named permissions for this product and for AI Guard. Token Observe adds a second axis because the corpus is different — an explicit list of team scopes on each human account, with the query predicate derived server-side and repeated on list, search, detail, single-trace export and the compliance bundle, and surfaces that join records with no trustworthy team key returning 403 to a team-scoped account rather than a narrower and misleading answer. Every one of those reads appends an audit entry naming the actor, the filter as interpreted, the teams the account was effectively authorised for and the number of rows returned.
- What a compliance export contains
- The traces and events for the period, the approvals with approver identity and rationale, the audit entries covering every governance-plane change, a chain verification result naming the sequence number of any break, and a SHA-256 digest of the bundle generated at a recorded time. The bundle is not itself signed.
- What the search box will not do
- Group, count or correlate across traces, because the model’s only permitted output is a JSON filter object validated against fourteen allow-listed keys before it reaches a parameterised query builder. Misinterpretation replaces injection as the failure mode, which is why the interpreted filter is shown beside the results as editable chips.
- What retention does not reach
- Erasure and retention act on the live primary database only, and reach neither approvals, radar findings nor webhook deliveries. That limit is published rather than discovered.
What running both actually looks like, and the one thing it does not buy
The arrangement is straightforward because the two products consume different inputs and neither needs the other to be absent. Your services keep `ddtrace`, `dd-trace` or the Java agent and keep sending spans to your Datadog site, so the dashboards, the `ml_obs.*` monitors, the evaluations and the correlation with APM and logs are all unchanged. Their base URL changes to point at Token Observe, so the same calls acquire an identity, a deny-by-default permission set, a budget reserved before egress, a policy verdict taken before the payload leaves your network and a hash-chained record of that verdict. Two records exist afterwards and they are not duplicates: Datadog holds what your application did in the detail an engineer needs, and Token Observe holds the decisions taken about it in the form a compliance officer can export.
Token Observe also ingests OpenTelemetry rather than competing for it, and the reason is worth stating because it is a strategic position rather than a gap. An OTLP over HTTP receiver takes bounded JSON and protobuf for logs, traces and metrics, answers in the request encoding, and attributes every write to the seat or agent credential that presented it. What that feeds is the observed view of a four-view reconciliation — what was declared in the registry, what a manifest locked, what is deployed, and what has actually been seen — where the difference between the views is the finding. The published limits belong in the same paragraph: protobuf interoperability has repository tests rather than a live collector and vendor compatibility matrix, and the receiver does not attest the device or exporter that produced the telemetry, so collection trust is an open gap and is named as one.
If both products are in the estate, the sensible division of the enforcement work is by what each check can see. AI Guard evaluates inside your application, where the surrounding steps of an agent run are visible, and Datadog’s hierarchical service-and-environment policy is the natural place to express a rule about a service. Token Observe evaluates at the boundary, where the payload is, and its per-agent permissions, payload-bound approvals and pre-flight budgets are the natural place to express a rule about an agent and about money. Running both means two policy surfaces, which is a genuine operational cost and should be a deliberate decision rather than an accident; the mitigation Token Observe offers is that scope and trigger matching can be compiled to digest-locked OPA Rego with reproducible positive and negative witnesses, so another enforcement point can be shown to match — while permissions, budgets, approval consumption, kill switches, action precedence and side effects are deliberately excluded from that artefact and stay authoritative here.
What running both does not buy is a single pane of glass, and implying otherwise would set up a disappointment in the second week. The two records have different shapes, different retention, different readers and different meters, and Token Observe has no trace viewer for application spans, no dataset management, no experiment runs and no evaluation harness to display anything in. The integration is deliberately shallow — a base-URL change on one side, an optional OTLP feed on the other — and the shallowness is the point, because the alternative is a coupling that makes your observability platform a dependency of your control plane.
Where each one is the right answer
Choose Datadog LLM Observability when
- You already run Datadog and the question you need answered spans more than the model — why an agent is slow when the slow part is a database, or which deployment changed the error rate. Token Observe records governed requests and their decisions and has no view of the rest of your estate.
- You need evaluations, topic clustering, anomaly detection over span patterns or human annotation and grading of outputs. Those were designed for in Token Observe’s data model and deliberately not built, and building another evaluation product is a stated non-goal rather than a backlog item.
- Framework depth is the requirement — you want the trace tree of a LangGraph or CrewAI run in the detail Datadog’s auto-instrumentation captures, rather than the record of what was sent to a provider and what came back.
- You cannot accept a fail-closed component in the request path, which is an entirely legitimate position for an estate whose agents draft text a human reads before anything happens.
- A third-party attestation is a gate on the purchase, or the deployment has to be a managed service somebody else operates with a contractual availability commitment. Token Observe is self-hosted only, holds no SOC 2, ISO 27001 or ISO 42001, has had no independent penetration test, and publishes no uptime SLA.
Choose Token Observe when
- The refusal has to happen at the boundary rather than inside the application, because the agents you are worried about include ones whose code you do not control and cannot instrument.
- A named human has to approve one specific action before it happens, bound to that exact payload, single-use and expiring, with the approver and their rationale recorded against the trace.
- Spend has to stop rather than be alerted on: per-agent budgets and rate limits for the request, hour, day and month, reserved before egress, with an unpriced resolved target refused outright.
- The reader of the record is an auditor or a compliance officer, reads of it need to be attributable to a named person, and the export needs to carry a chain verdict naming the sequence number of any break.
- You run more than one provider and the same rule has to fire identically on all of them, because a policy that fires on OpenAI but not on Gemini is worse than no policy.
When you would run both
Running both is the normal answer, and for an estate already paying Datadog it is close to the only sensible one. Keep Datadog where it is: the SDKs stay in your services, the spans keep flowing to your Datadog site, and the dashboards, monitors, evaluations and correlation with APM and logs are exactly the half of this problem Token Observe does not attempt and names as a non-goal. Put Token Observe at the base URL underneath, so the same calls acquire an identity, a deny-by-default permission set, a budget reserved before egress, one verdict of allow, block, redact or require approval taken before the payload leaves your network, and a hash-chained record of that verdict that an auditor can export with a chain verdict attached. Point an OTLP feed at Token Observe as well if you want the observed view of the reconciliation, remembering that it is an input rather than a competing destination and that the receiver does not attest the exporter that produced the telemetry. If you are also evaluating AI Guard, treat the two enforcement points as complementary rather than redundant and decide deliberately which rule lives where: their evaluation sees the surrounding steps of an agent run inside your application, and Token Observe’s sees the payload at the boundary and cannot be skipped. The division of labour is clean because the two products were built for different readers — Datadog answers what the agent did and how well, and Token Observe answers whether it was allowed to and who can prove it.
The category argument sits above this one: Token Observe and llm observability covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
LangSmith
LangSmith’s callback handler watches the call from beside it, and their newer LLM Gateway now stands in it too. The remaining differences are narrower than they were, and worth stating precisely.
Langfuse
Langfuse says in its own documentation that its SDKs are asynchronous and that blocking is a guardrail library’s job. That sentence is the whole comparison, and it is not a criticism.
Arize
Arize is instrumented into your application and reads what it did. Token Observe is a hop your agents call through and decides what they may do.
Braintrust
Braintrust is in the request path too. What it does there is deliver the call and record it; what it blocks is the release that would have made the call worse.
Can Datadog block a request, or does it only observe?
It can block, and a comparison that says otherwise is wrong. Datadog publishes inline enforcement as a separate product under Security: AI Guard, described in their documentation as sitting “inline with your AI app/agent” and operating “in the critical path”, evaluating user input, assistant output and tool calls and returning an action of `ALLOW`, `DENY` or `ABORT`. Their setup page states that evaluations return an action “but does not block requests” until blocking is explicitly configured, after which `DENY` and `ABORT` “actively prevent unsafe interactions from proceeding”, and that a Tool Blocklist can block a named tool for a chosen service and environment. What the LLM Observability material itself describes is monitoring, troubleshooting and evaluation. So the useful question is not whether Datadog enforces but where the check is made: theirs is a call your application makes, so its reach is the reach of your instrumentation, and Token Observe’s is made by the process holding the payload, so an agent that skips it fails to reach the provider instead of proceeding unchecked.
Does Token Observe replace Datadog LLM Observability?
No, and it is not trying to. There is no infrastructure monitoring, no log management, no APM correlation, no evaluation harness, no dataset management and no experiment runner, and the product’s roadmap names another standalone tracing and evaluation product as a strategic non-goal rather than something in the queue. The overlap is genuine — both keep a per-request record with tokens and cost on it — but Datadog’s exists so an engineer can work out why a chain regressed and correlate it with the rest of the estate, and Token Observe’s exists so a compliance officer can prove what was refused and by whom. Different readers, different retention defaults, different meters. The intended arrangement is both.
We already send LLM spans to Datadog. What does adding Token Observe change in our applications?
One environment variable per agent and one key. `OPENAI_BASE_URL` or `ANTHROPIC_BASE_URL` points at your deployment, and for supported OpenAI-compatible, Anthropic and Gemini ingress that is normally the whole integration rather than an application refactor. Your Datadog SDKs stay exactly where they are and keep exporting spans; nothing about `DD_LLMOBS_ENABLED`, `DD_SITE` or your retention filters changes. What changes is that the call now passes an identity, a deny-by-default permission set, a budget reserved before egress and a policy verdict before it reaches the provider, and that a refusal returns `ACP_POLICY_BLOCKED` with a trace id on `x-acp-trace-id` rather than a completion.
Both products calculate the cost of a call. Is that duplication?
It is the same arithmetic used for two different purposes, and the second purpose is why the numbers are computed twice. Datadog’s cost documentation describes an estimated cost per request from providers’ publicly available pricing across “800+ models”, with provider-specific cache read and cache write rates applied to cached input and an explicit warning that a partial cache breakdown can produce inaccurate token counts or cost discrepancies. Token Observe normalises provider usage into mutually exclusive buckets before any arithmetic, prices it, and reserves it against the agent’s budget in the same per-agent transaction before egress — and refuses a resolved target with no price outright for any budgeted agent rather than billing it at zero. Both are estimates against a provider invoice rather than the invoice. The difference is that one is a figure you read and the other is a figure the request has to pass.
What happens to our agents if Token Observe is unavailable?
They stop calling models, and that is deliberate rather than a defect. Being in the request path is what makes an inline refusal possible, and it makes availability a governance property of your environment: a boot-time streamed walk of the audit chain that finds corruption latches readiness and audit writes unavailable, and governed requests then receive a 503 carrying `ACP_AUDIT_UNAVAILABLE`. Restarting does not clear it — there is intentionally no online clear endpoint — so recovery means restoring a database whose chain and independently retained head both verify. A telemetry exporter has no equivalent problem because it is not holding anything back, which is a genuine advantage for Datadog and the reason this page recommends the observability half stay there. The product’s own support documentation asks you to plan for the outage in advance: run it close to the agents, watch the readiness endpoint, and decide before you need to who owns the emergency call.
Are the Datadog claims on this page tested?
No. Everything in the Datadog column paraphrases Datadog’s own published material read on 2 September 2026 — the Agent Observability product page, the documentation home, the SDK setup, instrumentation, auto-instrumentation, HTTP API, terms, evaluations, cost and metrics pages, the AI Guard overview, setup, SDK and onboarding pages, the RBAC permissions and Audit Trail documentation, the data-security page and the pricing page — and none of it has been independently verified. Datadog is mid-rename: the documentation and the marketing H1 now say Agent Observability, the old LLM Observability URLs redirect rather than 404, and the marketing page’s own title tag still reads LLM Observability, which is a reminder of how fast these pages move. Treat any cell that says something is not described in their published documentation as a question to put to Datadog in writing rather than as a finding — a capability that is merely undocumented reads identically to one that does not exist — and check pricing against their own page before it reaches a business case.
Prefer to ask a person? Write to us →
Tell us which one you are already running.
If Datadog LLM Observability is already in your stack, the useful question is not which to buy but what each is for, and where the seam between them sits. Say what you have and you will get a straight answer — including when the answer is that you do not need a second thing.
no form · no qualification step · no sales desk · the other three ways in