Token Observe vs Portkey
Both hold the payload before it reaches a provider. One is built to carry it to more than 250 models; the other is built to refuse it and prove afterwards who said it could go.
2 September 2026
- What is Portkey? — product overview
- AI Gateway — routing, fallbacks, limits
- Guardrails — actions and request hooks
- PII redaction guardrail
- Budget and rate limits on an API key
- Access control management
- Audit logs
- Enterprise architecture — data plane and control plane
- Deploy Portkey in your infrastructure
- SSO for the control plane
- Open source versus platform feature comparison
- Pricing
- Observability — logs, traces, metrics
- MCP Gateway — server and tool access control
- MCP Gateway guardrails, and what is coming
- Logs Export — JSONL export jobs
- Admin API — list audit logs
Their claims, not our testing. Verify anything that decides it for you.
On this page
For most teams shopping for an AI gateway, Portkey is the better purchase, and the reasons are not close
Start with the thing procurement asks about first. Portkey’s own material states ISO 27001 and SOC 2 certification and GDPR and HIPAA compliance, and their plan comparison lists SOC 2, ISO 27001, GDPR and HIPAA compliance certificates against the Enterprise tier. Token Observe holds none of those: no SOC 2, no ISO 27001, no ISO 42001, and no independent penetration test. If a certificate is a gate rather than a preference in your organisation, the comparison ends there in Portkey’s favour, and nothing further down this page changes that. Ask them for the scope and the date on each certificate, because that is the only version of the answer worth having and this page does not hold it on their behalf.
Then the operational half. Their overview describes the gateway processing requests through edge workers globally, cites 99.99 per cent uptime and over 25 million daily requests, and puts the added latency at 20 to 40 milliseconds compared with calling a provider directly. Their enterprise architecture page describes the data plane as containerised workloads on Kubernetes 1.20 and above with Helm 3.x, backed by S3-compatible object storage, MongoDB or DocumentDB, with the control plane hosting the dashboard. Token Observe at its current target scale is a single Node process over one SQLite file, with no replica, no clustering and no vendor-operated availability commitment, and the reason none is offered is published rather than hidden: the vendor does not operate your deployment and has no telemetry from it, so an uptime number from that party would be unmeasurable by either side. If the control has to be highly available on day one, choose differently.
Breadth is the third, and it is the one a developer feels weekly. Their overview describes a unified interface to over 250 AI models; their gateway documentation lists conditional routing, load balancing across keys, fallbacks between providers and models, automatic retries, request timeouts, circuit protection, simple and semantic caching, gRPC transport, canary testing of new models in production, MCP support, multimodality across vision, audio and image generation, and custom host URLs for privately hosted models. Their MCP Gateway is a genuine second product on top of that, documented as proxying MCP clients to MCP servers with credential injection, per-workspace and per-user server access, per-user enabling and disabling of individual tools, throttling per API key, server or tool, and every tool call logged with its user, parameters and response — which is nearer to what Token Observe does on the tool leg than a features grid makes it look. Their open-source gateway is described in their own words as open source and free to use, and their feature comparison lists universal API, automatic fallbacks, load balancing, conditional routing, automatic retries and request timeouts in that tier — so a team whose requirement is connectivity can have most of this for nothing. Token Observe registers six first-class upstreams plus any OpenAI-compatible endpoint you point it at, treats that list as table stakes rather than as the reason to buy, and does not attempt to match the catalogue. Every claim in these three paragraphs is Portkey’s own published wording as of 2026-09-02, has not been independently tested, and should be verified with them in writing rather than from a comparison page.
Token Observe and Portkey, capability by capability
The Portkey column paraphrases Portkey’s own published material as it stood on 2 September 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Check anything that decides it for you against their own current documentation.
Where it sits in the request path
A unified API their overview describes as a single interface to over 250 AI models, reached through their REST endpoint or their Python and Node SDKs, with a stated two-minute integration.
One base URL and one key. For supported OpenAI-compatible, Anthropic and Gemini ingress this is normally a base-URL change rather than an application refactor.
Their overview states the gateway adds 20 to 40 milliseconds compared with a direct API call, which they say is often offset by their caching and routing optimisations.
The only published figure is a laboratory baseline that travels with its conditions: 71.2 ms p50, 163.8 ms p95 and 223.4 ms p99 at concurrency 16 over 30.143 seconds, against a mock upstream on an Apple M1 Max. That is inline work, not end-user response time, and it is not a throughput commitment.
The two numbers are not comparable. Theirs is measured on their production edge against real providers; the Token Observe figure excludes provider latency, streaming, retries and failover by construction, and thirty seconds is not a soak.
Portkey-managed SaaS by default. On the hybrid deployment their documentation says to deploy the data plane in your VPC and route LLM traffic there, with all prompt content and LLM responses remaining within your network.
Self-hosted only, in your network, on your keys. The vendor receives no product telemetry, no phone-home data, no prompts, no keys and no trace database.
Their overview cites 99.99 per cent uptime, edge workers throughout the world, capacity for millions of requests per minute and over 25 million requests served daily. Their gateway page separately documents request timeouts, automatic retries and per-strategy circuit protection.
Fail-closed on purpose: if Token Observe stops, governed agents cannot call models. A boot-time audit walk that finds corruption latches readiness and returns ACP_AUDIT_UNAVAILABLE 503 to governed requests, and a restart does not clear the latch.
Considerably more than the gateway feature list alone suggests. Beyond MCP support in the gateway, their MCP Gateway documentation describes a proxy between MCP clients and MCP servers that handles credential injection, permission checks and request logging: control over which workspaces and users can access each server, the ability to enable or disable specific tools per user, throttling of tool calls by request count or token consumption per API key, server or tool, content filters and approval workflows per server, and every tool call logged with user, parameters and response, filterable by server, user or time range.
An MCP gateway on the same policy engine, plus evaluation of any tool call the model proposes on the response leg — so a rule about the arguments, such as a refund over a threshold, binds even when the agent executes the tool itself rather than only governing which tool it may reach. Token Observe can only refuse a proposal it is shown.
This is the closest row in the table and an earlier draft of this page understated their side of it. Their MCP Gateway already governs which server and which tool a user may reach, throttles per tool and logs every call. What Token Observe is claiming here is evaluation of the proposed arguments on the response leg, not the existence of tool governance.
What it enforces inline
Guardrails on the Gateway: deterministic and LLM-based checks that run as input guardrails in before_request_hooks and output guardrails in after_request_hooks. Asynchronous is the default and runs alongside the request without adding latency, logging the result without affecting orchestration; synchronous execution runs before the model call or before the response returns, and lets the request be orchestrated on the verdict.
One decision point rather than a set of hooks. Eleven ordered steps per governed request, in which Unicode sanitisation runs before any detector reads the payload, detection runs before the verdict, and a single evaluation returns allow, block or require-approval plus a redaction plan.
Their documented actions are deny, which blocks the request with a 446 status or allows it through flagged with a 246, asynchronous evaluation for logging, falling back to another LLM or prompt, retrying the request, and appending feedback to build an evaluation dataset.
Five actions — block, require a human approval, redact, warn, suspend the agent — resolved to one verdict in which a block beats an approval and an approval beats a redaction. Any rule can run in shadow mode first, recording what it would have done, and where the gate is enabled no rule may start enforcing until a backtest of that exact rule is acknowledged by a named person.
Their feature comparison lists role-based access control from the Pro plan with advanced RBAC at Enterprise, and their access control page — itself marked an Enterprise feature — documents organisation roles of owner, admin and member alongside API keys carrying permissions over metrics, completions, prompts, configs, guardrails, integrations, the model catalogue and user management. Their MCP Gateway documentation separately describes enabling or disabling specific tools per user, which is authority over a tool rather than over the console.
Deny-by-default permissions on the action itself: tool:payments/issue_refund and model:gpt-5o-mini are grants, an action no role names is refused, an explicit deny beats every allow wherever it is written, and a delegation chain intersects rather than unions so a low-privileged agent gains nothing by asking a higher-privileged one to act for it.
The two products mean different things by the same word, though less so than a features grid would suggest. Their organisation roles and key scopes govern who may use the platform and its resources, and their MCP Gateway governs which servers and tools a user may reach; Token Observe’s permissions govern which action a given agent may propose, including the arguments it proposes. Both are real controls, they overlap on the tool leg, and they are not substitutes for each other.
Their guardrail documentation describes orchestrating the request with deny, log, fallback, retry, evaluation-dataset and feedback actions. They are not silent on approvals beyond that: their MCP Gateway overview lists content filters and approval workflows per server among the guardrails available there, and their MCP Gateway guardrails page names approval workflows requiring explicit approval for high-risk operations such as deletions or write actions under a heading of what is coming. What such an approval is bound to — one exact payload, or a class of operations — and how it is redeemed are not described in their published documentation as of 2026-09-02.
A 403 carrying an approval id, a status URL and a resume contract. The approval is bound to the SHA-256 of the canonical action plus its execution context, is single-use through a compare-and-set so two concurrent retries cannot both execute, and expires at 60 minutes by default and from one minute to seven days by policy.
Their Pro PII guardrail detects phone numbers, email addresses, location information, IP addresses, Social Security Numbers, names and credit card information, replacing them with standardised identifiers such as {{EMAIL_ADDRESS_1}}, and can be attached to both input and output hooks. Their stated limits are that redaction patterns for pre-built guardrails are not customisable except through regex and that the transformation is one-way and non-reversible.
Eleven classes, three of them checksum-validated with Luhn, IBAN mod-97 and NHS mod-11, published with per-kind confidence scores — 0.70 for a phone number, 0.99 for a PEM private key — so a policy sets its own threshold. Free-text personal data is not detected at all: matching is regex plus checksum, which makes this a compensating control rather than a complete data-loss prevention layer. Streamed output passes a hold-back buffer with a 64-character floor and a separate buffer per tool-call argument channel, so a card number split across two chunks cannot escape.
Partner guardrails are integrated by adding the partner’s API key, with their documentation naming Aporia, SydeLabs and Pillar Security among the supported platforms.
Detection is one policy input among seven trigger kinds rather than the product. The roadmap names a generic AI firewall, prompt scanner or red-team platform as a strategic non-goal and points at consuming other vendors’ verdicts instead of reproducing them.
Budgets, rate limits and the stop button
Budget limits on an individual API key, set either as a maximum spend in USD with a $1 minimum or as a maximum token usage, which their documentation says automatically prevents further usage once the limit is reached; the same page states that the key continues to function until the full budget limit is reached. Limits can apply indefinitely, reset weekly on Sunday at 12 AM UTC, or reset monthly on the 1st at 12 AM UTC. Workspace-level limits are documented separately, and the page states the feature is available on the Enterprise plan and to select Pro customers.
Hard USD ceilings per request, per rolling hour, per UTC day and per UTC month in whatever combination you configure, reserved against the agent’s windows inside one per-agent database transaction before egress, priced at the most expensive of every provider and fallback the resolved route could reach.
Not described in their published documentation as of 2026-09-02.
A budgeted route with an unpriced reachable target is refused with a 409 before egress rather than priced at zero, because an empty price table is exactly how a ceiling was once silently disarmed. Fifty price rows ship and load additively at every boot.
Requests or tokens capped per minute, per hour and per day on an API key, with their documentation stating that subsequent requests are rejected until the time interval resets.
Requests, tool calls and tokens per minute per agent, evaluated after the kill switch, lifecycle and permissions and before the money verdict, so a refusal always names the first thing that actually refused it.
Configurable alert thresholds, where their documentation states that when usage reaches the threshold notifications will be sent to configured recipients, defaulting to organisation admins, owners and the key’s creator.
Webhook events on the trace lifecycle and on approval.requested. The ceiling itself refuses rather than warns, and the published cost of a hard ceiling is stated beside it: one billable egress, with no retry and no failover.
The nearest documented control is a budget limit on an API key, which their documentation says automatically prevents further usage once the limit is reached. A single control that halts one agent, a team or the whole estate is not described in their published documentation as of 2026-09-02.
A kill switch scoped to one agent, one team or everything, checked first in the pipeline ahead of lifecycle, permissions, budgets and policy, and reaching even the routes that execute nothing.
What it records, and who can rewrite it
Logs of multimodal requests and responses for viewing, monitoring and debugging; request tracing across the lifecycle of a request; custom metadata and tags for grouping; customisable filters; feedback values and weights; and analytics across what their documentation calls 21-plus key metrics.
One trace per governed request carrying the post-redaction prompt excerpt, the tool calls and their arguments, every policy decision including shadow-mode ones, the approvals, the tokens and the cost. The trace id is minted at step 3, before the verdict, so a blocked request is recorded rather than absent.
Audit logs capturing administrative activity with timestamp, user, workspace, action, resource, response status, client IP and country, filterable by method, request id, resource type, action, status, workspace, user, IP, country and time range, on the Enterprise plan.
Every governance-plane change — an agent created, a policy widened, the kill switch engaged, an approval decided — appended to a chain whose entry digest covers the previous entry’s hash plus the canonical JSON of that entry’s own content, so an edit or a deletion breaks verification at a named sequence number rather than merely somewhere.
Not described in their published documentation as of 2026-09-02. What that material does publish alongside the audit log is AES-256 encryption in transit and at rest, access restricted to org owners and admins, and indefinite retention; on whether a stored record can be shown afterwards not to have changed, it says nothing either way.
SHA-256 by default; HMAC-SHA256 under a MAC key held outside the database when one is configured, with the head sealed by a checkpoint MAC at every boot, and optional Ed25519 anchors signed on a schedule and published to a file or HTTP sink you site outside the database administrator’s reach. Tamper-evident, not tamper-proof — and the default is unkeyed, where a rewrite that re-hashes everything verifies clean.
More than an earlier draft of this page credited. Their Logs Export documentation describes an asynchronous export job over request logs that returns a signed download URL, with JSONL output, up to 50,000 logs per job, a choice of 18 fields, and an export_settings.deniedFields control an administrator can use to keep request and response bodies out of an export; the page states the feature is available on the Enterprise plan. Their admin API separately documents a GET /audit-logs endpoint returning audit records with filters and pagination. What is not described in their published documentation as of 2026-09-02 is a single bundle binding logs, approvals and an integrity result together, or a verification output an auditor could check an export against.
A compliance export of the traces and events for a period, the approvals with approver identity and rationale, the audit entries covering every governance-plane change, a chain verification result naming the sequence number of any break, and a SHA-256 digest of the bundle at a recorded time. Digest-sealed, not signed.
Audit log access is available to org owners and admins. Organisations are isolated, with their documentation stating that data, logs, analytics, prompts, integrations, providers, configs, guardrails and API keys are strictly confined within each organisation.
Trace list, search, detail and export reads are themselves attributable, spend figures and recertification evidence are gated on the reader’s team scopes as well as their role, and surfaces that join records with no trustworthy team key return 403 rather than a misleading partial view.
Customisable filters over logs and analytics, custom metadata and unique tags for grouping and troubleshooting, and dashboards over the metric set.
A plain-English question translated into a validated filter object over fourteen allow-listed fields, never into SQL, shown back as editable chips, with a deterministic keyword parser answering when no model is configured. The published cost: the filter cannot group, count or correlate across traces.
Published per plan: three days of logs and thirty days of metrics on the free Developer tier, thirty days of logs and ninety days of metrics on Production at $49 a month, custom retention on Enterprise, and indefinite retention named as an audit-log capability.
Trace retention is unset by default, and unset means keep forever. Setting a window turns on an hourly ageing pass; retention and erasure act on the live primary database only, so a restored pre-erasure backup can resurrect what was erased.
How it deploys, and what it costs
Portkey-managed SaaS, and a hybrid deployment their documentation summarises as deploying the data plane in your VPC to route LLM traffic, with the control plane hosted by Portkey and the gateway running as containerised workloads under Kubernetes or ECS.
Self-hosted only: one Node process, one SQLite file in WAL mode, five surfaces, on your infrastructure and your provider keys. PostgreSQL exists behind the store ports as an evaluation alternative and is explicitly not a supported HA topology.
Their architecture page describes the gateway periodically synchronising with the control plane at one-minute heartbeat intervals to fetch routing configs, templates, integrations and API keys, sending anonymous metrics such as model used, token counts and response times, and — where you keep logs in your own environment’s blob store — the control plane requesting them from the gateway when somebody views them in the dashboard. Their stated position is that all prompt content and LLM responses remain within your network and only anonymised metrics data cross network boundaries. The alternative they document is the gateway encrypting and sending logs to a Portkey log store instead, which they say needs no connection from Portkey into your environment.
Nothing leaves for the vendor. Governed payloads leave your network only for the model and tool providers you configure, after policy and redaction, and the runtime data flow is documented so you can verify that rather than take it on assurance. Whether a support arrangement makes the vendor a processor is still a question for your counsel.
Their documentation states that the gateway is open source and free to use, and their own feature comparison lists universal API, automatic fallbacks, load balancing, conditional routing, automatic retries and request timeouts in the open-source tier, with observability, prompt management, guardrails and the security and compliance features on the hosted plans. Their pricing page’s own open-source column is more generous than that table, listing guardrails and a basic dashboard as well, so establish the boundary with them rather than from either page.
Commercial source-available. Use, modify and self-host under a licence; redistribution and offering it as a competing hosted service are not permitted. Security research and publication of results are expressly permitted, with no gag clause and no pre-approval of results.
Developer free with 10,000 recorded logs a month; Production at $49 a month for 100,000 recorded logs, with overage stated as $9 per additional 100,000 requests; and Enterprise at custom pricing with custom retention periods for 10 million plus recorded logs a month. Their pricing page also lists a self-hosted open-source option, which their overview calls open source and free to use.
No published price. The licence itself is a template pending review by counsel in England and Wales rather than an executed grant of rights, so read it as the intended terms rather than the signed ones.
Their material states ISO 27001 and SOC 2 certification with GDPR and HIPAA compliance, AES-256 encryption in transit and at rest, a mode available on request that does not store request and response body objects, and their plan comparison lists SOC 2, ISO 27001, GDPR and HIPAA compliance certificates against Enterprise. Ask for the scope and the date.
None held: no SOC 2, no ISO 27001, no ISO 42001, no independent penetration test. Published instead are a residual-risk register with the role whose dated acceptance is required on each row, a confirmed defect list including the attacks that still work, and a 30-day evaluation written so your security team may read, run and attack the software before a purchase order is raised.
Control-plane SSO over OIDC and SAML 2.0 for enterprise customers, with first-time SSO users automatically provisioned into the organisation unless disable_auto_provision_user is set, and provisioning restricted to email domains verified in admin settings. Their plan comparison lists SSO with Okta auth at Enterprise.
OIDC SSO with directory groups mapped to roles, plus bounded SCIM Users provisioning for viewer accounts that hold no evidence scopes. Group claims are taken as a snapshot with a default staleness of 24 hours rather than read live from the directory, and full-profile SCIM, SCIM Groups and SAML are deliberately not built.
Their overview cites 99.99 per cent uptime, millions of requests per minute and edge workers globally.
One writer, one host at the current target scale. No replica, no clustering, no vendor-operated uptime SLA — and the reason none is offered is published: the vendor does not operate your deployment and has no telemetry from it.
The same slot, holding authority over different things
Both products stand between an agent and a provider, and both are reached by changing where the agent points. What differs is the object each one expresses authority over. Portkey’s published access control is a three-tier organisation role — owner, admin, member — with isolated organisations and API keys their documentation describes as carrying fine-grained scopes across metrics, completions, prompts, configs, guardrails, integrations and user management. That is authority over the platform: who may change a routing config, who may see the logs, who may mint a key. It is a real control and most estates need it.
Token Observe’s permissions name the action instead. A support agent holds an allow on tool:orderdb/get_details and model:gpt-5o-mini, while tool:payments/issue_refund is simply absent and therefore denied, because the default is deny rather than allow. An explicit deny beats every allow wherever it is written, including inside another role, so a narrow guardrail cannot be outvoted by a broad grant. When one agent delegates to another the chain intersects rather than unions: every hop must allow the action, which is what stops a low-privileged agent from escalating by routing the work through a higher-privileged one. Where the on-behalf-of header is switched to enforce, the roles mapped from a named human’s directory groups are appended as one more link that can only subtract.
The consequence for a buyer is that these two controls answer different questions and can sit in the same estate without contradicting each other. Portkey’s answers who in your organisation may operate the gateway. Token Observe’s answers whether this agent may issue this refund, and it answers it at the moment the model proposes the call. A grid row that scored those against each other would be measuring nothing, which is why the row above carries a note instead of a verdict.
- Deny-by-default
- An action no role names is refused. There is no implicit allow to be tightened later, so the failure mode of a forgotten grant is a blocked agent rather than an unbounded one.
- Precedence
- An explicit deny wins over every allow, in any role, in any order. Precedence is a property of the evaluator rather than of the order somebody happened to write the roles in.
- Delegation intersects
- Agent-to-agent calls carry a chain, and every hop must permit the action independently. Union semantics would make delegation an escalation primitive.
What a guardrail is bound to, and what an approval is bound to
Portkey’s guardrails are documented as running in two places and two modes: before_request_hooks and after_request_hooks, asynchronously by default so results are logged without affecting orchestration, or synchronously so the request can be orchestrated on the verdict. The documented outcomes on a failed synchronous check are deny with a 446, allow-and-flag with a 246, fall back to another LLM or prompt, retry, or record feedback for an evaluation dataset. That is a rich set of orchestration outcomes and it covers the cases where the right answer is to stop, to reroute, or to note. Their MCP Gateway material goes further and names approval workflows per server, with the detail page listing approval for high-risk operations such as deletions and write actions under what is coming rather than under what ships — so the gap described below is a gap in what is published today, and may not be one for long.
Token Observe’s difference is the outcome that is not on the guardrail list, and it is the one the whole product turns on. A policy whose action is require_approval refuses the request with a 403, mints an approval record, and holds the trace open until a person decides. The record is bound to the SHA-256 of the canonicalised action plus the execution context it was proposed in, which is what makes it an approval of that payload rather than of that action type: change one argument and the hash no longer matches, so the retry is refused as a mismatch rather than allowed as near enough. Consumption is a compare-and-set, so two concurrent retries cannot both execute on one approval. It expires — 60 minutes by default, one minute to seven days by policy — because an approval that never expires is a standing grant with extra paperwork.
The limit is stated in the same register as the claim, because it changes how you integrate. Approving does not push anything to the agent; Token Observe has no way to call an agent back. The agent redeems the approval by repeating the identical request with its id, once. An agent written to treat a 403 as fatal will need a small change to participate, and that is a real integration cost rather than a footnote. The compensating property is that the approval is verifiable afterwards: the approver’s identity and rationale land in the export beside the trace and the audit entries, so the question of who authorised this has an answer that does not depend on anyone’s memory.
- Bound to a payload, not a type
- The hash covers the canonical action and its execution context. An approval for a £180 refund on order 4417 does not authorise a £1,800 refund on order 4418.
- Single-use
- Consumption is a compare-and-set. Two retries racing each other cannot both redeem the same approval, which is the failure a naive status flag produces under concurrency.
- Expiring
- Sixty minutes by default. A policy can set anything from one minute to seven days, and the expiry is enforced at redemption rather than by a sweeper.
Both keep a record. The question is who can change it
Portkey publishes a detailed account of what their audit log captures — timestamp, user, workspace, action, resource, response status, client IP and country — and of how it is filtered, along with indefinite retention as an enterprise capability and access available to org owners and admins. On the metadata a security team wants for an administrative action, that is more than Token Observe advertises, and the country and client-IP fields in particular are useful in an investigation. Their log retention is published per plan, which is the kind of specificity a buyer can plan around: three days on the free tier, thirty days on Production, custom on Enterprise. They also document getting the record out again, which this page originally missed: a GET /audit-logs admin endpoint with filters and pagination, and a separate Enterprise Logs Export that runs an asynchronous job over request logs and hands back a signed URL to JSONL, up to fifty thousand logs at a time.
Token Observe’s record is built for a different adversary: the person who administers the database. Every governance-plane change is appended to a hash chain whose entry digest covers the previous entry’s hash plus the canonical JSON of the entry’s own content, and appends happen inside a transaction that also takes the chain tip, so concurrent writers cannot fork it. With an audit MAC key configured, the digests become HMAC-SHA256 under a key held outside the database and the head is sealed by a checkpoint MAC at every boot. Above that, an Ed25519 anchor signs a statement of the head on a schedule and publishes it to a file or HTTP sink. What the anchor buys is exactly one narrow claim, and it is the only one made: any copy of an anchor you kept off-box beats any rewrite made after you took it.
The honesty this page owes is that the guarantee is conditional and the default does not meet it. Unkeyed, the chain is plain SHA-256, and an operator with write access can rewrite a row and recompute every hash after it, and the result verifies clean. Token Observe reports which of the two you are holding in every verification result and every export, precisely because that difference is the whole guarantee. The anchor is off until a signing key is configured, and it is worth something only if the sink is somewhere the database administrator cannot reach. The word is tamper-evident, never tamper-proof, and the export is digest-sealed rather than signed. Anchoring also refuses more often than it signs: it will not anchor a chain that does not verify, a head that has moved backwards, or a rewritten anchored entry, because signing over a rewrite would launder it under a key auditors trust.
Two questions worth putting in writing to whichever proxy you run
These are not findings about Portkey and nothing here has been tested against their product. They are the two failure patterns Token Observe’s own engineering notes record having had to solve, and they are worth asking about any gateway, including this one, because the answers are specific and checkable and a good product will have them ready.
The first is what a fallback chain does with a refusal. Portkey documents fallbacks between providers and models for resilience and automatic retry strategies, which is what you want when a provider times out. The question is how the chain classifies a provider’s content-policy refusal, because if a refusal is treated as a retryable error then the next provider’s answer comes back as a success and nothing in the record says a refusal happened. Token Observe’s answer is seven typed failure classes of which three fail over and four do not: a timeout, a 429 and a 5xx move on, while a content-policy refusal, an authentication failure, an over-long context and a malformed request stop where they are.
The second is cache-token arithmetic. Anthropic reports cache reads and writes outside the input total while OpenAI and Gemini report them inside it, so any cost figure that adds the buckets without normalising first is wrong in one direction for one provider and wrong in the other for another, and the invoice is the thing that eventually disagrees. Token Observe normalises provider usage into mutually exclusive buckets before any arithmetic happens, and narrows the route to whichever provider actually served the request before pricing it, so the ledger prices against the vendor that will invoice you rather than the one that was tried first. Ask your gateway vendor how they do it; the answer is a paragraph, and it either exists or it does not.
Where each one is the right answer
Choose Portkey when
- Procurement requires SOC 2, ISO 27001 or a comparable certificate from the vendor. Portkey’s material states ISO 27001 and SOC 2 certification; Token Observe holds none and says so on the first call.
- You need breadth across models and providers as the primary requirement — their overview describes a unified interface to over 250 AI models with conditional routing, load balancing, caching, canary testing and multimodality.
- High availability is needed on day one. Their overview cites 99.99 per cent uptime and edge workers globally; Token Observe is a single-writer process on one host with no replica and no availability commitment.
- You want a published price and a self-serve path from free to production — theirs runs from a free tier at 10,000 recorded logs a month to $49 a month for 100,000, with an open-source gateway underneath it.
Choose Token Observe when
- A human has to authorise one exact payload rather than a class of requests, and the authorisation must be single-use, expiring and attributable to a named person against the trace.
- The agents take actions somebody answers for — a refund, a deployment, an email, a ticket transition, a database write — and permissions have to be expressed over the action rather than over the console.
- The record has to resist the person who administers the database, which means a hash chain with a MAC key held outside it and an anchor published somewhere that administrator cannot reach.
- Zero vendor egress is a hard requirement, including air-gapped operation, with no control plane to synchronise with and no metrics leaving the boundary.
When you would run both
Running both is the normal answer, and for an estate that already runs Portkey it is the only answer worth recommending: the base URL is already swapped, the keys are already central, the spend is already counted, and ripping that out to install something else is work with no governance outcome attached to it. The arrangement that costs least is to leave Portkey exactly where it is and route only the agents that take consequential actions through Token Observe, either in front of it or behind it. In front, Token Observe holds the agent credential and takes the governance verdict, then routes to Portkey as an OpenAI-compatible upstream so provider keys, fallbacks and caching stay in one place. Behind, Portkey keeps the ingress and forwards the agents that need governing. Both shapes add a hop and a second failure domain, so this is a decision to take deliberately rather than by default — a chat assistant does not need both, and a refund agent might. Where you need a third enforcement point to be shown to agree, Token Observe can compile scope and trigger matching to digest-locked OPA Rego with reproducible positive and negative witnesses, though that artefact deliberately excludes permissions, budgets, approval consumption, kill switches, action precedence and side effects, which stay authoritative in Token Observe.
The category argument sits above this one: Token Observe and llm gateways covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
LiteLLM
If you already run LiteLLM, keep it. The question that decides whether you need anything more is about the actions your agents take, not about the proxy.
Kong AI Gateway
Kong governs the traffic. Token Observe governs the action. If you already run Kong, the first one is nearly free and the second one is the only reason to read further.
Cloudflare AI Gateway
Cloudflare’s gateway decides what the payload contains. Token Observe decides whether the agent that sent it was allowed to.
Envoy AI Gateway
Both hold the request. One charges the token budget once the response completes; the other reserves the money before the request leaves your network.
MuleSoft AI Gateway
Both refuse the call inline. One refuses on behalf of an endpoint, the other on behalf of an agent that has an owner.
Is Token Observe trying to replace Portkey?
No, and for most readers it should not. Portkey publishes a unified API over more than 250 models, conditional routing, load balancing, fallbacks, caching, an open-source gateway, a published price list and ISO 27001 and SOC 2 certification; Token Observe registers six first-class upstreams plus any OpenAI-compatible endpoint, holds no certifications and publishes no price. The product’s own roadmap says to integrate above or beside an existing gateway rather than compete on connectivity, and any gateway you already run can be registered as an upstream or can forward to Token Observe for the agents that need governing. The narrow thing Token Observe adds is the decision and the evidence: deny-by-default action-level permissions, an approval bound to one payload hash, and a hash-chained record that can be anchored off the box.
Portkey has guardrails. What does Token Observe do that a guardrail does not?
Their documented guardrail outcomes are deny with a 446, allow-and-flag with a 246, asynchronous logging, fallback to another LLM or prompt, retry, and feedback into an evaluation dataset, running as input or output hooks synchronously or asynchronously. Token Observe’s difference is the outcome that pauses rather than decides: a 403 carrying an approval id, bound to the SHA-256 of the canonical action plus its execution context, single-use through a compare-and-set and expiring at 60 minutes by default. Alongside that sits authority over the action itself — an agent that holds no grant on tool:payments/issue_refund cannot issue one whatever the payload says — and a delegation chain that intersects rather than unions. Portkey is not silent on approvals, and this page should not imply they are: their MCP Gateway overview lists approval workflows per server among its guardrails, and their MCP Gateway guardrails page names approval workflows for high-risk operations such as deletions and write actions under a heading of what is coming. What such an approval binds to — one exact payload, or a class of operations — and how it is redeemed are not described in their published documentation as of 2026-09-02, and an undocumented detail reads identically to an absent one, which is why this page will not assert either. Put it to them in writing.
Their hybrid deployment keeps prompts in our VPC. Is that not the same as self-hosting?
It is close on the part that matters most and different on one part worth knowing about. Their architecture page describes the data plane running in your infrastructure with all prompt content and LLM responses remaining within your network, which is the substantive privacy claim. The same page describes what does cross the boundary: the gateway synchronises with the control plane at one-minute heartbeat intervals to fetch routing configs, templates, integrations and API keys, sends non-sensitive operational metrics such as model used, token counts and response times, and, where logs are kept in your own blob store, requests them from your gateway when somebody opens them in the dashboard. Token Observe has no control plane to synchronise with, so nothing leaves for the vendor at all — no telemetry, no prompts, no keys, no trace database — and the runtime data flow is documented so you can verify it. Whether the difference matters depends on whether your constraint is prompt confidentiality or complete network isolation, and the honest note on the second is that Token Observe still cannot promise the vendor is legally never a processor, because support handling and contracts are your counsel’s analysis rather than a runtime property.
Their audit log has indefinite retention. Why is a hash chain better?
It is not better at the things their audit log is good at, and this page should not pretend otherwise. Their published audit record carries client IP and country alongside user, action, resource and status, is filterable across all of those, is retained indefinitely on the Enterprise plan, and can be pulled programmatically through a documented admin endpoint alongside an Enterprise export job for request logs; that is a strong administrative record. The hash chain answers a narrower question: whether the record you are reading is the record that was written. Each entry’s digest covers the previous entry’s hash plus the canonical JSON of its own content, so an edit or a deletion breaks verification at a named sequence number; with a MAC key held outside the database the digests become HMAC-SHA256, and with a signing key configured an Ed25519 anchor is published off the box on a schedule. The caveats belong in the same sentence: the default is unkeyed and a rewrite that re-hashes everything verifies clean, the anchor is off until you configure it and only counts if the sink is out of the database administrator’s reach, and the export is digest-sealed rather than signed. Tamper-evident, not tamper-proof.
Have you tested Portkey against Token Observe?
No. Every statement about Portkey on this page paraphrases their own published material read on 2026-09-02 — the product overview, the gateway and guardrails documentation, the budget and rate limit page, access control, audit logs, the enterprise architecture and deployment pages, SSO, the open-source feature comparison, observability, pricing, the MCP Gateway pages, and the logs-export and admin audit-log API references — and none of it has been independently tested. There has been no witnessed bake-off, and the product’s own launch gates name a competitor bake-off as evidence that does not yet exist. Where a cell says a capability is not described in their published documentation, that is a statement about the documentation and not about the product: read it as a question to put to Portkey in writing. What is offered on the Token Observe side instead is a 30-day evaluation written so your security team can read, run and attack the software before a purchase order is raised, with no gag clause and no pre-approval of results — under a licence that is a template pending review by counsel in England and Wales rather than an executed grant of rights.
Prefer to ask a person? Write to us →
Tell us which one you are already running.
If Portkey is already in your stack, the useful question is not which to buy but what each is for, and where the seam between them sits. Say what you have and you will get a straight answer — including when the answer is that you do not need a second thing.
no form · no qualification step · no sales desk · the other three ways in