Token Observe vs Portkey (Prisma AIRS AI Gateway)
Compare the AI bill you can explain and the routed requests you need to govern, alongside your existing Portkey gateway.
For most teams shopping for an AI gateway, Portkey is the better purchase
Start with the thing procurement asks about first. Portkey’s own material states ISO 27001 and SOC 2 certification and GDPR and HIPAA compliance, their plan comparison lists SOC 2, ISO 27001, GDPR and HIPAA compliance certificates against the Enterprise tier, and their pricing page’s Enterprise tier lists advanced compliance as SOC 2 Type 2, GDPR and HIPAA, with custom BAAs. The product now also carries Palo Alto Networks’ name, whose announcement describes it as the Portkey acquisition brought into Prisma AIRS. Token Observe holds none of those certifications: no SOC 2, no ISO 27001, no ISO 42001, and no independent penetration test. If a certificate is a gate rather than a preference in your organisation, the comparison ends there in Portkey’s favour, and nothing further down this page changes that. Ask them for the scope and the date on each certificate, because that is the only version of the answer worth having and this page does not hold it on their behalf.
Then the operational half. Their overview describes the gateway processing requests through edge workers globally, cites 99.99 per cent uptime and over 25 million daily requests, and puts the added latency at 20 to 40 milliseconds compared with calling a provider directly. Their enterprise architecture page describes the data plane as containerised workloads on Kubernetes 1.20 and above with Helm 3.x, backed by S3-compatible object storage, MongoDB or DocumentDB, with the control plane hosting the dashboard. The cloud pilot runs on one shared host without an availability SLA, qualified customer capacity or live recovery proof. Customer-hosted runtime operation and recovery remain the customer’s responsibility, with no supported HA or multi-replica topology. If the control has to be highly available on day one, choose differently.
Breadth is the third, and it is the one a developer feels weekly. Their overview describes a unified interface to over 250 AI models; their gateway documentation lists conditional routing, load balancing across keys, fallbacks between providers and models, automatic retries, request timeouts, circuit protection, simple and semantic caching, gRPC transport, canary testing of new models in production, MCP support, multimodality across vision, audio and image generation, and custom host URLs for privately hosted models. Their MCP Gateway is a second product on top of that, documented as proxying MCP clients to MCP servers with credential injection, per-workspace and per-user server access, per-user enabling and disabling of individual tools, guardrail checks (regex, JSON schema, required keys, a webhook to your own validator) on a tool call’s arguments before it executes and on its result afterwards, throttling per API key, server or tool, and every tool call logged with its user, parameters and response. That is nearer to what Token Observe does on the tool leg than a features grid makes it look. Their open-source gateway is described in their own words as open source and free to use, and their feature comparison lists universal API, automatic fallbacks, load balancing, conditional routing, automatic retries and request timeouts in that tier, so a team whose requirement is connectivity can have most of this for nothing. Token Observe registers six first-class upstreams plus any OpenAI-compatible endpoint you point it at, treats that list as table stakes rather than as the reason to buy, and does not attempt to match the catalogue. Every claim in these three paragraphs is Portkey’s own published wording as of 2026-10-05, has not been independently tested, and should be verified with them in writing rather than from a comparison page.
Token Observe and Portkey (Prisma AIRS AI Gateway), capability by capability
The Portkey (Prisma AIRS AI Gateway) column paraphrases Palo Alto Networks’s own published material as it stood on 5 October 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Token Observe’s technical comparison describes optional customer-hosted capabilities; the default cloud pilot has separate published setup, custody and traffic limits. Check anything that decides it for you against their own current documentation.
Where it sits in the request path
| Capability | Portkey (Prisma AIRS AI Gateway) | Token Observe |
|---|---|---|
| How you integrate | A unified API their overview describes as a single interface to over 250 AI models, reached through their REST endpoint or their Python and Node SDKs, with a stated two-minute integration. | One base URL and one key. For supported OpenAI-compatible, Anthropic and Gemini ingress this is normally a base-URL change rather than an application refactor. |
| Added latency | Their overview states the gateway adds 20 to 40 milliseconds compared with a direct API call, which they say is often offset by their caching and routing optimisations. | The only published figure is a laboratory baseline that travels with its conditions: 71.2 ms p50, 163.8 ms p95 and 223.4 ms p99 at concurrency 16 over 30.143 seconds, against a mock upstream on an Apple M1 Max. That is inline work, not end-user response time, and it is not a throughput commitment. |
| Note | The two numbers are not comparable. Theirs is measured on their production edge against real providers; the Token Observe figure excludes provider latency, streaming, retries and failover by construction, and thirty seconds is not a soak. | |
| Who holds the payload | Portkey-managed SaaS by default. On the hybrid deployment their documentation says to deploy the data plane in your VPC and route LLM traffic there, with all prompt content and LLM responses remaining within your network. | Cloud pilot or customer hosting. The cloud service holds workspace evidence and supplied credentials; customer-hosted installations keep runtime state on your infrastructure without vendor phone-home. These data-custody statements depend on the chosen mode. |
| Failure posture | Their overview cites 99.99 per cent uptime, edge workers throughout the world, capacity for millions of requests per minute and over 25 million requests served daily. Their gateway page separately documents request timeouts, automatic retries and per-strategy circuit protection. | Fail-closed on purpose: if Token Observe stops, governed agents cannot call models. A boot-time audit walk that finds corruption latches readiness and returns ACP_AUDIT_UNAVAILABLE 503 to governed requests, and a restart does not clear the latch. |
| The tool leg | Considerably more than the gateway feature list alone suggests. Beyond MCP support in the gateway, their MCP Gateway documentation describes a proxy between MCP clients and MCP servers that handles credential injection, permission checks and request logging: control over which workspaces and users can access each server, the ability to enable or disable specific tools per user, throttling of tool calls by request count or token consumption per API key, server or tool, content filters and approval workflows per server, guardrail checks that run on a tool call’s arguments before the tool executes and on its result afterwards, scoped to a server or to individual tools, and every tool call logged with user, parameters and response, filterable by server, user or time range. The documented checks for tool calls include regex match and replace, JSON schema validation, required keys, content and length checks and a webhook to an external validator; the same page states that LLM-specific checks such as PII detection and content moderation are not available for MCP tool calls, and documents the JSON-RPC error a blocked call returns. | An MCP gateway on the same policy engine, plus evaluation of any tool call the model proposes on the response leg, before it is handed back to the agent. A rule about the arguments, such as a refund over £200, can block the call or hold it for a named person whose approval binds to that exact payload, and it applies even when the agent executes the tool itself. Token Observe can only refuse a proposal it is shown. |
| Note | This is the closest row in the table. Both products check a tool call’s arguments before the tool runs, and their MCP Gateway also governs which server and tool a user may reach, throttles per tool and logs every call. Two differences remain on the published record. Token Observe evaluates the arguments a model proposes on the response leg, which covers an agent that executes the tool itself; their MCP guardrails are documented on calls that pass through their MCP Gateway. And a failed argument rule in Token Observe can wait for a person’s approval of that exact payload, an outcome their MCP guardrails page does not describe as of 2026-10-05. | |
What it enforces inline
| Capability | Portkey (Prisma AIRS AI Gateway) | Token Observe |
|---|---|---|
| The guardrail model | Guardrails on the Gateway: deterministic and LLM-based checks that run as input guardrails in before_request_hooks and output guardrails in after_request_hooks. Asynchronous is the default and runs alongside the request without adding latency, logging the result without affecting orchestration; synchronous execution runs before the model call or before the response returns, and lets the request be orchestrated on the verdict. | One decision point rather than a set of hooks. Eleven ordered steps per governed request, in which Unicode sanitisation runs before any detector reads the payload, detection runs before the verdict, and a single evaluation returns allow, block or require-approval plus a redaction plan. |
| What happens on a failed check | Their documented actions are deny, which blocks the request with a 446 status or allows it through flagged with a 246, asynchronous evaluation for logging, falling back to another LLM or prompt, retrying the request, and appending feedback to build an evaluation dataset. | Five actions — block, require a human approval, redact, warn, suspend the agent — resolved to one verdict in which a block beats an approval and an approval beats a redaction. Any rule can run in shadow mode first, recording what it would have done, and where the gate is enabled no rule may start enforcing until a backtest of that exact rule is acknowledged by a named person. |
| What an agent is allowed to do | Their feature comparison lists role-based access control from the Pro plan with advanced RBAC at Enterprise, and their access control page — itself marked an Enterprise feature — documents organisation roles of owner, admin and member alongside API keys carrying permissions over metrics, completions, prompts, configs, guardrails, integrations, the model catalogue and user management. Their MCP Gateway documentation separately describes enabling or disabling specific tools per user, which is authority over a tool rather than over the console. | Deny-by-default permissions on the action itself: tool:payments/issue_refund and model:gpt-5o-mini are grants, an action no role names is refused, an explicit deny beats every allow wherever it is written, and a delegation chain intersects rather than unions so a low-privileged agent gains nothing by asking a higher-privileged one to act for it. |
| Note | The two products mean different things by the same word, though less so than a features grid would suggest. Their organisation roles and key scopes govern who may use the platform and its resources, and their MCP Gateway governs which servers and tools a user may reach and can check the arguments sent to them; Token Observe’s permissions govern which action a given agent may propose, including the arguments it proposes. Both are real controls, they overlap on the tool leg, and they are not substitutes for each other. | |
| A human decision in the loop | Their guardrail documentation describes orchestrating the request with deny, log, fallback, retry, evaluation-dataset and feedback actions. They are not silent on approvals beyond that: their MCP Gateway overview lists content filters and approval workflows per server among its capabilities, though the MCP Gateway guardrails page that entry links to documents checks on tool inputs and outputs and a blocked-request error, and an approval workflow is not described there. What such an approval is bound to — one exact payload, or a class of operations — and how it is redeemed are not described in their published documentation as of 2026-10-05. | A 403 carrying an approval id, a status URL and a resume contract. The approval is bound to the SHA-256 of the canonical action plus its execution context, is single-use through a compare-and-set so two concurrent retries cannot both execute, and expires at 60 minutes by default and from one minute to seven days by policy. |
| Sensitive data in the payload | Their Pro PII guardrail detects phone numbers, email addresses, location information, IP addresses, Social Security Numbers, names and credit card information, replacing them with standardised identifiers such as {{EMAIL_ADDRESS_1}}, and can be attached to both input and output hooks. Their stated limits are that redaction patterns for pre-built guardrails are not customisable except through regex and that the transformation is one-way and non-reversible. | Eleven classes, three of them checksum-validated with Luhn, IBAN mod-97 and NHS mod-11, published with per-kind confidence scores — 0.70 for a phone number, 0.99 for a PEM private key — so a policy sets its own threshold. Free-text personal data is not detected at all: matching is regex plus checksum, which makes this a compensating control rather than a complete data-loss prevention layer. Streamed output passes a hold-back buffer with a 64-character floor and a separate buffer per tool-call argument channel, so a card number split across two chunks cannot escape. |
| Third-party detection | Partner guardrails are integrated by adding the partner’s API key, with their documentation naming Aporia, SydeLabs and Pillar Security among the supported platforms. | Detection is one policy input among seven trigger kinds rather than the product. The roadmap names a generic AI firewall, prompt scanner or red-team platform as a strategic non-goal and points at consuming other vendors’ verdicts instead of reproducing them. |
Budgets, rate limits and the stop button
| Capability | Portkey (Prisma AIRS AI Gateway) | Token Observe |
|---|---|---|
| Spend ceiling | Budget limits on an individual API key, set either as a maximum spend in USD with a $1 minimum or as a maximum token usage, which their documentation says automatically prevents further usage once the limit is reached; the same page states that the key continues to function until the full budget limit is reached. Limits can apply indefinitely, reset weekly on Sunday at 12 AM UTC, or reset monthly on the 1st at 12 AM UTC. Workspace-level limits are documented separately, and the page states the feature is available on the Enterprise plan and to select Pro customers. | Hard USD ceilings per request, per rolling hour, per UTC day and per UTC month in whatever combination you configure, reserved against the agent’s windows inside one per-agent database transaction before egress, priced at the most expensive of every provider and fallback the resolved route could reach. |
| An unpriced model inside a budget | Not described in their published documentation as of 2026-10-05. | A budgeted route with an unpriced reachable target is refused with a 409 before egress rather than priced at zero, because an empty price table once silently disarmed a ceiling. Fifty price rows ship and load additively at every boot. |
| Rate ceilings | Requests or tokens capped per minute, per hour and per day on an API key, with their documentation stating that subsequent requests are rejected until the time interval resets. | Requests, tool calls and tokens per minute per agent, evaluated after the kill switch, lifecycle and permissions and before the money verdict, so a refusal always names the first thing that refused it. |
| Warning before the wall | Configurable alert thresholds, where their documentation states that when usage reaches the threshold notifications will be sent to configured recipients, defaulting to organisation admins, owners and the key’s creator. | Webhook events on the trace lifecycle and on approval.requested. The ceiling itself refuses rather than warns, and the published cost of a hard ceiling is stated beside it: one billable egress, with no retry and no failover. |
| Stopping everything at once | The nearest documented control is a budget limit on an API key, which their documentation says automatically prevents further usage once the limit is reached. A single control that halts one agent, a team or the whole estate is not described in their published documentation as of 2026-10-05. | A kill switch scoped to one agent, one team or everything, checked first in the pipeline ahead of lifecycle, permissions, budgets and policy, and reaching even the routes that execute nothing. |
What it records, and who can rewrite it
| Capability | Portkey (Prisma AIRS AI Gateway) | Token Observe |
|---|---|---|
| The request record | Logs of multimodal requests and responses for viewing, monitoring and debugging; request tracing across the lifecycle of a request; custom metadata and tags for grouping; customisable filters; feedback values and weights; and analytics across what their documentation calls 21-plus key metrics. | One trace per governed request carrying the tool calls and their arguments, every policy decision including shadow-mode ones, the approvals, the tokens and the cost. Prompt and reply text is kept only if you switch conversation capture on, redacted before truncation; the default keeps none. The trace id is minted at step 3, before the verdict, so a blocked request is recorded rather than absent. |
| Administrative audit | Audit logs capturing administrative activity with timestamp, user, workspace, action, resource, response status, client IP and country, filterable by method, request id, resource type, action, status, workspace, user, IP, country and time range, on the Enterprise plan. | Every governance-plane change — an agent created, a policy widened, the kill switch engaged, an approval decided — appended to a chain whose entry digest covers the previous entry’s hash plus the canonical JSON of that entry’s own content, so an edit or a deletion breaks verification at a named sequence number rather than merely somewhere. |
| Whether the record resists an administrator | Not described in their published documentation as of 2026-10-05. What that material does publish is AES-256 encryption in transit and at rest, stated in their overview, and, on the audit-log page, access available to org owners and admins and indefinite retention; on whether a stored record can be shown afterwards not to have changed, it says nothing either way. | SHA-256 by default; HMAC-SHA256 under a MAC key held outside the database when one is configured, with the head sealed by a checkpoint MAC at every boot, and optional Ed25519 anchors signed on a schedule and published to a file or HTTP sink you site outside the database administrator’s reach. Tamper-evident, not tamper-proof — and the default is unkeyed, where a rewrite that re-hashes everything verifies clean. |
| An evidence bundle for an auditor | More than an earlier draft of this page credited. Their Logs Export documentation describes an asynchronous export job over request logs that returns a signed download URL, with JSONL output, up to 50,000 logs per job, a choice of 17 fields, and an export_settings.deniedFields control an administrator can use to keep request and response bodies out of an export; the page states the feature is available on the Enterprise plan. Their admin API separately documents a GET /audit-logs endpoint returning audit records with filters and pagination. What is not described in their published documentation as of 2026-10-05 is a single bundle binding logs, approvals and an integrity result together, or a verification output an auditor could check an export against. | A compliance export of the traces and events for a period, the approvals with approver identity and rationale, the audit entries covering every governance-plane change, a chain verification result naming the sequence number of any break, and a SHA-256 digest of the bundle at a recorded time. Digest-sealed, not signed. |
| Who may read the record | Audit log access is available to org owners and admins. Organisations are isolated, with their documentation stating that data, logs, analytics, prompts, integrations, providers, configs, guardrails and API keys are strictly confined within each organisation. | Trace list, search, detail and export reads are themselves attributable, spend figures and recertification evidence are gated on the reader’s team scopes as well as their role, and surfaces that join records with no trustworthy team key return 403 rather than a misleading partial view. |
| Asking the record a question | Customisable filters over logs and analytics, custom metadata and unique tags for grouping and troubleshooting, and dashboards over the metric set. | A plain-English question translated into a validated filter object over fourteen allow-listed fields, never into SQL, shown back as editable chips, with a deterministic keyword parser answering when no model is configured. The published cost: the filter cannot group, count or correlate across traces. |
| Retention | Published per plan: three days of logs and thirty days of metrics on the free Developer tier, thirty days of logs and ninety days of metrics on Production at $49 a month, custom retention on Enterprise, and indefinite retention named as an audit-log capability. | Cloud activity retention is configured to 30 days. For customer hosting: Trace retention is unset by default, and unset means keep forever. Setting a window turns on an hourly ageing pass; retention and erasure act on the live primary database only, so a restored pre-erasure backup can resurrect what was erased. |
How it deploys, and what it costs
| Capability | Portkey (Prisma AIRS AI Gateway) | Token Observe |
|---|---|---|
| Deployment shapes | Portkey-managed SaaS, and a hybrid deployment their documentation summarises as deploying the data plane in your VPC to route LLM traffic, with the control plane hosted by Portkey and the gateway running as containerised workloads under Kubernetes or ECS. | Cloud pilot or customer hosting. The cloud service holds workspace evidence and supplied credentials; customer-hosted installations keep runtime state on your infrastructure without vendor phone-home. These data-custody statements depend on the chosen mode. |
| What crosses the boundary on the hybrid deployment | Their architecture page describes the gateway periodically synchronising with the control plane at one-minute heartbeat intervals to fetch routing configs, templates, integrations and API keys, sending anonymous metrics such as model used, token counts and response times, and — where you keep logs in your own environment’s blob store — the control plane requesting them from the gateway when somebody views them in the dashboard. Their stated position is that all prompt content and LLM responses remain within your network and only anonymised metrics data cross network boundaries. The alternative they document is the gateway encrypting and sending logs to a Portkey log store instead, which they say needs no connection from Portkey into your environment. | Cloud pilot or customer hosting. The cloud service holds workspace evidence and supplied connection credentials; customer-hosted installations keep runtime state on your infrastructure without vendor phone-home. Cloud hosting makes Token Observe the custodian of workspace data and credentials; procurement and data-processing terms require review for the chosen mode. |
| Open source | Their documentation states that the gateway is open source and free to use, and their own feature comparison lists universal API, automatic fallbacks, load balancing, conditional routing, automatic retries and request timeouts in the open-source tier, with observability, prompt management, guardrails and the security and compliance features on the hosted plans. Their pricing page’s own open-source column is more generous than that table, listing guardrails and a basic dashboard as well, so establish the boundary with them rather than from either page. | Commercial source-available. Use, modify and self-host under a licence; redistribution and offering it as a competing hosted service are not permitted. Security research on your own deployment is expressly permitted; benchmark results are publishable with no pre-approval once the version and configuration tested are named, and security findings after coordinated disclosure. |
| Published price | Developer free with 10,000 recorded logs a month; Production at $49 a month for 100,000 recorded logs, with overage stated as $9 per additional 100,000 requests; and Enterprise at custom pricing with custom retention periods for 10 million plus recorded logs a month. Their pricing page also lists a self-hosted open-source option, which their overview calls open source and free to use. | No published price. The licence itself is a template pending review by counsel in England and Wales rather than an executed grant of rights, so read it as the intended terms rather than the signed ones. |
| Assurance you can buy today | Their material states ISO 27001 and SOC 2 certification with GDPR and HIPAA compliance, AES-256 encryption in transit and at rest, a mode available on request that does not store request and response body objects, and their plan comparison lists SOC 2, ISO 27001, GDPR and HIPAA compliance certificates against Enterprise. Their pricing page’s Enterprise tier names the SOC 2 report as Type 2. Ask for the scope and the date. | None held: no SOC 2, no ISO 27001, no ISO 42001, no independent penetration test. Published instead are a residual-risk register with the role whose dated acceptance is required on each row, a confirmed defect list including the attacks that still work, and a 30-day evaluation written so your security team may read, run and attack the software before a purchase order is raised. |
| Identity | Control-plane SSO over OIDC and SAML 2.0 for enterprise customers, with first-time SSO users automatically provisioned into the organisation unless disable_auto_provision_user is set, and provisioning restricted to email domains verified in admin settings. Their plan comparison lists SSO with Okta auth at Enterprise. | In customer-hosted mode: OIDC SSO with directory groups mapped to roles, plus bounded SCIM Users provisioning for viewer accounts that hold no evidence scopes. Group claims are taken as a snapshot with a default staleness of 24 hours rather than read live from the directory, and full-profile SCIM, SCIM Groups and SAML are deliberately not built. The cloud pilot uses managed Supabase identity and blocks independent child SSO and identity management. |
| Availability | Their overview cites 99.99 per cent uptime, millions of requests per minute and edge workers globally. | No availability SLA is offered. The cloud pilot runs on one shared host and has not qualified customer capacity or live recovery. In customer-hosted mode, operating the runtime and its recovery is your responsibility. The gateway fails closed, so an unavailable runtime stops governed model calls. |
The same slot, holding authority over different things
Both products stand between an agent and a provider, and both are reached by changing where the agent points. What differs is the object each one expresses authority over. Portkey’s published access control is a three-tier organisation role — owner, admin, member — with isolated organisations and API keys their documentation describes as carrying fine-grained scopes across metrics, completions, prompts, configs, guardrails, integrations and user management. That is authority over the platform: who may change a routing config, who may see the logs, who may mint a key. It is a real control and most estates need it.
Token Observe’s permissions name the action instead. A support agent holds an allow on tool:orderdb/get_details and model:gpt-5o-mini, while tool:payments/issue_refund is absent and therefore denied, because the default is deny rather than allow. An explicit deny beats every allow wherever it is written, including inside another role, so a narrow guardrail cannot be outvoted by a broad grant. When one agent delegates to another the chain intersects rather than unions: every hop must allow the action, which is what stops a low-privileged agent from escalating by routing the work through a higher-privileged one. Where the on-behalf-of header is switched to enforce, the roles mapped from a named human’s directory groups are appended as one more link that can only subtract.
The consequence for a buyer is that these two controls answer different questions and can sit in the same estate without contradicting each other. Portkey’s roles answer who in your organisation may operate the gateway, and its MCP Gateway answers which tools a user may reach and which arguments pass a check. Token Observe’s answers whether this agent may issue this refund, and it answers it at the moment the model proposes the call. A grid row that scored those against each other would be measuring nothing, which is why the row above carries a note instead of a verdict.
- Deny-by-default
- An action no role names is refused. There is no implicit allow to be tightened later, so the failure mode of a forgotten grant is a blocked agent rather than an unbounded one.
- Precedence
- An explicit deny wins over every allow, in any role, in any order. Precedence is a property of the evaluator rather than of the order somebody happened to write the roles in.
- Delegation intersects
- Agent-to-agent calls carry a chain, and every hop must permit the action independently. Union semantics would make delegation an escalation primitive.
What a guardrail is bound to, and what an approval is bound to
Portkey’s guardrails are documented as running in two places and two modes: before_request_hooks and after_request_hooks, asynchronously by default so results are logged without affecting orchestration, or synchronously so the request can be orchestrated on the verdict. The documented outcomes on a failed synchronous check are deny with a 446, allow-and-flag with a 246, fall back to another LLM or prompt, retry, or record feedback for an evaluation dataset. That is a rich set of orchestration outcomes and it covers the cases where the right answer is to stop, to reroute, or to note. Their MCP Gateway material goes further: its overview names approval workflows per server, and its guardrails page documents checks on a tool call’s arguments before the tool executes and on its result afterwards, blocking a failed call with a JSON-RPC error. An approval workflow is not described on that guardrails page as of 2026-10-05 — so the gap described below is a gap in what is published today, and may not be one for long.
Token Observe’s difference is the outcome that is not on the guardrail list, and it is the one the whole product turns on. A policy whose action is require_approval refuses the request with a 403, mints an approval record, and holds the trace open until a person decides. The record is bound to the SHA-256 of the canonicalised action plus the execution context it was proposed in, which is what makes it an approval of that payload rather than of that action type: change one argument and the hash no longer matches, so the retry is refused as a mismatch rather than allowed as near enough. Consumption is a compare-and-set, so two concurrent retries cannot both execute on one approval. It expires — 60 minutes by default, one minute to seven days by policy — because an approval that never expires is a standing grant with extra paperwork.
The limit is stated in the same register as the claim, because it changes how you integrate. Approving does not push anything to the agent; Token Observe has no way to call an agent back. The agent redeems the approval by repeating the identical request with its id, once. An agent written to treat a 403 as fatal will need a small change to participate, and that is a real integration cost rather than a footnote. The compensating property is that the approval is verifiable afterwards: the approver’s identity and rationale land in the export beside the trace and the audit entries, so the question of who authorised this has an answer that does not depend on anyone’s memory.
- Bound to a payload, not a type
- The hash covers the canonical action and its execution context. An approval for a £180 refund on order 4417 does not authorise a £1,800 refund on order 4418.
- Single-use
- Consumption is a compare-and-set. Two retries racing each other cannot both redeem the same approval, which is the failure a naive status flag produces under concurrency.
- Expiring
- Sixty minutes by default. A policy can set anything from one minute to seven days, and the expiry is enforced at redemption rather than by a sweeper.
Both keep a record. The question is who can change it
Portkey publishes a detailed account of what their audit log captures — timestamp, user, workspace, action, resource, response status, client IP and country — and of how it is filtered, along with indefinite retention as an enterprise capability and access available to org owners and admins. On the metadata a security team wants for an administrative action, that is more than Token Observe advertises, and the country and client-IP fields in particular are useful in an investigation. Their log retention is published per plan, which is the kind of specificity a buyer can plan around: three days on the free tier, thirty days on Production, custom on Enterprise. They also document getting the record out again, which this page originally missed: a GET /audit-logs admin endpoint with filters and pagination, and a separate Enterprise Logs Export that runs an asynchronous job over request logs and hands back a signed URL to JSONL, up to fifty thousand logs at a time.
Token Observe’s record is built for a different adversary: the person who administers the database. Every governance-plane change is appended to a hash chain whose entry digest covers the previous entry’s hash plus the canonical JSON of the entry’s own content, and appends happen inside a transaction that also takes the chain tip, so concurrent writers cannot fork it. With an audit MAC key configured, the digests become HMAC-SHA256 under a key held outside the database and the head is sealed by a checkpoint MAC at every boot. Above that, an Ed25519 anchor signs a statement of the head on a schedule and publishes it to a file or HTTP sink. The anchor buys one narrow claim, and it is the only one made: any copy of an anchor you kept off-box beats any rewrite made after you took it.
The guarantee is conditional, and the default does not meet it. Unkeyed, the chain is plain SHA-256, and an operator with write access can rewrite a row and recompute every hash after it, and the result verifies clean. Token Observe reports which of the two you are holding in every verification result and every export, because that difference is the guarantee. The anchor is off until a signing key is configured, and it is worth something only if the sink is somewhere the database administrator cannot reach. The word is tamper-evident, never tamper-proof, and the export is digest-sealed rather than signed. Anchoring also refuses more often than it signs: it will not anchor a chain that does not verify, a head that has moved backwards, or a rewritten anchored entry, because signing over a rewrite would launder it under a key auditors trust.
Two questions worth putting in writing to whichever proxy you run
These are not findings about Portkey and nothing here has been tested against their product. They are the two failure patterns Token Observe’s own engineering notes record having had to solve, and they are worth asking about any gateway, including this one, because the answers are specific and checkable and a good product will have them ready.
The first is what a fallback chain does with a refusal. Portkey documents fallbacks between providers and models for resilience and automatic retry strategies, which is what you want when a provider times out. The question is how the chain classifies a provider’s content-policy refusal, because if a refusal is treated as a retryable error then the next provider’s answer comes back as a success and nothing in the record says a refusal happened. Token Observe’s answer is seven typed failure classes of which three fail over and four do not: a timeout, a 429 and a 5xx move on, while a content-policy refusal, an authentication failure, an over-long context and a malformed request stop where they are.
The second is cache-token arithmetic. Anthropic reports cache reads and writes outside the input total while OpenAI and Gemini report them inside it, so any cost figure that adds the buckets without normalising first is wrong in one direction for one provider and wrong in the other for another, and the invoice is the thing that eventually disagrees. Token Observe normalises provider usage into mutually exclusive buckets before any arithmetic happens, and narrows the route to whichever provider actually served the request before pricing it, so the ledger prices against the vendor that will invoice you rather than the one that was tried first. Ask your gateway vendor how they do it; the answer is a paragraph, and it either exists or it does not.
Where each one is the right answer
Choose Portkey (Prisma AIRS AI Gateway) when
- Procurement requires SOC 2, ISO 27001 or a comparable certificate from the vendor. Portkey’s material states ISO 27001 and SOC 2 certification; Token Observe holds none and says so on the first call.
- You need breadth across models and providers as the primary requirement — their overview describes a unified interface to over 250 AI models with conditional routing, load balancing, caching, canary testing and multimodality.
- High availability is needed on day one. Their overview cites 99.99 per cent uptime and edge workers globally; Token Observe is a single-writer process on one host with no replica and no availability commitment.
- You want a published price and a self-serve path from free to production — theirs runs from a free tier at 10,000 recorded logs a month to $49 a month for 100,000, with an open-source gateway underneath it.
Choose Token Observe when
- A human has to authorise one exact payload rather than a class of requests, and the authorisation must be single-use, expiring and attributable to a named person against the trace.
- The agents take actions somebody answers for — a refund, a deployment, an email, a ticket transition, a database write — and permissions have to be expressed over the action rather than over the console.
- The record has to resist the person who administers the database, which means a hash chain with a MAC key held outside it and an anchor published somewhere that administrator cannot reach.
- Zero vendor egress is a hard requirement, including air-gapped operation, with no control plane to synchronise with and no metrics leaving the boundary.
When you would run both
For an estate that already runs Portkey, adding Token Observe beside it is the only arrangement to recommend: the base URL is already swapped, the keys are already central, the spend is already counted, and ripping that out to install something else is work with no governance outcome attached to it. The arrangement that costs least is to leave Portkey where it is and route only the agents that take consequential actions through Token Observe, either in front of it or behind it. In front, Token Observe holds the agent credential and takes the governance verdict, then routes to Portkey as an OpenAI-compatible upstream so provider keys, fallbacks and caching stay in one place. Behind, Portkey keeps the ingress and forwards the agents that need governing. Both shapes add a hop and a second failure domain, so this is a decision to take deliberately rather than by default — a chat assistant does not need both, and a refund agent might. Where you need a third enforcement point to be shown to agree, Token Observe can compile scope and trigger matching to digest-locked OPA Rego with reproducible positive and negative witnesses, though that artefact deliberately excludes permissions, budgets, approval consumption, kill switches, action precedence and side effects, which stay authoritative in Token Observe.
The category argument sits above this one: Token Observe and llm gateways covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
LiteLLM
Keep LiteLLM where it meets your gateway needs. Compare Token Observe when your IT team also needs provider billing evidence and scoped agent decisions.
Kong AI Gateway
Kong decides which caller may use which tool. Token Observe decides whether this one call, with these arguments, goes ahead, and checks afterwards that it happened. If you already run Kong, the first is nearly free and the second is the reason to read further.
Cloudflare AI Gateway
Cloudflare’s gateway decides what the payload contains. Token Observe decides whether the agent that sent it was allowed to.
Agent Router
Both hold the request. One charges the token budget once the response completes; the other reserves the money before the request leaves your network.
MuleSoft Omni Gateway
Both refuse the call inline, and both now say they govern agents. MuleSoft governs an agent as an instance in API Manager; Token Observe governs it as a record with a named owner, a budget and a human gate.
Is Token Observe trying to replace Portkey?
No, and for most readers it should not. Portkey publishes a unified API over more than 250 models, conditional routing, load balancing, fallbacks, caching, an open-source gateway, a published price list and ISO 27001 and SOC 2 certification; Token Observe registers six first-class upstreams plus any OpenAI-compatible endpoint, holds no certifications and publishes no price. The product’s own roadmap says to integrate above or beside an existing gateway rather than compete on connectivity, and any gateway you already run can be registered as an upstream or can forward to Token Observe for the agents that need governing. The narrow thing Token Observe adds is the decision and the evidence: deny-by-default action-level permissions, an approval bound to one payload hash, and a hash-chained record that can be anchored off the box.
Portkey has guardrails. What does Token Observe do that a guardrail does not?
Their documented guardrail outcomes are deny with a 446, allow-and-flag with a 246, asynchronous logging, fallback to another LLM or prompt, retry, and feedback into an evaluation dataset, running as input or output hooks synchronously or asynchronously. Token Observe’s difference is the outcome that pauses rather than decides: a 403 carrying an approval id, bound to the SHA-256 of the canonical action plus its execution context, single-use through a compare-and-set and expiring at 60 minutes by default. Alongside that sits authority over the action itself — an agent that holds no grant on tool:payments/issue_refund cannot issue one whatever the payload says — and a delegation chain that intersects rather than unions. Portkey is not silent on approvals, and this page should not imply they are: their MCP Gateway overview lists approval workflows per server alongside content filters, and their MCP Gateway guardrails page documents checks on tool-call arguments and results, though an approval workflow is not described on that page. What such an approval binds to — one exact payload, or a class of operations — and how it is redeemed are not described in their published documentation as of 2026-10-05, and an undocumented detail reads identically to an absent one, which is why this page will not assert either. Put it to them in writing.
Their hybrid deployment keeps prompts in our VPC. Is that not the same as self-hosting?
It is close on the part that matters most and different on one part. Their architecture page describes the data plane running in your infrastructure with all prompt content and LLM responses remaining within your network, which is the substantive privacy claim. The same page describes what does cross the boundary: the gateway synchronises with the control plane at one-minute heartbeat intervals to fetch routing configs, templates, integrations and API keys, sends non-sensitive operational metrics such as model used, token counts and response times, and, where logs are kept in your own blob store, requests them from your gateway when somebody opens them in the dashboard. Token Observe has no control plane to synchronise with, so nothing leaves for the vendor at all (no telemetry, no prompts, no keys, no trace database), and the runtime data flow is documented so you can verify it. Whether the difference matters depends on whether your constraint is prompt confidentiality or complete network isolation, and on the second, Token Observe still cannot promise the vendor is legally never a processor, because support handling and contracts are your counsel’s analysis rather than a runtime property.
Their audit log has indefinite retention. Why is a hash chain better?
It is not better at the things their audit log is good at, and this page should not pretend otherwise. Their published audit record carries client IP and country alongside user, action, resource and status, is filterable across all of those, is retained indefinitely on the Enterprise plan, and can be pulled programmatically through a documented admin endpoint alongside an Enterprise export job for request logs; that is a strong administrative record. The hash chain answers a narrower question: whether the record you are reading is the record that was written. Each entry’s digest covers the previous entry’s hash plus the canonical JSON of its own content, so an edit or a deletion breaks verification at a named sequence number; with a MAC key held outside the database the digests become HMAC-SHA256, and with a signing key configured an Ed25519 anchor is published off the box on a schedule. The caveats belong in the same sentence: the default is unkeyed and a rewrite that re-hashes everything verifies clean, the anchor is off until you configure it and only counts if the sink is out of the database administrator’s reach, and the export is digest-sealed rather than signed. Tamper-evident, not tamper-proof.
Have you tested Portkey against Token Observe?
No. Every statement about Portkey on this page paraphrases their own published material read on 2026-10-05 — the homepage, the product overview, the gateway and guardrails documentation, the budget and rate limit page, access control, audit logs, the enterprise architecture and deployment pages, SSO, the open-source feature comparison, observability, pricing, the MCP Gateway pages, and the logs-export and admin audit-log API references, plus Palo Alto Networks’ announcement of Prisma AIRS AI Gateway — and none of it has been independently tested. There has been no witnessed bake-off, and the product’s own launch gates name a competitor bake-off as evidence that does not yet exist. Where a cell says a capability is not described in their published documentation, that is a statement about the documentation and not about the product: read it as a question to put to Portkey in writing. What is offered on the Token Observe side instead is a 30-day evaluation written so your security team can read, run and attack the software before a purchase order is raised, with benchmark results publishable with no pre-approval once the version and configuration tested are named and security findings publishable after coordinated disclosure — under a licence that is a template pending review by counsel in England and Wales rather than an executed grant of rights.
Prefer to ask a person? Write to James
Anything that speaks an OpenAI-compatible, Anthropic or Gemini dialect. One base URL and one key change.
Authorise, redact, budget, decide. Then forward, refuse, or hold for a human.
Fails closed. A decision it cannot make is a request that does not go.
OpenAI, Anthropic, Google Gemini, OpenRouter, Amazon Bedrock, Azure OpenAI, MCP tool servers, and any OpenAI-compatible endpoint you host.
Registry, policy decisions, the hash-chained audit log and the flight recorder. Held in your database, on your disk. Nothing goes to the vendor.
Tell us which one you are already running.
If Portkey (Prisma AIRS AI Gateway) is already in your stack, the useful question is what each is for and where the seam between them sits. Say what you have and the reply will say where each fits, including when you do not need a second thing.