Token Observe vs LiteLLM
If you already run LiteLLM, keep it. The question that decides whether you need anything more is about the actions your agents take, not about the proxy.
2 September 2026
- LiteLLM product and pricing page
- LiteLLM Enterprise
- Enterprise features and support SLAs (docs)
- Proxy architecture and request flow
- Guardrails quick start
- Audit logs
- Budgets and rate limits
- MCP permission management
- Proxy logging
- MCP overview and tool-call approval
- Responses API and the MCP approval round trip
- Custom LLM pricing and zero-cost models
- Production deployment guide
- Repository LICENSE
Their claims, not our testing. Verify anything that decides it for you.
On this page
For most teams shopping for a gateway, LiteLLM is the better purchase, and it is not a close call
Start with reach and price, because between them they settle the question for a large share of readers. LiteLLM’s site advertises one OpenAI-compatible API to 140+ providers and 1,800+ models, and its free tier is listed at $0, “Free forever”, carrying virtual keys, budgets and teams, load balancing and RPM/TPM limits, guardrails, and logging into Langfuse, Arize Phoenix, LangSmith and OTEL. The repository LICENSE puts everything outside the enterprise/ directory under MIT. Token Observe has six first-class upstreams plus whatever OpenAI-compatible endpoints you register, and publishes no price list. If what you need is one key store, one retry policy, one spend counter and a dashboard, LiteLLM is the proportionate control and Token Observe would be an expensive way to add nothing to it.
The second advantage is operational, and this is the part that should stop a reader who needs the request path to be dependable on day one. LiteLLM documents a Postgres-backed proxy with Redis or in-memory caching in front of the key lookup, official images on ghcr.io/berriai with Helm charts for EKS, GKE and AKS and Terraform modules for AWS and GCP, air-gapped deployment, support for the four most recent stable minor lines, and 24/7 support with published response targets — one hour at Sev 0, six hours at Sev 1, twenty-four at Sev 2 and 3, and seventy-two for a vulnerability. Their enterprise page lists SOC 2 Type 2 and ISO 27001. Token Observe is a single-writer SQLite process on one host at its current target scale, with no replica, no clustering and no vendor-operated uptime SLA, no SOC 2 or ISO certification and no independent penetration test, and a published licence its own repository describes as a template pending counsel. Against that list, choosing LiteLLM is a defensible decision to write down.
The third is guardrail choice, and it is a genuine architectural advantage rather than a longer feature list. LiteLLM’s guardrail model is an integration surface: pre_call, during_call, post_call and logging_only modes wrapped around Aporia, AWS Bedrock, Guardrails AI, Lakera, Presidio, Azure Content Safety, OpenAI Moderation, Cato Networks and a generic API for anything else, so you can put the detector your security team has already bought into the path. Token Observe’s detection is its own and it is heuristic — eleven data classes of which three are checksum-validated, and nine weighted injection patterns rather than a model — which is a narrower and more opinionated thing to own. Everything said about LiteLLM on this page is taken from the pages listed in the sources, read on 2 September 2026, and none of it has been tested; where a row reads as an absence, read it as a question to put to the vendor in writing rather than as a finding.
Token Observe and LiteLLM, capability by capability
The LiteLLM column paraphrases BerriAI’s own published material as it stood on 2 September 2026. None of it has been independently tested here, products in this category ship quickly, and a capability that is absent from a vendor’s documentation is not the same thing as a capability the product lacks. Check anything that decides it for you against their own current documentation.
Where it sits in the request path
One OpenAI-compatible API in front of 140+ providers; their site’s claim is that you can swap models without changing app code.
The same swap: one base URL and one key for supported OpenAI-compatible, Anthropic and Gemini ingress, normally rather than an application refactor.
Their architecture page: the bearer token is checked as valid and under budget, the parallel-request limiter checks rpm and tpm across global, key, user and team, then the router handles load balancing, fallbacks and retries, then the provider is called.
Eleven ordered steps to one decision point: authenticate, resolve the agent, open the trace, sanitise Unicode, scan, take the verdict at step 6, enact it, route, call upstream, govern any proposed tool call, then meter and record.
After the response. Logging to external services, rate-limit accounting and spend tracking run as asynchronous background tasks, and their architecture page states that no database write sits in the request path.
The trace is opened at step 3, before the verdict, so a request that is refused at step 6 is still recorded with a trace id returned to the caller.
These are two different design goals rather than two attempts at one. Keeping writes off the request path is how a carrying proxy stays fast; opening the trace first is how a refusal becomes evidence. Neither choice is free.
post_call guardrails “run after LLM call, on input & output”; during_call runs in parallel with the request, on the input.
Streamed output passes a hold-back buffer with a 64-character floor and a per-tool-call-argument channel, and a blocking data class ends the stream with an in-band ACP_POLICY_BLOCKED frame; the response-side plan is fixed before the first byte, because a status line is spent once it is written.
An MCP gateway with server-level and tool-level permissions, and lists intersected across key, team, end user, agent and internal user, with the organisation acting as a ceiling.
A tool call the model proposes is evaluated against tool_call policies on the way back, before it reaches the caller, and a tool executed through the gateway is authorised again at execution rather than only filtered out of the catalogue.
What it enforces before the payload leaves
Four modes — pre_call, during_call, post_call, logging_only — with a violation answered as “Violated guardrail policy”. Documented providers include Aporia, AWS Bedrock, Guardrails AI, Lakera, Presidio, Azure Content Safety, OpenAI Moderation, Cato Networks and a generic API.
Detection is built in and heuristic: eleven data classes with Luhn, IBAN mod-97 and NHS mod-11 checksums on three of them, and nine weighted injection patterns scored 1.25× when the text is a tool result.
Their MCP permissions page: lists intersect and most-restrictive wins, but “If no level has a list, the request can access every MCP server (open by default).”
Deny by default. An action no role names is refused, an explicit deny beats every allow wherever it is written, and a delegation chain intersects at every hop rather than unions.
Hard budgets per key, team, org and model with daily and monthly resets — “At the cap, requests stop” — enforced against spend read from the database. Their documentation heads that section “Budgets require a database” and states that “every budget on this page is enforced against spend read from the database, so none of them cap anything on a DB-less deployment”, adding that litellm_settings.max_budget “fails open there rather than erroring”.
The money verdict is taken last, after the route is resolved: every provider and fallback that route could execute is priced, and the most expensive of those rates is reserved against the agent’s hour, day and month windows inside one per-agent transaction before egress.
Their custom-pricing page addresses this directly. A model configured at zero cost makes LiteLLM “automatically skip ALL budget checks (user, team, team member, end-user, organization, and global proxy budget)” for requests to that model, and it warns that “Both costs must be explicitly set to 0. If costs are null or undefined, the model will be treated as having cost and budget checks will apply.” Which dollar figure those checks then charge a request to a model carrying no price is not described on that page as of 2026-09-02.
A budgeted agent whose resolved route has an unpriced reachable target is refused with a 409 before egress rather than priced at zero.
This row exists because of Token Observe’s own defect history rather than anything found in LiteLLM’s. The shipped price rows once loaded only under the demo seeder that the setup guide tells production operators not to run, so a documented install metered every trace at $0 and every per-request ceiling admitted every request. Put the question to whichever proxy you run.
TPM and RPM limits, max parallel requests and per-model rate limits, configurable at team, user, key or agent level.
Requests, tool calls and tokens per minute per agent, checked before the route is resolved, under a kill switch scoped to one agent, one team or the whole estate that is checked first in the pipeline.
Documented for MCP tool calls. Setting require_approval to “never” makes the proxy execute the returned tool calls itself; their MCP page states that if you omit require_approval or set any other value, “the MCP tool calls are returned to the client so that you can review and execute them manually, matching the upstream OpenAI behavior”. Their Responses API page shows that round trip: an mcp_approval_request output carrying an id, then a followup input of type mcp_approval_response with approve and approval_request_id.
A 403 carrying an approval id, bound to the SHA-256 of the canonicalised action plus the execution context it was proposed in, single-use by compare-and-set, expiring at 60 minutes by default and configurable from one minute to seven days.
Both are approval mechanisms and the difference is what holds the decision. LiteLLM’s is a round trip inside one conversation: the tool call comes back to the client, and the approval travels in the client’s next request. Token Observe’s is a server-side record bound to a digest of one exact canonicalised action, consumed once, expiring, with the approver’s identity and rationale written into the trace. Which of the two you need depends on whether the record of who approved has to outlive the conversation.
What it records, and what the record is worth
Audit logs record keys, teams, users and models across create, update, delete and regenerate, with who performed the action. On by default under an enterprise licence; otherwise set store_audit_logs in config.yaml.
Every governance-plane change — an agent created, a policy widened, the kill switch engaged, an approval decided — appended to a hash chain whose entry digest covers the previous entry’s hash plus the canonical JSON of that entry’s own content.
Their enterprise page advertises “Audit logs on every request” while their audit-log documentation describes the entities and actions above. The two read differently, so it is worth asking which is current before you rely on either.
Audit rows are stored in the proxy database and can additionally be exported to an external backend such as S3, batched and uploaded asynchronously. That page publishes the full row spec — id, timestamp, who changed it, the action, the table, the object id and the before and after values — and does not describe a hash chain, MAC or signature over those rows as of 2026-09-02.
SHA-256 by default and HMAC-SHA256 when an audit MAC key is configured outside the database, the head sealed by a checkpoint MAC at every boot, and optional Ed25519 anchors published off-box on a schedule. Tamper-evident, not tamper-proof: unkeyed, an operator who rewrites a row and recomputes every hash after it verifies clean, and Token Observe reports which of the two you hold in every verification result.
Callbacks build a standard logging object per call and ship it to Langfuse, OpenTelemetry, Datadog, GCS, S3, Azure Blob, AWS SQS, Langsmith, MLflow and others.
One trace per governed request holding events, usage, policy decisions and tool calls, searchable in English through a validated filter object over fourteen allow-listed fields, never through generated SQL. An OTLP-over-HTTP receiver takes JSON and protobuf as an input rather than competing for the destination.
litellm.turn_off_message_logging=True “will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tracked”, with per-request control via the x-litellm-enable-message-redaction header, which their documentation marks as being in beta.
Redaction happens inline before storage rather than at the logging boundary: detected values are tokenised or blocked before the payload leaves your network, secrets are never tokenised reversibly, and the stored trace holds the redacted text.
Audit logs export to an external storage backend in addition to the database, batched and uploaded asynchronously so they do not block proxy requests.
Evidence exports are sealed with a SHA-256 digest and carry the audit-chain verdict, and an offline verifier runs as one Node script with no install, no database and no network. The exports are digest-sealed, not signed.
RBAC by key, team and org, listed on their enterprise page as “RBAC by key / team / org” alongside SSO and SCIM; their enterprise documentation adds SSO for the admin UI across Okta, Azure AD, Google Workspace and any OIDC or SAML provider, and IP-address-based access control lists.
Trace list, search, detail and export reads are themselves attributable; spend figures and recertification evidence are gated on the reader’s team scopes as well as their role, and organisation-wide views return 403 rather than a misleading partial answer without an explicit organisation-wide scope.
How it deploys, how it is licensed, what it costs
Self-host anywhere, including air-gapped. Their enterprise page describes a gateway that “runs in your own infrastructure, so your data and your keys never leave it”, and their production deployment guide publishes official images to ghcr.io/berriai with Helm on EKS, GKE and AKS and Terraform modules for AWS and GCP. Enterprise is the same self-hosted deployment, activated by a LITELLM_LICENSE key.
Self-hosted only and bring-your-own-key. The vendor receives no product telemetry, phone-home data, prompts, keys or trace database; governed payloads leave only for the providers you configure, after policy and redaction.
A Postgres-backed proxy with Redis or in-memory caching ahead of the key lookup, and support for “the four most recent stable minor lines”.
A single-writer SQLite process on one host at the current target scale. No replica, no clustering and no vendor-operated uptime SLA, and because it fails closed, its availability is a governance property of your environment.
140+ LLM providers and 1,800+ models on their site, behind one key.
OpenAI, Anthropic, Gemini, OpenRouter, Bedrock and Azure OpenAI as first-class upstreams, plus any OpenAI-compatible endpoint you register — including a LiteLLM proxy you already run.
The repository LICENSE states that content under the enterprise/ directory is licensed under enterprise/LICENSE, and that content outside that directory is under the MIT License.
Self-hosted and source-available to the customer under a licence the repository itself describes as a template pending counsel.
$0, “Free forever”, for the open-source tier — provider coverage, virtual keys, budgets and teams, load balancing and RPM/TPM limits, guardrails and the logging integrations. Enterprise is quoted rather than listed: “Pricing is based on usage.”
No published price list. You pay your providers directly, because the deployment holds your keys.
Their enterprise page lists SSO and SCIM, OIDC/JWT auth, secret managers and key rotation, audit logs, and RBAC by key, team and org under Enterprise, alongside 24/7 support and response-time SLAs. Their enterprise documentation adds that “SSO is free for up to 5 users. Beyond that, an enterprise license is required.”
Permissions, policy, approvals, budgets, the flight recorder, the audit chain and anchoring are one product with no feature tier; anchoring and the on-behalf-of intersection default to off because they need a key or an identity provider, not because they are sold separately.
Their enterprise page lists SOC 2 Type 2 and ISO 27001.
None published: no SOC 2, no ISO 27001, no ISO 42001 and no independent penetration test.
If you already run LiteLLM, the question is not about the proxy
The objection is real rather than rhetorical, and it deserves the concession first: LiteLLM sits in exactly the slot Token Observe asks for, the base URL is already swapped, the keys are already central, and the spend is already counted against a virtual key. Pulling it out to install something else is work with a migration attached and no governance outcome attached, and nothing on this page recommends it.
The question that decides whether you need anything more is about what a governed call can end in. If every call ends in text that a human reads before anything happens — drafting, summarising, chat, code suggestions someone reviews — then a routing proxy with a spend counter and a guardrail plugin is a proportionate control, and it is the one to keep. If a call can end in a refund being issued, a pull request being merged, an email leaving the building, a ticket transitioning or a row being written, the interesting question stops being what did it cost and becomes whether the thing that was authorised is the thing that happened. That is a different product rather than a larger configuration of the same one.
Three questions are worth putting in writing to whichever proxy you run, and they are questions rather than findings, drawn from failure patterns recorded in Token Observe’s own engineering notes rather than from anything observed in LiteLLM. Does the fallback chain treat a provider’s content-policy refusal as a retryable error, so the next provider’s answer returns as a success and nothing in the record says a refusal happened? Does cache-token accounting add Anthropic’s cache read and write buckets to a total that already includes them, or fail to add OpenAI’s, and produce a figure the invoice disagrees with? Does an approval, where one exists, authorise an action type rather than an exact payload, so a retry with one argument changed is still allowed? The answers are specific and checkable, and a good product will have them to hand.
Which side of LiteLLM’s own line the governance features sit on
“We already have LiteLLM” usually means the MIT proxy, and the features a governance buyer is shopping for are mostly on the other side of LiteLLM’s published line. Their pricing panel puts virtual keys, budgets and teams, load balancing, RPM and TPM limits, guardrails and the logging integrations in the $0 tier. Their enterprise page and enterprise documentation put SSO for the admin UI across Okta, Azure AD, Google Workspace and OIDC or SAML, SCIM, JWT-based authentication, role-based access control, IP-based access lists, key rotation, secret-manager integration and audit logs with retention policies under Enterprise, deployed self-hosted with a licence key. One qualification belongs in the same sentence rather than a footnote, because it is theirs and it moves the line: their enterprise documentation states that “SSO is free for up to 5 users. Beyond that, an enterprise license is required.” That is not a criticism of the split — it is a sensible place to draw it, and the free tier is unusually generous by the standards of the category — but it changes the comparison a buyer is actually making.
The practical consequence is that a shortlist which reads “Token Observe, or the free thing we already run” is comparing against something narrower than the enterprise pages describe. If the requirement that started the evaluation is single sign-on, an audit trail of who changed which key, or role-based separation between the platform team and the application teams, then the comparison is between two commercial purchases and the cost line on the LiteLLM side is a quote rather than zero. Ask for it early; their documentation says pricing is based on usage.
The licence boundary is worth reading for yourself rather than taking second-hand. The repository LICENSE says content under the enterprise/ directory is licensed under enterprise/LICENSE and everything outside it is MIT, and the enterprise documentation describes a self-hosted Docker deployment activated by a licence key. Which specific behaviours that key turns on in a given release is a question for the vendor and for your counsel, and it is the sort of question that is much cheaper to answer before a deployment than after one.
- In the free tier, per their pricing panel
- 140+ provider integrations, virtual keys, budgets and teams, load balancing with RPM and TPM limits, LLM guardrails, and logging into Langfuse, Arize Phoenix, LangSmith and OTEL.
- Listed under Enterprise, per their enterprise page and docs
- SSO and SCIM, OIDC and JWT auth, RBAC by key, team and org, audit logs with retention policies, IP access lists, key rotation, secret managers, and 24/7 support with response-time SLAs — with SSO itself free for up to five users, per their enterprise documentation.
- What that means for the shortlist
- If the evaluation was triggered by an identity, audit or access-separation requirement, both columns carry a price, and the LiteLLM one is quoted on usage rather than published.
Accounting after the response versus deciding before it
The sharpest technical difference is one LiteLLM states plainly about itself, and it is a strength in its own frame: no database write sits in the request path, and logging, rate-limit accounting and spend tracking run as asynchronous background tasks once the provider has answered. A carrying proxy should be built that way. Budgets are then enforced, in their words, against spend read from the database — their page is headed “Budgets require a database” and says that without one “none of them cap anything”, with the global check failing open rather than erroring. What that buys is throughput; what it costs is that the figure a budget check reads is the figure the last completed write left behind.
Token Observe defers the money verdict for the opposite reason. Permissions, policy and rate limits are decided first, then the route is resolved, then every provider and fallback in that route is priced and the most expensive candidate rate is projected and reserved against the agent’s rolling hour, UTC day and UTC month inside a single per-agent database transaction — before the call goes out. A budgeted agent whose resolved route contains a target with no price row is refused with a 409 rather than admitted at an assumed zero. The cost of that design is stated in the same breath as the claim: a hard ceiling buys you one billable egress, with no retry and no failover behind it, and the ceiling is conservative rather than optimistic by construction.
The approval gate is the second half of the same argument, and it is the capability this page turns on — so the concession goes first. LiteLLM documents an approval path of its own for MCP tool calls: omit require_approval, or set it to anything but “never”, and their MCP page says the tool calls are returned to the client for you to review and execute manually, with the Responses API carrying that round trip as an mcp_approval_request and a matching mcp_approval_response. The decision there lives in the client’s next request; what follows is a different object rather than a missing one. A policy whose action is require_approval refuses the request with a 403, mints an approval bound to the SHA-256 of the canonicalised action plus the execution context it was proposed in, and holds the trace open until a named person decides. That approval authorises that payload and nothing else — change one argument and the hash no longer matches, so the retry is refused as a mismatch rather than allowed as near enough — it is spendable once by compare-and-set so two concurrent retries cannot both execute, and it expires at 60 minutes by default. The honest limit sits beside it: approving pushes nothing to the agent, because Token Observe has no way to call an agent back, so the approval takes effect only when the agent repeats the identical request carrying its id.
Where each one is the right answer
Choose LiteLLM when
- The requirement is connectivity and cost visibility — one key store, one retry policy, one dashboard across many models — and nobody has yet asked you to prove what an agent did.
- You need breadth. LiteLLM advertises 140+ providers and 1,800+ models; Token Observe has six first-class upstreams plus the OpenAI-compatible endpoints you add yourself.
- You need the request path to be dependable now, against a vendor publishing a one-hour Sev 0 response target. Token Observe is one writer on one host at this scale, with no replica and no uptime SLA.
- Procurement needs a supplier with published certifications: their enterprise page lists SOC 2 Type 2 and ISO 27001, and Token Observe publishes none.
- The traffic is chat or drafting, where a human reads every output before anything happens, so an inline refusal buys less than the outage risk of a fail-closed dependency.
Choose Token Observe when
- The agents take actions somebody has to answer for — a refund, a deployment, an email, a ticket transition, a database write — and an API returning 200 is not acceptable proof that it happened.
- You need a human decision bound to one exact payload rather than to an action type, spendable once, expiring, with the approver recorded against the trace.
- Somebody will eventually ask who says the head you are showing me is the head, and a log table plus a bucket export is not an answer to that question.
- The ceiling has to hold before the money is spent rather than be reconciled from the last completed write, including for a model whose price you have not loaded.
- Permissions have to be deny-by-default across model calls and tool calls alike, with an explicit deny beating every allow and delegation intersecting rather than accumulating.
When you would run both
Running both is the normal answer, and the cheapest arrangement is to leave LiteLLM exactly where it is and register it as an OpenAI-compatible upstream, so Token Observe holds the agent credential and takes the verdict, and LiteLLM keeps the provider keys, the 140+ upstreams, the load balancing and the retries behind it. That way the governance decision happens before the hop and the connectivity investment you have already made is untouched. Point only the consequential agents at Token Observe — the ones that issue refunds, merge code, send mail or write rows — and let everything else keep pointing where it points now; that is the pilot boundary the product is designed around, roughly five to fifty agents owned by one platform team. Two proxies in series is a second failure domain and a second hop of latency, so make it a deliberate decision rather than a default: a chat assistant does not need both, and a refund agent might. LiteLLM’s logging callbacks and Token Observe’s OTLP-over-HTTP receiver are complementary rather than competing, and Token Observe’s roadmap treats being another standalone tracing product as a named non-goal.
The category argument sits above this one: Token Observe and llm gateways covers what the whole category does and does not do, which is the better page to read if you have not yet shortlisted a product.
The others in the same slot
Portkey
Both hold the payload before it reaches a provider. One is built to carry it to more than 250 models; the other is built to refuse it and prove afterwards who said it could go.
Kong AI Gateway
Kong governs the traffic. Token Observe governs the action. If you already run Kong, the first one is nearly free and the second one is the only reason to read further.
Cloudflare AI Gateway
Cloudflare’s gateway decides what the payload contains. Token Observe decides whether the agent that sent it was allowed to.
Envoy AI Gateway
Both hold the request. One charges the token budget once the response completes; the other reserves the money before the request leaves your network.
MuleSoft AI Gateway
Both refuse the call inline. One refuses on behalf of an endpoint, the other on behalf of an agent that has an owner.
We already run LiteLLM. Do we have to replace it?
No, and replacing it is not the recommendation. The usual arrangement is to register your LiteLLM proxy as an OpenAI-compatible upstream and route only the agents that take consequential actions through Token Observe first, so LiteLLM keeps the provider keys, the breadth of coverage, the load balancing and the retries, and the policy verdict happens before that hop. The cost of the arrangement is a second failure domain and a second hop of latency, which is a reason to route selectively rather than universally.
Is LiteLLM free?
Partly, and the split matters to this comparison. Their pricing panel lists a $0 “Free forever” tier carrying 140+ provider integrations, virtual keys, budgets and teams, load balancing with RPM and TPM limits, guardrails and the logging integrations, and the repository LICENSE places everything outside the enterprise/ directory under MIT. Their enterprise page and enterprise documentation list SSO and SCIM, OIDC and JWT auth, RBAC by key, team and org, audit logs with retention policies, secret managers, key rotation and 24/7 support under Enterprise, deployed self-hosted with a licence key and priced on usage rather than published — though their enterprise documentation also says “SSO is free for up to 5 users. Beyond that, an enterprise license is required.” If your requirement is audit, access separation or single sign-on beyond five people, budget for a quote.
Does LiteLLM block requests inline, or only observe them?
It blocks. Their guardrails documentation describes four modes — pre_call before the call on the input, during_call in parallel with the call on the input, post_call after the call on input and output, and logging_only which masks in logs without stopping anything — and a violation is answered as “Violated guardrail policy”. Budgets stop requests too: their site says that at the cap, requests stop. The difference this page argues is not whether LiteLLM can refuse, but what a refusal is decided from and what it leaves behind — spend read from a database written asynchronously after previous responses, versus a projected and reserved figure taken inside one transaction before egress, and a decision appended to a hash chain rather than a row in a log table.
What does LiteLLM’s audit log actually record?
Their audit-log documentation describes keys, teams, users and models across create, update, delete and regenerate, with a record of who performed the action; it is on by default under an enterprise licence and otherwise enabled with store_audit_logs in config.yaml, and rows can be exported to an external backend such as S3, batched and uploaded asynchronously. Their enterprise page uses the phrase “Audit logs on every request”, which reads differently, so it is worth asking the vendor which is current for the release you would deploy. Per-request content in LiteLLM goes through the logging callbacks instead, into Langfuse, OpenTelemetry, Datadog, GCS, S3 and similar destinations, with litellm.turn_off_message_logging available to keep messages and responses out of them while still tracking spend.
Can Token Observe do what LiteLLM does?
For a much narrower set of providers, yes — six first-class upstreams with typed failover, per-agent budgets and rate limits, key custody and a cost ledger — and the product’s own material treats all of that as table stakes rather than as the reason to buy. What it does not have is LiteLLM’s reach, its operational maturity or its support commitments: 140+ providers against six, a Postgres-backed proxy with a documented support window against a single-writer SQLite process on one host, and published 24/7 response targets against no vendor-operated uptime commitment at all. If connectivity is the requirement, that comparison decides it.
Prefer to ask a person? Write to us →
Tell us which one you are already running.
If LiteLLM is already in your stack, the useful question is not which to buy but what each is for, and where the seam between them sits. Say what you have and you will get a straight answer — including when the answer is that you do not need a second thing.
no form · no qualification step · no sales desk · the other three ways in