Token Observe for Anthropic
The Messages API, unchanged, with a decision point in front of it and a record behind it.
On this page
What moves in a Anthropic client
ANTHROPIC_BASE_URL="https://gateway.example.com" # was https://api.anthropic.com
ANTHROPIC_API_KEY="acp_agent_…" # the agent key, not the Anthropic key
# Claude Code takes the credential under a different name
export ANTHROPIC_BASE_URL="https://gateway.example.com"
export ANTHROPIC_AUTH_TOKEN="acp_agent_…"
# Per project, the same pair in .claude/settings.json
{ "env": { "ANTHROPIC_BASE_URL": "https://gateway.example.com",
"ANTHROPIC_API_KEY": "acp_agent_…" } }Where that traffic lands
- POST /v1/messages
- The Anthropic dialect. max_tokens is required, system may be a string or a block array, tool use travels as tool_use and tool_result blocks, and the anthropic-version and anthropic-beta headers are passed through verbatim. The stream is named-event server-sent events — message_start, content_block_delta and the rest, through to message_stop.
- POST /v1/messages/count_tokens
- Token counting, and the pre-flight budget primitive. It is a governed endpoint rather than a passthrough: it authenticates the agent, asserts the agent is operable, and runs the deny-by-default role check on model invocation, because an endpoint that priced prompts and enumerated models past a frozen agent would be a hole in the kill switch.
- GET /v1/models
- Registry-filtered to what the calling agent’s roles permit. Anthropic’s SDK sends cursor parameters here, which are accepted rather than rejected — refusing them would break model discovery in the client for no gain.
What is true of Anthropic and not of the others
Every provider in this list behaves differently somewhere that matters, and those differences are the reason a single general integration page is not enough.
- Cache tokens are exclusive
- cache_read_input_tokens and cache_creation_input_tokens sit alongside input_tokens rather than inside it, so the buckets are read directly with no subtraction. OpenAI and Gemini report cached tokens inside the prompt total, so reading either vendor with the other’s assumption misprices cache-heavy traffic by between half and nine tenths.
- max_tokens is required and is defaulted
- The upstream refuses a request without it. A canonical request that omits it is given 4,096 here rather than a 400, and that figure is chosen to match what LiteLLM injects, so an OpenAI-shaped client that never sets a cap behaves the same through this gateway as through the proxies operators are migrating from.
- The system prompt is a top-level field
- It is not a message. Sending it as one is a 400 upstream, so the translation lifts it out. Where a client sends system as blocks, the joined text is kept for every existing reader and the per-block cache breakpoints are preserved separately, because the joined form cannot express them.
- cache_control breakpoints survive translation
- Anthropic caching is opt-in per block, so a lost breakpoint silently disables caching and bills the whole prefix at the uncached rate every turn. The marker is carried opaquely on every content variant, on tool definitions and on structured system blocks, and re-emitted on the way out. The system prefix is emitted as one block built from the governed text, never from the raw blocks, because the pipeline redacts the joined string.
- The version header is pinned, betas are forwarded
- anthropic-version is sent as 2023-06-01 and feature gating is left to anthropic-beta, which is forwarded verbatim because files, compaction, fallbacks and the connector all negotiate through it and break silently when it is dropped. It is the only inbound header that reaches the upstream.
- The typed code rides in an extension object
- Anthropic’s error envelope has no code field, so a governance refusal once reached the caller as a bare permission_error. The typed code — ACP_RBAC_DENIED, ACP_KILL_SWITCH_ENGAGED and the rest — now travels in an acp extension object on both dialects, rather than in an invented top-level field that would break strict SDK parsing.
- Streaming usage arrives in two places
- Input usage comes on message_start and final usage on message_delta, so both are needed to meter a stream. Anthropic types its stream errors, and the type is more reliable than the status: overloaded_error and api_error are server errors, rate_limit_error is a rate limit, authentication_error and permission_error are auth, timeout_error is a timeout.
The change, and the trailing slash that costs people an afternoon
Point the client at your gateway and give it an agent key where the Anthropic key used to be. The dialect is unchanged, the SDK is unchanged, and the response comes back in the shape the SDK expects with the governance information in headers rather than in the body. The agent key is the agent’s identity: it is minted against a registry record by an administrator, it is stored as a SHA-256 digest with a display prefix, and it will not authenticate against api.anthropic.com.
The detail that catches people is that this dialect takes no trailing /v1 on the base URL where the OpenAI dialect does, and the asymmetry is real rather than a documentation slip — the Anthropic SDK appends its own path. The product’s own troubleshooting names it as one of the top three causes of an invalid-key 401, which is a more useful sentence than any amount of prose about how easy the integration is. A base-URL change is the normal way in for the supported dialects, and when it does not take effect the first thing to check is your own SDK and its version rather than the gateway: libraries disagree about which variable wins and whether a constructor argument overrides the environment, and Claude Code reads ANTHROPIC_AUTH_TOKEN for the credential rather than ANTHROPIC_API_KEY.
There is one thing the gateway will not do, and it is a limit of the vendor’s terms rather than of the software. A subscription cannot be used as a credential. A signed-in client pointed at the gateway without an agent key is rejected and recorded on the shadow-AI radar as an unrecognised caller, because relaying a consumer subscription is prohibited by Anthropic’s terms. For subscription tools that cannot be pointed anywhere, the seat-policy route exists instead: a hook deployed through device management asks the policy engine for a decision before a tool call executes, and the developer’s own login is never seen, stored or relayed.
Roll the two variables out through device management rather than asking each developer to set them. An opt-in redirect is the single largest source of shadow-AI findings, for the ordinary reason that a control somebody has to remember to switch on is a control most people will not switch on. And tell the team first: once a coding client is given a gateway credential it stops using that developer’s own subscription, and the work is billed per token to the account behind the install.
Cache accounting, which on this provider is most of the bill
Anthropic reports cache reads and cache writes as their own fields beside input_tokens, so the three do not overlap and the adapter reads them straight into the mutually exclusive buckets the cost engine requires: uncached input, cache reads, cache writes and output. OpenAI and Gemini report cached tokens inside the prompt total and are normalised by subtraction instead. Both conventions produce the same four buckets, which is the point of normalising in the adapter rather than branching in the ledger.
The reason this is not a footnote is that the error is large and directional. Reading Anthropic’s numbers with the inclusive assumption undercounts the input side; reading an inclusive vendor with the exclusive assumption double-counts every cached token. The product’s own cost module states the range plainly — getting it wrong misprices cache-heavy agent traffic by between fifty and ninety per cent — and cache-heavy is the normal shape of an agent, because an agent re-sends its system prompt, its tool definitions and its conversation prefix on every turn.
The second half of the same problem is the breakpoint itself. Anthropic caching is opt-in per block, so a marker lost in translation does not degrade caching, it disables it. That was once a live defect here: the canonical request carried no field for a cache breakpoint, so the marker was dropped on the way in and could not reappear when the outbound body was rebuilt, and a fifty-turn session over a hundred-thousand-token prefix cost roughly fifteen dollars instead of one dollar eighty-five — with a zero in the cache-read bucket as the only visible signal. The fix carries the marker opaquely on every canonical content variant, on tool definitions and on structured system blocks, and re-emits it on the way out.
One subtlety in that fix is worth knowing if you audit the outbound body. Several breakpoints on the system prefix collapse into one, which keeps the marker where it earns its keep and leaves the cached prefix byte-stable across turns, and the single emitted block is built from the governed text rather than from the caller’s raw blocks — because the pipeline redacts the joined string, and building from the blocks would have routed an unredacted system prompt to the provider.
- Cache pricing defaults
- Where a price row states no explicit cached-read or cache-write rate, both bill at the full input rate. That is conservative by construction: the ledger never under-bills relative to the invoice, and a missing rate reads slightly high rather than silently free.
- The count_tokens endpoint
- POST /v1/messages/count_tokens is the pre-flight budget primitive, which is why it is governed rather than proxied. The crude four-characters-per-token estimate elsewhere in the product exists for pre-flight checks only and is never used for billing.
- Metering runs on every terminal path
- Including a stream the client disconnected from halfway through, and including a refused proposed tool call. The tokens were spent either way, and a partial trace is still evidence.
What fails over from Anthropic, and what stays refused
Failover is decided by the class of the failure rather than by a retry count, and Anthropic’s typed error objects make that classification more accurate here than on providers that answer with a bare status. A rate_limit_error is rate-limited, overloaded_error and api_error are server errors, authentication_error and permission_error are auth failures, and timeout_error is a timeout. Only the first three of the seven classes overall — timeout, rate_limited and server_error — walk down a route rule’s fallback chain.
The four that do not are context_too_long, content_policy, auth and invalid_request, and each has a specific reason. A content-policy refusal is a governance signal, and retrying it at another vendor is a second attempt at the same action with the objection hidden behind a recorded success. An auth failure means the gateway’s Anthropic key is wrong, revoked or scope-limited, which fails identically everywhere that credential is used. An over-long context is a property of the payload rather than the provider, and fallbacks usually have similar or smaller windows, so failing over pays a full input-token charge to receive the same error.
Provider-specific nuance is flattened by that classification, and the design says so rather than implying otherwise: an overloaded_error and a generic 529 both become server_error. So does the case that undermines the control — a provider returning 5xx for what is really a refusal will be failed over, producing exactly the laundering the classification prevents elsewhere. The control depends on upstream error hygiene nobody here controls, and that residual is published rather than assumed away.
Above the chain, one circuit breaker per provider opens after five consecutive failures and stays open for thirty seconds before allowing a probe. It survives configuration refreshes deliberately, because rebuilding a breaker whenever somebody edits a provider row gives a flapping upstream a clean slate on every edit, which is how a breaker stops working.
What this ingress will not do
Embeddings are the clearest boundary. POST /v1/embeddings is proxied with the same governance and metering as completions for openai and openrouter provider rows only, and Anthropic, Google, Azure and Bedrock routes fail closed before credential resolution rather than being sent a request in a dialect they do not speak. The refusal happens before any environment value is read, which is the honest place for it.
Media parts are refused on this dialect as they are on the others: an Anthropic image block is rejected for the same reason as a Chat image_url or a Gemini inlineData part, because there is not yet bounded media decoding and OCR under the data-loss and injection policies, and a caller-supplied MIME type is not proof that opaque bytes are safe.
The heuristic limits are worth stating in the same register as the claims. Personally identifiable information detection and prompt-injection detection are heuristic and have false negatives; an injection finding is one input to a policy rather than the control itself, and what actually bounds the damage a missed injection can do is the deny-by-default role check, the tool scoping and the approval branch downstream of it. Evidence exports are digest-sealed rather than signed, and the audit chain is tamper-evident rather than tamper-proof: a hash chain catches any alteration that does not also recompute every downstream digest, which is exactly the alteration somebody with write access to the database will not make.
Finally, the default that surprises people in the other direction: trace retention is unset by default, and unset means keep forever. That is deliberate — an upgrade that silently began deleting a customer’s evidence would be the worse failure — but it means a retention window is a decision you make rather than one you inherit, and a volume has to be sized against the window you set.
The parts of the platform this uses
Model routing
Six upstreams behind one set of policies, and a fallback chain that will not launder a refusal.
Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
Flight recorder
Every governed request in a timeline a compliance officer can read, and a search box that never writes SQL.
Endpoint seats
Policy enforced inside each vendor’s own administrator hook, decided offline against a signed bundle, because a hook that phones home fails open.
The same policies apply identically whichever provider serves the request, and that equivalence is enforced by a test over every provider kind rather than asserted.
See the request pathThe rest of the upstreams
Why does the Anthropic base URL have no /v1 when the OpenAI one does?
Because the two SDKs append different paths. The Anthropic client builds /v1/messages onto whatever base URL you give it, so adding /v1 yourself produces a doubled segment and a 404 that presents as an authentication problem; the OpenAI client expects the version segment to already be in the base URL. Token Observe accepts what each SDK actually sends rather than asking you to normalise them, which is why the two variables look inconsistent side by side. The product’s onboarding troubleshooting lists this asymmetry among the top three causes of an invalid-key 401 — and if it still does not work after you have checked it, the next thing to check is your own SDK and its version rather than the gateway.
Does prompt caching still work through the gateway?
Yes, and it is carried deliberately rather than incidentally. Anthropic caching is opt-in per block, so the cache_control marker is preserved on every canonical content variant, on tool definitions and on structured system blocks, then re-emitted when the outbound body is rebuilt. Several breakpoints on the system prefix collapse into one, which keeps the cached prefix byte-stable across turns. This was once a live defect — the marker was dropped on the way in, a long-context agent silently paid the full input rate on every turn, and a zero in the cache-read bucket was the only signal — which is why the behaviour is stated here rather than assumed.
What happens if my request omits max_tokens?
It is defaulted to 4,096 rather than refused. Anthropic requires the field upstream, and the OpenAI-shaped clients that get pointed at this dialect routinely leave it off, so refusing would turn a working migration into a wall of 400s. The figure matches what LiteLLM injects, which means an uncapped client behaves the same through this gateway as through the proxy it is being migrated from. If you care about the cap, set it: a default is a default, and a budget ceiling is a better control over spend than an output cap in any case.
Can Claude Code be governed this way?
Yes, when it is given a gateway credential: set ANTHROPIC_BASE_URL to your gateway and ANTHROPIC_AUTH_TOKEN to an agent key, either through device management or per project in .claude/settings.json. What cannot be done is relaying the developer’s own subscription — a signed-in client pointed at the gateway without an agent key is rejected and recorded on the shadow-AI radar as an unrecognised caller, because relaying a consumer subscription is prohibited by Anthropic’s terms. For that case the seat-policy route applies instead: a hook deployed through device management asks the policy engine for a decision before a tool call executes, and the subscription credential is never seen, stored or relayed. Tell the team before you switch either on, because a governed client bills per token to the install’s account rather than to the developer’s subscription.
How do governance refusals reach an Anthropic SDK?
As an error in the dialect the caller is speaking, with the typed code in an acp extension object. Anthropic’s envelope has no code field, so a refusal once arrived as a bare permission_error whether it was a role denial or an engaged kill switch; the typed code now travels in the extension rather than in an invented top-level field that would break strict SDK parsing. On a stream the refusal cannot be a status code at all, because the reply is hijacked before the first byte, so it arrives in band as a typed error frame carrying the same code a buffered refusal would have carried. Every governed outcome, refusals included, comes back with x-acp-trace-id.
Name the SDK and the version.
A base-URL change is the normal case for the Anthropic dialect, and whether your own client library and version behave that way is the first thing worth checking. Say which you use and you will get a straight answer.
no form · no qualification step · no sales desk · the other three ways in