Token Observe for Azure OpenAI
Your agents name a model; the gateway resolves the deployment behind it.
On this page
What moves in a Azure OpenAI client
# On the agent — the plain OpenAI client, not the Azure one
OPENAI_BASE_URL="https://gateway.example.com/v1"
OPENAI_API_KEY="acp_agent_…"
# On the gateway. The provider row names the key; two more derive from it.
AZURE_OPENAI_API_KEY="…"
AZURE_OPENAI_DEPLOYMENTS="gpt-5.6-sol=sol-prod, gpt-5.6-luna=luna-eu"
AZURE_OPENAI_API_VERSION="2024-10-21" # optional; this is the pinned default
# Bare AZURE_ is not an allowed credential prefix. AZURE_OPENAI_ is.Where that traffic lands
- POST /v1/chat/completions
- The ingress. Your agent speaks the plain OpenAI dialect and names a model id; the deployment is resolved on the way out. Streaming, tools, response formats and the rest behave exactly as they do on a direct OpenAI row, because both are built from the same client.
- GET /v1/models
- Registry-filtered to the models the calling agent’s roles permit. Where a deployment map is configured, that map is the catalogue for this provider: the model ids the operator wrote are what is listed, because listing Azure’s deployment names would advertise ids the resolver then refuses.
- POST /v1/embeddings
- Not available on Azure rows. The embeddings adapter speaks the OpenAI-compatible contract for openai and openrouter rows only, and Azure fails closed before credential resolution rather than being sent a request in a shape it would not answer the same way.
- {baseUrl}/openai/deployments/{deployment}/chat/completions
- The egress, with the api-version as a query parameter. The deployment name is operator-supplied and is percent-encoded rather than interpolated, because a raw slash would steer the request at a different Azure operation with the gateway’s credential attached.
What is true of Azure OpenAI and not of the others
Every provider in this list behaves differently somewhere that matters, and those differences are the reason a single general integration page is not enough.
- The URL names a deployment, not a model
- A deployment is an operator’s private name: one deployment may serve a model whose id looks nothing like it, and two deployments of the same model routinely have different quotas and regions. Assuming the two names match is the classic Azure integration bug, so the mapping is explicit and configurable, with identity used only where no map is configured at all.
- A malformed map entry throws rather than being skipped
- A dropped pair would route that model to a deployment name that does not exist and surface as an opaque Azure 404 at request time, long after the operator could connect it to the typo. The parser refuses the whole value instead, naming the entry. Entries are capped at 200, which is far more than a resource holds.
- An unmapped model is refused, and the map is not enumerated back
- A request for a model with no mapping fails with the number of configured mappings and no list of them. The operator reads the names from their own configuration; the caller does not learn what else the resource serves.
- api-key, not a bearer token
- Azure authenticates with its own header. Sending a bearer token as well would put the credential on the wire twice and invites the endpoint to validate it as a Microsoft Entra token and fail, so the credential header is replaced rather than added to.
- max_tokens, not max_completion_tokens
- Azure’s deployment surface documents max_tokens, and the newer name is accepted only on the newest api-versions — it returns a 400 on the pinned GA one. The shared client’s field name is overridden for this provider kind rather than the version being moved forward.
- The api-version is pinned, not tracked
- A widely supported GA contract, deliberately not the newest. Azure changes response shapes between versions, and a gateway that silently followed the latest would change its metering behaviour without a deploy. A resource that needs a later contract sets the variable, which is a visible operator decision rather than a silent upgrade.
- Cached tokens are inclusive
- Azure follows OpenAI’s convention, so cached prompt tokens sit inside the prompt total and are subtracted into their own bucket before pricing. Anthropic is the exception among the providers here, reporting cache tokens beside the input total rather than inside it.
The deployment problem, which is the whole Azure integration
Every other provider on this site takes a model id. Azure takes a deployment name, which is an operator’s private label for a model they have provisioned in a particular resource, in a particular region, with a particular quota. The name is arbitrary: a deployment called for production in Europe may serve any model at all, and two deployments of the same model in the same subscription routinely differ in both quota and region. Assuming the deployment name equals the model id is the classic Azure integration bug, and it is a bug precisely because it works right up until somebody names a deployment sensibly.
So the mapping is explicit and lives on the provider row as a model-to-deployment map. Identity is used only when no map is configured at all, which is the convention Azure’s own quickstarts produce and therefore the state most first installs are in. Once a map exists it is authoritative in both directions: a request for an unmapped model is refused with the number of configured mappings and not a list of them, and GET /v1/models answers with the model ids the operator wrote rather than with Azure’s deployment names — because listing the deployment names would advertise ids the resolver then refuses, which is worse than listing nothing.
The parser is strict on purpose. A malformed entry throws rather than being skipped, because a silently dropped pair routes that model to a deployment that does not exist, and the failure then arrives as an opaque Azure 404 at request time, long after anybody could connect it to a typo in a configuration value. The value is capped at 200 entries, which bounds a pathological value while sitting far above what a real resource holds.
One more detail is a security property rather than a convenience: the deployment name is operator-supplied, so it is percent-encoded into the URL rather than interpolated. A raw slash in a deployment name would otherwise steer the request at a different Azure operation with the gateway’s credential attached, which is the same class of problem the base-URL allowlist exists to prevent at a coarser grain.
What changes on the agent, and the client you must not use
On the agent side this is the ordinary OpenAI change: point OPENAI_BASE_URL at the gateway with the trailing /v1, and put an agent key where the key used to be. The agent names a model id, not a deployment, which is a genuine simplification — the deployment name is an operator concern and stops being distributed across every application that calls the model.
There is one client-side trap, and it is specific to Azure. The Azure-aware client classes in the OpenAI SDKs build the deployment URL themselves, appending the /openai/deployments/… path and the api-version query parameter. The gateway’s ingress is /v1/chat/completions, so a client doing that will not find a route. Switch to the plain OpenAI client pointed at the gateway. That is the whole change, and it removes rather than adds configuration: the endpoint, the deployment name and the api-version all stop being application concerns.
A base-URL change is the normal way in for the supported dialects, and when it does not take effect the first thing to check is your own SDK and its version. This one is worth checking twice, because an environment already configured for Azure often carries several variables at once — an endpoint, a deployment, an api-version and a key — and a client that reads any of them may keep constructing an Azure-shaped request while the base URL points at the gateway. Clear the ones the plain client does not need.
What arrives at the upstream is then indistinguishable in shape from a direct OpenAI call, because Azure speaks that dialect and the adapter is composed from the same client. The three deltas are the URL, the credential header and the output-cap field name; the body translation, the streaming discipline and the usage normalisation are shared, which is the reason those three are the only Azure-specific code paths worth reasoning about.
- Credential prefix
- AZURE_OPENAI_ is an allowed provider key prefix; bare AZURE_ is deliberately absent, because it would make a host’s ambient cloud credentials nameable from a provider row. The deployment map and the api-version derive their variable names from the credential name on the row rather than being separate unaudited fields.
- Host allowlist
- Azure is per-tenant, so it has no single hostname. The defaults are single-label wildcards over the customer resource domains, and a wildcard never spans a dot — so a lookalike domain ending in the same characters does not match.
- No inbound headers are forwarded
- Nothing an inbound client sends is meaningful to this upstream, and the outbound request carries the gateway’s credential, so the forwardable-header list is empty for Azure rows rather than inheriting OpenAI’s three.
The pinned api-version, and what it buys
Azure pins the request and response contract with an api-version query parameter, and the version shipped as the default is a widely supported GA contract chosen deliberately over the newest one. The reason is metering. Azure changes response shapes between versions, and usage is a response shape: a gateway that silently followed the latest version would change how it counts tokens — and therefore what it charges an agent’s budget — without anybody deploying anything.
A resource that needs a later contract sets the api-version variable, which makes the change a visible operator decision recorded in configuration rather than a silent upgrade. An empty value is refused outright when the client is built, because every Azure endpoint requires one and an empty string would fail every request with an error that names the request rather than the configuration.
The version pin is also why the output-cap field differs from the direct OpenAI route. Azure’s deployment surface documents max_tokens; max_completion_tokens is accepted only on the newest api-versions and returns a 400 on the pinned GA one. Rather than moving the pin forward to accommodate a field name, the field name is overridden for this provider kind, which keeps the metering-stability argument intact.
Accounting itself is OpenAI’s. Cached prompt tokens are reported inside the prompt total and are subtracted into their own bucket before pricing, producing the same four mutually exclusive buckets every provider produces here — uncached input, cache reads, cache writes and output. Anthropic is the exception among these providers, reporting cache tokens alongside the input total rather than inside it, and mixing the two conventions is the most common way a cost figure stops reconciling against an invoice. Prices are matched on provider kind as well as model, so an Azure-served model is priced against your Azure row rather than against a direct OpenAI row carrying the same model name.
What fails over, and the Azure cases that will not
Failover is decided by failure class rather than by a retry count. Timeout, rate_limited and server_error walk down a route rule’s fallback chain; context_too_long, content_policy, auth and invalid_request do not, because each fails identically at the next provider and failing over either wastes budget or hides the cause. Azure inherits the shared HTTP classifier: 408 is a timeout, 429 is a rate limit, 5xx is a server error, 401 and 403 are auth, a 400 naming a context length is an over-long context, a message matching the content-policy pattern is a policy refusal, and everything else is an invalid request.
Two Azure-specific failures land in the no-failover set by design and are worth recognising. An unmapped model is refused as an invalid request before egress, with the number of configured mappings and no list of them, because the caller does not need to learn what else the resource serves. And a deployment quota exhausted on one deployment is not, in general, a reason to fail over to a provider that would serve a different model — which is why quota separation between deployments belongs in your route rules as an explicit fallback rather than being inferred.
The circuit breaker sits above the chain: five consecutive failures opens it for thirty seconds, and it is keyed by provider id in a map that configuration refreshes never rebuild. That last property matters more on Azure than elsewhere, because Azure rows get edited — a new deployment, a changed api-version — and rebuilding a breaker on every edit would hand a flapping resource a clean slate each time.
Failures surface as ACP_UPSTREAM_TIMEOUT at 504, ACP_PROVIDER_UNAVAILABLE at 502 and ACP_INVALID_REQUEST at 400, with the typed code carried in an extension object so a strict SDK parser is not broken by an invented top-level field. Every governed outcome, refusals included, carries x-acp-trace-id.
The parts of the platform this uses
Model routing
Six upstreams behind one set of policies, and a fallback chain that will not launder a refusal.
Spend controls
Hard USD ceilings, per-minute rate limits and a kill switch, all decided before the request leaves your network.
Agent permissions
Deny by default, explicit deny wins, and delegation intersects — so an agent cannot borrow authority it was never granted.
Flight recorder
Every governed request in a timeline a compliance officer can read, and a search box that never writes SQL.
The same policies apply identically whichever provider serves the request, and that equivalence is enforced by a test over every provider kind rather than asserted.
See the request pathThe rest of the upstreams
Do my agents keep using the Azure OpenAI client?
No — switch them to the plain OpenAI client pointed at the gateway. The Azure-aware client classes build the deployment URL and the api-version query parameter themselves, and the gateway’s ingress is POST /v1/chat/completions, so those requests will not find a route. The change removes configuration rather than adding it: the endpoint, the deployment name and the api-version all stop being application concerns and become one provider row. Worth clearing the leftover Azure variables in the same edit, because a client that still reads them may keep constructing an Azure-shaped request while the base URL points at the gateway — and as always with a base-URL change, your own SDK and its version is the first thing to check when it does not take effect.
How does the gateway know which deployment to use?
From a model-to-deployment map on the provider row, written as model=deployment pairs. Where no map is configured at all, the requested model is used as the deployment name, which is the convention Azure’s own quickstarts produce. Where a map exists it is authoritative: an unmapped model is refused with the number of configured mappings and no list of them, and GET /v1/models answers with the model ids you wrote rather than with Azure’s deployment names, since listing deployment names would advertise ids the resolver then refuses. A malformed entry throws rather than being skipped, because a dropped pair would surface as an opaque Azure 404 at request time long after anyone could connect it to the typo.
Why is the api-version pinned to an older contract?
Because usage is part of the response shape, and Azure changes response shapes between versions. A gateway that silently followed the newest version would change how it counts tokens — and therefore what it charges against an agent’s budget — with nobody deploying anything. Pinning a widely supported GA contract makes any change to that behaviour a deliberate act: a resource that needs a later contract sets the api-version variable, which is a visible operator decision recorded in configuration. The pin is also why this provider sends max_tokens rather than max_completion_tokens, since the newer field name returns a 400 on the pinned contract.
Can one gateway front several Azure resources?
Yes — each is its own provider row with its own base URL, its own credential variable name, its own deployment map and its own priority, and route rules decide which models go where. That is also the sane way to express quota separation, because two deployments of the same model with different quotas are two deployments rather than one, and a fallback between them is a route-rule decision you make explicitly rather than something inferred from an error. Each row is bounded by the same two allowlists as every other integration: which credential variable names it may read, and which hosts it may send to. Bare AZURE_ is deliberately not an allowed credential prefix, since it would make a host’s ambient cloud credentials nameable from a provider row.
Are Azure’s cached tokens counted like OpenAI’s or like Anthropic’s?
Like OpenAI’s, because Azure serves the OpenAI dialect: cached prompt tokens are reported inside the prompt total and are subtracted into their own bucket before pricing. Anthropic is the exception among the providers here, reporting cache reads and cache writes alongside input tokens rather than inside them, and mixing the two conventions misprices cache-heavy agent traffic by between half and nine tenths. Both end up as the same four mutually exclusive buckets, which is why the normalisation lives in each adapter rather than in the ledger. Prices are matched on provider kind as well as model, so a model served through Azure is priced against your Azure row rather than against a direct OpenAI row with the same model name.
Name the SDK and the version.
A base-URL change is the normal case for the Azure OpenAI dialect, and whether your own client library and version behave that way is the first thing worth checking. Say which you use and you will get a straight answer.
no form · no qualification step · no sales desk · the other three ways in