--- title: "Glossary: Typed failover" canonical_url: "https://tokenobserve.com/glossary/typed-failover" published: 2026-09-02 last_updated: 2026-10-21 --- # Glossary: Typed failover Typed failover is a fallback policy that decides whether to try the next provider from the class of the failure rather than from a retry count: transient classes (a timeout, a rate limit, a server error) move to the next candidate, while failures that are properties of the request or the credential stop where they are. It exists because some failures are identical at every vendor, and because one of them is a refusal that a second attempt would convert into an apparent success. ## Also called - typed fallback - classified failover - provider failover ## In practice A retry counter cannot tell the difference between a provider being down and a provider saying no. Both arrive as a non-2xx response. That means the fallback chain built to absorb an outage is also the mechanism that will replay a declined request at a vendor with a different safety configuration, take the second answer, and record a success — with the objection nowhere in the record. Nothing was bypassed on purpose; the counter never knew what it was counting. Classifying the failure first, and letting the class decide, is the smallest change that closes it. Four classes should not fail over, each for its own reason. A content-policy refusal is a governance signal: one vendor’s safety system declined, and sending the same payload onward is a second attempt at the same action. An authentication failure means the credential is wrong, revoked or scoped, so it fails identically everywhere that credential is used and failing over hides a broken key behind a more expensive provider until the invoice arrives. An over-long context is a property of the payload rather than the provider, and fallbacks usually have similar or smaller windows, so trying one pays a full input-token charge to receive the same error. A malformed request is malformed at every vendor. Note that the first of those is the only one that might have succeeded elsewhere, which is why it is the one not tried. Three classes should fail over: timeout, rate limit and server error. These are transient and provider-specific, and they are the outages a chain exists for. One ambiguity is worth naming rather than burying: a timeout cannot prove the upstream did not execute and bill the request, so failing one over risks paying two vendors for one call. That is an acceptable trade for ordinary traffic and not for a call under a hard money ceiling, which is why a strictly budgeted request is often capped at a single potentially billable attempt across the whole chain — no retry, no failover — and the ceiling is paid for in availability. Make the classes a closed set and the decision exhaustive. A switch over a closed union with no default branch means that adding an eighth class fails to compile until somebody has decided its failover behaviour, which is a language-level guard rather than a review convention that erodes. Classification itself should read the status first and the body only for what the status leaves undecided: 408 is a timeout, 429 is a rate limit, any 5xx is a server error, 401 and 403 are authentication, and only then is the message text consulted to tell a context-length complaint from a content refusal from an ordinary bad request. One case needs explicit handling: a provider that returns a blocked prompt as a successful HTTP response carrying a block reason and no content must be mapped to the refusal class deliberately, or it is indistinguishable from a success. The limits are the reason to trust the design rather than a reason to doubt it. Classification is a lossy mapping from heterogeneous vendor error shapes onto a handful of classes and it will get cases wrong — a content refusal returned as a bare 400 looks exactly like a malformed request, and although both correctly decline to fail over, the reason recorded in the record will be the wrong one. A provider that returns a 5xx for a refusal will be failed over, reproducing the very laundering the design prevents, and no downstream component can fix upstream error hygiene. An unclassifiable exception treated as a server error is failover-eligible, which is the availability-safe reading and the evidence-unsafe one. And availability is traded away: some requests that a counting gateway would have completed on a second provider now fail, and that will be reported as a bug by the people whose requests they are. ## Example: The refusal that would have been laundered An agent submits a payload that the primary vendor’s safety system declines, returning a typed content-policy error. Under a retry count, the chain sends the identical payload to the second vendor, which answers; the caller gets a 200, the record shows one successful completion, and no artefact anywhere says that a vendor objected. Under typed failover the request fails with a typed error naming the class, the chain is not tried, and the refusal is what the evidence holds. The cost is real: if the first vendor’s refusal was a false positive, the caller is now blocked from an answer the second vendor would have given, and someone has to decide whether to re-word the request or route that traffic elsewhere deliberately. ## Routinely confused with Retry with backoff: A retry stays with the same provider and the same credential for a bounded number of attempts; failover moves to a different provider entirely. The two questions are separate in principle — a failure could be worth retrying but not worth moving — though in practice the transient set is the same set, and a failure not worth retrying here is not worth trying there either. Circuit breaker: A breaker decides whether a candidate is called at all, from its recent failure history, before any request is sent. Typed failover decides where the request goes after an attempt has failed, from the class of that single failure. They compose: an open breaker skips the candidate; a non-failover class stops the chain even though candidates remain.