LLM API error · 529 / 503
Overloaded / service unavailable (529, 503)
The provider is at capacity. Why this is not your fault, why it still needs handling, and how to make an application survive it.
The message you are seeing
overloaded_error: Overloaded
The server had an error while processing your request. Sorry about that!
What it means
The provider is temporarily unable to serve your request because of load on their side. Nothing about your request is wrong, and the same request will usually succeed shortly.
It is worth distinguishing from a rate limit: a 429 means you sent too much, a 529 or 503 means they cannot take it right now. Both need retrying, and only one is within your control.
Why it happens
Most likely cause first.
- 1
Provider capacity, usually at peak hours
Demand on frontier models is spiky and correlated across customers. The most capable and newest models are the most affected.
- 2
A newly released model
Launch capacity is frequently tight for the first weeks. Building a production dependency on a model released last week is a decision with an availability cost.
How to fix it
Retry with exponential backoff and jitter
These usually clear in seconds. Retry three to five times with doubling waits and randomness before surfacing a failure.
Fall back to another model
A cheaper or older model that answers is better than a flagship that does not. Build the fallback path before you need it, since capacity events do not give notice.
LLM Model FinderSet a budget and a context requirement, get the shortlist. Sorted by your own input-to-output ratio.Queue rather than fail, where you can
If nobody is waiting on the response, a queue with a retry converts an outage into a delay. Much of what looks interactive is not.
Batch API Savings CalculatorThe 50% discount most teams never claim, priced at your volume.
Stopping it happening again
- Always have a second model configured and tested, not just documented as an option.
- Treat provider availability as a dependency in your own SLA rather than as an assumption.
- Move anything not user-facing to batch processing, which is both cheaper and less exposed to interactive capacity.
Tools that help
- LLM Model FinderSet a budget and a context requirement, get the shortlist. Sorted by your own input-to-output ratio.
- Model Routing Savings CalculatorTwo-tier routing with escalation cost, and the break-even escalation rate.
- Batch API Savings CalculatorThe 50% discount most teams never claim, priced at your volume.
Frequently asked questions
- Is this counted against my rate limit?
- A failed request generally does not consume quota, but the retries do count as requests. Aggressive retrying against an overloaded service can produce a rate limit error on top of the capacity one.
- How long do these usually last?
- Typically seconds to minutes. If it persists for longer, check the provider status page — at that point it is an incident rather than transient load.