TokenPad

LLM API error · 429

Rate limit exceeded (429)

A 429 means you exceeded tokens per minute or requests per minute. How to work out which, how to handle it correctly, and how to raise your effective ceiling.

The message you are seeing

OpenAI
Rate limit reached for gpt-5 in organization org-xxxx on tokens per min (TPM): Limit 30000, Used 28500, Requested 3000.
Anthropic
rate_limit_error: Number of request tokens has exceeded your per-minute rate limit.

What it means

You are sending more than your quota allows in a rolling window. Providers enforce at least two limits at once — tokens per minute and requests per minute — and you are bound by whichever you hit first.

This is unrelated to the context window, which limits the size of one request. A 429 is about aggregate throughput, and it is usually the tokens-per-minute quota that binds for anything with substantial prompts.

Why it happens

Most likely cause first.

  1. 1

    Peak traffic, not average traffic

    The most common cause is planning against a monthly average divided by minutes. Real traffic is bursty — a batch job, a marketing send or one enterprise customer running an import produces a minute that is twenty times the mean.

  2. 2

    max_tokens counted against quota

    Several providers count your requested ceiling rather than what was generated. Setting max_tokens to the model maximum "to be safe" can halve your effective throughput while costing nothing on the bill.

  3. 3

    Naive retries amplifying the problem

    An immediate retry from every concurrent worker turns a brief overage into a sustained outage. Without backoff, the retries themselves keep you over the limit.

  4. 4

    A new account on a low tier

    Provider quotas rise with spend and account age. A new key has limits far below what the same account will have in three months.

How to fix it

  1. Work out which limit you hit

    The error usually names it. Tokens per minute binds for large prompts; requests per minute binds for high-volume small ones. The fix differs completely.

    API Rate Limit CalculatorConvert traffic into TPM and RPM, and find the ceiling before production does.

  2. Implement exponential backoff with jitter

    Not a fixed retry delay — every worker would retry in lockstep. Double the wait each attempt and add randomness so the retries spread out.

  3. Lower max_tokens to something realistic

    Derive it from the 95th percentile of your observed answer lengths with a margin. If your provider counts the ceiling, this alone can double your throughput.

    max_tokens PlannerA ceiling derived from your p95, checked against window and quota.

  4. Reduce tokens per request

    Every token you remove raises your TPM ceiling proportionally. Payload minification and a shorter system prompt convert directly into throughput here, not just savings.

    Prompt Token OptimizerCuts whitespace, minifies JSON, collapses blank lines. Shows tokens saved and the annual value.

  5. Move non-urgent work to a batch endpoint

    Batch processing usually sits outside the interactive quota entirely, and comes with a discount of around 50%. Anything not blocking a user belongs there.

    Batch API Savings CalculatorThe 50% discount most teams never claim, priced at your volume.

Stopping it happening again

  • Plan against your peak minute, not your monthly average. If you have no production data, assume peak is five to ten times the mean.
  • Spread load across models where you can — quotas are typically per model, so routing simple work elsewhere frees quota on the expensive one.
  • Track your headroom as a metric, not just your errors. Hitting 80% of quota is the signal to act; hitting 100% is the incident.

Tools that help

Frequently asked questions

What is the difference between a rate limit and a quota error?
A rate limit is temporary — you sent too much too fast and can retry shortly. A quota error means you have run out of credit or hit a hard billing cap, and retrying will not help until you resolve the billing side.
How long should I wait before retrying?
Start around one second, double each attempt, and add random jitter of up to a few hundred milliseconds. Cap the total at something your users will tolerate and fail cleanly after that rather than retrying forever.
Do output tokens count towards the rate limit?
On most providers yes — the tokens-per-minute quota covers input and output together. Some count your requested max_tokens rather than what was actually generated, which is worth checking for your provider.