EN ▾
Get API key

LLM API Error 429: Causes and How to Fix It

An llm api error 429 means the server is refusing a request for now, not for good. On this API there are three distinct causes, and each needs a slightly different fix.

Updated

Packets of light queuing at a gate with eight lanes, illustrating API rate limits

Key points

The three kinds of 429

HTTP 429 is the standard "too many requests" status. Here it covers three situations, and the JSON error body tells you which one you are in.

Error codeWhat it meansTypical fix
rate_limitedToo many requests per minuteSlow down, spread requests over the minute
concurrencyToo many requests running at the same timeLower parallelism, wait for slots to free
Overload, with Retry-After: 5Temporary load on the service, not your faultWait the number of seconds in the header, then retry

Read the code first, because a fix for one cause does nothing for another. Halving your requests per minute will not help if the real problem is 20 workers firing at once.

Know your limits

The limits depend on the key type:

  • Full key: 8 concurrent requests and 300 requests per minute.
  • Trial key: 2 concurrent requests and 60 requests per minute.

Every response carries X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency, so your client can read the ceiling instead of hard-coding it. The trial key is the usual culprit for new users: a script that runs fine at 1 request at a time starts failing when you add a thread pool of 4. The trial is meant for testing, and a full key comes with any top-up from $20 (see the pricing page for amounts and bonuses).

The numbers above are per key. If one key is shared by a web app, a cron job and a teammate's notebook, they all draw on the same pool, and the 429s will look random until you add the three workloads together.

Why streaming makes concurrency errors more likely

A normal request holds a concurrency slot for a second or two. A streamed request (stream: true) holds it until the last token arrives and the stream closes. A long answer can keep a slot busy for tens of seconds, so 8 slots disappear quickly in a chat app with several active users.

Two mistakes cause most of the trouble here. The first is abandoning a stream without closing it: if the client stops reading but leaves the connection open, the slot stays occupied until the connection ends. Always close the response in a finally block, or use the SDK's context manager. The second is retrying a stream by opening a second one while the first is still alive, which doubles the slot use exactly when you are short of slots.

If your app needs more simultaneous long streams than your key allows, queue new chats on your side and show a short waiting state. That is better than letting users see raw errors.

Python retry code with backoff

The pattern is simple: catch the 429, read the code, wait, try again with a growing delay and some randomness, and give up after a few attempts. This version uses the official openai package against the service's base URL:

import random
import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    base_url="https://api.onekeyllmapi.com/v1",
    api_key="YOUR_KEY",
    max_retries=0,  # we handle retries ourselves
)

def chat(messages, attempts=5):
    for i in range(attempts):
        try:
            r = client.chat.completions.create(
                model="uncensored",
                messages=messages,
            )
            return r.choices[0].message.content
        except RateLimitError as e:
            headers = e.response.headers
            wait = float(headers.get("Retry-After", 0))
            if not wait:
                wait = min(30, 2 ** i)  # 1, 2, 4, 8, 16 seconds
            wait += random.uniform(0, 1)  # jitter
            time.sleep(wait)
    raise RuntimeError("still rate limited after retries")

print(chat([{"role": "user", "content": "Reply with the word ok."}]))

When the header is present, use it as the minimum wait. When it is missing, the exponential delay works for rate_limited because the per-minute window moves on. The jitter keeps many workers from retrying at the same instant.

Cap parallel calls so you rarely see a 429

Retries treat the symptom. A semaphore prevents most of it by never sending more than your slot count. Set it a little under the limit so a retry or a stray request still has room:

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://api.onekeyllmapi.com/v1",
    api_key="YOUR_KEY",
)
sem = asyncio.Semaphore(6)  # full key allows 8; use 1 on a trial key

async def ask(prompt):
    async with sem:
        r = await client.chat.completions.create(
            model="uncensored",
            messages=[{"role": "user", "content": prompt}],
        )
        return r.choices[0].message.content

async def main():
    prompts = [f"Give one fact about the number {n}." for n in range(50)]
    results = await asyncio.gather(*(ask(p) for p in prompts))
    print(len(results))

asyncio.run(main())

For the per-minute limit, 300 requests a minute is 5 per second on average. If you batch thousands of short jobs, add a small sleep or a token bucket in front of the semaphore. Because the SDK also retries on its own by default, set max_retries explicitly so your own logic and the SDK's do not stack and multiply the waits.

Reproduce and inspect a 429 with curl

Before you change code, look at the raw response. Add -i to curl so the headers print, and fire a few requests in parallel to provoke the error on purpose:

for i in 1 2 3 4; do
  curl -i -s https://api.onekeyllmapi.com/v1/chat/completions     -H "Authorization: Bearer $API_KEY"     -H "Content-Type: application/json"     -d '{"model":"uncensored","messages":[{"role":"user","content":"Count to 200 slowly."}]}' &
done
wait

On a trial key, which allows 2 requests at once, the third and fourth calls should come back with status 429 and an error code of concurrency. On a full key you would need more than 8 at once. Look at three things in each reply: the status line, the Retry-After header if there is one, and the error.code field in the JSON body. That small experiment tells you more than an hour of guessing, and it gives you a reproducible case to show support if something looks wrong.

Design for it, not around it

Treat a 429 as a normal event, like a timeout. Put model calls behind one function, log every retry with its reason, and expose a counter so you notice when retries climb. A rising retry rate is an early sign that your traffic has outgrown a single key's limits, long before users complain. For chat products, a small in-memory queue per user keeps one busy account from taking all 8 slots from everyone else.

A short debugging checklist

  1. Log the full error body and the X-Request-Id header for each 429. The code field separates rate_limited from concurrency.
  2. Check if your key is a trial key. If yes, expect 2 concurrent requests and 60 per minute.
  3. Count every program that uses the key, not only the one that failed.
  4. Look for unclosed streams, and for loops that retry instantly with no delay. An instant retry loop turns one 429 into a hundred.
  5. Confirm the balance with X-Balance-USD or GET /v1/balance. An empty balance is a different problem from a 429, but checking takes seconds.

Failed requests are not charged, so retrying does not cost you tokens for the attempts that were rejected. For the full list of status codes and headers, see the documentation. If you need a key with higher limits than a trial, get an API key and top up.

Questions and answers

Is a 429 my fault or the provider's?

Both are possible. The codes rate_limited and concurrency mean your key went over its limit. A 429 with a Retry-After header can also come from temporary overload, and waiting the stated time then retrying is the right response.

How long should I wait before retrying?

If the response has a Retry-After header, wait at least that many seconds. Otherwise start at one second and double the delay each time, with a little random jitter.

Do rejected requests cost money?

No. Failed or refused requests are not charged.

Can I raise my limits?

A full key allows 8 concurrent requests and 300 per minute, against 2 and 60 for a trial key. Beyond that, spread work across a queue on your side.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.