A batch script starts fine, then every call returns 429 Too Many Requests /
rate_limit_error. A quick "fix" — wrapping the call in while True: retry —
makes the failures continuous. After a pause the script works again for a minute,
then collapses into 429s again. The key is valid; single manual calls succeed.
The first real batch job: classify or summarise a few thousand rows in a tight
for loop, as fast as the loop can spin. Every API account carries per-minute
quotas — requests per minute and tokens per minute, sized by tier. A loop with no
pauses burns the whole minute's quota in seconds. Common in nightly jobs over
support tickets, shipment records, or KYC rows. Linked module: 3.2.
A 429 is not a broken key (401) and not a struggling server (5xx) — it is the
provider saying "you are over your per-minute allowance, slow down." The naive
instant retry is petrol on the fire: each retry is itself a request, counted
against the same quota, so the storm feeds itself. Confirm the mechanism: read
the error body and the retry-after header if present, and plot your request
timestamps — you will see a burst, a wall of 429s, a quota reset, another burst.
The same shape appears locally: hammer Ollama with parallel calls and responses
queue and slow, because every server has a rate it can sustain.
Three changes turn the job into a clean run. (1) Backoff: on 429 or 5xx, wait
before retrying, doubling the wait each attempt (1s, 2s, 4s) with a little random
jitter; honour retry-after when the server sends it. (2) Pacing: add a small
fixed sleep between calls so the steady rate sits under your quota — for a batch
job, finishing in forty minutes instead of "fast but never" is a win. (3)
Checkpointing: write results as you go and resume from the last completed row, so
a failure at row 1,900 never re-buys the first 1,899 calls. For genuinely large
offline volumes, move to the provider's batch endpoint and collect results later.
Know your tier's limits before you write the loop, and do the arithmetic: rows × calls per row ÷ quota per minute = minimum runtime. Write the backoff wrapper once and reuse it in every project. Log every status code, and treat a rising 429 ratio as an alarm, not noise. Never blanket-retry 4xx errors other than 429 — a 400 or 404 repeated is the same mistake at higher volume.
Q: "Your overnight batch died in a wall of 429s. What happened, and how do you
redesign it?" Strong answer: explain per-minute quotas, why instant retries
amplify the failure, then name backoff with jitter, retry-after, pacing under
the quota, and checkpoint-resume. Bonus signal: distinguishing 429 from 401 and
5xx by whose move it is, and knowing that batch endpoints exist for exactly this
workload.