What is exponential backoff?
A busy API needs breathing room, not a thousand clients retrying at the same second.
Exponential backoff
backoff with jitter
Exponential backoff is a retry strategy where a client waits longer after each failed attempt, usually doubling the delay every time. A random offset, called jitter, is added to that delay so many clients failing at once don't retry in the same instant.
A simple retry loop waits a fixed second and tries again. That works for one client. It fails for a thousand, because they all picked the same second.
Exponential backoff grows the wait: one second, then two, then four, then eight. Jitter breaks the pattern further by adding a small random amount to each wait, so no two clients land on the exact same retry moment.
Without jitter, retries cause the outage they're trying to survive
A payment API has a rough thirty seconds and starts returning errors. Every app that called it in that window now retries. If they all wait the same fixed delay, they all come back at once, and the API sees the exact same spike again.
This is called a thundering herd. The service was recovering, then a thousand identical retries arrive in the same second and knock it back down. Each retry wave gets weaker but the pattern repeats until something breaks the sync.
Growing the delay spreads the herd out over time. Jitter spreads it further by scattering clients that would otherwise land on the same second. AWS's architecture blog simulated 100 contending clients and found jitter cut the total call count by more than half compared with plain exponential backoff.
- 01A doubling delay with a cap (30 or 60 seconds is common) stops the wait from growing forever on a service that's down for good.
- 02Jitter is usually a random value between zero and the calculated delay, not a fixed offset added on top.
- 03Most SDKs from AWS, Stripe, and Google Cloud already retry with backoff built in; check before writing your own loop.
- RequestCall goes out, server is overloaded.
- Fail503 or timeout comes back.
- Wait1s + jitter, then retry.
- Fail againStill overloaded.
- Wait longer4s + jitter, then retry.
- SucceedServer had time to recover.
The delay doubles each time. Jitter means your client's wait and another client's wait land a little apart, not on the same tick.
Common questions
01How is exponential backoff different from a fixed retry delay?
A fixed delay waits the same amount of time before every retry, so many clients failing together tend to retry together too. Exponential backoff grows the wait after each failed attempt. That spreads retries out and gives an overloaded service time to recover instead of facing another synchronized wave.
02Why add jitter if the delay is already growing?
Growing the delay alone still leaves clients that failed at the same moment retrying at the same moment, just later. Jitter adds a small random offset to each wait so those clients spread across a window instead of landing on the same second. In AWS's own simulation of 100 contending clients, adding jitter cut the total call count by more than half.
03Should every failed request be retried with backoff?
No. A 400 Bad Request means the request itself was wrong, and retrying it fails the same way every time. Backoff belongs on errors that might clear on their own: timeouts, 429 Too Many Requests, and 5xx server errors. A request that isn't safe to repeat also needs an idempotency key alongside the delay.
04What's a reasonable cap on the retry delay?
Most systems cap the wait somewhere between 30 and 60 seconds and stop retrying after a fixed number of attempts, often five or six. A doubling delay with no cap quickly grows past the point of being useful. And with no limit on attempts, a client can retry a dead service forever.
