Retry Pattern
Automatically retry transient failures using exponential backoff with jitter, bounded attempts, and idempotent operations to avoid duplicate side effects.
By Ratish Jain · Updated
Definition
The retry pattern automatically re-attempts an operation that failed for a *transient* reason — a brief network glitch, a timeout, or a temporarily overloaded service. Rather than surfacing the error immediately, the caller waits and tries again a bounded number of times.
Good retries use exponential backoff (increasing delays) plus jitter (randomized delay) and require the operation to be idempotent.
The Problem It Solves
Distributed systems fail transiently all the time: a packet drops, a leader election happens, a node briefly GCs. Failing the whole request on the first blip wastes a recoverable situation and degrades user experience.
Retries turn transient failures into successes transparently — but done naively they amplify outages by hammering an already-struggling service.
How It Works
- 1The operation fails; the caller checks whether the error is transient/retryable.
- 2If retryable and attempts remain, it waits
base * 2^attempt(exponential backoff). - 3It adds random jitter to the delay so many clients don't retry in sync.
- 4It re-attempts the operation, incrementing the attempt counter.
- 5On success it returns; on non-transient errors it fails immediately.
- 6After max attempts it gives up — surfacing the error or routing to a dead-letter queue.
Idempotency is mandatory
A retry may fire because the response was lost, not the request — the first call may have already succeeded. Without idempotency (via idempotency keys or safe operations), retries cause duplicate charges, double sends, or corrupted state.
Tradeoffs
- Transparently recovers from transient failures.
- Improves perceived reliability and success rate.
- Backoff + jitter avoids synchronized retry storms.
- Simple to add around most network calls.
- Can amplify load and prolong outages if unbounded.
- Requires idempotent operations to be safe.
- Adds latency to eventually-failing requests.
- Retrying non-transient errors is wasted effort.
Interview Questions
Without jitter, many clients that failed at the same moment retry at the same computed delays, creating synchronized spikes (thundering herd) that re-overload the recovering service. Random jitter spreads retries out, smoothing load.
Non-transient errors: 4xx client errors like 400/401/404, validation failures, and business-rule rejections. Retrying them can't succeed and only wastes resources. Retry timeouts, 429s (with backoff), 503s, and connection resets.
- →Retry only transient, retryable failures.
- →Use exponential backoff + jitter, bounded attempts.
- →Operations must be idempotent (idempotency keys).
- →Combine with circuit breakers to avoid amplifying outages.
- →Route exhausted retries to a dead-letter queue.
Related
Everything on Archprimer is connected — follow the graph.