How to Implement a Robust Webhook Retry Strategy (with Exponential Backoff) Webhooks have become the nervous system of the modern internet.

How to Implement a Robust Webhook Retry Strategy (with Exponential Backoff)

Webhooks have become the nervous system of the modern internet. From payment processors notifying your application of a successful transaction to CRM systems triggering marketing workflows, webhooks are the glue that holds microservices and third-party integrations together. They allow real-time, event-driven architecture to thrive, replacing the old, inefficient model of constant API polling.

But there is a dark side to webhooks: they are fundamentally unreliable. Because webhooks operate over the public internet and bridge entirely separate systems, they are subject to the chaos of distributed networks. Endpoints go down. Servers get overloaded. Networks experience transient blips. When you send a webhook, you are firing a payload into the void and hoping the receiving server is ready, willing, and able to catch it.

When a webhook fails — and it will fail — how your system responds determines whether your application stays consistent or quietly drifts out of sync. If a payment-success webhook is dropped, a user might not get access to the product they just paid for. If an inventory-update webhook fails, you might oversell a product you don't have.

This guide covers why webhooks fail, why naive retry logic makes things worse, how exponential backoff and jitter actually work (with the real formulas AWS uses in production), what major providers like Stripe and GitHub actually do today, and the architectural patterns — idempotency, dead-letter queues, circuit breakers — that turn "we'll retry it" into a system you can trust.

1. Why Webhooks Fail

Failures in a distributed system aren't anomalies — they're the expected state. Here are the main reasons webhooks fail to arrive or fail to be processed correctly.

Network blips and transient DNS errors

Packets get dropped. BGP routes change. DNS resolution hiccups. These blips often last milliseconds to a few seconds, but if your webhook's POST request is in flight during one, the connection times out or is refused.

Server restarts and deployments

Rolling restarts of Kubernetes pods or EC2 instances can momentarily reject incoming traffic during modern CI/CD-driven deployment cycles. If a webhook hits an endpoint exactly as a container is rotating, you'll likely see an abrupt connection termination.

Service unavailability (502, 503, 504)

502 Bad Gateway — the reverse proxy (NGINX, Cloudflare) can't reach the upstream application server.
503 Service Unavailable — the server is temporarily down or overloaded.
504 Gateway Timeout — the server took too long to respond, often because a downstream database is locked or slow.

Rate limiting (429 Too Many Requests)

If you're sending a high volume of webhooks, you can trip the receiver's rate limiter. A 429 is the receiver saying "slow down." Ignoring it just compounds the failure.

Client-side application errors (500)

Sometimes the webhook arrives fine but the receiver's code crashes while processing it — an unhandled exception, a database constraint violation, a bug in a recent deploy.

2. The Naive Approaches: Immediate and Linear Retries

Immediate retries: a self-inflicted DDoS

The simplest retry logic just fires the request again the instant it fails. If the failure was a millisecond network blip, that might work. But if the receiver returned a 503 because it's already overloaded, an immediate retry is the worst possible response — you're now hammering a struggling server with relentless traffic, which is functionally a denial-of-service attack against your own customer or partner.

Linear retries: the stubborn march

A step up is a fixed-interval retry — say, every 10 minutes, up to 5 times. This is safer than immediate retries but still inefficient in both directions:

Too slow for short outages. If the receiver's server restarted in 30 seconds, waiting a full 10 minutes to retry leaves data stale far longer than necessary.
Too wasteful for long outages. If the receiver's database is down for 6 hours, pinging every 10 minutes burns your outbound bandwidth, clogs your workers, and fills the receiver's error logs with noise.

3. Exponential Backoff: What It Is and How Real Providers Use It

Exponential backoff increases the wait time between retries exponentially rather than linearly, so a struggling server gets progressively more room to recover.

The standard formula:

Code example

Wait Time = Base Interval × (Multiplier ^ Attempt Number)

With a 10-second base and a multiplier of 2:

Attempt	Calculation	Wait
Retry 1	10 × 2⁰	10 sec
Retry 2	10 × 2¹	20 sec
Retry 3	10 × 2²	40 sec
Retry 4	10 × 2³	1 min 20 sec
Retry 5	10 × 2⁴	2 min 40 sec
Retry 6	10 × 2⁵	5 min 20 sec
Retry 10	10 × 2⁹	~1.4 hours
Retry 15	10 × 2¹⁴	~45 hours

In practice, almost every production implementation caps this growth at a maximum delay (capped exponential backoff) rather than letting it run unbounded — otherwise a single stuck webhook could theoretically schedule a retry decades out.

What real providers actually do

Stripe retries failed webhook deliveries for up to three days in live mode using exponential backoff, and disables the endpoint (with an email notification) if it keeps failing for that long. In sandbox/test mode, it retries only three times over a few hours. Stripe doesn't publish its exact delay schedule in its official docs, but third-party integration guides commonly report a cadence along the lines of: immediately, then roughly 5 minutes, 30 minutes, 2 hours, 5 hours, 10 hours, and then every 12 hours until the 3-day window closes. Stripe guarantees at-least-once delivery, never exactly-once, so duplicate deliveries are expected behavior, not a bug — more on this in the idempotency section below.

GitHub, somewhat surprisingly, does not automatically retry failed webhook deliveries at all. If your endpoint is down when GitHub tries to deliver an event, that delivery simply fails, and GitHub does not attempt it again on its own. GitHub's own documentation suggests building your own recovery script (e.g., a scheduled GitHub Actions workflow) that polls the Deliveries API for failed attempts and calls the redelivery endpoint yourself. Delivery history is retained for a limited window (a few days via the UI, longer via the newer Deliveries API), after which failed events are simply gone. This is a meaningfully different reliability model than Stripe's, and it's worth checking explicitly for any provider you integrate with — "webhooks" as a term doesn't imply any particular retry guarantee.

AWS services (SNS, EventBridge, SQS-backed delivery, and the AWS SDKs generally) implement exponential backoff with jitter as a first-class, built-in retry behavior in their "standard" and "adaptive" client retry modes, so you often don't have to hand-roll the backoff math yourself when working with AWS-native event delivery.

The takeaway: "webhooks" is not one contract. Always check what your specific event source actually guarantees before you design retry logic around it.

4. Jitter: Preventing the Thundering Herd

Exponential backoff alone isn't enough at scale.

The thundering herd problem

Say you send 5,000 webhooks a minute to one client's endpoint, and it goes down. All 5,000 failures get scheduled for their first retry exactly 10 seconds later. Ten seconds pass, and your system fires a synchronized wave of 5,000 requests at a server that may have just come back online in a fragile state — immediately knocking it back over. Because the backoff is deterministic, these synchronized waves recur at 20 seconds, 40 seconds, 80 seconds, and so on.

This isn't a hypothetical — it's exactly the problem Amazon engineers documented in their widely cited 2015 AWS Architecture Blog post on backoff and jitter, based on real production incidents. Their conclusion: capped exponential backoff without jitter reduces call volume, but it doesn't eliminate the synchronized retry clusters — it just relocates them in time.

Adding jitter

Jitter adds randomness to the wait time so retries spread out instead of arriving in lockstep. AWS's post (still referenced as the canonical source, and updated as recently as 2023 to note that most AWS SDKs now bake this in) compares three approaches:

Full Jitter — pick a uniformly random delay between 0 and the full capped exponential value:

Code example

exponential_delay = min(cap, base * 2 ** attempt)
actual_wait_time = random.uniform(0, exponential_delay)

Equal Jitter — keep half the exponential delay as a guaranteed floor, and randomize only the other half (avoids retrying too fast):

Code example

exponential_delay = min(cap, base * 2 ** attempt)
half_delay = exponential_delay / 2
actual_wait_time = half_delay + random.uniform(0, half_delay)

Decorrelated Jitter — doesn't use the attempt number directly; instead it grows from the previous sleep value, which produces a smoother, less clustered distribution:

Code example

sleep = min(cap, random.uniform(base, previous_sleep * 3))

According to AWS's own measurements, Full Jitter does the least total client-side work and produces the lowest server load, at the cost of slightly more variance in total completion time. Decorrelated Jitter finishes slightly faster but does a bit more total work. Equal Jitter is the weakest of the three on both counts. For most webhook systems, Full Jitter is the sensible default; Decorrelated Jitter is worth considering if you care more about tail latency than about minimizing peak load on the receiver.

Respect `Retry-After` when the server gives it to you

Before falling back to your own backoff math, check for a Retry-After header. It's defined in RFC 9110 (the current HTTP semantics spec) and is commonly sent on 429 and 503 responses. It can be either an integer number of seconds or an absolute HTTP-date, and when it's present, it's a more authoritative signal than anything you'd compute yourself — a well-behaved retry client honors it rather than layering its own backoff on top.

5. Architectural Best Practices for Webhook Retries

Differentiate retriable from non-retriable errors

Not every failure should trigger a retry. Inspect the status code:

Retriable (trigger backoff):

408 Request Timeout
429 Too Many Requests
500 Internal Server Error
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
Connection refused / DNS failures

Non-retriable (fail immediately, don't burn retry budget):

400 Bad Request — the payload is malformed; retrying the same bad payload just repeats the failure.
401 Unauthorized / 403 Forbidden — credentials or signatures are invalid; retries won't fix this without human intervention.
404 Not Found — the endpoint doesn't exist.
410 Gone — the endpoint was intentionally removed.

Idempotency on the receiving end

Because at-least-once delivery is the norm, duplicate deliveries are expected — not a corner case. If your worker sends a webhook and the receiver takes 31 seconds to respond with a 200 but your timeout is 30 seconds, you'll likely resend an already-processed event.

This is exactly why an emerging industry convention called Standard Webhooks exists — an open specification, led by the webhook infrastructure company Svix, that defines consistent headers for exactly this problem: webhook-id (a unique, retry-stable identifier), webhook-timestamp, and webhook-signature (HMAC-SHA256 by default). The webhook-id stays the same across every retry attempt of the same logical event, so a receiver can deduplicate by storing seen IDs (e.g., in Redis with a short TTL) and short-circuiting on repeats. The spec has been adopted by a notable range of companies — including OpenAI, Anthropic, Google Gemini, Twilio, PagerDuty, Supabase, and others — specifically to make this pattern consistent across providers instead of every API reinventing its own idempotency header.

Maximum retry limits and dead-letter queues (DLQs)

You can't retry forever. A common ceiling is somewhere around 3–7 days or roughly 15–20 attempts — Stripe's own 3-day cutoff is a reasonable real-world reference point.

When a webhook exhausts its retries, don't discard it — move it to a Dead-Letter Queue: a table or dedicated queue holding permanently failed deliveries so a human can inspect what went wrong and replay it manually once the receiver's issue is fixed.

Circuit breakers for chronically failing endpoints

If an endpoint fails 100% of its requests over some window (say, 10 minutes), tripping a circuit breaker to stop attempting new deliveries — and alerting the customer — is more useful than continuing to generate events destined to fail. Once the customer confirms a fix, reset the circuit and release the backlog.

6. Build vs. Buy

Building this in-house requires, at minimum:

A delivery-tracking table — payload, destination, status, attempt_count, next_retry_at.
Background workers — Celery, Sidekiq, BullMQ, or similar; this can't run synchronously in your request path.
A scheduler — something has to poll WHERE status = 'failed' AND next_retry_at <= NOW(), and at scale this query becomes a real bottleneck if not indexed and sharded carefully.
Retry and jitter logic — the math above, plus correct classification of retriable vs. non-retriable errors.
Signature verification — HMAC-based, ideally following the Standard Webhooks conventions so your consumers get a familiar integration experience.
Observability — a dashboard for "did that webhook actually send," searchable by payload, with a manual-replay button, because support teams and customers will ask this constantly.

None of this is exotic, but it is genuinely time-consuming to get right — particularly avoiding race conditions where two workers grab the same delivery and send duplicates.

If you'd rather not build it, the realistic options in this space today include:

Managed webhook-delivery platforms such as Svix (which also stewards the open Standard Webhooks spec) or similar providers, which handle retries, jitter, signature verification, and delivery dashboards as a hosted service you call into.
Cloud-native primitives — AWS EventBridge, SNS, or SQS with a Lambda consumer, which already implement backoff-with-jitter in their SDKs, if your event delivery is happening within AWS anyway.
Adopting Standard Webhooks on the receiving or sending side regardless of which platform you choose, so your integration is compatible with the conventions your customers' other providers already expect.

Whichever direction you go, the underlying requirements don't change: honor Retry-After when it's given, back off exponentially with jitter when it isn't, distinguish retriable from non-retriable failures, make your receiver idempotent, and give failed events somewhere to land besides "silently dropped."

Conclusion

A webhook retry strategy isn't a nice-to-have — it's a mandatory piece of any resilient API or SaaS product. The internet is unreliable by nature, and outages, however brief, are inevitable.

Immediate or linear retries create self-inflicted outages and waste resources. Exponential backoff with jitter — the same approach documented in AWS's production engineering and implemented, in some form, by providers like Stripe — solves both problems: fast recovery from brief blips, and automatic load-shedding during real outages. Layer on top of that a clear split between retriable and non-retriable errors, idempotent receivers, a dead-letter queue for anything that never makes it through, and a circuit breaker for chronically broken endpoints, and you have a system that degrades gracefully instead of catastrophically.

Whether you build this yourself or lean on a managed platform, the goal is the same: your data should always eventually arrive, without turning a brief outage on the other end into an outage of your own making.