Anycast Routing for Global Webhook Ingress: Minimizing Latency and Packet Loss for Worldwide Senders

Modern SaaS ecosystems rely heavily on webhooks. When a payment succeeds on Stripe, an order is created on Shopify, or a push lands on GitHub, the platform sends an asynchronous HTTP POST request to an endpoint you control. The sender usually expects a quick 2xx response, and it treats anything else, including a slow response, as a failed delivery.

Scaling webhook ingestion globally is therefore a problem of physics and transport reliability. If your endpoint lives in a single region, every sender outside that region pays for distance on every connection. This guide explains how BGP Anycast shortens that distance, where it helps, and where it doesn’t.

+-----------------------------------------------------------------------------------+
|                         UNICAST INGRESS (illustrative)                            |
|                                                                                   |
|  Sender (Tokyo) ---[ TCP + TLS handshake over a long path ]---> Origin (Virginia) |
+-----------------------------------------------------------------------------------+

+-----------------------------------------------------------------------------------+
|                         ANYCAST EDGE INGRESS (illustrative)                       |
|                                                                                   |
|  Sender (Tokyo) ---[ short handshake ]---> Nearby PoP                             |
|                                               |                                   |
|                                  [ warm connections / private backbone ]          |
|                                               v                                   |
|                                          Origin / Queue                           |
+-----------------------------------------------------------------------------------+

What Webhook Senders Expect

Before designing ingress, it helps to know what the biggest providers require. These figures come from their public documentation:

• GitHub expects a 2xx response within 10 seconds. A slower response is treated as a failed delivery, and you can redeliver failed webhooks later. GitHub lowered this limit from 30 seconds in 2017.
• Shopify fails a delivery if your app doesn’t respond within five seconds. It retries failed deliveries up to eight times over about four hours, and it removes the subscription if failures persist.
• Stripe recommends returning a 2xx quickly, before any complex logic that could cause a timeout. In live mode it retries delivery for up to three days with exponential backoff. In sandbox it retries three times over a few hours. Stripe doesn’t publish an exact timeout value, so don’t design around a specific number.

Every one of these providers tells you to acknowledge first and process later. Anycast ingress applies that advice at the network layer.

The Problem with Unicast Ingress for Global Webhooks

1. Handshake Latency Scales with Distance

A unicast IP routes traffic to one location. If your ingestion service runs in a single region such as us-east-1 and the sender dispatches from Tokyo, every new connection pays the full round-trip time (RTT) several times before any payload is processed.

Before the request can be sent, a new HTTPS connection needs:

• One RTT for the TCP handshake.
• One RTT for the TLS 1.3 handshake. TLS 1.2 needs two.

Client (Tokyo) <------------------ ~160 ms RTT (illustrative) ------------------> Origin (Virginia)

1. TCP SYN --------------------------------------------------------------->
2. <---------------------------------------------------------- TCP SYN-ACK
3. TCP ACK + TLS ClientHello ---------------------------------------------->
4. <----------------------------------- TLS ServerHello, Certificate, Finished
5. TLS Finished + HTTP POST /webhook --------------------------------------->
6. <------------------------------------------------------- HTTP 2xx response

At an illustrative 160 ms RTT, the arithmetic looks like this:

These numbers are far below a 5-second or 10-second timeout on a healthy path. The real danger is variance: loss, congestion, and slow handlers stack on top of that baseline. A sender that sees a failure retries, and a large burst of retries can add load to an endpoint that is already struggling. Providers use exponential backoff to soften this, but your endpoint still absorbs the duplicates.

2. Loss Hurts More on Long Paths

Traffic over the public internet crosses multiple autonomous systems (AS) with varying congestion. Two effects make loss more costly on long paths:

• Recovery time scales with RTT. Loss in the middle of a transfer is typically recovered after roughly one RTT, so a 160 ms path recovers far more slowly than a 10 ms one. Loss-based congestion control such as CUBIC also shrinks the congestion window after loss, and on long paths it takes longer to grow back.
• Handshake loss is the most expensive kind. If a SYN is lost, RFC 6298 sets the initial retransmission timeout at one second. A single dropped SYN can add a full second before the connection even starts.

3. Longer Paths Mean More Exposure

The farther a connection travels over networks you don’t control, the more opportunities there are for congestion, rerouting, and jitter. Internet paths are also often asymmetric, meaning the forward and return routes differ. That is normal, and it applies to anycast too. Anycast doesn’t remove path problems. It shortens the portion of the path where they can hurt a new connection.

How BGP Anycast Works

With anycast, the same IP prefix is announced from multiple physical locations (Points of Presence, or PoPs). Each network on the internet picks a route to that prefix using BGP, so traffic from a given sender typically flows to whichever announcing location its network prefers.

                      Sender (US-West)          Sender (Europe)
                            |                         |
                     (BGP-selected route)      (BGP-selected route)
                            v                         v
                  +------------------+      +------------------+
                  | PoP: San Jose    |      | PoP: Frankfurt   |
                  | 198.51.100.0/24  |      | 198.51.100.0/24  |
                  +--------+---------+      +--------+---------+
                           \                       /
                            \    private / warm   /
                             v    connections    v
                          +--------------------------+
                          |  Central origin / queue  |
                          +--------------------------+

(198.51.100.0/24 is a documentation-only prefix and is not routable on the real internet.)

“Nearest” Means Nearest in BGP, Not in Kilometers

BGP selects routes using policy first, then attributes such as AS-path length. It knows nothing about geography. Most of the time the chosen PoP is close to the sender, but not always. A sender on a network with unusual peering can land at a distant PoP. Anycast catchments are therefore something to measure and tune, not something to assume.

Operators influence catchments by choosing where to announce, which transit providers and exchange points (IXPs) to peer with, and by using techniques such as AS-path prepending or BGP communities.

Terminate Connections at the Edge

Instead of completing TCP and TLS across an ocean, the sender completes them with the nearest PoP. If the sender is 10 ms from that PoP, the handshake costs roughly 20 ms instead of several hundred.

1. Local handshake: TCP and TLS 1.3 complete in about two short RTTs.
2. Local receipt: The edge reads the POST body over a short path.
3. Backbone forwarding: The edge hands the event to your systems over pre-established, long-lived connections on a path you control. This might be a private backbone or a provider network such as AWS’s.

Loss Recovery Happens Over a Shorter Segment

The sender-to-edge leg is still the public internet, but it is shorter. Because retransmissions are paced by RTT, recovery on a 10 ms path is much faster than on a 160 ms path. The long-haul leg moves onto infrastructure with more predictable behavior. It can still suffer loss, but it is no longer the part the sender’s timeout clock is watching.

Managed Anycast vs. Running Your Own

Announcing your own anycast prefix requires an ASN, provider-independent address space (a /24 is the smallest IPv4 prefix generally accepted in the global routing table), and BGP relationships at every site. Many teams use a managed service instead:

• Cloudflare runs its network on anycast. It was connected to more than 300 cities as of its 2023 announcement.
• AWS Global Accelerator provides two static anycast IPv4 addresses announced from multiple AWS edge locations. Traffic enters the AWS network near the sender and travels to your regional endpoint over AWS’s backbone.

Check where your actual senders dispatch from before investing. If most of your webhook volume originates in one or two regions, a regional endpoint near those regions may deliver most of the benefit at lower cost.

Architecture of an Anycast-Fronted Webhook Ingress

An anycast ingress stack has three layers: edge routing, edge processing, and internal transport and buffering.

+-----------------------------------------------------------------------------------+
| 1. ANYCAST EDGE ROUTING                                                           |
|    - Prefix announced via BGP (e.g., BIRD or ExaBGP)                              |
|    - L4 load balancing inside the PoP (e.g., Maglev-style consistent hashing,     |
|      IPVS, or XDP/DPDK-based balancers)                                           |
+-----------------------------------------------------------------------------------+
                                       |
                                       v
+-----------------------------------------------------------------------------------+
| 2. EDGE PROCESSING                                                                |
|    - TLS termination, HTTP/1.1 + HTTP/2 (HTTP/3 optional)                         |
|    - Signature verification (optional)                                            |
|    - Durable enqueue, then 2xx acknowledgment                                     |
+-----------------------------------------------------------------------------------+
                                       |
                                       v
+-----------------------------------------------------------------------------------+
| 3. TRANSPORT AND BUFFER                                                           |
|    - Persistent connections over private links or a provider backbone             |
|    - Durable queue (Kafka, Redpanda, SQS, etc.), consumed asynchronously          |
+-----------------------------------------------------------------------------------+

Layer 1: BGP at the Edge

Each PoP runs a BGP daemon such as BIRD or ExaBGP that announces the anycast prefix to upstream providers and peers. A minimal BIRD 2 sketch looks like this:

# Illustrative BIRD 2 sketch. All ASNs and prefixes are placeholders.
protocol static anycast_vip {
    ipv4;
    # Anycast service address, bound to a loopback interface on the PoP's servers
    route 198.51.100.0/24 via "lo";
}

protocol bgp upstream_1 {
    local as 65001;
    neighbor 192.0.2.1 as 64512;

    ipv4 {
        import none;
        export filter {
            if net = 198.51.100.0/24 then accept;
            reject;
        };
    };
}

In production you would use your own public ASN and prefix, add session authentication, prefix limits, and RPKI validation, and tune announcements per upstream.

Layer 2: Fast-Ack Ingestion

To keep the sender’s clock short, the edge should do as little as possible before acknowledging:

1. Terminate TLS.
2. Optionally verify the signature using a cached secret, so obviously forged traffic is rejected cheaply. Each provider signs differently:
   • GitHub sends X-Hub-Signature-256, an HMAC-SHA256 of the raw body prefixed with sha256=.
   • Shopify sends X-Shopify-Hmac-Sha256, a base64-encoded HMAC-SHA256 of the raw body.
   • Stripe sends a Stripe-Signature header containing a timestamp and signature(s), computed over the timestamp and the payload. Its libraries also enforce a timestamp tolerance.In every case, verify against the raw, unparsed body.
3. Durably enqueue the event.
4. Return a 2xx. The acknowledgment should never come before the event is durably stored.

Example: Edge Ingress Handler for GitHub Webhooks

import { createHmac, timingSafeEqual } from "node:crypto";

interface WebhookRequest {
  headers: Record<string, string | undefined>;
  body: string; // raw, unparsed body
}

// Application-defined: durably writes to a queue near this PoP.
declare function publishToQueue(msg: {
  rawBody: string;
  headers: Record<string, string | undefined>;
  receivedAt: number;
  pop: string | undefined;
}): Promise<boolean>;

export async function handleGitHubWebhook(req: WebhookRequest): Promise<Response> {
  const signature = req.headers["x-hub-signature-256"];
  const secret = process.env.GITHUB_WEBHOOK_SECRET;

  if (!signature || !secret) {
    return new Response(JSON.stringify({ error: "Unauthorized" }), { status: 401 });
  }

  // 1. Verify the HMAC over the raw body using a constant-time comparison
  const expected = "sha256=" + createHmac("sha256", secret).update(req.body).digest("hex");
  const a = Buffer.from(signature);
  const b = Buffer.from(expected);

  if (a.length !== b.length || !timingSafeEqual(a, b)) {
    return new Response(JSON.stringify({ error: "Invalid signature" }), { status: 403 });
  }

  // 2. Durably enqueue before acknowledging
  const ok = await publishToQueue({
    rawBody: req.body,
    headers: req.headers,
    receivedAt: Date.now(),
    pop: process.env.POP_REGION_CODE,
  });

  if (!ok) {
    // A non-2xx makes the sender retry, which is what you want if nothing was stored
    return new Response(JSON.stringify({ error: "Queue unavailable" }), { status: 503 });
  }

  // 3. Acknowledge; heavy processing happens asynchronously downstream
  return new Response(JSON.stringify({ status: "accepted" }), { status: 202 });
}

Two caveats apply to this design:

• Where the queue lives matters. If every PoP must synchronously write to one distant queue, you have moved the long round trip from before the acknowledgment to inside it. Write to a regional or PoP-local durable buffer and replicate or forward asynchronously, or accept the cross-region write latency as a known cost.
• Duplicates are normal. Senders retry on timeouts and ambiguous failures, and Shopify’s documentation notes that duplicate deliveries can occur. Deduplicate downstream, using identifiers such as GitHub’s X-GitHub-Delivery header or the event ID in a Stripe payload.

Anycast vs. Unicast: What Changes

Operational Challenges and Mitigations

1. Route Changes Can Break In-Flight Connections

Anycast sends each packet toward whichever PoP the network currently prefers. If routing shifts mid-connection, subsequent packets can arrive at a different PoP that has no state for that TCP connection, and it answers with a reset.

How common is this? A measurement study by Wei and Heidemann covering more than 9,000 vantage points and 11 anycast services found that about 1% of vantage points saw frequent flips between anycast sites. TCP connections were disrupted far less often, in about 0.15% of vantage point and service combinations. Those results come from DNS root services, so your own measurements may differ.

Practical mitigations:

• Keep connections short. Webhook deliveries are brief request/response exchanges, so the exposure window is small. A rare reset simply causes a sender retry, which your deduplication already handles.
• Use consistent hashing inside each PoP. Maglev-style balancing keeps a connection on the same backend server within a PoP. Within-PoP ECMP does not help when traffic moves between PoPs.
• Consider cross-PoP forwarding only if you need it. Encapsulating and forwarding misdirected packets to the PoP that owns the connection (for example, over GRE or GUE tunnels) is possible, but it adds significant complexity.
• Don’t count on QUIC to solve this. QUIC connection IDs support migration, but it only helps if the sender’s HTTP client speaks HTTP/3. A server that receives packets for a connection it doesn’t know still cannot continue it without extra routing infrastructure.
• Lean on idempotency and retries. Providers already retry failed deliveries, so a robust consumer treats a rare reset as just another duplicate.

2. Health Checks and Route Withdrawal

If a PoP’s application fails while BGP keeps announcing the prefix, traffic still arrives and is dropped. Withdraw the route when the local service is unhealthy.

• Run health checks that cover the ingress service, its link to the queue, and upstream connectivity.
• Have the checker withdraw the announcement (ExaBGP’s process API is commonly used for this) when checks fail, and re-announce when they recover.
• Add hysteresis so a flapping service doesn’t cause a flapping route. Frequent announcements and withdrawals can trigger route-flap damping at upstream providers.

Be realistic about timing. Withdrawing a route isn’t instant, because the news must propagate across the internet. Measurements discussed by APNIC’s Geoff Huston put typical BGP convergence at tens of seconds, and older academic studies recorded minutes in bad cases. Vendors that advertise anycast failover in under 30 seconds are describing good-case behavior. Sub-second failover is not a safe assumption.

During that window, senders will see timeouts or resets and rely on their retry policies. Stripe’s three days and Shopify’s eight attempts over four hours are far longer than any realistic convergence period.

 PoP A: healthy ---- announces prefix ----> stays in service
 PoP B: unhealthy -- withdraws prefix ----> traffic shifts to other PoPs
                                             as the withdrawal propagates
                                             (tens of seconds typical; can be longer)

3. Edge-to-Origin Backhaul

Terminating at the edge solves the sender-facing leg. The data still has to reach your origin.

• Keep persistent, pre-established connections from each PoP to your origin or queue so forwarding never pays a fresh handshake.
• Where possible, use a private or provider backbone, such as AWS’s network behind Global Accelerator or a dedicated interconnect, rather than the public internet.
• Make backhaul failures survivable. The edge should buffer or fail over when the link to the origin degrades, and it should return a 5xx if it truly cannot store the event, so the sender retries.

When Anycast Is (and Isn’t) Worth It

Anycast ingress is a strong fit when:

• Senders are geographically diverse, or you receive webhooks from many sources worldwide.
• Providers enforce tight timeouts, such as Shopify’s five seconds.
• You already need a global edge for DDoS absorption or for static IPs.

It is less compelling when:

• Nearly all senders dispatch from a single region you can colocate with.
• Your real bottleneck is slow handler code, not the network. Moving to a queue-then-process pattern fixes that regardless of anycast.
• You don’t have the operational capacity to run BGP at multiple sites and would need to depend on a managed provider anyway.

Conclusion

Webhook senders enforce short acknowledgment windows: five seconds for Shopify and ten for GitHub, with Stripe asking for a quick 2xx and not publishing a number. A single-region endpoint makes distant senders pay several long round trips just to connect, and it exposes them to loss recovery that scales with distance.

BGP anycast moves the sender’s handshake to a nearby PoP, shortens the segment where loss recovery happens, and lets you move events to your origin over connections you control. It does not give you instant failover or immunity from route changes. The realistic goal is to accept events locally and durably, acknowledge quickly, deduplicate downstream, and treat the occasional reset or slow convergence as a normal retry.