InstaWebhook
November 10, 2026By InstaWebhook TeamWebhook Reliability

Bypassing the Kernel for Webhook Gateways: Implementing AF\_XDP in Rust for 1M+ RPS Ingestion

Learn how to build a high-performance webhook gateway capable of 1M+ RPS using AF\_XDP, eBPF maps, and Rust to bypass Linux kernel networking overhead.

1m rps webhook ingestionaf_xdp packet processingaf_xdp rustbypass linux kernel networkingebpf maps rusthigh performance webhook gatewayhigh throughput webhook receiverkernel bypass rustlinux network stack bypasslow latency webhook processingxdp ebpf rustxdp socket programming rust
Bypassing the Kernel for Webhook Gateways: Implementing AF\_XDP in Rust for 1M+ RPS Ingestion

Bypassing the Kernel for Webhook Gateways: Implementing AF_XDP in Rust for 1M+ RPS Ingestion

Webhook traffic is bursty by nature. Payment processors, code hosts, e-commerce platforms and in-house enterprise APIs (Stripe, GitHub and Shopify among them) can all retry or fan out events at once, and an ingestion gateway has to absorb the spike. At very high request rates, the Linux networking stack’s per-packet cost becomes a real limit. AF_XDP, an address family built on the eXpress Data Path (XDP) and eBPF, is one way to reduce that cost. It is a powerful tool, but it is not a drop-in replacement for sockets.

This article covers what AF_XDP does, what it doesn’t do, how to set up the pieces in Rust, and what to expect in practice.

1. Where the Standard Stack Spends Its Time

On the normal receive path, the NIC DMAs a frame into kernel memory and the driver builds an sk_buff. The packet then moves through IP and TCP processing, lands in a socket receive queue, and is copied into your application’s buffer when you call recv/read (typically after an epoll wakeup). Three costs add up at high packet rates:

  • Per-packet allocation and traversal. Every packet pays for sk_buff handling and passes through the protocol layers, even if your application only wants a small JSON body.
  • Copies and cache pressure. Data is copied from kernel buffers to user buffers, and the kernel/user transitions disturb CPU caches.
  • Shared state. Sockets, connection tables and queues are shared structures, and contention grows with core count. Techniques such as SO_REUSEPORT and per-queue RSS spread the load, but the per-packet work is still there.

Modern drivers use NAPI, which combines interrupts with polling, so the cost is not one hardware interrupt per packet. The remaining cost is mostly per-packet software work in the stack.

What AF_XDP Changes

The kernel documentation describes AF_XDP as an address family optimized for high-performance packet processing. An XDP program runs in the driver’s receive path, before the normal stack, and can use bpf_redirect_map() to send a frame to an AF_XDP socket (XSK), which exposes it to a user-space buffer. Frames that the program does not redirect continue through the kernel normally.

Code example
Standard stack:
[NIC] -> DMA -> driver -> sk_buff -> IP/TCP -> socket queue -> recv() -> app

AF_XDP:
[NIC] -> DMA -> driver -> XDP program -> XSKMAP -> UMEM + RX ring -> app
                                   \-> XDP_PASS -> normal kernel stack (everything else)

The Catch: You Receive Raw Ethernet Frames

AF_XDP delivers L2 frames, not TCP streams. Redirected traffic skips the kernel’s TCP, and TLS if you use it is not handled either. For an HTTP(S) webhook endpoint that means your program has to supply:

  • ARP/NDP and ICMP handling (or let the kernel handle them by passing that traffic with XDP_PASS);
  • the TCP state machine: handshakes, retransmission, congestion control, segmentation and reassembly;
  • TLS, if the gateway terminates it itself;
  • HTTP parsing on top of the reassembled stream.

Projects exist that put a user-space TCP stack on AF_XDP. Academic work has combined mTCP with AF_XDP, and Rust projects have planned or built bridges to smoltcp. But this is a substantial engineering investment, and it is the main reason AF_XDP is not a simple “faster socket.”

It also matters that 1M packets per second is not 1M requests per second. A single HTTP request involves several packets: a new TLS connection adds handshake flights, and every request and response adds data and acknowledgement packets. Published AF_XDP benchmarks, quoted below, measure raw packets, not completed HTTP transactions.

2. Core Components of an AF_XDP Setup

The kernel documentation defines these objects:

  1. UMEM. A contiguous region of virtual memory divided into equal-sized chunks (frames), registered with the XDP_UMEM_REG socket option. A UMEM is bound to a network device and a queue ID. Chunk sizes are typically 2 KiB or 4 KiB in aligned mode.
  2. Four rings, all single-producer/single-consumer:
    • Fill ring: user space hands the kernel empty chunk addresses to receive into.
    • RX ring: the kernel hands user space descriptors (UMEM offset plus length) for received packets.
    • TX ring: user space submits descriptors for packets to send.
    • Completion ring: the kernel returns chunk addresses of frames that have finished transmitting. Fill and completion rings belong to the UMEM; RX and TX rings belong to each socket.
  3. XDP program and BPF_MAP_TYPE_XSKMAP. The program decides which packets go to user space and redirects them through the map. A packet received on a given RX queue can only be redirected to a socket bound to that same queue, otherwise the kernel drops it. In practice that means one XSK per hardware queue, usually one per worker thread.

Ring sizes must be powers of two. Because the rings are single-producer/single-consumer, you need your own synchronization if several threads touch the same ring.

3. Implementing the Pieces in Rust

Rust is a good fit for the user-space side because you handle raw buffers and ring indices, which is easy to get wrong in C. Memory safety does not remove the unsafe code, though. The code below is an abridged blueprint, not a complete implementation.

Step 1: Allocate the UMEM

Use anonymous memory and pre-fault it with MAP_POPULATE so you avoid page faults during a burst. Memory from mmap is page-aligned, which AF_XDP requires.

Code example
use libc::{mmap, munmap, MAP_ANONYMOUS, MAP_FAILED, MAP_POPULATE, MAP_PRIVATE, PROT_READ, PROT_WRITE};
use std::ptr;

const UMEM_SIZE: usize = 64 * 1024 * 1024; // 64 MiB
pub const FRAME_SIZE: usize = 2048;        // 2 KiB chunks

pub struct UmemBuffer {
    pub addr: *mut u8,
    pub size: usize,
}

impl UmemBuffer {
    /// # Safety
    /// The caller must keep the buffer alive for as long as the kernel may use it.
    pub unsafe fn new() -> Result<Self, std::io::Error> {
        let addr = mmap(
            ptr::null_mut(),
            UMEM_SIZE,
            PROT_READ | PROT_WRITE,
            MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE,
            -1,
            0,
        );
        if addr == MAP_FAILED {
            return Err(std::io::Error::last_os_error());
        }
        Ok(Self { addr: addr as *mut u8, size: UMEM_SIZE })
    }
}

impl Drop for UmemBuffer {
    fn drop(&mut self) {
        unsafe { munmap(self.addr as *mut _, self.size); }
    }
}

Registering the UMEM with the socket (XDP_UMEM_REG) only works while the memory stays valid, so the buffer must outlive the socket. Close the socket before unmapping.

Step 2: The XDP Program with Aya

The Aya project provides aya-ebpf for writing the kernel side in Rust, including an XskMap type. Its documentation shows the basic pattern: look up the receive queue index and redirect to the socket at that index. The program below extends that pattern with a simple IPv4/TCP destination-port check. It is a sketch, and you will need to adapt it to your Aya version and verifier constraints.

Code example
#![no_std]
#![no_main]

use aya_ebpf::{
    bindings::xdp_action,
    macros::{map, xdp},
    maps::XskMap,
    programs::XdpContext,
};

#[map]
static XSKS: XskMap = XskMap::with_max_entries(64, 0);

const ETH_HDR_LEN: usize = 14;
const ETH_P_IP: u16 = 0x0800;
const IPPROTO_TCP: u8 = 6;

#[inline(always)]
fn byte_at(ctx: &XdpContext, off: usize) -> Result<u8, ()> {
    let p = ctx.data() + off;
    if p + 1 > ctx.data_end() {
        return Err(());
    }
    Ok(unsafe { *(p as *const u8) })
}

#[xdp]
pub fn webhook_classifier(ctx: XdpContext) -> u32 {
    try_classify(&ctx).unwrap_or(xdp_action::XDP_PASS)
}

fn try_classify(ctx: &XdpContext) -> Result<u32, ()> {
    // EtherType (bytes 12-13) must be IPv4.
    let ethertype = ((byte_at(ctx, 12)? as u16) << 8) | byte_at(ctx, 13)? as u16;
    if ethertype != ETH_P_IP {
        return Ok(xdp_action::XDP_PASS); // ARP, IPv6, etc. stay with the kernel
    }
    // IPv4 protocol byte and header length.
    if byte_at(ctx, ETH_HDR_LEN + 9)? != IPPROTO_TCP {
        return Ok(xdp_action::XDP_PASS);
    }
    let ihl = (byte_at(ctx, ETH_HDR_LEN)? & 0x0f) as usize * 4;
    if ihl < 20 {
        return Ok(xdp_action::XDP_PASS);
    }
    // TCP destination port is at offset 2 of the TCP header.
    let tcp = ETH_HDR_LEN + ihl;
    let dport = ((byte_at(ctx, tcp + 2)? as u16) << 8) | byte_at(ctx, tcp + 3)? as u16;

    if dport == 443 || dport == 8443 {
        let queue = unsafe { (*ctx.ctx).rx_queue_index };
        // Falls back to XDP_PASS if no socket is registered for this queue.
        return Ok(XSKS.redirect(queue, xdp_action::XDP_PASS as u64).unwrap_or(xdp_action::XDP_PASS));
    }
    Ok(xdp_action::XDP_PASS)
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    loop {}
}

Be aware of what this does: once port 443 traffic is redirected to an XSK, the kernel no longer sees those packets. Your user-space program owns that port’s TCP connections entirely. Test on a dedicated interface or a dedicated IP, not on a machine you manage over the same port.

In user space, Aya’s XskMap type lets you load the program and insert each socket’s file descriptor at the index of its queue.

Step 3: Create the Socket and Rings

The socket setup follows the kernel’s documented sequence: create the socket with socket(AF_XDP, SOCK_RAW, 0), register the UMEM with XDP_UMEM_REG, size the fill and completion rings (XDP_UMEM_FILL_RING, XDP_UMEM_COMPLETION_RING) and the RX/TX rings (XDP_RX_RING, XDP_TX_RING), mmap the rings, and then bind() to an interface and queue ID. In bind you choose the mode: XDP_COPY or XDP_ZEROCOPY. Binding fails if you request zero-copy on a driver that does not support it. You can also set XDP_USE_NEED_WAKEUP, which lets the kernel tell you when you need to make a syscall to kick the driver.

Most teams do not hand-roll the ring bookkeeping. They use libxdp/libbpf’s XSK helpers through FFI, or a Rust crate that wraps AF_XDP (several exist, such as netring), and check each crate’s maturity before depending on it. Running the process needs privileges such as CAP_NET_RAW, and the BPF and memory-lock limits (ulimit -l) must allow the maps and UMEM.

Step 4: The Receive Loop

The loop below is deliberately schematic. Its shape is correct, but the stack and parsing layers are the hard part.

Code example
pub fn run_worker(xsk: &mut XdpSocket /* one per queue */) {
    loop {
        // 1. Peek the RX ring for a batch of descriptors (addr, len).
        // 2. For each descriptor, take the frame at umem_base + addr.
        // 3. Run it through your user-space stack: Ethernet -> IP -> TCP
        //    (-> TLS, if terminated here) -> HTTP.
        // 4. Hand complete webhook bodies to a worker pool over a bounded queue.
        // 5. Release the RX descriptors, then return frame addresses to the
        //    Fill ring so the kernel can keep receiving.
        // 6. Reap the Completion ring if you transmit (ACKs, responses).
        // 7. If the socket reports that it needs a wakeup, call poll()/sendto().
    }
}

Two details are easy to get wrong. First, the application must keep the fill ring stocked, because with no fill descriptors the kernel has nowhere to put new packets and the RX ring stays empty. Second, with XDP_USE_NEED_WAKEUP you only make a syscall when the kernel’s flag asks for it. Busy-polling is another option, and its benefit depends on the driver and configuration.

“Zero-copy parsing” needs a caveat as well. A frame stays in the UMEM, but TCP payload may arrive in several segments, and TLS decryption produces new plaintext. Beyond the packet-delivery layer you will usually copy or reassemble data somewhere.

4. Copy Mode vs. Zero-Copy Mode

The kernel documentation names two operating modes: the generic mode (XDP_SKB), which works on any network device but still uses sk_buffs and copies data to user space, and native driver mode (XDP_DRV), used when the driver supports XDP. Within native mode, zero-copy additionally requires driver support, and the NIC’s DMA writes directly into UMEM.

The following kernel-developer benchmark numbers (Intel i40e 40 GbE, 64-byte packets, from the original AF_XDP patch discussions in 2018) show the relative gap for a pure rxdrop test. The application receives packets and drops them, and does nothing else:

Moderxdrop (Mpps, one test setup, 2018)
Generic (XDP_SKB)about 3
Native, copy (XDP_DRV)about 9.6
Native, zero-copyabout 21.5

These are best-case microbenchmarks, not predictions for a webhook gateway. A more recent open-source Go AF_XDP library reports that a single queue on a 100 Gbit Mellanox mlx5 setup tops out near 7.7 Mpps. Real numbers depend on the NIC, driver, kernel version, packet size and, above all, what your application does with each packet.

Driver Support

A kernel patch enabling XDP feature flags lists native and zero-copy support for the Intel i40e, ice, ixgbe and igc drivers, Mellanox mlx5, stmmac and Netronome nfp. It lists native XDP (without zero-copy) for ena, bnxt, virtio_net, mlx4 and others. Features can vary by kernel version and configuration, so verify on your kernel with your NIC before committing to a design.

On AWS, the out-of-tree ENA driver has shipped AF_XDP zero-copy support in some releases. Its release notes show it was added in 2.7.0, later made experimental, and re-enabled in release 2.13.0, so check the exact driver version on your instances. Even in generic or copy mode, AF_XDP skips the TCP/IP stack for redirected traffic, but you then pay for implementing that functionality yourself.

5. Production Checklist

  1. One XSK per RX queue, one thread per socket. Because redirects only work to a socket on the same queue, align your threads, queues and IRQ affinity. Pin threads to cores on the NIC’s NUMA node.
  2. Steer the right traffic to the right queues. Use RSS and flow steering (ethtool -N) so webhook traffic lands on the queues that have AF_XDP sockets, and keep management traffic elsewhere.
  3. Process in batches. Read and release ring entries in bursts rather than one by one.
  4. Size rings and the UMEM generously. Use power-of-two ring sizes (for example 2048 or 4096), and size the UMEM for in-flight packets across all stages. If the application falls behind, the first symptoms are a full RX ring or an empty fill ring, and packets are dropped.
  5. Decide where TLS terminates. A common layout puts a layer-4 load balancer in front of layer-7 proxies that terminate TLS. Meta’s open-source Katran is an XDP-based L4 load balancer used this way. It forwards packets using transport-layer information and does not terminate TLS or parse HTTP. If your gateway terminates TLS itself on an AF_XDP stack, you must supply TLS and handle its per-connection and per-record CPU cost.
  6. Bound your queues. Put a bounded queue between the network threads and the business-logic workers so a slow downstream (signature verification, database writes) turns into explicit backpressure rather than growing memory.
  7. Measure the real workload. Test with realistic connection counts, TLS settings and payload sizes, and compare against a well-tuned kernel-socket baseline (SO_REUSEPORT, multiple queues, a modern async runtime) before you commit.

6. When AF_XDP Is, and Isn’t, the Right Choice

AF_XDP is a strong choice when packets need to reach user space with minimal overhead and the per-packet logic does not fit in an eBPF program. The library authors list examples such as tunnels and crypto, QUIC, a user-space TCP/TLS stack, stateful inspection and traffic generation. If you only need to drop, forward or rewrite packets, doing it inside the XDP program itself is faster than a round trip to user space.

For a conventional HTTPS webhook gateway, the practical picture is different. The TCP and TLS work does not go away, it moves into your code. Many teams will get further by tuning the kernel path, scaling horizontally, and putting an XDP-based L4 balancer in front of ordinary proxies. Consider AF_XDP for the ingress tier only if profiling shows the kernel stack really is your bottleneck and you are prepared to own a user-space network stack.

7. Conclusion

AF_XDP, eBPF and Rust together give you a safe-as-possible path to receive packets directly from the driver into user-space memory, with benchmark results far above the generic socket path in raw packets per second. The trade-off is that you take over everything the kernel stack did for you, from TCP to TLS. Whether that reaches 1M+ HTTP requests per second depends on your hardware, drivers, traffic mix and how well that user-space stack is built. Treat the packet-rate benchmarks as an upper bound, and validate the design with measurements from your own environment.

References