Re: [PATCH bpf-next 00/15] xdp: add SKB core extensions to keep RX metadata until skb creation

[Jesper Dangaard Brouer <hawk@kernel.org>]

On 11/10/26 04:56, Tianyi Gao wrote:
> An skb gets its RX hash, VLAN tag and checksum status from the RX
> descriptor, which describes the packet before the XDP program ran.
> 
> Without this series:
> 
> - On XDP_PASS, the values are stale. After the program strips a UDP
>    tunnel header, the stack applies the outer CHECKSUM_UNNECESSARY to the
>    inner packet [1]. If the outer UDP checksum is 0, a packet with a bad
>    inner checksum reaches the socket.

Our XDP Unimog load-balancer uses GUE like packet encapsulation, so it
hits above case all the time. If packet is end-host, it modify packet by
popping header and returns XDP_PASS for local netstack delivery.

> - On XDP_REDIRECT, the values are lost. The skb that cpumap or veth
>    builds from the xdp_frame has no RX hash and no VLAN tag, and its
>    checksum status is CHECKSUM_NONE.
> 
> In one use case, an XDP program redirects packets into a veth device.
> The checksum status is lost, so GRO verifies every checksum in software.
> With one TCP flow, this costs 34% of the receiving CPU.
> 
> Another use case [2] is load balancing IPsec tunnel traffic with cpumap.
> Without the RX hash, GRO on cpumap puts all flows into the same bucket.
> In a cpumap test with 16 TCP flows and no IPsec, the throughput is
> 17.6 Gbit/s instead of 23.4.
> 

Yes, GRO missing RX-hash for bucket split is real. Above IPsec XDP load-
balacing have been running in production for a year by now.
Our more general RX-pipeline CPUMAP redirect is currently being tested
in production.  This means all of our traffic will soon be hitting this
GRO limitation.

> This series introduces SKB core extensions for XDP. It reserves an skb
> core extension area in the headroom, right after the space for struct
> xdp_frame. An XDP program stores RX metadata in the area with kfuncs,
> and the kernel sets the skb fields from it. This design was initially
> proposed in [3].
> 

Yes, I agree. We don't want to add static members to xdp_frame, as it
will always reserve the space.  Instead this only takes up the space
when needed.  The most obvious case is the VLAN.


> The layout of the area follows skb extensions (struct skb_ext). A small
> header gives the offset of each entry. An entry takes headroom only when
> the program stores it, so the area does not grow with each new type of
> RX metadata, unlike [4]. Unlike skb_ext, the area needs no allocation.
> 

Hopefully other maintainers like this encoding format better than our
previous attempt.  As this is already used by existing skb extensions.
This is not expose as UAPI and BPF access this via helpers.


> This series supports three types of RX metadata. Each type has a store
> kfunc, and bpf_xdp_skb_ext_del() deletes an entry:
> 
> - The RX hash and its type: bpf_xdp_skb_ext_store_rx_hash().
> - The VLAN tag and its protocol: bpf_xdp_skb_ext_store_rx_vlan().
> - The checksum status: bpf_xdp_skb_ext_store_rx_csum(). Only
>    XDP_CHECKSUM_NONE, or XDP_CHECKSUM_UNNECESSARY with a csum_level up to
>    SKB_MAX_CSUM_LEVEL, are allowed [5][6].
> 
> The kernel sets the skb fields from the area:
> 
> - After XDP_REDIRECT, in __xdp_build_skb_from_frame().
> - On XDP_PASS, in the NIC driver. A driver that does this sets the new
>    XDP feature skb-ext-pass. On other native XDP drivers, the kernel
>    warns once when the program returns XDP_PASS with stored values, and
>    ignores the area. This series adds the support to ice.
> 
> The existing XDP RX metadata kfuncs are bpf_xdp_metadata_rx_*(). Only
> a device-bound program can call them. Programs on cpumap and devmap are
> not device-bound, so they cannot call them. Relaxing this check would
> change their semantics. So this series adds three load kfuncs,
> bpf_xdp_skb_ext_load_rx_*(). They read the stored values.
> 
> bpf_xdp_adjust_head(), bpf_xdp_adjust_meta() and bpf_xdp_pull_data() do
> not move the packet or the metadata into the area. In this series, the
> area takes at most 20 bytes of headroom. With the hardware RX timestamp,
> the RX queue index and CHECKSUM_COMPLETE, it would take up to 40 bytes.
> 

I like how the size usage is limited in this patchset. As I understood, 
this was previously a concern from Jakub/kuba in my previous patchset.
In my previous patchset we allowed more flexibility and others to use 
this area. This has been dropped. Notice how this patchset focus only on 
the fields needed for the SKB.  This is basically working towards that
xdp_frame becomes the "mini-SKB" concept.


> Lorenzo's RX checksum kfunc series v5 [7] also defines enum
> xdp_checksum. The series that is merged second drops its copy.
> 
> Performance:
> 
> We measured on ice (E810, 25 Gbit/s) with bpf-next 0fb18ce8a2e9, patches
> 1-3 of [7] and this series. In the RX metadata column, the XDP program
> reads a value with bpf_xdp_metadata_rx_*() and stores it. The veth test
> stores the checksum status, and the cpumap test stores the RX hash. The
> kernel then sets the value on the skb. Baseline does not store it. Each
> value is the median of 5 rounds:
> 
> test                  metric          baseline     RX metadata
> veth, 1 TCP flow      RX CPU busy     100%         66% (-34%)
> veth, 1 TCP flow      RX cycles/pkt   2027         1343 (-34%)
> cpumap, 16 TCP flows  throughput      17.6 Gbit/s  23.4 (+33%)
> cpumap, 16 TCP flows  pkts/GRO skb    3.4          7.5 (+122%)
> 
> Both veth columns reach line rate, so the gain is in CPU time. A store
> kfunc takes 2.9 to 4.9 ns. Details are in [8].
> 

Nice performance gains :-)

--Jesper

[... cut ...]
 > [1]https://lore.kernel.org/all/20260227153231.78d16b69@kernel.org/
 > 
[2]https://lore.kernel.org/all/ca38f2ed-999f-4ce1-8035-8ee9247f27f2@kernel.org/
 > 
[3]https://lore.kernel.org/all/1d5521eb-84fc-4417-a3f5-6366694d160a@kernel.org/
 > [4]https://lore.kernel.org/
 > all/175146824674.1421237.18351246421763677468.stgit@firesoul/
 > [5]https://lore.kernel.org/all/20260722113525.34092660@kernel.org/
 > [6]https://lore.kernel.org/all/20260721183256.6f49d6e1@kernel.org/
 > [7]https://lore.kernel.org/all/20260715-bpf-xdp-meta-rxcksum-
 > v5-0-623d5c0d0ab7@kernel.org/
 > [8]https://github.com/tgao-cf/skb-core-extensions-bench