[PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jason Xing]

From: Jason Xing <kernelxing@tencent.com>

Since Eric proposed an idea about adding indirect call for UDP and
managed to see a huge improvement[1], the same situation can also be
applied in xsk scenario.

This patch adds an indirect call for xsk and helps current copy mode
improve the performance by around 1% stably which was observed with
IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect
will be magnified. I applied this patch on top of batch xmit series[2],
and was able to see <5% improvement from our internal application
which is a little bit unstable though.

Use INDIRECT wrappers to keep xsk_destruct_skb static as it used to
be when the mitigation config is off.

[1]: https://lore.kernel.org/netdev/20251006193103.2684156-2-edumazet@google.com/
[2]: https://lore.kernel.org/all/20251021131209.41491-1-kerneljasonxing@gmail.com/

Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Signed-off-by: Jason Xing <kernelxing@tencent.com>
---
v2
Link: https://lore.kernel.org/all/20251023085843.25619-1-kerneljasonxing@gmail.com/
1. use INDIRECT helpers (Alexander)
---
 include/net/xdp_sock.h | 7 +++++++
 net/core/skbuff.c      | 8 +++++---
 net/xdp/xsk.c          | 3 ++-
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h
index ce587a225661..23e8861e8b25 100644
--- a/include/net/xdp_sock.h
+++ b/include/net/xdp_sock.h
@@ -125,6 +125,7 @@ struct xsk_tx_metadata_ops {
 int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp);
 int __xsk_map_redirect(struct xdp_sock *xs, struct xdp_buff *xdp);
 void __xsk_map_flush(struct list_head *flush_list);
+INDIRECT_CALLABLE_DECLARE(void xsk_destruct_skb(struct sk_buff *));
 
 /**
  *  xsk_tx_metadata_to_compl - Save enough relevant metadata information
@@ -218,6 +219,12 @@ static inline void __xsk_map_flush(struct list_head *flush_list)
 {
 }
 
+#ifdef CONFIG_MITIGATION_RETPOLINE
+static inline void xsk_destruct_skb(struct sk_buff *skb)
+{
+}
+#endif
+
 static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta,
 					    struct xsk_tx_metadata_compl *compl)
 {
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 5b4bc8b1c7d5..00ea38248bd6 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -81,6 +81,7 @@
 #include <net/page_pool/helpers.h>
 #include <net/psp/types.h>
 #include <net/dropreason.h>
+#include <net/xdp_sock.h>
 
 #include <linux/uaccess.h>
 #include <trace/events/skb.h>
@@ -1140,12 +1141,13 @@ void skb_release_head_state(struct sk_buff *skb)
 	if (skb->destructor) {
 		DEBUG_NET_WARN_ON_ONCE(in_hardirq());
 #ifdef CONFIG_INET
-		INDIRECT_CALL_3(skb->destructor,
+		INDIRECT_CALL_4(skb->destructor,
 				tcp_wfree, __sock_wfree, sock_wfree,
+				xsk_destruct_skb,
 				skb);
 #else
-		INDIRECT_CALL_1(skb->destructor,
-				sock_wfree,
+		INDIRECT_CALL_2(skb->destructor,
+				sock_wfree, xsk_destruct_skb,
 				skb);
 
 #endif
diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index 7b0c68a70888..9451b090db16 100644
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -605,7 +605,8 @@ static u32 xsk_get_num_desc(struct sk_buff *skb)
 	return XSKCB(skb)->num_descs;
 }
 
-static void xsk_destruct_skb(struct sk_buff *skb)
+INDIRECT_CALLABLE_SCOPE
+void xsk_destruct_skb(struct sk_buff *skb)
 {
 	struct xsk_tx_metadata_compl *compl = &skb_shinfo(skb)->xsk_meta;
 
-- 
2.41.3

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Alexander Lobakin]

From: Jason Xing <kerneljasonxing@gmail.com>
Date: Sun, 26 Oct 2025 22:58:24 +0800

> From: Jason Xing <kernelxing@tencent.com>
> 
> Since Eric proposed an idea about adding indirect call for UDP and
> managed to see a huge improvement[1], the same situation can also be
> applied in xsk scenario.
> 
> This patch adds an indirect call for xsk and helps current copy mode
> improve the performance by around 1% stably which was observed with
> IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect
> will be magnified. I applied this patch on top of batch xmit series[2],
> and was able to see <5% improvement from our internal application
> which is a little bit unstable though.
> 
> Use INDIRECT wrappers to keep xsk_destruct_skb static as it used to
> be when the mitigation config is off.
> 
> [1]: https://lore.kernel.org/netdev/20251006193103.2684156-2-edumazet@google.com/
> [2]: https://lore.kernel.org/all/20251021131209.41491-1-kerneljasonxing@gmail.com/
> 
> Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>

Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com>

> ---
> v2
> Link: https://lore.kernel.org/all/20251023085843.25619-1-kerneljasonxing@gmail.com/
> 1. use INDIRECT helpers (Alexander)
> ---
>  include/net/xdp_sock.h | 7 +++++++
>  net/core/skbuff.c      | 8 +++++---
>  net/xdp/xsk.c          | 3 ++-
>  3 files changed, 14 insertions(+), 4 deletions(-)
Thanks,
Olek

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jakub Kicinski]

On Sun, 26 Oct 2025 22:58:24 +0800 Jason Xing wrote:
> Since Eric proposed an idea about adding indirect call for UDP and
> managed to see a huge improvement[1], the same situation can also be
> applied in xsk scenario.
> 
> This patch adds an indirect call for xsk and helps current copy mode
> improve the performance by around 1% stably which was observed with
> IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect
> will be magnified. I applied this patch on top of batch xmit series[2],
> and was able to see <5% improvement from our internal application
> which is a little bit unstable though.
> 
> Use INDIRECT wrappers to keep xsk_destruct_skb static as it used to
> be when the mitigation config is off.

FTR I don't think this code complication is worth "stable 1%" win
on the slowpath. But maybe it's just me so I'll let Paolo decide.

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jason Xing]

On Wed, Oct 29, 2025 at 8:30 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Sun, 26 Oct 2025 22:58:24 +0800 Jason Xing wrote:
> > Since Eric proposed an idea about adding indirect call for UDP and
> > managed to see a huge improvement[1], the same situation can also be
> > applied in xsk scenario.
> >
> > This patch adds an indirect call for xsk and helps current copy mode
> > improve the performance by around 1% stably which was observed with
> > IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect
> > will be magnified. I applied this patch on top of batch xmit series[2],
> > and was able to see <5% improvement from our internal application
> > which is a little bit unstable though.
> >
> > Use INDIRECT wrappers to keep xsk_destruct_skb static as it used to
> > be when the mitigation config is off.
>
> FTR I don't think this code complication is worth "stable 1%" win
> on the slowpath. But maybe it's just me so I'll let Paolo decide.

For xdp or af_xdp, the best practice is to turn off mitigation since
it has a noticeable impact. But in some cases we still keep this
config on for safety. This patch is one of small optimizations that
mitigate the impact because I'm trying to optimize the af_xdp in every
possible aspect. Besides, adding this one will not disrupt the benefit
which Eric brought in his commit. Please review.

Thanks,
Jason

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Paolo Abeni]

On 10/26/25 3:58 PM, Jason Xing wrote:
> From: Jason Xing <kernelxing@tencent.com>
> 
> Since Eric proposed an idea about adding indirect call for UDP and

Minor nit:                          ^^^^^^

either 'remove an indirect call' or 'adding indirect call wrappers'

> managed to see a huge improvement[1], the same situation can also be
> applied in xsk scenario.
> 
> This patch adds an indirect call for xsk and helps current copy mode
> improve the performance by around 1% stably which was observed with
> IXGBE at 10Gb/sec loaded. 

If I follow the conversation correctly, Jakub's concern is mostly about
this change affecting only the copy mode.

Out of sheer ignorance on my side is not clear how frequent that
scenario is. AFAICS, applications could always do zero-copy with proper
setup, am I correct?!?

In such case I think this patch is not worth.

Otherwise, please describe/explain the real-use case needing the copy mode.

Thanks,

Paolo

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jason Xing]

On Thu, Oct 30, 2025 at 6:15 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> On 10/26/25 3:58 PM, Jason Xing wrote:
> > From: Jason Xing <kernelxing@tencent.com>
> >
> > Since Eric proposed an idea about adding indirect call for UDP and
>
> Minor nit:                          ^^^^^^
>
> either 'remove an indirect call' or 'adding indirect call wrappers'

Oh, right!

>
> > managed to see a huge improvement[1], the same situation can also be
> > applied in xsk scenario.
> >
> > This patch adds an indirect call for xsk and helps current copy mode
> > improve the performance by around 1% stably which was observed with
> > IXGBE at 10Gb/sec loaded.
>
> If I follow the conversation correctly, Jakub's concern is mostly about
> this change affecting only the copy mode.

Copy mode is worth optimization really. Please see below.

>
> Out of sheer ignorance on my side is not clear how frequent that
> scenario is. AFAICS, applications could always do zero-copy with proper
> setup, am I correct?!?

In my env, around 2,000,000 packets are sent per second which in turn
means the destruction function gets called the same number of times.

>
> In such case I think this patch is not worth.
>
> Otherwise, please describe/explain the real-use case needing the copy mode.

I gave a detailed explanation in the cover letter [1]. The real use
case from my side is to support the virtio_net and veth scenario. This
topic has been discussed in the version 1 of [1] and Jesper also
acknowledged this point. I also noticed that there remain some
physical nics that haven't supported zerocopy mode yet and some of
them[2] are still in progress.

[1]: https://lore.kernel.org/all/20251021131209.41491-1-kerneljasonxing@gmail.com/
[2]: https://lore.kernel.org/all/20251014105613.2808674-1-m-malladi@ti.com/

Thanks,
Jason

>
> Thanks,
>
> Paolo
>

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Alexander Lobakin]

From: Paolo Abeni <pabeni@redhat.com>
Date: Thu, 30 Oct 2025 11:15:18 +0100

> On 10/26/25 3:58 PM, Jason Xing wrote:
>> From: Jason Xing <kernelxing@tencent.com>
>>
>> Since Eric proposed an idea about adding indirect call for UDP and
> 
> Minor nit:                          ^^^^^^
> 
> either 'remove an indirect call' or 'adding indirect call wrappers'
> 
>> managed to see a huge improvement[1], the same situation can also be
>> applied in xsk scenario.
>>
>> This patch adds an indirect call for xsk and helps current copy mode
>> improve the performance by around 1% stably which was observed with
>> IXGBE at 10Gb/sec loaded. 
> 
> If I follow the conversation correctly, Jakub's concern is mostly about
> this change affecting only the copy mode.
> 
> Out of sheer ignorance on my side is not clear how frequent that
> scenario is. AFAICS, applications could always do zero-copy with proper
> setup, am I correct?!?

It is correct only when the target driver implements zero-copy
driver-side XSk. While it's true for modern Ethernet drivers for real
NICs, "virtual" drivers like virtio-net, veth etc. usually don't have it.
It's not as common usecase as using XSk on real NICs, but still valid
and widely used.
For example, virtio-net has a shortcut where it can send XSk skbs
without copying everything from the userspace (the no-linear-head mode),
so there generic XSk mode is way faster there than usually.

Also worth noting that there were attempts to introduce driver-side XSk
zerocopy for virtio-net (and probably veth, I don't remember) on LKML,
but haven't been upstreamed yet.

> 
> In such case I think this patch is not worth.
> 
> Otherwise, please describe/explain the real-use case needing the copy mode.
> 
> Thanks,
> 
> Paolo

Thanks,
Olek

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jason Xing]

On Thu, Oct 30, 2025 at 7:00 PM Alexander Lobakin
<aleksander.lobakin@intel.com> wrote:
>
> From: Paolo Abeni <pabeni@redhat.com>
> Date: Thu, 30 Oct 2025 11:15:18 +0100
>
> > On 10/26/25 3:58 PM, Jason Xing wrote:
> >> From: Jason Xing <kernelxing@tencent.com>
> >>
> >> Since Eric proposed an idea about adding indirect call for UDP and
> >
> > Minor nit:                          ^^^^^^
> >
> > either 'remove an indirect call' or 'adding indirect call wrappers'
> >
> >> managed to see a huge improvement[1], the same situation can also be
> >> applied in xsk scenario.
> >>
> >> This patch adds an indirect call for xsk and helps current copy mode
> >> improve the performance by around 1% stably which was observed with
> >> IXGBE at 10Gb/sec loaded.
> >
> > If I follow the conversation correctly, Jakub's concern is mostly about
> > this change affecting only the copy mode.
> >
> > Out of sheer ignorance on my side is not clear how frequent that
> > scenario is. AFAICS, applications could always do zero-copy with proper
> > setup, am I correct?!?
>
> It is correct only when the target driver implements zero-copy
> driver-side XSk. While it's true for modern Ethernet drivers for real
> NICs, "virtual" drivers like virtio-net, veth etc. usually don't have it.
> It's not as common usecase as using XSk on real NICs, but still valid
> and widely used.
> For example, virtio-net has a shortcut where it can send XSk skbs
> without copying everything from the userspace (the no-linear-head mode),
> so there generic XSk mode is way faster there than usually.
>
> Also worth noting that there were attempts to introduce driver-side XSk
> zerocopy for virtio-net (and probably veth, I don't remember) on LKML,
> but haven't been upstreamed yet.

Thanks for the added context. One minor thing I need to say is that
virtio_net has zc mode but it requires the host to support a series of
features which means at a hyperscaler it's really hard to ask hosts to
upgrade their kernels and on the contrary it's effortless to upgrade
VMs. Indeed, veth doesn't have zc mode.

Thanks,
Jason

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jakub Kicinski]

On Thu, 30 Oct 2025 11:59:58 +0100 Alexander Lobakin wrote:
> >> managed to see a huge improvement[1], the same situation can also be
> >> applied in xsk scenario.
> >>
> >> This patch adds an indirect call for xsk and helps current copy mode
> >> improve the performance by around 1% stably which was observed with
> >> IXGBE at 10Gb/sec loaded.   
> >
> > If I follow the conversation correctly, Jakub's concern is mostly about
> > this change affecting only the copy mode.
> >
> > Out of sheer ignorance on my side is not clear how frequent that
> > scenario is. AFAICS, applications could always do zero-copy with proper
> > setup, am I correct?!?  
>
> It is correct only when the target driver implements zero-copy
> driver-side XSk. While it's true for modern Ethernet drivers for real
> NICs, "virtual" drivers like virtio-net, veth etc. usually don't have it.
> It's not as common usecase as using XSk on real NICs, but still valid
> and widely used.

To be clear my main concern is that the XDP<>skb conversions are 
an endless source of bugs and complexity. We have one fix for XDP->skb
on the list from Maciej and another for AF_XDP from Fernando which
tried to create an XDP skb_ext. We are digging a deeper and deeper
hole with all this fallback stuff, and it will affect performance
of both normal skb and XDP paths. Optimizing AF_XDP fallback is
shortsighted.

Re: [PATCH net-next v2] xsk: add indirect call for xsk_destruct_skb

[Jason Xing]

On Thu, Oct 30, 2025 at 11:55 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 30 Oct 2025 11:59:58 +0100 Alexander Lobakin wrote:
> > >> managed to see a huge improvement[1], the same situation can also be
> > >> applied in xsk scenario.
> > >>
> > >> This patch adds an indirect call for xsk and helps current copy mode
> > >> improve the performance by around 1% stably which was observed with
> > >> IXGBE at 10Gb/sec loaded.
> > >
> > > If I follow the conversation correctly, Jakub's concern is mostly about
> > > this change affecting only the copy mode.
> > >
> > > Out of sheer ignorance on my side is not clear how frequent that
> > > scenario is. AFAICS, applications could always do zero-copy with proper
> > > setup, am I correct?!?
> >
> > It is correct only when the target driver implements zero-copy
> > driver-side XSk. While it's true for modern Ethernet drivers for real
> > NICs, "virtual" drivers like virtio-net, veth etc. usually don't have it.
> > It's not as common usecase as using XSk on real NICs, but still valid
> > and widely used.
>
> To be clear my main concern is that the XDP<>skb conversions are
> an endless source of bugs and complexity. We have one fix for XDP->skb
> on the list from Maciej and another for AF_XDP from Fernando which
> tried to create an XDP skb_ext. We are digging a deeper and deeper
> hole with all this fallback stuff, and it will affect performance
> of both normal skb and XDP paths. Optimizing AF_XDP fallback is
> shortsighted.

Sorry, I have to say I'm against the whole point :(

1. Every feature has bugs. Just taking a look at the history of those
well-known features like LRO/GRO/GSO, you must be aware that so many
nasty bugs have been found.
2. The so-called fallback is not something that nobody uses and nobody
cares. It has its right valid position and has actually been used by
many applications. Searching in the github, you can find many related
supports in their own repos. Being generic doesn't mean being
shortsighted and useless. There are a few so-called fallback features
other than this, like SMC-R for RDMA.
3. Back to the main point we've discussed, there are many cases that
don't support zerocopy mode and I don't see anyone who volunteers to
complete the rest of them. So currently and at least in these recent
years I can predict that copy mode can be used widely.
4. I'm working on something that is really beneficial to me and some
people like me who need to accelerate the transmission in copy mode.
There's still a huge gap between zc and copy mode, but sadly we have
no chance to use zc mode. Optimizing the copy mode then becomes my
primary job after work.

I fully understand what you meant here. But the fact is the fact...

As to the maintenance, I believe we're able to cover this area and
make it better and better in every aspect. I really hope we don't get
stuck in these minor optimizations, shall we? I have more copy mode
patches on the way... I really appreciate that we can merge it and
move on. Many thanks here!

Thanks,
Jason