From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 175C53043BE for ; Thu, 23 Oct 2025 08:58:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761209934; cv=none; b=i4b0NS9lLPNx3MLEyuw6FMZ5lfCw6WN+hAVwyx3KKZDUz1cZzGiUTCarTNVwcF3NjjwFBM9FJ3zC/0umLFbK2fXpdF4hnqzjbNfWl9YsFqbruU4bxHfFyT6Gnf28zANsjGnO2YVRhQekf/CEymRa9ZQsWMg7HrSmUnUo3l4LO/Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761209934; c=relaxed/simple; bh=7+QdE7u95F2srOVEyYJsJ3K+vNzuW01jxQiAAFCgJZQ=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=gM/4+IgfQVgxsUGYdg0UAVcCQuABl5xVWgSF+C3eK5N9R1sLsgKvXRBAgAHNdP/6Bpc9ugA2DPE2Xp2yNXnIEjXuJqHIw+H7Fq47eaSO+OXb1eHZC3zchHGtg4EJ4/w82v08DFKCfsa1LHLBpYIjKJF7hYbrXDuCToWBWITPuEE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FuKtiwkt; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FuKtiwkt" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-27ee41e0798so7859565ad.1 for ; Thu, 23 Oct 2025 01:58:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1761209932; x=1761814732; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=1RU4IedZsSTtJdpKj+rxbznt86HzuF4sa6O2zsfwDOE=; b=FuKtiwktcS/S7+3zUX0bzd8Q8g5y6mHRuzHm1yznUrcCfZ47HXJzwjmEPk9wj4t5bW NQYLWhMNGD5/RBwczLZDZ8ydsrfXu/yXwEwa5lqhbolwVo4WVmJ7P4/3HusNOcATsB7E OEAntUXDb/ZeEa/euj96kX7QDpcAtYkMAgAk3FF6efxReQIBSOaKHmW62MgD+G0vgxdy 38IFN9DuXPJulAngtsMy/+GXatrtdpAT9RYDggdVxLBvcvzIq7j5qT+OIXiXwTB8R46y YqEzK5tHMXVywrBrPXEcM7h9LsvmyS9P0YDPvCos2cRXg1XkO/VH+VNZ0SnrOywKkWyq 6leQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761209932; x=1761814732; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1RU4IedZsSTtJdpKj+rxbznt86HzuF4sa6O2zsfwDOE=; b=oJofA70CYBEw0FaXisVV0EmFA8PzhHncHRb7R8Z8OrsllHefc9FApPUa5lYNCBlmmJ g+jW3wYquAYFEJFiceQ083Oq54aR8EGkev3O+kZik0ESQrcUJQ+PSBkKRND9I3LruX2v wRS1dxxPTG84lchusUC1vztFAaD9EmdqIqh//XfejDZpLekKeIx08wI2v0WtAw4ngUTX 9oOrBQff5iQkRbq6xIZSFEyKk/CrvzAstio9CZCxs8GBW/jlg1DnlLDl3Hq7340tfwhX kaVgKTdeOfRL6S6JJ91beG8Dh6Z79gVpjH4dtHj8z7sUIi/I1k6TTszh3ef4icmOnXWM ztRQ== X-Forwarded-Encrypted: i=1; AJvYcCUwLbfsl1WR66w/gsTV/KoWiqA4Q4lxBs/V8jutekMyG2AYnGHyc4b8+jlpOALqhBDlDjWB6o8=@vger.kernel.org X-Gm-Message-State: AOJu0YxXrarFihDsslX32LnJ7GdpmElkHNWtDFWznwVASrXv0HkkA937 ovyTxhBA4HPI4G1DLNQ0mfjWG8MY3Jdvm4O/5c9RpFvOeiFs/bMQq7pt X-Gm-Gg: ASbGncsQY/EFS3OJfniuzx6exq/9CzIVS8bQsfsau0t2SxYsD6d4MqSDj7mNKk+tTHV O6kOcwAQ75CSnL2JmOFvJPR0cLfV2ujuBaehdbA8m6IEW+pgfAUv1Eib8KGEPDS7AvWvzkVedaH JVZvqb31HKzbeGhwfyXXAgqi201WI5BAtYI4yAntJE31Udf/smCzqbMqktpmIkuHMV4Zvth7Dba Eci58/a/IRstIPgkZjnhS+/faVM/PahZeqp3pYiU9u2W9jvQ9nw+lAFhW+b7KcCaJYp+oKStGND WTVV4xOob69GUgtMPhSSbWZ3AlKz1hodhrkpwBSGxwht4xtRCchX+5mZjk8FHTq+OvjRKtEoY3u 9F3AiIObuK04yfRK47+XC7ot7NvCUG2OnJqgp1ic+pAk1RKIBbg4yRbYkpvUa3Et/zaspj1ALWP +fyoUYK4TFc4Bmp2U6Qqdkj3R5UvgaZZKxHyeupIXWtgSskljxfScYM+/S+pxZrIa7fBnqNEpg X-Google-Smtp-Source: AGHT+IEVUoymZeiSVlE/Uk+XC59V5IvcJHH/cBg60zVNVyFpWSnT3ANMTmgq33TFbobFjHJXmeQC8A== X-Received: by 2002:a17:903:2990:b0:264:8a8d:92e8 with SMTP id d9443c01a7336-290ccab59b1mr303357615ad.59.1761209932109; Thu, 23 Oct 2025 01:58:52 -0700 (PDT) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b6cf4e12d6asm1448513a12.21.2025.10.23.01.58.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Oct 2025 01:58:51 -0700 (PDT) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, bjorn@kernel.org, magnus.karlsson@intel.com, maciej.fijalkowski@intel.com, jonathan.lemon@gmail.com, sdf@fomichev.me, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, joe@dama.to, willemdebruijn.kernel@gmail.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing , Alexander Lobakin Subject: [PATCH net-next] xsk: add indirect call for xsk_destruct_skb Date: Thu, 23 Oct 2025 16:58:43 +0800 Message-Id: <20251023085843.25619-1-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jason Xing Since Eric proposed an idea about adding indirect call for UDP and managed to see a huge improvement[1], the same situation can also be applied in xsk scenario. This patch adds an indirect call for xsk and helps current copy mode improve the performance by around 1% stably which was observed with IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect will be magnified. I applied this patch on top of batch xmit series[2], and was able to see <5% improvement. [1]: https://lore.kernel.org/netdev/20251006193103.2684156-2-edumazet@google.com/ [2]: https://lore.kernel.org/all/20251021131209.41491-1-kerneljasonxing@gmail.com/ Suggested-by: Alexander Lobakin Signed-off-by: Jason Xing --- include/net/xdp_sock.h | 5 +++++ net/core/skbuff.c | 8 +++++--- net/xdp/xsk.c | 2 +- 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index ce587a225661..431de372d0a0 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -125,6 +125,7 @@ struct xsk_tx_metadata_ops { int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp); int __xsk_map_redirect(struct xdp_sock *xs, struct xdp_buff *xdp); void __xsk_map_flush(struct list_head *flush_list); +void xsk_destruct_skb(struct sk_buff *skb); /** * xsk_tx_metadata_to_compl - Save enough relevant metadata information @@ -218,6 +219,10 @@ static inline void __xsk_map_flush(struct list_head *flush_list) { } +static inline void xsk_destruct_skb(struct sk_buff *skb) +{ +} + static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta, struct xsk_tx_metadata_compl *compl) { diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 5b4bc8b1c7d5..00ea38248bd6 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -81,6 +81,7 @@ #include #include #include +#include #include #include @@ -1140,12 +1141,13 @@ void skb_release_head_state(struct sk_buff *skb) if (skb->destructor) { DEBUG_NET_WARN_ON_ONCE(in_hardirq()); #ifdef CONFIG_INET - INDIRECT_CALL_3(skb->destructor, + INDIRECT_CALL_4(skb->destructor, tcp_wfree, __sock_wfree, sock_wfree, + xsk_destruct_skb, skb); #else - INDIRECT_CALL_1(skb->destructor, - sock_wfree, + INDIRECT_CALL_2(skb->destructor, + sock_wfree, xsk_destruct_skb, skb); #endif diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 7b0c68a70888..8e6ccb2f79c0 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -605,7 +605,7 @@ static u32 xsk_get_num_desc(struct sk_buff *skb) return XSKCB(skb)->num_descs; } -static void xsk_destruct_skb(struct sk_buff *skb) +void xsk_destruct_skb(struct sk_buff *skb) { struct xsk_tx_metadata_compl *compl = &skb_shinfo(skb)->xsk_meta; -- 2.41.3