Linux Documentation
 help / color / mirror / Atom feed
From: Tianyi Gao <tianyi@cloudflare.com>
To: netdev@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
	Paolo Abeni <pabeni@redhat.com>,
	Eric Dumazet <edumazet@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
	"Simon Horman" <horms@kernel.org>,
	"David Keogh" <dkeogh@amazon.com>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"David Woodhouse" <dwmw2@infradead.org>,
	"Dragos Tatulea" <dtatulea@nvidia.com>,
	"Nimrod Oren" <noren@nvidia.com>,
	"Alasdair McWilliam" <alasdair@mcwilliam.dev>,
	"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
	"Maciej Fijalkowski" <maciej.fijalkowski@intel.com>,
	"Stanislav Fomichev" <sdf@fomichev.me>,
	"Magnus Karlsson" <magnus.karlsson@intel.com>,
	"David Ahern" <dsahern@kernel.org>,
	"Ido Schimmel" <idosch@nvidia.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Donald Hunter" <donald.hunter@gmail.com>,
	"Andrew Lunn" <andrew+netdev@lunn.ch>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Shuah Khan" <skhan@linuxfoundation.org>,
	"Shuah Khan" <shuah@kernel.org>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	"Breno Leitao" <leitao@debian.org>,
	"Asbjørn Sloth Tønnesen" <ast@fiberby.net>,
	"Danielle Ratson" <danieller@nvidia.com>,
	"Kuniyuki Iwashima" <kuniyu@google.com>,
	bpf@vger.kernel.org, linux-doc@vger.kernel.org,
	linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com,
	nh-open-source@amazon.com,
	"Stephen Hemminger" <stephen@networkplumber.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>,
	"Remy D. Farley" <one-d-wide@protonmail.com>,
	"Amery Hung" <ameryhung@gmail.com>,
	"Andrii Nakryiko" <andrii@kernel.org>,
	"Andy Roulin" <aroulin@nvidia.com>,
	"Anil Kaushik" <anilkaushikwireless@gmail.com>,
	"Demos Ra" <demosthenes.ra@gmail.com>,
	"Eduard Zingerman" <eddyz87@gmail.com>,
	"Emil Tsalapatis" <emil@etsalapatis.com>,
	"Hangbin Liu" <hangbin.liu@linux.dev>,
	"Ihor Solodrai" <ihor.solodrai@linux.dev>,
	"Jiayuan Chen" <jiayuan.chen@linux.dev>,
	"Jiri Olsa" <jolsa@kernel.org>, "Joe Damato" <joe@dama.to>,
	"Kumar Kartikeya Dwivedi" <memxor@gmail.com>,
	"Li RongQing" <lirongqing@baidu.com>,
	"Louis Scalbert" <louis.scalbert@6wind.com>,
	"Marc Kleine-Budde" <mkl@pengutronix.de>,
	"Martin KaFai Lau" <martin.lau@linux.dev>,
	"Martin Karsten" <mkarsten@uwaterloo.ca>,
	"Matt Bobrowski" <mattbobrowski@google.com>,
	"Michael Bommarito" <michael.bommarito@gmail.com>,
	"Nikolay Aleksandrov" <razor@blackwall.org>,
	"Runyu Xiao" <runyu.xiao@seu.edu.cn>,
	"Song Liu" <song@kernel.org>,
	"Thorsten Leemhuis" <linux@leemhuis.info>,
	"Wang Zhan" <wang.zhan@smartx.com>,
	"Yonghong Song" <yonghong.song@linux.dev>,
	"Zhou Qiuyue" <n0ps1edzz@gmail.com>,
	"Tianyi Gao" <tianyi.gao@linux.dev>,
	"Tianyi Gao" <tianyi@cloudflare.com>
Subject: [PATCH net-next v2 03/11] net: set tx_extra_headroom of the peer on XDP attach
Date: Thu, 19 Nov 2026 01:38:22 +0000	[thread overview]
Message-ID: <20261119013830.36935-4-tianyi@cloudflare.com> (raw)
In-Reply-To: <20261119013830.36935-1-tianyi@cloudflare.com>

XDP needs XDP_PACKET_HEADROOM (256 bytes), but locally generated packets
only get needed_headroom. XDP on the peer of a veth or netkit device
then copies every packet. Generic XDP copies it in
netif_skb_check_for_xdp(), and native XDP on veth in
veth_convert_skb_to_xdp_buff().

While the peer has an XDP program, set tx_extra_headroom of the device
to XDP_PACKET_HEADROOM minus needed_headroom.
netdev_update_tx_extra_headroom() recalculates it when an XDP program
is attached or detached, also when an upper device such as a bond
propagates its program to the peer with netif_xdp_propagate(). It does
not recalculate it:

- for IFLA_NETKIT_HEADROOM, which cannot change after the netkit
  device is created,
- in dev_xdp_uninstall(), because veth and netkit unregister both ends
  together,
- when bridge or openvswitch change needed_headroom later.

openvswitch changes needed_headroom under ovs_mutex, without RTNL.
There are three ways to handle a later change of needed_headroom:

- Recalculate tx_extra_headroom in netdev_set_rx_headroom(). This
  races with an XDP attach or detach under RTNL. The last write can
  leave tx_extra_headroom at 0 while XDP is attached.
- Do not subtract needed_headroom. This always reserves needed_headroom
  bytes more than XDP needs.
- Subtract needed_headroom only when tx_extra_headroom is recalculated.

This patch uses the last one, because it does not race and reserves
only what XDP needs. If needed_headroom shrinks later, XDP on the peer
copies packets again until the next recalculation.

tx_extra_headroom is used rather than needed_headroom, because raising
needed_headroom can regress the performance of other paths.

Signed-off-by: Tianyi Gao <tianyi@cloudflare.com>
---
For native XDP on veth, this patch removes one of two copy
conditions, the missing headroom. The other condition is a head that is
not a page fragment. A follow-up veth series will fix that.

v2:
- New patch, replaces IFLA_TX_EXTRA_HEADROOM for XDP on the peer
  (Stephen).
---
 Documentation/networking/netdevices.rst | 11 +++++++
 include/linux/netdevice.h               |  1 +
 net/core/dev.c                          | 38 +++++++++++++++++++++++++
 3 files changed, 50 insertions(+)

diff --git a/Documentation/networking/netdevices.rst b/Documentation/networking/netdevices.rst
index f593b242da9c..53ca8439f73b 100644
--- a/Documentation/networking/netdevices.rst
+++ b/Documentation/networking/netdevices.rst
@@ -454,6 +454,17 @@ devices do not inherit it. It is not part of ``LL_RESERVED_SPACE()``, so a
 packet that reaches the device with less headroom, such as a forwarded
 packet, is not reallocated because of it.
 
+The kernel sets ``tx_extra_headroom`` of a device to the sum of these
+values:
+
+* ``XDP_PACKET_HEADROOM`` (256 bytes) minus ``needed_headroom``, when the
+  peer of a veth or netkit device has an XDP program
+
+When the peer has an XDP program, locally generated UDP, raw and
+AF_PACKET packets and AF_XDP packets in copy mode have enough headroom
+for it. ``tx_extra_headroom`` is not recalculated when a bridge or
+openvswitch changes ``needed_headroom`` later.
+
 NETDEV_INTERNAL symbol namespace
 ================================
 
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 392e6fc81e34..786b63f21d4f 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -4484,6 +4484,7 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf);
 int dev_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf);
 u8 dev_xdp_sb_prog_count(struct net_device *dev);
 u32 dev_xdp_prog_id(struct net_device *dev, enum bpf_xdp_mode mode);
+void netdev_update_tx_extra_headroom(struct net_device *dev);
 
 u32 dev_get_min_mp_channel_count(const struct net_device *dev);
 
diff --git a/net/core/dev.c b/net/core/dev.c
index 85b7c61015c7..6e053f612d72 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -10666,6 +10666,7 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
 	if (old_prog)
 		bpf_prog_put(old_prog);
 
+	netdev_update_tx_extra_headroom(dev);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(netif_xdp_propagate);
@@ -10677,6 +10678,41 @@ u32 dev_xdp_prog_id(struct net_device *dev, enum bpf_xdp_mode mode)
 	return prog ? prog->aux->id : 0;
 }
 
+static void __netdev_update_tx_extra_headroom(struct net_device *dev,
+					      struct net_device *peer)
+{
+	unsigned int needed_headroom = READ_ONCE(dev->needed_headroom);
+	unsigned int tx_extra_headroom = 0;
+
+	if (peer && dev_xdp_prog_count(peer) &&
+	    needed_headroom < XDP_PACKET_HEADROOM)
+		tx_extra_headroom = XDP_PACKET_HEADROOM - needed_headroom;
+
+	WRITE_ONCE(dev->tx_extra_headroom, tx_extra_headroom);
+}
+
+/**
+ *	netdev_update_tx_extra_headroom - recalculate tx_extra_headroom
+ *	@dev: the device to check
+ *
+ *	Recalculate tx_extra_headroom of @dev and its peer. Should be called
+ *	after conditions that influence tx_extra_headroom might have changed.
+ *	The caller must hold the RTNL lock.
+ */
+void netdev_update_tx_extra_headroom(struct net_device *dev)
+{
+	struct net_device *peer = NULL;
+
+	rcu_read_lock();
+	if (dev->netdev_ops->ndo_get_peer_dev)
+		peer = dev->netdev_ops->ndo_get_peer_dev(dev);
+	__netdev_update_tx_extra_headroom(dev, peer);
+	if (peer)
+		__netdev_update_tx_extra_headroom(peer, dev);
+	rcu_read_unlock();
+}
+EXPORT_SYMBOL_GPL(netdev_update_tx_extra_headroom);
+
 static int dev_xdp_install(struct net_device *dev, enum bpf_xdp_mode mode,
 			   bpf_op_t bpf_op, struct netlink_ext_ack *extack,
 			   u32 flags, struct bpf_prog *prog)
@@ -10887,6 +10923,7 @@ static int dev_xdp_attach(struct net_device *dev, struct netlink_ext_ack *extack
 	if (cur_prog)
 		bpf_prog_put(cur_prog);
 
+	netdev_update_tx_extra_headroom(dev);
 	return 0;
 }
 
@@ -10913,6 +10950,7 @@ static int dev_xdp_detach_link(struct net_device *dev,
 	bpf_op = dev_xdp_bpf_op(dev, mode);
 	WARN_ON(dev_xdp_install(dev, mode, bpf_op, NULL, 0, NULL));
 	dev_xdp_set_link(dev, mode, NULL);
+	netdev_update_tx_extra_headroom(dev);
 	return 0;
 }
 
-- 
2.56.0


  parent reply	other threads:[~2026-11-19  1:38 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  1:38 [PATCH net-next v2 00/11] net: extra TX headroom for local egress Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 01/11] net: add tx_extra_headroom and netdev_get_alloc_headroom() Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 02/11] net: reserve extra TX headroom for local egress Tianyi Gao
2026-11-19  1:38 ` Tianyi Gao [this message]
2026-11-19  1:38 ` [PATCH net-next v2 04/11] selftests: net: add flag to send a frame in xdp_helper Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 05/11] selftests: net: add tx_extra_headroom test Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 06/11] bpf, tcx: Set tx_extra_headroom from a decl tag on egress programs Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 07/11] selftests: net: tx_extra_headroom: test a push at TC egress Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 08/11] selftests/bpf: Test the tx_extra_headroom decl tag Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 09/11] veth: add VETH_INFO_TX_EXTRA_HEADROOM Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 10/11] netlink: specs: rt-link: add veth attributes Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 11/11] selftests: net: tx_extra_headroom: test the veth attribute Tianyi Gao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119013830.36935-4-tianyi@cloudflare.com \
    --to=tianyi@cloudflare.com \
    --cc=alasdair@mcwilliam.dev \
    --cc=ameryhung@gmail.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=andrii@kernel.org \
    --cc=anilkaushikwireless@gmail.com \
    --cc=aroulin@nvidia.com \
    --cc=ast@fiberby.net \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=corbet@lwn.net \
    --cc=daniel@iogearbox.net \
    --cc=danieller@nvidia.com \
    --cc=davem@davemloft.net \
    --cc=demosthenes.ra@gmail.com \
    --cc=dkeogh@amazon.com \
    --cc=donald.hunter@gmail.com \
    --cc=dsahern@kernel.org \
    --cc=dtatulea@nvidia.com \
    --cc=dwmw2@infradead.org \
    --cc=eddyz87@gmail.com \
    --cc=edumazet@kernel.org \
    --cc=emil@etsalapatis.com \
    --cc=gustavoars@kernel.org \
    --cc=hangbin.liu@linux.dev \
    --cc=hawk@kernel.org \
    --cc=horms@kernel.org \
    --cc=idosch@nvidia.com \
    --cc=ihor.solodrai@linux.dev \
    --cc=jiayuan.chen@linux.dev \
    --cc=joe@dama.to \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kernel-team@cloudflare.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=leitao@debian.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux@leemhuis.info \
    --cc=lirongqing@baidu.com \
    --cc=louis.scalbert@6wind.com \
    --cc=maciej.fijalkowski@intel.com \
    --cc=magnus.karlsson@intel.com \
    --cc=martin.lau@linux.dev \
    --cc=mattbobrowski@google.com \
    --cc=memxor@gmail.com \
    --cc=michael.bommarito@gmail.com \
    --cc=mkarsten@uwaterloo.ca \
    --cc=mkl@pengutronix.de \
    --cc=n0ps1edzz@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=nh-open-source@amazon.com \
    --cc=noren@nvidia.com \
    --cc=one-d-wide@protonmail.com \
    --cc=pabeni@redhat.com \
    --cc=razor@blackwall.org \
    --cc=rdunlap@infradead.org \
    --cc=runyu.xiao@seu.edu.cn \
    --cc=sdf@fomichev.me \
    --cc=shuah@kernel.org \
    --cc=skhan@linuxfoundation.org \
    --cc=song@kernel.org \
    --cc=stephen@networkplumber.org \
    --cc=tianyi.gao@linux.dev \
    --cc=wang.zhan@smartx.com \
    --cc=willemdebruijn.kernel@gmail.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox