From: Tianyi Gao <tianyi@cloudflare.com>
To: netdev@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
Paolo Abeni <pabeni@redhat.com>,
Eric Dumazet <edumazet@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
"Simon Horman" <horms@kernel.org>,
"David Keogh" <dkeogh@amazon.com>,
"Jesper Dangaard Brouer" <hawk@kernel.org>,
"Daniel Borkmann" <daniel@iogearbox.net>,
"David Woodhouse" <dwmw2@infradead.org>,
"Dragos Tatulea" <dtatulea@nvidia.com>,
"Nimrod Oren" <noren@nvidia.com>,
"Alasdair McWilliam" <alasdair@mcwilliam.dev>,
"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
"Maciej Fijalkowski" <maciej.fijalkowski@intel.com>,
"Stanislav Fomichev" <sdf@fomichev.me>,
"Magnus Karlsson" <magnus.karlsson@intel.com>,
"David Ahern" <dsahern@kernel.org>,
"Ido Schimmel" <idosch@nvidia.com>,
"Alexei Starovoitov" <ast@kernel.org>,
"John Fastabend" <john.fastabend@gmail.com>,
"Donald Hunter" <donald.hunter@gmail.com>,
"Andrew Lunn" <andrew+netdev@lunn.ch>,
"Jonathan Corbet" <corbet@lwn.net>,
"Shuah Khan" <skhan@linuxfoundation.org>,
"Shuah Khan" <shuah@kernel.org>,
"Randy Dunlap" <rdunlap@infradead.org>,
"Breno Leitao" <leitao@debian.org>,
"Asbjørn Sloth Tønnesen" <ast@fiberby.net>,
"Danielle Ratson" <danieller@nvidia.com>,
"Kuniyuki Iwashima" <kuniyu@google.com>,
bpf@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com,
nh-open-source@amazon.com,
"Stephen Hemminger" <stephen@networkplumber.org>,
"Gustavo A. R. Silva" <gustavoars@kernel.org>,
"Remy D. Farley" <one-d-wide@protonmail.com>,
"Amery Hung" <ameryhung@gmail.com>,
"Andrii Nakryiko" <andrii@kernel.org>,
"Andy Roulin" <aroulin@nvidia.com>,
"Anil Kaushik" <anilkaushikwireless@gmail.com>,
"Demos Ra" <demosthenes.ra@gmail.com>,
"Eduard Zingerman" <eddyz87@gmail.com>,
"Emil Tsalapatis" <emil@etsalapatis.com>,
"Hangbin Liu" <hangbin.liu@linux.dev>,
"Ihor Solodrai" <ihor.solodrai@linux.dev>,
"Jiayuan Chen" <jiayuan.chen@linux.dev>,
"Jiri Olsa" <jolsa@kernel.org>, "Joe Damato" <joe@dama.to>,
"Kumar Kartikeya Dwivedi" <memxor@gmail.com>,
"Li RongQing" <lirongqing@baidu.com>,
"Louis Scalbert" <louis.scalbert@6wind.com>,
"Marc Kleine-Budde" <mkl@pengutronix.de>,
"Martin KaFai Lau" <martin.lau@linux.dev>,
"Martin Karsten" <mkarsten@uwaterloo.ca>,
"Matt Bobrowski" <mattbobrowski@google.com>,
"Michael Bommarito" <michael.bommarito@gmail.com>,
"Nikolay Aleksandrov" <razor@blackwall.org>,
"Runyu Xiao" <runyu.xiao@seu.edu.cn>,
"Song Liu" <song@kernel.org>,
"Thorsten Leemhuis" <linux@leemhuis.info>,
"Wang Zhan" <wang.zhan@smartx.com>,
"Yonghong Song" <yonghong.song@linux.dev>,
"Zhou Qiuyue" <n0ps1edzz@gmail.com>,
"Tianyi Gao" <tianyi.gao@linux.dev>,
"Tianyi Gao" <tianyi@cloudflare.com>
Subject: [PATCH net-next v2 00/11] net: extra TX headroom for local egress
Date: Thu, 19 Nov 2026 01:38:19 +0000 [thread overview]
Message-ID: <20261119013830.36935-1-tianyi@cloudflare.com> (raw)
This is v2 of the series that David Keogh posted as an RFC [1].
v1 is [2].
Locally generated UDP, raw and AF_PACKET packets, and AF_XDP packets in
copy mode, get only the headroom that needed_headroom asks for. If an
XDP or TC program needs more, the skb is reallocated or copied for every
packet. This happens in three cases:
- An XDP program on a veth peer needs XDP_PACKET_HEADROOM (256 bytes).
The peer copies the packet.
- A program at TC egress adds headers. The skb is reallocated.
- A namespace sends through a veth pair, and a program at TC egress of
another device, such as the host uplink, adds headers. The skb is
reallocated on that device.
(patches 1-2, from David Keogh) This series adds tx_extra_headroom to
struct net_device. It is added only where a locally generated skb is
allocated.
Each case sets tx_extra_headroom in its own way, to keep the new UAPI
minimal:
- (patches 3-5) The kernel sets it when XDP is attached to the peer of
a veth or netkit device. New skbs on the device then reserve enough
headroom for XDP_PACKET_HEADROOM.
- (patches 6-8) Add a BTF decl tag "tx_extra_headroom:<bytes>" as
UAPI. A tcx egress program declares it, and the verifier reads it at
load.
- (patches 9-11) Add a VETH_INFO_TX_EXTRA_HEADROOM UAPI for headers that
another device pushes, such as the host uplink. It is like
IFLA_NETKIT_HEADROOM for netkit [3]. The CNI plugin sets it at veth
creation, as Cilium does with netkit [4].
tx_extra_headroom is the sum of the extra headroom that each case
requires, limited to 512 bytes. For example, a program at TC egress
pushes a header, and XDP on the peer then still needs
XDP_PACKET_HEADROOM in front of it. The kernel recalculates
tx_extra_headroom when XDP or a tcx program is attached or detached.
For all three cases, one option is to raise needed_headroom instead.
This causes a regression for TCP and forwarding, because
needed_headroom is also the minimum that ip_finish_output2() and
ip_forward() require.
Patch 3 depends on commit f564749b3b26 ("net: record XDP programs
propagated to lower devices"), which is only in net-next, so the series
targets net-next. Patches 6-8 change tcx and the verifier. An Ack from
the BPF maintainers on them would be appreciated.
Performance:
All tests run on a Xeon E5-2640 v4 with a ConnectX-4 Lx NIC, on
net-next at d6816d9a93b6. Details of the artifacts are in [5].
The first table shows the regression when needed_headroom is raised
to 256 on both ends of a veth pair. XDP runs on the namespace side:
test net-next needed_headroom 256
netns sends TCP to host 32.1 Gbit/s 30.8 Gbit/s
host forwards 1400B UDP into netns 0.583 Mpps 0.366 Mpps
The second table shows the gain from tx_extra_headroom in all three
cases, with 1400-byte UDP payloads. All numbers come from one kernel
with this series applied, with tx_extra_headroom off and on. Between
two kernels, the code layout alone changed the rate by up to 13%.
Off uses a benchmark-only patch for the XDP case [5], the push program
without its decl tag, and a veth pair without
VETH_INFO_TX_EXTRA_HEADROOM:
test off on
host to netns, generic XDP_PASS in netns 0.237 Mpps 0.283 Mpps
host to netns, generic XDP_DROP in netns 0.386 Mpps 0.465 Mpps
host to netns, 20-byte push at TC egress 0.385 Mpps 0.468 Mpps
netns to uplink, 20-byte push on the uplink 0.190 Mpps 0.219 Mpps
Out of scope:
- Auto-tuning for the second and third cases [6].
- Backlinking the originating netdev to set tx_extra_headroom for the
third case [7].
- Native XDP on veth. veth_convert_skb_to_xdp_buff() still copies
these packets. A follow-up veth patch changes this for XDP_PASS and
XDP_DROP.
Changes:
v2:
- Drop "configurable" from the series title and rework the cover
letter.
- Drop IFLA_TX_EXTRA_HEADROOM (Willem, Stephen).
- Set tx_extra_headroom on veth XDP attach (Stephen), from the BPF decl
tag "tx_extra_headroom:<bytes>" on tcx attach (Dragos, Willem), and
with VETH_INFO_TX_EXTRA_HEADROOM at veth creation.
- Turn the LL_ALLOC_SPACE() and LL_ALLOC_SPACE_EX() macros into static
inline functions (Willem).
- Test the veth attribute with YNL only (Jakub).
v1 [2]:
- Add the veth XDP case and the needed_headroom regression data to the
cover letter.
- Remove LL_RESERVED_SPACE_EXTRA(), which has no user.
- Replace the RFC data-path selftests with a test of a push at TC
egress.
- Add tests of generic XDP on a veth peer and of setting both ends of
a veth pair.
RFC: [1]
[1] https://lore.kernel.org/all/20261015235335.4162993-1-dkeogh@amazon.com/
[2] https://lore.kernel.org/all/cover.1794020868.git.tianyi@cloudflare.com/
[3] https://git.kernel.org/torvalds/c/b9ed315d3c4c
[4] https://github.com/cilium/cilium/blob/4a5cd2a20e31/pkg/datapath/connector/netkit.go#L63-L67
[5] https://github.com/tgao-cf/tx-extra-headroom-bench/tree/main
[6] https://lore.kernel.org/all/20261108214356.95406-1-tianyi@cloudflare.com/
[7] https://lore.kernel.org/all/20261110201250.95225-1-tianyi@cloudflare.com/
---
David Keogh will post a TCP test as a reply to this series.
The iproute2 patch for VETH_INFO_TX_EXTRA_HEADROOM will be posted after
this series is accepted.
David Keogh (2):
net: add tx_extra_headroom and netdev_get_alloc_headroom()
net: reserve extra TX headroom for local egress
Tianyi Gao (9):
net: set tx_extra_headroom of the peer on XDP attach
selftests: net: add flag to send a frame in xdp_helper
selftests: net: add tx_extra_headroom test
bpf, tcx: Set tx_extra_headroom from a decl tag on egress programs
selftests: net: tx_extra_headroom: test a push at TC egress
selftests/bpf: Test the tx_extra_headroom decl tag
veth: add VETH_INFO_TX_EXTRA_HEADROOM
netlink: specs: rt-link: add veth attributes
selftests: net: tx_extra_headroom: test the veth attribute
Documentation/netlink/specs/rt-link.yaml | 14 +
.../networking/net_cachelines/net_device.rst | 1 +
Documentation/networking/netdevices.rst | 44 ++
drivers/net/veth.c | 44 ++
include/linux/bpf.h | 1 +
include/linux/netdevice.h | 40 +-
include/net/tcx.h | 6 +
include/uapi/linux/veth.h | 5 +-
kernel/bpf/tcx.c | 24 +
kernel/bpf/verifier.c | 31 ++
net/core/dev.c | 44 ++
net/ipv4/ip_output.c | 2 +-
net/ipv4/raw.c | 2 +-
net/ipv6/ip6_output.c | 2 +-
net/ipv6/raw.c | 2 +-
net/packet/af_packet.c | 7 +-
net/xdp/xsk.c | 6 +-
tools/net/ynl/Makefile.deps | 3 +-
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../bpf/progs/verifier_tx_extra_headroom.c | 40 ++
tools/testing/selftests/net/Makefile | 1 +
tools/testing/selftests/net/lib/xdp_helper.c | 82 +++-
.../selftests/net/tx_extra_headroom.bpf.c | 138 ++++++
.../selftests/net/tx_extra_headroom.py | 426 ++++++++++++++++++
24 files changed, 944 insertions(+), 23 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/verifier_tx_extra_headroom.c
create mode 100644 tools/testing/selftests/net/tx_extra_headroom.bpf.c
create mode 100755 tools/testing/selftests/net/tx_extra_headroom.py
base-commit: d6816d9a93b66a12bf01e10037e377f3333ad6cc
--
2.56.0
next reply other threads:[~2026-11-19 1:38 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 1:38 Tianyi Gao [this message]
2026-11-19 1:38 ` [PATCH net-next v2 01/11] net: add tx_extra_headroom and netdev_get_alloc_headroom() Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 02/11] net: reserve extra TX headroom for local egress Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 03/11] net: set tx_extra_headroom of the peer on XDP attach Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 04/11] selftests: net: add flag to send a frame in xdp_helper Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 05/11] selftests: net: add tx_extra_headroom test Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 06/11] bpf, tcx: Set tx_extra_headroom from a decl tag on egress programs Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 07/11] selftests: net: tx_extra_headroom: test a push at TC egress Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 08/11] selftests/bpf: Test the tx_extra_headroom decl tag Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 09/11] veth: add VETH_INFO_TX_EXTRA_HEADROOM Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 10/11] netlink: specs: rt-link: add veth attributes Tianyi Gao
2026-11-19 1:38 ` [PATCH net-next v2 11/11] selftests: net: tx_extra_headroom: test the veth attribute Tianyi Gao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119013830.36935-1-tianyi@cloudflare.com \
--to=tianyi@cloudflare.com \
--cc=alasdair@mcwilliam.dev \
--cc=ameryhung@gmail.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=anilkaushikwireless@gmail.com \
--cc=aroulin@nvidia.com \
--cc=ast@fiberby.net \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=daniel@iogearbox.net \
--cc=danieller@nvidia.com \
--cc=davem@davemloft.net \
--cc=demosthenes.ra@gmail.com \
--cc=dkeogh@amazon.com \
--cc=donald.hunter@gmail.com \
--cc=dsahern@kernel.org \
--cc=dtatulea@nvidia.com \
--cc=dwmw2@infradead.org \
--cc=eddyz87@gmail.com \
--cc=edumazet@kernel.org \
--cc=emil@etsalapatis.com \
--cc=gustavoars@kernel.org \
--cc=hangbin.liu@linux.dev \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=idosch@nvidia.com \
--cc=ihor.solodrai@linux.dev \
--cc=jiayuan.chen@linux.dev \
--cc=joe@dama.to \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kernel-team@cloudflare.com \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=leitao@debian.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux@leemhuis.info \
--cc=lirongqing@baidu.com \
--cc=louis.scalbert@6wind.com \
--cc=maciej.fijalkowski@intel.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=mattbobrowski@google.com \
--cc=memxor@gmail.com \
--cc=michael.bommarito@gmail.com \
--cc=mkarsten@uwaterloo.ca \
--cc=mkl@pengutronix.de \
--cc=n0ps1edzz@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=nh-open-source@amazon.com \
--cc=noren@nvidia.com \
--cc=one-d-wide@protonmail.com \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
--cc=rdunlap@infradead.org \
--cc=runyu.xiao@seu.edu.cn \
--cc=sdf@fomichev.me \
--cc=shuah@kernel.org \
--cc=skhan@linuxfoundation.org \
--cc=song@kernel.org \
--cc=stephen@networkplumber.org \
--cc=tianyi.gao@linux.dev \
--cc=wang.zhan@smartx.com \
--cc=willemdebruijn.kernel@gmail.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox