Netdev List
 help / color / mirror / Atom feed
From: Tianyi Gao <tianyi@cloudflare.com>
To: netdev@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
	Paolo Abeni <pabeni@redhat.com>,
	Eric Dumazet <edumazet@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
	"Simon Horman" <horms@kernel.org>,
	"David Keogh" <dkeogh@amazon.com>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"David Woodhouse" <dwmw2@infradead.org>,
	"Dragos Tatulea" <dtatulea@nvidia.com>,
	"Nimrod Oren" <noren@nvidia.com>,
	"Alasdair McWilliam" <alasdair@mcwilliam.dev>,
	"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
	"Maciej Fijalkowski" <maciej.fijalkowski@intel.com>,
	"Stanislav Fomichev" <sdf@fomichev.me>,
	"Magnus Karlsson" <magnus.karlsson@intel.com>,
	"David Ahern" <dsahern@kernel.org>,
	"Ido Schimmel" <idosch@nvidia.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Donald Hunter" <donald.hunter@gmail.com>,
	"Andrew Lunn" <andrew+netdev@lunn.ch>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Shuah Khan" <skhan@linuxfoundation.org>,
	"Shuah Khan" <shuah@kernel.org>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	"Breno Leitao" <leitao@debian.org>,
	"Asbjørn Sloth Tønnesen" <ast@fiberby.net>,
	"Danielle Ratson" <danieller@nvidia.com>,
	"Kuniyuki Iwashima" <kuniyu@google.com>,
	bpf@vger.kernel.org, linux-doc@vger.kernel.org,
	linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com,
	nh-open-source@amazon.com,
	"Stephen Hemminger" <stephen@networkplumber.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>,
	"Remy D. Farley" <one-d-wide@protonmail.com>,
	"Amery Hung" <ameryhung@gmail.com>,
	"Andrii Nakryiko" <andrii@kernel.org>,
	"Andy Roulin" <aroulin@nvidia.com>,
	"Anil Kaushik" <anilkaushikwireless@gmail.com>,
	"Demos Ra" <demosthenes.ra@gmail.com>,
	"Eduard Zingerman" <eddyz87@gmail.com>,
	"Emil Tsalapatis" <emil@etsalapatis.com>,
	"Hangbin Liu" <hangbin.liu@linux.dev>,
	"Ihor Solodrai" <ihor.solodrai@linux.dev>,
	"Jiayuan Chen" <jiayuan.chen@linux.dev>,
	"Jiri Olsa" <jolsa@kernel.org>, "Joe Damato" <joe@dama.to>,
	"Kumar Kartikeya Dwivedi" <memxor@gmail.com>,
	"Li RongQing" <lirongqing@baidu.com>,
	"Louis Scalbert" <louis.scalbert@6wind.com>,
	"Marc Kleine-Budde" <mkl@pengutronix.de>,
	"Martin KaFai Lau" <martin.lau@linux.dev>,
	"Martin Karsten" <mkarsten@uwaterloo.ca>,
	"Matt Bobrowski" <mattbobrowski@google.com>,
	"Michael Bommarito" <michael.bommarito@gmail.com>,
	"Nikolay Aleksandrov" <razor@blackwall.org>,
	"Runyu Xiao" <runyu.xiao@seu.edu.cn>,
	"Song Liu" <song@kernel.org>,
	"Thorsten Leemhuis" <linux@leemhuis.info>,
	"Wang Zhan" <wang.zhan@smartx.com>,
	"Yonghong Song" <yonghong.song@linux.dev>,
	"Zhou Qiuyue" <n0ps1edzz@gmail.com>,
	"Tianyi Gao" <tianyi.gao@linux.dev>,
	"Tianyi Gao" <tianyi@cloudflare.com>
Subject: [PATCH net-next v2 00/11] net: extra TX headroom for local egress
Date: Thu, 19 Nov 2026 01:38:19 +0000	[thread overview]
Message-ID: <20261119013830.36935-1-tianyi@cloudflare.com> (raw)

This is v2 of the series that David Keogh posted as an RFC [1].
v1 is [2].

Locally generated UDP, raw and AF_PACKET packets, and AF_XDP packets in
copy mode, get only the headroom that needed_headroom asks for. If an
XDP or TC program needs more, the skb is reallocated or copied for every
packet. This happens in three cases:

- An XDP program on a veth peer needs XDP_PACKET_HEADROOM (256 bytes).
  The peer copies the packet.
- A program at TC egress adds headers. The skb is reallocated.
- A namespace sends through a veth pair, and a program at TC egress of
  another device, such as the host uplink, adds headers. The skb is
  reallocated on that device.

(patches 1-2, from David Keogh) This series adds tx_extra_headroom to
struct net_device. It is added only where a locally generated skb is
allocated.

Each case sets tx_extra_headroom in its own way, to keep the new UAPI
minimal:

- (patches 3-5) The kernel sets it when XDP is attached to the peer of
  a veth or netkit device. New skbs on the device then reserve enough
  headroom for XDP_PACKET_HEADROOM.
- (patches 6-8) Add a BTF decl tag "tx_extra_headroom:<bytes>" as
  UAPI. A tcx egress program declares it, and the verifier reads it at
  load.
- (patches 9-11) Add a VETH_INFO_TX_EXTRA_HEADROOM UAPI for headers that
  another device pushes, such as the host uplink. It is like
  IFLA_NETKIT_HEADROOM for netkit [3]. The CNI plugin sets it at veth
  creation, as Cilium does with netkit [4].

tx_extra_headroom is the sum of the extra headroom that each case
requires, limited to 512 bytes. For example, a program at TC egress
pushes a header, and XDP on the peer then still needs
XDP_PACKET_HEADROOM in front of it. The kernel recalculates
tx_extra_headroom when XDP or a tcx program is attached or detached.

For all three cases, one option is to raise needed_headroom instead.
This causes a regression for TCP and forwarding, because
needed_headroom is also the minimum that ip_finish_output2() and
ip_forward() require.

Patch 3 depends on commit f564749b3b26 ("net: record XDP programs
propagated to lower devices"), which is only in net-next, so the series
targets net-next. Patches 6-8 change tcx and the verifier. An Ack from
the BPF maintainers on them would be appreciated.

Performance:

All tests run on a Xeon E5-2640 v4 with a ConnectX-4 Lx NIC, on
net-next at d6816d9a93b6. Details of the artifacts are in [5].

The first table shows the regression when needed_headroom is raised
to 256 on both ends of a veth pair. XDP runs on the namespace side:

test                                 net-next       needed_headroom 256
netns sends TCP to host              32.1 Gbit/s    30.8 Gbit/s
host forwards 1400B UDP into netns   0.583 Mpps     0.366 Mpps

The second table shows the gain from tx_extra_headroom in all three
cases, with 1400-byte UDP payloads. All numbers come from one kernel
with this series applied, with tx_extra_headroom off and on. Between
two kernels, the code layout alone changed the rate by up to 13%.
Off uses a benchmark-only patch for the XDP case [5], the push program
without its decl tag, and a veth pair without
VETH_INFO_TX_EXTRA_HEADROOM:

test                                          off          on
host to netns, generic XDP_PASS in netns      0.237 Mpps   0.283 Mpps
host to netns, generic XDP_DROP in netns      0.386 Mpps   0.465 Mpps
host to netns, 20-byte push at TC egress      0.385 Mpps   0.468 Mpps
netns to uplink, 20-byte push on the uplink   0.190 Mpps   0.219 Mpps

Out of scope:

- Auto-tuning for the second and third cases [6].
- Backlinking the originating netdev to set tx_extra_headroom for the
  third case [7].
- Native XDP on veth. veth_convert_skb_to_xdp_buff() still copies
  these packets. A follow-up veth patch changes this for XDP_PASS and
  XDP_DROP.

Changes:

v2:
- Drop "configurable" from the series title and rework the cover
  letter.
- Drop IFLA_TX_EXTRA_HEADROOM (Willem, Stephen).
- Set tx_extra_headroom on veth XDP attach (Stephen), from the BPF decl
  tag "tx_extra_headroom:<bytes>" on tcx attach (Dragos, Willem), and
  with VETH_INFO_TX_EXTRA_HEADROOM at veth creation.
- Turn the LL_ALLOC_SPACE() and LL_ALLOC_SPACE_EX() macros into static
  inline functions (Willem).
- Test the veth attribute with YNL only (Jakub).

v1 [2]:
- Add the veth XDP case and the needed_headroom regression data to the
  cover letter.
- Remove LL_RESERVED_SPACE_EXTRA(), which has no user.
- Replace the RFC data-path selftests with a test of a push at TC
  egress.
- Add tests of generic XDP on a veth peer and of setting both ends of
  a veth pair.

RFC: [1]

[1] https://lore.kernel.org/all/20261015235335.4162993-1-dkeogh@amazon.com/
[2] https://lore.kernel.org/all/cover.1794020868.git.tianyi@cloudflare.com/
[3] https://git.kernel.org/torvalds/c/b9ed315d3c4c
[4] https://github.com/cilium/cilium/blob/4a5cd2a20e31/pkg/datapath/connector/netkit.go#L63-L67
[5] https://github.com/tgao-cf/tx-extra-headroom-bench/tree/main
[6] https://lore.kernel.org/all/20261108214356.95406-1-tianyi@cloudflare.com/
[7] https://lore.kernel.org/all/20261110201250.95225-1-tianyi@cloudflare.com/
---
David Keogh will post a TCP test as a reply to this series.

The iproute2 patch for VETH_INFO_TX_EXTRA_HEADROOM will be posted after
this series is accepted.

David Keogh (2):
  net: add tx_extra_headroom and netdev_get_alloc_headroom()
  net: reserve extra TX headroom for local egress

Tianyi Gao (9):
  net: set tx_extra_headroom of the peer on XDP attach
  selftests: net: add flag to send a frame in xdp_helper
  selftests: net: add tx_extra_headroom test
  bpf, tcx: Set tx_extra_headroom from a decl tag on egress programs
  selftests: net: tx_extra_headroom: test a push at TC egress
  selftests/bpf: Test the tx_extra_headroom decl tag
  veth: add VETH_INFO_TX_EXTRA_HEADROOM
  netlink: specs: rt-link: add veth attributes
  selftests: net: tx_extra_headroom: test the veth attribute

 Documentation/netlink/specs/rt-link.yaml      |  14 +
 .../networking/net_cachelines/net_device.rst  |   1 +
 Documentation/networking/netdevices.rst       |  44 ++
 drivers/net/veth.c                            |  44 ++
 include/linux/bpf.h                           |   1 +
 include/linux/netdevice.h                     |  40 +-
 include/net/tcx.h                             |   6 +
 include/uapi/linux/veth.h                     |   5 +-
 kernel/bpf/tcx.c                              |  24 +
 kernel/bpf/verifier.c                         |  31 ++
 net/core/dev.c                                |  44 ++
 net/ipv4/ip_output.c                          |   2 +-
 net/ipv4/raw.c                                |   2 +-
 net/ipv6/ip6_output.c                         |   2 +-
 net/ipv6/raw.c                                |   2 +-
 net/packet/af_packet.c                        |   7 +-
 net/xdp/xsk.c                                 |   6 +-
 tools/net/ynl/Makefile.deps                   |   3 +-
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_tx_extra_headroom.c    |  40 ++
 tools/testing/selftests/net/Makefile          |   1 +
 tools/testing/selftests/net/lib/xdp_helper.c  |  82 +++-
 .../selftests/net/tx_extra_headroom.bpf.c     | 138 ++++++
 .../selftests/net/tx_extra_headroom.py        | 426 ++++++++++++++++++
 24 files changed, 944 insertions(+), 23 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_tx_extra_headroom.c
 create mode 100644 tools/testing/selftests/net/tx_extra_headroom.bpf.c
 create mode 100755 tools/testing/selftests/net/tx_extra_headroom.py


base-commit: d6816d9a93b66a12bf01e10037e377f3333ad6cc
-- 
2.56.0


             reply	other threads:[~2026-11-19  1:38 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  1:38 Tianyi Gao [this message]
2026-11-19  1:38 ` [PATCH net-next v2 01/11] net: add tx_extra_headroom and netdev_get_alloc_headroom() Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 02/11] net: reserve extra TX headroom for local egress Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 03/11] net: set tx_extra_headroom of the peer on XDP attach Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 04/11] selftests: net: add flag to send a frame in xdp_helper Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 05/11] selftests: net: add tx_extra_headroom test Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 06/11] bpf, tcx: Set tx_extra_headroom from a decl tag on egress programs Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 07/11] selftests: net: tx_extra_headroom: test a push at TC egress Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 08/11] selftests/bpf: Test the tx_extra_headroom decl tag Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 09/11] veth: add VETH_INFO_TX_EXTRA_HEADROOM Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 10/11] netlink: specs: rt-link: add veth attributes Tianyi Gao
2026-11-19  1:38 ` [PATCH net-next v2 11/11] selftests: net: tx_extra_headroom: test the veth attribute Tianyi Gao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119013830.36935-1-tianyi@cloudflare.com \
    --to=tianyi@cloudflare.com \
    --cc=alasdair@mcwilliam.dev \
    --cc=ameryhung@gmail.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=andrii@kernel.org \
    --cc=anilkaushikwireless@gmail.com \
    --cc=aroulin@nvidia.com \
    --cc=ast@fiberby.net \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=corbet@lwn.net \
    --cc=daniel@iogearbox.net \
    --cc=danieller@nvidia.com \
    --cc=davem@davemloft.net \
    --cc=demosthenes.ra@gmail.com \
    --cc=dkeogh@amazon.com \
    --cc=donald.hunter@gmail.com \
    --cc=dsahern@kernel.org \
    --cc=dtatulea@nvidia.com \
    --cc=dwmw2@infradead.org \
    --cc=eddyz87@gmail.com \
    --cc=edumazet@kernel.org \
    --cc=emil@etsalapatis.com \
    --cc=gustavoars@kernel.org \
    --cc=hangbin.liu@linux.dev \
    --cc=hawk@kernel.org \
    --cc=horms@kernel.org \
    --cc=idosch@nvidia.com \
    --cc=ihor.solodrai@linux.dev \
    --cc=jiayuan.chen@linux.dev \
    --cc=joe@dama.to \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kernel-team@cloudflare.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=leitao@debian.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux@leemhuis.info \
    --cc=lirongqing@baidu.com \
    --cc=louis.scalbert@6wind.com \
    --cc=maciej.fijalkowski@intel.com \
    --cc=magnus.karlsson@intel.com \
    --cc=martin.lau@linux.dev \
    --cc=mattbobrowski@google.com \
    --cc=memxor@gmail.com \
    --cc=michael.bommarito@gmail.com \
    --cc=mkarsten@uwaterloo.ca \
    --cc=mkl@pengutronix.de \
    --cc=n0ps1edzz@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=nh-open-source@amazon.com \
    --cc=noren@nvidia.com \
    --cc=one-d-wide@protonmail.com \
    --cc=pabeni@redhat.com \
    --cc=razor@blackwall.org \
    --cc=rdunlap@infradead.org \
    --cc=runyu.xiao@seu.edu.cn \
    --cc=sdf@fomichev.me \
    --cc=shuah@kernel.org \
    --cc=skhan@linuxfoundation.org \
    --cc=song@kernel.org \
    --cc=stephen@networkplumber.org \
    --cc=tianyi.gao@linux.dev \
    --cc=wang.zhan@smartx.com \
    --cc=willemdebruijn.kernel@gmail.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox