[PATCH net 0/2] veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

[PATCH net 0/2] veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

[Tianyi Gao]

When an XDP program on a veth device returns XDP_TX and the peer has
GRO enabled but no XDP program, the frames are queued on the peer's
ring, but the peer's NAPI is never scheduled to process them. Once the
ring is full, every XDP_TX frame is dropped.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before
xmit of XDP frame") made such a peer work as an XDP_REDIRECT target,
where veth_xdp_xmit() both queues the frames and schedules the peer's
NAPI. XDP_TX queues its frames through the same function but schedules
the NAPI in veth_xdp_flush(), which still checks xdp_prog.

The usual workaround is to load a dummy XDP program on the peer [1].
That turns off TSO on the device, and every skb it sends then goes
through the peer's XDP path, where most of them are copied. In our
tests, TCP sent from a namespace dropped from 22.1 to 14.8 Gbit/s with
the dummy program. With this fix the program is no longer needed.

Patch 1 makes veth_xdp_flush() check the peer's napi pointer, as
veth_xdp_xmit() does. Patch 2 adds a test for this case to veth.sh.

[1] https://lore.kernel.org/all/fb2d324b-35fb-802d-2e1d-1ee1aa234c16@gmail.com/

---
Bochao Cao's migration of the XDP feature test [2] skips XDP_TX on veth
when the peer has no XDP program, which is the case patch 1 fixes.

[2] https://lore.kernel.org/netdev/20261010-xdp-v5-send-20261008-v5-0-de687cb67c40@gmail.com/

Tianyi Gao (2):
  veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
  selftests: net: veth: test XDP_TX to a peer with GRO enabled

 drivers/net/veth.c                            |  2 +-
 .../testing/selftests/net/lib/xdp_dummy.bpf.c |  6 +++++
 tools/testing/selftests/net/veth.sh           | 25 +++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)


base-commit: 513444cc62e231a11c9b15ea88de0890da9b2b4f
-- 
2.55.0

[PATCH net 1/2] veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

[Tianyi Gao]

veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has
NAPI. A peer with GRO enabled and no XDP program also has NAPI.

veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's
xdp_prog is set, so for such a peer the frames are never processed.
Nothing else schedules that NAPI by default.

Once the ring is full, every XDP_TX frame is dropped, and so are the
skbs that veth_xmit() queues on the same ring.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before
xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one
in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer
instead of xdp_prog, as veth_xdp_xmit() does.

Fixes: 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")
Signed-off-by: Tianyi Gao <tianyi@cloudflare.com>
---
 drivers/net/veth.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 643b97dc5245..9405ffac598a 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq)
 	rcv_priv = netdev_priv(rcv);
 	rcv_rq = &rcv_priv->rq[veth_select_rxq(rcv)];
 	/* xdp_ring is initialized on receive side? */
-	if (unlikely(!rcu_access_pointer(rcv_rq->xdp_prog)))
+	if (unlikely(!rcu_access_pointer(rcv_rq->napi)))
 		goto out;
 
 	__veth_xdp_flush(rcv_rq);
-- 
2.55.0

[PATCH net 2/2] selftests: net: veth: test XDP_TX to a peer with GRO enabled

[Tianyi Gao]

Test that XDP_TX frames reach a peer that has GRO enabled and no XDP
program. GRO enables NAPI on the peer, so the frames are queued on the
peer's ring, and the peer's NAPI must be scheduled to process them.

Attach a program that returns XDP_TX for every frame, send 5 echo
requests and check that all of them were sent back and that the peer's
NAPI received them, as counted in its xdp_packets statistic. IPv6 is
disabled on both devices, so that only the echo requests are counted.

Add the program to xdp_dummy.bpf.c in a section of its own, so that the
tests that load "section xdp" still get XDP_PASS.

Signed-off-by: Tianyi Gao <tianyi@cloudflare.com>
---
The XDP_TX program goes into xdp_dummy.bpf.c as a new section instead of
a new file, since it is a single line; 75cc19c8ff89 ("selftests:
drv-net: add xdp cases for ping.py") added the xdp.frags variant there
the same way. libbpf does not know the section name "xdp_tx", but
iproute2 loads it, as tc loads the custom sections in nat6to4.bpf.c.

 .../testing/selftests/net/lib/xdp_dummy.bpf.c |  6 +++++
 tools/testing/selftests/net/veth.sh           | 25 +++++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/tools/testing/selftests/net/lib/xdp_dummy.bpf.c b/tools/testing/selftests/net/lib/xdp_dummy.bpf.c
index e73fab3edd9f..95abd67cc184 100644
--- a/tools/testing/selftests/net/lib/xdp_dummy.bpf.c
+++ b/tools/testing/selftests/net/lib/xdp_dummy.bpf.c
@@ -16,4 +16,10 @@ int xdp_dummy_prog_frags(struct xdp_md *ctx)
 	return XDP_PASS;
 }
 
+SEC("xdp_tx")
+int xdp_dummy_tx(struct xdp_md *ctx)
+{
+	return XDP_TX;
+}
+
 char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/net/veth.sh b/tools/testing/selftests/net/veth.sh
index 8b867e0675a7..0167f78604ef 100755
--- a/tools/testing/selftests/net/veth.sh
+++ b/tools/testing/selftests/net/veth.sh
@@ -334,6 +334,31 @@ ip netns exec $NS_DST ethtool -K veth$DST rx-udp-gro-forwarding on
 chk_gro "        - aggregation with TSO off" 1
 cleanup
 
+create_ns
+ip netns exec $NS_SRC sysctl -qw net.ipv6.conf.veth$SRC.disable_ipv6=1
+ip netns exec $NS_DST sysctl -qw net.ipv6.conf.veth$DST.disable_ipv6=1
+ip netns exec $NS_SRC ethtool -K veth$SRC gro on
+ip -n $NS_DST link set dev veth$DST xdp object ${BPF_FILE} section xdp_tx
+# veth$DST sends every frame back, so neither ARP nor ping gets an answer
+mac=`ip netns exec $NS_DST cat /sys/class/net/veth$DST/address`
+ip -n $NS_SRC neigh replace $BM_NET_V4$DST lladdr $mac dev veth$SRC
+ip netns exec $NS_SRC ping -qc 5 -i 0.01 -W 1 $BM_NET_V4$DST >/dev/null
+tx=`ip netns exec $NS_DST ethtool -S veth$DST | \
+    awk '/rx_queue_[0-9]+_xdp_tx:/ {s += $2} END {print s + 0}'`
+rx=`ip netns exec $NS_SRC ethtool -S veth$SRC | \
+    awk '/rx_queue_[0-9]+_xdp_packets:/ {s += $2} END {print s + 0}'`
+printf "%-60s" "xdp_tx to peer with gro on"
+if [ $tx -ne 5 ]; then
+	echo " fail - $tx frames sent back, expected 5"
+	ret=1
+elif [ $rx -ne 5 ]; then
+	echo " fail - $rx frames received by the peer NAPI, expected 5"
+	ret=1
+else
+	echo " ok "
+fi
+cleanup
+
 create_ns
 
 CUR_TX=1
-- 
2.55.0