Re: [PATCH net-next v8 9/9] selftests: net: Add a test for BIG TCP in UDP tunnels

["Alice Mikityanska" <alice.kernel@fastmail.im>]

On Tue, Jul 7, 2026, at 11:06, Paolo Abeni wrote:
> On 7/6/26 8:19 PM, Alice Mikityanska wrote:
>> From: Alice Mikityanska <alice@isovalent.com>
>> 
>> The test sets up VXLAN and GENEVE tunnels over IPv4 and IPv6 and runs
>> IPv4 and IPv6 traffic through them with BIG TCP enabled. It checks that
>> a non-negligible amount of big aggregated packets are seen in tcpdump.
>> 
>> Check the number of packets on both TX and RX sides to verify that GSO
>> packets are valid and not dropped. Capture on the lower netdev (veth),
>> when checksum offload is on, to verify that encapsulated BIG TCP packets
>> can get to their destination. In the test with TX checksum offload off,
>> software GSO splits aggregated VXLAN packets before passing them to
>> veth, so capture inside the tunnel instead to check that the big packets
>> are not dropped.
>> 
>> Check that the amount of SACKs is negligible. On unsupported kernels,
>> some amount of broken GSO packets bigger than 65536 bytes can be
>> produced in VXLAN tunnels, but they don't reach the destination. Seeing
>> TCP SACKs is a sign that such packets could have been dropped (in such
>> cases, the amount of SACKs is a few times bigger than the number of
>> attempts to send BIG TCP packets).
>> 
>> Signed-off-by: Alice Mikityanska <alice@isovalent.com>
>> ---
>>  tools/testing/selftests/net/Makefile          |   1 +
>>  .../testing/selftests/net/big_tcp_tunnels.sh  | 183 ++++++++++++++++++
>>  2 files changed, 184 insertions(+)
>>  create mode 100755 tools/testing/selftests/net/big_tcp_tunnels.sh
>> 
>> diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
>> index 708d960ae07d..4cb0a0bc1eea 100644
>> --- a/tools/testing/selftests/net/Makefile
>> +++ b/tools/testing/selftests/net/Makefile
>> @@ -13,6 +13,7 @@ TEST_PROGS := \
>>  	arp_ndisc_untracked_subnets.sh \
>>  	bareudp.sh \
>>  	big_tcp.sh \
>> +	big_tcp_tunnels.sh \
>>  	bind_bhash.sh \
>>  	bpf_offload.py \
>>  	bridge_stp_mode.sh \
>> diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
>> new file mode 100755
>> index 000000000000..128a710e74ad
>> --- /dev/null
>> +++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
>> @@ -0,0 +1,183 @@
>> +#!/usr/bin/env bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +#
>> +# Testing for IPv4 and IPv6 BIG TCP over VXLAN and GENEVE tunnels.
>> +
>> +SERVER_NS=$(mktemp -u server-XXXXXXXX)
>> +SERVER_IP4="192.168.1.1"
>> +SERVER_IP6="2001:db8::1:1"
>> +SERVER_IP4_TUN="192.168.2.1"
>> +SERVER_IP6_TUN="2001:db8::2:1"
>> +
>> +CLIENT_NS=$(mktemp -u client-XXXXXXXX)
>> +CLIENT_IP4="192.168.1.2"
>> +CLIENT_IP6="2001:db8::1:2"
>> +CLIENT_IP4_TUN="192.168.2.2"
>> +CLIENT_IP6_TUN="2001:db8::2:2"
>> +
>> +: "${PACKETS_THRESHOLD:=1000}"
>> +
>> +# Kselftest framework requirement - SKIP code is 4.
>> +ksft_skip=4
>> +
>> +setup() {
>> +	ip netns add "$SERVER_NS"
>> +	ip netns add "$CLIENT_NS"
>> +	ip -netns "$SERVER_NS" link add link1 type veth peer name link0 netns "$CLIENT_NS"
>> +
>> +	ip -netns "$CLIENT_NS" link set link0 up
>> +	ip -netns "$CLIENT_NS" addr replace "$CLIENT_IP4/24" dev link0
>> +	ip -netns "$CLIENT_NS" addr replace "$CLIENT_IP6/112" dev link0 nodad
>> +	ip -netns "$CLIENT_NS" link set link0 \
>> +		gso_max_size 196608 gso_ipv4_max_size 196608 \
>> +		gro_max_size 196608 gro_ipv4_max_size 196608
>> +	ip -netns "$SERVER_NS" link set link1 up
>> +	ip -netns "$SERVER_NS" addr replace "$SERVER_IP4/24" dev link1
>> +	ip -netns "$SERVER_NS" addr replace "$SERVER_IP6/112" dev link1 nodad
>> +	ip -netns "$SERVER_NS" link set link1 \
>> +		gso_max_size 196608 gso_ipv4_max_size 196608 \
>> +		gro_max_size 196608 gro_ipv4_max_size 196608
>> +
>> +	ip netns exec "$SERVER_NS" netserver >/dev/null
>> +}
>> +
>> +setup_tunnel() {
>> +	if [ "$2" = 4 ]; then
>> +		SERVER_IP="$SERVER_IP4"
>> +		CLIENT_IP="$CLIENT_IP4"
>> +		echo "Setting up ${1^^} over IPv4, veth tx csum offload $3"
>> +	else
>> +		SERVER_IP="$SERVER_IP6"
>> +		CLIENT_IP="$CLIENT_IP6"
>> +		echo "Setting up ${1^^} over IPv6, veth tx csum offload $3"
>> +	fi
>> +
>> +	if [ "$1" = vxlan ]; then
>> +		ip -netns "$CLIENT_NS" link add tun0 type vxlan \
>> +			id 5001 remote "$SERVER_IP" local "$CLIENT_IP" dev link0 dstport 4789
>> +	else
>> +		ip -netns "$CLIENT_NS" link add tun0 type geneve \
>> +			id 5001 remote "$SERVER_IP"
>> +	fi
>> +	ip -netns "$CLIENT_NS" link set tun0 up
>> +	ip -netns "$CLIENT_NS" addr replace "$CLIENT_IP4_TUN/24" dev tun0
>> +	ip -netns "$CLIENT_NS" addr replace "$CLIENT_IP6_TUN/112" dev tun0 nodad
>> +	ip -netns "$CLIENT_NS" link set tun0 \
>> +		gso_max_size 196608 gso_ipv4_max_size 196608 \
>> +		gro_max_size 196608 gro_ipv4_max_size 196608
>> +	if [ "$1" = vxlan ]; then
>> +		ip -netns "$SERVER_NS" link add tun1 type vxlan \
>> +			id 5001 remote "$CLIENT_IP" local "$SERVER_IP" dev link1 dstport 4789
>> +	else
>> +		ip -netns "$SERVER_NS" link add tun1 type geneve \
>> +			id 5001 remote "$CLIENT_IP"
>> +	fi
>> +	ip -netns "$SERVER_NS" link set tun1 up
>> +	ip -netns "$SERVER_NS" addr replace "$SERVER_IP4_TUN/24" dev tun1
>> +	ip -netns "$SERVER_NS" addr replace "$SERVER_IP6_TUN/112" dev tun1 nodad
>> +	ip -netns "$SERVER_NS" link set tun1 \
>> +		gso_max_size 196608 gso_ipv4_max_size 196608 \
>> +		gro_max_size 196608 gro_ipv4_max_size 196608
>> +
>> +	ip netns exec "$CLIENT_NS" ethtool -K link0 tx-checksumming "$3" > /dev/null
>> +	ip netns exec "$SERVER_NS" ethtool -K link1 tx-checksumming "$3" > /dev/null
>> +}
>> +
>> +cleanup_tunnel() {
>> +	ip -netns "$CLIENT_NS" link del tun0
>> +	ip -netns "$SERVER_NS" link del tun1
>> +}
>> +
>> +cleanup() {
>> +	ip netns pids "$SERVER_NS" | xargs -r kill
>> +	ip netns pids "$CLIENT_NS" | xargs -r kill
>> +	ip netns del "$SERVER_NS"
>> +	ip netns del "$CLIENT_NS"
>> +	rm -rf "$WORKDIR"
>> +}
>> +
>> +do_test() {
>> +	# When tx csum offload is off, software GSO is performed before passing the
>> +	# packet to veth. Check BIG TCP packets inside the VXLAN tunnel to verify
>> +	# the software checksum path: if the checksum code is broken, these packets
>> +	# will be dropped.
>> +	if [ "$2" = on ]; then
>> +		CAPTURE_IFACE='link'
>> +	else
>> +		CAPTURE_IFACE='tun'
>> +	fi
>> +
>> +	ip netns exec "$SERVER_NS" tcpdump -nn -s 256 -i "${CAPTURE_IFACE}1" greater 65536 -w "$WORKDIR/server.pcap" 2> /dev/null &
>> +	TCPDUMP_SERVER_PID="$!"
>> +	ip netns exec "$CLIENT_NS" tcpdump -nn -s 256 -i "${CAPTURE_IFACE}0" greater 65536 -w "$WORKDIR/client.pcap" 2> /dev/null &
>> +	TCPDUMP_CLIENT_PID="$!"
>> +
>> +	# This filter doesn't capture all possible variants of SACK, but it's aimed
>> +	# at the typical one where SACK follows after [nop, nop, timestamp, nop,
>> +	# nop] (14 bytes after the 20-byte TCP header). IPv6 needs a separate match,
>> +	# because man tcpdump says:
>> +	# > Arithmetic expression against transport layer headers, like tcp[0], does
>> +	# > not work against IPv6 packets.  It only looks at IPv4 packets.
>> +	ip netns exec "$SERVER_NS" tcpdump -nn -s 256 -i "tun1" '(tcp[tcpflags] & (tcp-syn|tcp-ack) = tcp-ack and tcp[34:2] & 0xffc3 = 0x0502) or (ip6[6] = 0x06 and ip6[53] & 0x12 = 0x10 and ip6[74:2] & 0xffc3 = 0x0502)' -w "$WORKDIR/sack.pcap" 2> /dev/null &
>> +	TCPDUMP_SACK_PID="$!"
>> +
>> +	if [ "$1" = 4 ]; then
>> +		SERVER_IP="$SERVER_IP4_TUN"
>> +		echo "Running IPv4 traffic in the tunnel"
>> +	else
>> +		SERVER_IP="$SERVER_IP6_TUN"
>> +		echo "Running IPv6 traffic in the tunnel"
>> +	fi
>> +
>> +	sleep 1 # Give tcpdump a second to spin up.
>> +	ip netns exec "$CLIENT_NS" netperf -t TCP_STREAM -l 5 -H "$SERVER_IP" -- \
>> +		-m 80000 > /dev/null
>> +	sleep 1 # Give tcpdump a second to process buffered packets.
>> +	kill "$TCPDUMP_SERVER_PID" "$TCPDUMP_CLIENT_PID" "$TCPDUMP_SACK_PID"
>> +	wait "$TCPDUMP_SERVER_PID" "$TCPDUMP_CLIENT_PID" "$TCPDUMP_SACK_PID"
>> +	PACKETS_SERVER=$(tcpdump --count -r "$WORKDIR/server.pcap" 2> /dev/null | cut -d ' ' -f 1)
>> +	PACKETS_CLIENT=$(tcpdump --count -r "$WORKDIR/client.pcap" 2> /dev/null | cut -d ' ' -f 1)
>> +	PACKETS_SACK=$(tcpdump --count -r "$WORKDIR/sack.pcap" 2> /dev/null | cut -d ' ' -f 1)
>> +
>> +	echo "Captured BIG TCP RX packets: $PACKETS_SERVER"
>> +	echo "Captured BIG TCP TX packets: $PACKETS_CLIENT"
>> +	echo "Captured TCP SACK packets: $PACKETS_SACK"
>> +	[ "$PACKETS_SERVER" -gt "$PACKETS_THRESHOLD" ] || return 1
>> +	[ "$PACKETS_CLIENT" -gt "$PACKETS_THRESHOLD" ] || return 1
>> +	[ "$PACKETS_SACK" -lt "$(( PACKETS_CLIENT / 2 ))" ] || return 1
>
> KCI fails here, see:
>
> https://netdev-ctrl.bots.linux.dev/logs/vmksft/net/results/722863/156-big-tcp-tunnels-sh/stdout
>
> possibly the above parsing is a bit fragile/tcpdump version's dependent?!?

TL/DR: I've seen this when I ran out of space in /tmp before adding -s 256.

Hmm, the changes I made in this iteration were aimed at addressing version-
dependent behavior of tcpdump... I used to count the lines of its
output. Newer tcpdump prints two lines per encapsulated packet. Older
tcpdump can't parse BIG TCP in this test and prints one line per packet.
To make it more robust, I decided to store the pcap and count the
packets there (--count doesn't work with live capture when tcpdump is
interrupted). The pitfall is that now it takes a few hundred megabytes
in /tmp to store those pcaps.

Now, seeing empty stdout from tcpdump could be if the pcap file is empty
(doesn't contain the pcap header), because we ran out of space in /tmp
(first two tcpdumps took all space, the third didn't write the header -
I observed this behavior before I added -s 256). We don't see the
corresponding error messages, because tcpdump starts with 2> /dev/null
(otherwise it floods the screen with statistics). Could this be the
reason? How much space is allocated for /tmp in the CI runners?

I can probably address it by limiting the overall number of captured
packets with -c, but then I won't be able to compare $PACKETS_SACK
relative to $PACKETS_CLIENT.

> Note that KCI automatically drops from PW series with failing test.
>
> /P