[PATCH net-next 0/4] Improve stability of big_tcp_tunnels.sh selftest

[PATCH net-next 0/4] Improve stability of big_tcp_tunnels.sh selftest

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

This series addresses flakiness of big_tcp_tunnels.sh discussed at:

https://lore.kernel.org/netdev/349c582c-73e3-468c-91cd-ad6cc3562700@app.fastmail.com/

Lower the thresholds to avoid failures like this (number of packets
below the threshold):

https://netdev-ctrl.bots.linux.dev/logview.html?f=/logs/vmksft/net-dbg/results/754376/28-big-tcp-tunnels-sh/stdout

Tweak tcp_min_tso_segs and initcwnd to avoid failures like this (BIG TCP
doesn't kick in in slow configurations):

https://netdev-ctrl.bots.linux.dev/logview.html?f=/logs/vmksft/net-dbg/results/751124/21-big-tcp-tunnels-sh/stdout

Alice Mikityanska (4):
  selftests: net: Wait for netserver to launch
  selftests: net: Lower threshold on debug kernels for
    big_tcp_tunnels.sh
  selftests: net: Lower threshold with csum offload off in
    big_tcp_tunnels.sh
  selftests: net: Fix slow configurations in big_tcp_tunnels.sh

 .../testing/selftests/net/big_tcp_tunnels.sh  | 45 +++++++++++++++++--
 1 file changed, 41 insertions(+), 4 deletions(-)

-- 
2.55.0

[PATCH net-next 1/4] selftests: net: Wait for netserver to launch

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

Use wait_local_port_listen after starting netserver in
big_tcp_tunnels.sh to ensure it's listening when the test starts.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index d6513ed8d4e8..9b7d0456bcc1 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -3,6 +3,8 @@
 #
 # Testing for IPv4 and IPv6 BIG TCP over VXLAN and GENEVE tunnels.
 
+source "$(dirname "$0")/lib.sh"
+
 SERVER_NS=$(mktemp -u server-XXXXXXXX)
 SERVER_IP4="192.168.1.1"
 SERVER_IP6="2001:db8::1:1"
@@ -39,6 +41,7 @@ setup() {
 		gro_max_size 196608 gro_ipv4_max_size 196608
 
 	ip netns exec "$SERVER_NS" netserver >/dev/null
+	wait_local_port_listen "$SERVER_NS" 12865 tcp
 }
 
 setup_tunnel() {
-- 
2.55.0

[PATCH net-next 2/4] selftests: net: Lower threshold on debug kernels for big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

CONFIG_DEBUG_NET kernels run slower and generate fewer BIG TCP packets,
making the test flaky on upstream CI runners. Lower the default
threshold for those kernels.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index 9b7d0456bcc1..e27774eae20e 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -17,11 +17,25 @@ CLIENT_IP6="2001:db8::1:2"
 CLIENT_IP4_TUN="192.168.2.2"
 CLIENT_IP6_TUN="2001:db8::2:2"
 
-: "${PACKETS_THRESHOLD:=1000}"
-
 # Kselftest framework requirement - SKIP code is 4.
 ksft_skip=4
 
+is_debug() {
+	local config="/proc/config.gz"
+	[ -f "$config" ] || config="/boot/config-$(uname -r)"
+	[ -f "$config" ] || return 1 # No config found; assume non-debug.
+	gzip -dcfq "$config" | grep -qx 'CONFIG_DEBUG_NET=y' || return 1
+}
+
+if [ -z "$PACKETS_THRESHOLD" ]; then
+	if is_debug; then
+		echo 'Debug kernel detected, lowering the default threshold'
+		PACKETS_THRESHOLD=100
+	else
+		PACKETS_THRESHOLD=1000
+	fi
+fi
+
 setup() {
 	ip netns add "$SERVER_NS"
 	ip netns add "$CLIENT_NS"
-- 
2.55.0

[PATCH net-next 3/4] selftests: net: Lower threshold with csum offload off in big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

With checksum offload disabled, much fewer BIG TCP packets are generated
due to overall loss of throughput. Use a separate threshold in these
tests, which is 1/10 of the threshold set for the rest of tests.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index e27774eae20e..1f1c811c563c 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -114,6 +114,8 @@ cleanup() {
 }
 
 do_test() {
+	local packets_threshold="$PACKETS_THRESHOLD"
+
 	# When tx csum offload is off, software GSO is performed before passing the
 	# packet to veth. Check BIG TCP packets inside the VXLAN tunnel to verify
 	# the software checksum path: if the checksum code is broken, these packets
@@ -132,6 +134,7 @@ do_test() {
 		else
 			IPTABLES=ip6tables
 		fi
+		packets_threshold=$(( PACKETS_THRESHOLD / 10 ))
 	fi
 	if [ "$2" = 4 ]; then
 		IPTABLES_SACK=iptables
@@ -164,8 +167,8 @@ do_test() {
 	echo "Captured BIG TCP RX packets: $PACKETS_SERVER"
 	echo "Captured BIG TCP TX packets: $PACKETS_CLIENT"
 	echo "Captured TCP SACK packets: $PACKETS_SACK"
-	[ "$PACKETS_SERVER" -gt "$PACKETS_THRESHOLD" ] || return 1
-	[ "$PACKETS_CLIENT" -gt "$PACKETS_THRESHOLD" ] || return 1
+	[ "$PACKETS_SERVER" -gt "$packets_threshold" ] || return 1
+	[ "$PACKETS_CLIENT" -gt "$packets_threshold" ] || return 1
 	[ "$PACKETS_SACK" -lt "$(( PACKETS_CLIENT / 2 ))" ] || return 1
 }
 
-- 
2.55.0

[PATCH net-next 4/4] selftests: net: Fix slow configurations in big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

The combination of checksum offload disabled (that causes software GSO)
and CONFIG_DEBUG_NET=y is inherently slow. Depending on the CPU power
and load, RTT may increase, limiting sk_pacing_rate, so tcp_tso_autosize
caps SKBs at around 40 segments, and zero BIG TCP packets are produced.

Increase sysctl net.ipv4.tcp_min_tso_segs and set a bigger initial value
of CWND in these configurations to force BIG TCP.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index 1f1c811c563c..222de7c0c294 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -56,6 +56,8 @@ setup() {
 
 	ip netns exec "$SERVER_NS" netserver >/dev/null
 	wait_local_port_listen "$SERVER_NS" 12865 tcp
+
+	DEFAULT_TCP_MIN_TSO_SEGS=$(ip netns exec "$CLIENT_NS" sysctl -n net.ipv4.tcp_min_tso_segs)
 }
 
 setup_tunnel() {
@@ -142,6 +144,21 @@ do_test() {
 		IPTABLES_SACK=ip6tables
 	fi
 
+	if [ "$3" != 'on' ] && is_debug; then
+		echo 'Slow configuration; increasing net.ipv4.tcp_min_tso_segs and initcwnd'
+		ip netns exec "$CLIENT_NS" sysctl -w net.ipv4.tcp_min_tso_segs=52
+		if [ "$2" = 4 ]; then
+			ip -netns "$CLIENT_NS" \
+			    route change 192.168.2.0/24 dev tun0 initcwnd 100
+		else
+			ip -netns "$CLIENT_NS" -6 \
+			    route change 2001:db8::2:0/112 dev tun0 initcwnd 100
+		fi
+	else
+		ip netns exec "$CLIENT_NS" \
+		    sysctl -w net.ipv4.tcp_min_tso_segs="$DEFAULT_TCP_MIN_TSO_SEGS"
+	fi
+
 	ip netns exec "$SERVER_NS" "$IPTABLES" -w -t raw -I PREROUTING -i "${CAPTURE_IFACE}1" -m length ! --length 0:65535 -m comment --comment "bigtcp"
 	ip netns exec "$CLIENT_NS" "$IPTABLES" -w -t raw -I OUTPUT -o "${CAPTURE_IFACE}0" -m length ! --length 0:65535 -m comment --comment "bigtcp"
 	ip netns exec "$SERVER_NS" "$IPTABLES_SACK" -w -t raw -I OUTPUT -o "tun1" -p tcp -m tcp --tcp-flags ACK ACK --tcp-option 5 -m comment --comment "sack"
-- 
2.55.0

Re: [PATCH net-next 2/4] selftests: net: Lower threshold on debug kernels for big_tcp_tunnels.sh

[Matthieu Baerts]

Hi Alice,

On 14/08/2026 21:41, Alice Mikityanska wrote:
> From: Alice Mikityanska <alice@isovalent.com>
> 
> CONFIG_DEBUG_NET kernels run slower and generate fewer BIG TCP packets,
> making the test flaky on upstream CI runners. Lower the default
> threshold for those kernels.

Thank you for looking at that!

> Signed-off-by: Alice Mikityanska <alice@isovalent.com>
> ---
>  tools/testing/selftests/net/big_tcp_tunnels.sh | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
> index 9b7d0456bcc1..e27774eae20e 100755
> --- a/tools/testing/selftests/net/big_tcp_tunnels.sh
> +++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
> @@ -17,11 +17,25 @@ CLIENT_IP6="2001:db8::1:2"
>  CLIENT_IP4_TUN="192.168.2.2"
>  CLIENT_IP6_TUN="2001:db8::2:2"
>  
> -: "${PACKETS_THRESHOLD:=1000}"
> -
>  # Kselftest framework requirement - SKIP code is 4.
>  ksft_skip=4
>  
> +is_debug() {
> +	local config="/proc/config.gz"
> +	[ -f "$config" ] || config="/boot/config-$(uname -r)"
> +	[ -f "$config" ] || return 1 # No config found; assume non-debug.
> +	gzip -dcfq "$config" | grep -qx 'CONFIG_DEBUG_NET=y' || return 1

Are you sure it is CONFIG_DEBUG_NET that causes that? It shouldn't be
that heavy.

Do you not want to change the thresholds only for the -dbg runners using
kernel/configs/debug.config? If yes, you should check if
KSFT_MACHINE_SLOW == yes:

https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style#dealing-with-slow-runners-in-performancelatency-tests

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH net-next 4/4] selftests: net: Fix slow configurations in big_tcp_tunnels.sh

[Matthieu Baerts]

Hi Alice,

On 14/08/2026 21:41, Alice Mikityanska wrote:
> From: Alice Mikityanska <alice@isovalent.com>
> 
> The combination of checksum offload disabled (that causes software GSO)
> and CONFIG_DEBUG_NET=y is inherently slow. Depending on the CPU power
> and load, RTT may increase, limiting sk_pacing_rate, so tcp_tso_autosize
> caps SKBs at around 40 segments, and zero BIG TCP packets are produced.
> 
> Increase sysctl net.ipv4.tcp_min_tso_segs and set a bigger initial value
> of CWND in these configurations to force BIG TCP.
Just an innocent question: would it affect the test somehow if the
sysctl and initial cwnd is always modified, not just for a specific config?

(Note that apart from looking at ${KSFT_MACHINE_SLOW} in patch 2, I
think the series looks good to me, and can probably be sent to net after
the merge with net-next.)

Cheers,
Matt

Re: [PATCH net-next 2/4] selftests: net: Lower threshold on debug kernels for big_tcp_tunnels.sh

[Alice Mikityanska]

On Sat, Aug 15, 2026, at 02:07, Matthieu Baerts wrote:
> Hi Alice,
>
> On 14/08/2026 21:41, Alice Mikityanska wrote:
>> From: Alice Mikityanska <alice@isovalent.com>
>> 
>> CONFIG_DEBUG_NET kernels run slower and generate fewer BIG TCP packets,
>> making the test flaky on upstream CI runners. Lower the default
>> threshold for those kernels.
>
> Thank you for looking at that!
>
>> Signed-off-by: Alice Mikityanska <alice@isovalent.com>
>> ---
>>  tools/testing/selftests/net/big_tcp_tunnels.sh | 18 ++++++++++++++++--
>>  1 file changed, 16 insertions(+), 2 deletions(-)
>> 
>> diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
>> index 9b7d0456bcc1..e27774eae20e 100755
>> --- a/tools/testing/selftests/net/big_tcp_tunnels.sh
>> +++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
>> @@ -17,11 +17,25 @@ CLIENT_IP6="2001:db8::1:2"
>>  CLIENT_IP4_TUN="192.168.2.2"
>>  CLIENT_IP6_TUN="2001:db8::2:2"
>>  
>> -: "${PACKETS_THRESHOLD:=1000}"
>> -
>>  # Kselftest framework requirement - SKIP code is 4.
>>  ksft_skip=4
>>  
>> +is_debug() {
>> +	local config="/proc/config.gz"
>> +	[ -f "$config" ] || config="/boot/config-$(uname -r)"
>> +	[ -f "$config" ] || return 1 # No config found; assume non-debug.
>> +	gzip -dcfq "$config" | grep -qx 'CONFIG_DEBUG_NET=y' || return 1
>
> Are you sure it is CONFIG_DEBUG_NET that causes that? It shouldn't be
> that heavy.
>
> Do you not want to change the thresholds only for the -dbg runners using
> kernel/configs/debug.config? If yes, you should check if
> KSFT_MACHINE_SLOW == yes:

Thanks for the suggestion! I'll switch to KSFT_MACHINE_SLOW.

> https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style#dealing-with-slow-runners-in-performancelatency-tests
>
> Cheers,
> Matt
> -- 
> Sponsored by the NGI0 Core fund.

Re: [PATCH net-next 4/4] selftests: net: Fix slow configurations in big_tcp_tunnels.sh

[Alice Mikityanska]

On Wed, Aug 19, 2026, at 12:25, Matthieu Baerts wrote:
> Hi Alice,
>
> On 14/08/2026 21:41, Alice Mikityanska wrote:
>> From: Alice Mikityanska <alice@isovalent.com>
>> 
>> The combination of checksum offload disabled (that causes software GSO)
>> and CONFIG_DEBUG_NET=y is inherently slow. Depending on the CPU power
>> and load, RTT may increase, limiting sk_pacing_rate, so tcp_tso_autosize
>> caps SKBs at around 40 segments, and zero BIG TCP packets are produced.
>> 
>> Increase sysctl net.ipv4.tcp_min_tso_segs and set a bigger initial value
>> of CWND in these configurations to force BIG TCP.
> Just an innocent question: would it affect the test somehow if the
> sysctl and initial cwnd is always modified, not just for a specific config?

That would guarantee that BIG TCP kicks in, but IMO it reduces the
coverage by not letting BIG TCP kick in naturally. I'd keep the defaults
in fast tests.

> (Note that apart from looking at ${KSFT_MACHINE_SLOW} in patch 2, I
> think the series looks good to me, and can probably be sent to net after
> the merge with net-next.)

Thanks! I'll wait for the selftest to be merged to net and send the follow-
ups there.

> Cheers,
> Matt