[PATCH net-next 0/4] Improve stability of big_tcp_tunnels.sh selftest

[PATCH net-next 0/4] Improve stability of big_tcp_tunnels.sh selftest

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

This series addresses flakiness of big_tcp_tunnels.sh discussed at:

https://lore.kernel.org/netdev/349c582c-73e3-468c-91cd-ad6cc3562700@app.fastmail.com/

Lower the thresholds to avoid failures like this (number of packets
below the threshold):

https://netdev-ctrl.bots.linux.dev/logview.html?f=/logs/vmksft/net-dbg/results/754376/28-big-tcp-tunnels-sh/stdout

Tweak tcp_min_tso_segs and initcwnd to avoid failures like this (BIG TCP
doesn't kick in in slow configurations):

https://netdev-ctrl.bots.linux.dev/logview.html?f=/logs/vmksft/net-dbg/results/751124/21-big-tcp-tunnels-sh/stdout

Alice Mikityanska (4):
  selftests: net: Wait for netserver to launch
  selftests: net: Lower threshold on debug kernels for
    big_tcp_tunnels.sh
  selftests: net: Lower threshold with csum offload off in
    big_tcp_tunnels.sh
  selftests: net: Fix slow configurations in big_tcp_tunnels.sh

 .../testing/selftests/net/big_tcp_tunnels.sh  | 45 +++++++++++++++++--
 1 file changed, 41 insertions(+), 4 deletions(-)

-- 
2.55.0

[PATCH net-next 1/4] selftests: net: Wait for netserver to launch

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

Use wait_local_port_listen after starting netserver in
big_tcp_tunnels.sh to ensure it's listening when the test starts.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index d6513ed8d4e8..9b7d0456bcc1 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -3,6 +3,8 @@
 #
 # Testing for IPv4 and IPv6 BIG TCP over VXLAN and GENEVE tunnels.
 
+source "$(dirname "$0")/lib.sh"
+
 SERVER_NS=$(mktemp -u server-XXXXXXXX)
 SERVER_IP4="192.168.1.1"
 SERVER_IP6="2001:db8::1:1"
@@ -39,6 +41,7 @@ setup() {
 		gro_max_size 196608 gro_ipv4_max_size 196608
 
 	ip netns exec "$SERVER_NS" netserver >/dev/null
+	wait_local_port_listen "$SERVER_NS" 12865 tcp
 }
 
 setup_tunnel() {
-- 
2.55.0

[PATCH net-next 2/4] selftests: net: Lower threshold on debug kernels for big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

CONFIG_DEBUG_NET kernels run slower and generate fewer BIG TCP packets,
making the test flaky on upstream CI runners. Lower the default
threshold for those kernels.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index 9b7d0456bcc1..e27774eae20e 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -17,11 +17,25 @@ CLIENT_IP6="2001:db8::1:2"
 CLIENT_IP4_TUN="192.168.2.2"
 CLIENT_IP6_TUN="2001:db8::2:2"
 
-: "${PACKETS_THRESHOLD:=1000}"
-
 # Kselftest framework requirement - SKIP code is 4.
 ksft_skip=4
 
+is_debug() {
+	local config="/proc/config.gz"
+	[ -f "$config" ] || config="/boot/config-$(uname -r)"
+	[ -f "$config" ] || return 1 # No config found; assume non-debug.
+	gzip -dcfq "$config" | grep -qx 'CONFIG_DEBUG_NET=y' || return 1
+}
+
+if [ -z "$PACKETS_THRESHOLD" ]; then
+	if is_debug; then
+		echo 'Debug kernel detected, lowering the default threshold'
+		PACKETS_THRESHOLD=100
+	else
+		PACKETS_THRESHOLD=1000
+	fi
+fi
+
 setup() {
 	ip netns add "$SERVER_NS"
 	ip netns add "$CLIENT_NS"
-- 
2.55.0

[PATCH net-next 3/4] selftests: net: Lower threshold with csum offload off in big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

With checksum offload disabled, much fewer BIG TCP packets are generated
due to overall loss of throughput. Use a separate threshold in these
tests, which is 1/10 of the threshold set for the rest of tests.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index e27774eae20e..1f1c811c563c 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -114,6 +114,8 @@ cleanup() {
 }
 
 do_test() {
+	local packets_threshold="$PACKETS_THRESHOLD"
+
 	# When tx csum offload is off, software GSO is performed before passing the
 	# packet to veth. Check BIG TCP packets inside the VXLAN tunnel to verify
 	# the software checksum path: if the checksum code is broken, these packets
@@ -132,6 +134,7 @@ do_test() {
 		else
 			IPTABLES=ip6tables
 		fi
+		packets_threshold=$(( PACKETS_THRESHOLD / 10 ))
 	fi
 	if [ "$2" = 4 ]; then
 		IPTABLES_SACK=iptables
@@ -164,8 +167,8 @@ do_test() {
 	echo "Captured BIG TCP RX packets: $PACKETS_SERVER"
 	echo "Captured BIG TCP TX packets: $PACKETS_CLIENT"
 	echo "Captured TCP SACK packets: $PACKETS_SACK"
-	[ "$PACKETS_SERVER" -gt "$PACKETS_THRESHOLD" ] || return 1
-	[ "$PACKETS_CLIENT" -gt "$PACKETS_THRESHOLD" ] || return 1
+	[ "$PACKETS_SERVER" -gt "$packets_threshold" ] || return 1
+	[ "$PACKETS_CLIENT" -gt "$packets_threshold" ] || return 1
 	[ "$PACKETS_SACK" -lt "$(( PACKETS_CLIENT / 2 ))" ] || return 1
 }
 
-- 
2.55.0

[PATCH net-next 4/4] selftests: net: Fix slow configurations in big_tcp_tunnels.sh

[Alice Mikityanska]

From: Alice Mikityanska <alice@isovalent.com>

The combination of checksum offload disabled (that causes software GSO)
and CONFIG_DEBUG_NET=y is inherently slow. Depending on the CPU power
and load, RTT may increase, limiting sk_pacing_rate, so tcp_tso_autosize
caps SKBs at around 40 segments, and zero BIG TCP packets are produced.

Increase sysctl net.ipv4.tcp_min_tso_segs and set a bigger initial value
of CWND in these configurations to force BIG TCP.

Signed-off-by: Alice Mikityanska <alice@isovalent.com>
---
 tools/testing/selftests/net/big_tcp_tunnels.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/testing/selftests/net/big_tcp_tunnels.sh b/tools/testing/selftests/net/big_tcp_tunnels.sh
index 1f1c811c563c..222de7c0c294 100755
--- a/tools/testing/selftests/net/big_tcp_tunnels.sh
+++ b/tools/testing/selftests/net/big_tcp_tunnels.sh
@@ -56,6 +56,8 @@ setup() {
 
 	ip netns exec "$SERVER_NS" netserver >/dev/null
 	wait_local_port_listen "$SERVER_NS" 12865 tcp
+
+	DEFAULT_TCP_MIN_TSO_SEGS=$(ip netns exec "$CLIENT_NS" sysctl -n net.ipv4.tcp_min_tso_segs)
 }
 
 setup_tunnel() {
@@ -142,6 +144,21 @@ do_test() {
 		IPTABLES_SACK=ip6tables
 	fi
 
+	if [ "$3" != 'on' ] && is_debug; then
+		echo 'Slow configuration; increasing net.ipv4.tcp_min_tso_segs and initcwnd'
+		ip netns exec "$CLIENT_NS" sysctl -w net.ipv4.tcp_min_tso_segs=52
+		if [ "$2" = 4 ]; then
+			ip -netns "$CLIENT_NS" \
+			    route change 192.168.2.0/24 dev tun0 initcwnd 100
+		else
+			ip -netns "$CLIENT_NS" -6 \
+			    route change 2001:db8::2:0/112 dev tun0 initcwnd 100
+		fi
+	else
+		ip netns exec "$CLIENT_NS" \
+		    sysctl -w net.ipv4.tcp_min_tso_segs="$DEFAULT_TCP_MIN_TSO_SEGS"
+	fi
+
 	ip netns exec "$SERVER_NS" "$IPTABLES" -w -t raw -I PREROUTING -i "${CAPTURE_IFACE}1" -m length ! --length 0:65535 -m comment --comment "bigtcp"
 	ip netns exec "$CLIENT_NS" "$IPTABLES" -w -t raw -I OUTPUT -o "${CAPTURE_IFACE}0" -m length ! --length 0:65535 -m comment --comment "bigtcp"
 	ip netns exec "$SERVER_NS" "$IPTABLES_SACK" -w -t raw -I OUTPUT -o "tun1" -p tcp -m tcp --tcp-flags ACK ACK --tcp-option 5 -m comment --comment "sack"
-- 
2.55.0