[PATCH net v4] selftests/net: big_tcp: return xfail on slow machines

[PATCH net v4] selftests/net: big_tcp: return xfail on slow machines

[Pablo Martin Medrano]

After debugging the following output for big_tcp.sh on a board:

CLI GSO | GW GRO | GW GSO | SER GRO
on        on       on       on      : [PASS]
on        off      on       off     : [PASS]
off       on       on       on      : [FAIL_on_link1]
on        on       off      on      : [FAIL_on_link1]

Davide Caratti found that by default the test duration 1s is too short
in slow systems to reach the correct cwd size necessary for tcp/ip to
generate at least one packet bigger than 65536 (matching the iptables
match on length rule the test evaluates)

This skips (with xfail) the aforementioned failing combinations when
KSFT_MACHINE_SLOW is set. For that the test has been modified to use
facilities from net/lib.sh.

The new output for the test will look like this (example with a forced
XFAIL)

Testing for BIG TCP:
      CLI GSO | GW GRO | GW GSO | SER GRO
TEST: on        on       on       on                    [ OK ]
TEST: on        off      on       off                   [ OK ]
TEST: off       on       on       on                    [XFAIL]

Fixes: a19747c3b9bf ("selftests: net: let big_tcp test cope with slow env")
Suggested-by: Davide Caratti <dcaratti@redhat.com>
Suggested-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Pablo Martin Medrano <pablmart@redhat.com>
---
Changes in v4:
- Remove unneded lines; local variable and return check, thanks Petr!

Changes in v3:
- Added 12 characters of length to the Fixes: tag
- Exit test on the first fail

Changes in v2:
- Don't break the loop and use lib.sh facilities (thanks Peter Machata)
- Rephrased the subject from "longer netperf session on slow machines"
  as the patch is not configuring a longer session but skipping
- Added tags and SOB and the Fixes: hash (thank you Davide Caratti)
- Link to v1: https://lore.kernel.org/all/b800a71479a24a4142542051636e980c3b547434.1739794830.git.pablmart@redhat.com/
---
 tools/testing/selftests/net/big_tcp.sh | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/tools/testing/selftests/net/big_tcp.sh b/tools/testing/selftests/net/big_tcp.sh
index 2db9d15cd45f..de9bda5f4b7c 100755
--- a/tools/testing/selftests/net/big_tcp.sh
+++ b/tools/testing/selftests/net/big_tcp.sh
@@ -21,8 +21,7 @@ CLIENT_GW6="2001:db8:1::2"
 MAX_SIZE=128000
 CHK_SIZE=65535
 
-# Kselftest framework requirement - SKIP code is 4.
-ksft_skip=4
+source lib.sh
 
 setup() {
 	ip netns add $CLIENT_NS
@@ -132,7 +131,6 @@ do_test() {
 	local gw_gro=$2
 	local gw_tso=$3
 	local ser_gro=$4
-	local ret="PASS"
 
 	ip net exec $CLIENT_NS ethtool -K link0 tso $cli_tso
 	ip net exec $ROUTER_NS ethtool -K link1 gro $gw_gro
@@ -143,21 +141,19 @@ do_test() {
 	start_counter link3 $SERVER_NS
 	do_netperf $CLIENT_NS
 
-	if check_counter link1 $ROUTER_NS; then
-		check_counter link3 $SERVER_NS || ret="FAIL_on_link3"
-	else
-		ret="FAIL_on_link1"
-	fi
+	check_counter link1 $ROUTER_NS
+	check_err $? "fail on link1"
+	check_counter link3 $SERVER_NS
+	check_err $? "fail on link3"
 
 	stop_counter link1 $ROUTER_NS
 	stop_counter link3 $SERVER_NS
-	printf "%-9s %-8s %-8s %-8s: [%s]\n" \
-		$cli_tso $gw_gro $gw_tso $ser_gro $ret
-	test $ret = "PASS"
+	log_test "$(printf "%-9s %-8s %-8s %-8s" \
+			$cli_tso $gw_gro $gw_tso $ser_gro)"
 }
 
 testup() {
-	echo "CLI GSO | GW GRO | GW GSO | SER GRO" && \
+	echo "      CLI GSO | GW GRO | GW GSO | SER GRO" && \
 	do_test "on"  "on"  "on"  "on"  && \
 	do_test "on"  "off" "on"  "off" && \
 	do_test "off" "on"  "on"  "on"  && \
@@ -176,7 +172,8 @@ if ! ip link help 2>&1 | grep gso_ipv4_max_size &> /dev/null; then
 fi
 
 trap cleanup EXIT
+xfail_on_slow
 setup && echo "Testing for BIG TCP:" && \
 NF=4 testup && echo "***v4 Tests Done***" && \
 NF=6 testup && echo "***v6 Tests Done***"
-exit $?
+exit $EXIT_STATUS
-- 
2.49.0

Re: [PATCH net v4] selftests/net: big_tcp: return xfail on slow machines

[Petr Machata]

Pablo Martin Medrano <pablmart@redhat.com> writes:

> After debugging the following output for big_tcp.sh on a board:
>
> CLI GSO | GW GRO | GW GSO | SER GRO
> on        on       on       on      : [PASS]
> on        off      on       off     : [PASS]
> off       on       on       on      : [FAIL_on_link1]
> on        on       off      on      : [FAIL_on_link1]
>
> Davide Caratti found that by default the test duration 1s is too short
> in slow systems to reach the correct cwd size necessary for tcp/ip to
> generate at least one packet bigger than 65536 (matching the iptables
> match on length rule the test evaluates)
>
> This skips (with xfail) the aforementioned failing combinations when
> KSFT_MACHINE_SLOW is set. For that the test has been modified to use
> facilities from net/lib.sh.
>
> The new output for the test will look like this (example with a forced
> XFAIL)
>
> Testing for BIG TCP:
>       CLI GSO | GW GRO | GW GSO | SER GRO
> TEST: on        on       on       on                    [ OK ]
> TEST: on        off      on       off                   [ OK ]
> TEST: off       on       on       on                    [XFAIL]
>
> Fixes: a19747c3b9bf ("selftests: net: let big_tcp test cope with slow env")
> Suggested-by: Davide Caratti <dcaratti@redhat.com>
> Suggested-by: Petr Machata <petrm@nvidia.com>
> Signed-off-by: Pablo Martin Medrano <pablmart@redhat.com>

Reviewed-by: Petr Machata <petrm@nvidia.com>

Re: [PATCH net v4] selftests/net: big_tcp: return xfail on slow machines

[Jakub Kicinski]

On Mon, 24 Mar 2025 19:54:02 +0100 Pablo Martin Medrano wrote:
> After debugging the following output for big_tcp.sh on a board:
> 
> CLI GSO | GW GRO | GW GSO | SER GRO
> on        on       on       on      : [PASS]
> on        off      on       off     : [PASS]
> off       on       on       on      : [FAIL_on_link1]
> on        on       off      on      : [FAIL_on_link1]
> 
> Davide Caratti found that by default the test duration 1s is too short
> in slow systems to reach the correct cwd size necessary for tcp/ip to
> generate at least one packet bigger than 65536 (matching the iptables
> match on length rule the test evaluates)
> 
> This skips (with xfail) the aforementioned failing combinations when
> KSFT_MACHINE_SLOW is set. For that the test has been modified to use
> facilities from net/lib.sh.
> 
> The new output for the test will look like this (example with a forced
> XFAIL)

Sorry I was AFK for 2 weeks, what happened to my suggestions of using
byte count rather than time ?