[PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs

[PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs

[Matthieu Baerts (NGI0)]

The initial goal of this series is to improve the results when slow test
environments, e.g. when using QEmu without KVM.

I looked at the results from LKFT [1] and noticed that some mptcp_join
subtests were skipped because it was not possible to add IPTables rules.
It looks like some KConfig were missing. The 3 first patches should fix
that.

I also noticed that mptcp_join selftest stopped after a timeout of 20
minutes. We keep adding tests there, it is normal to increase this
timeout at some points, especially because we forced many subtests to
have a longer connections, to have enough time for some actions to
happen. Patch 4 fixes that.

Patch 5 decreases the BW in simult flows, hopefully reducing the load on
the CPU, and improving the stability.

Patch 6 is needed because our CI run MPTCP Connect twice: the second
time with '-m mmap'. When parsing the results, the tests from the two
runs have the same title, hence the same "fingerprint". Instead of
having a post-processing step modifying the results for this specific
case, it is now possible to set a prefix.

When looking at the results of mptcp_join, I also realised that the
'userspace' tests were blocking after having displayed the results.
That's because they wait for the end of the transfer, while they don't
need to: tests are the end are ignored because this is done in the
background (but that's fine, other tests check that). Patch 7 improves
the situation there.

Link: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240108/testrun/21966973/suite/kselftest-net-mptcp/tests/ [1]
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Matthieu Baerts (NGI0) (7):
      selftests: mptcp: add missing kconfig for NF Filter
      selftests: mptcp: add missing kconfig for NF Filter in v6
      selftests: mptcp: add missing kconfig for NF Mangle
      selftests: mptcp: increase timeout to 30 min
      selftests: mptcp: decrease BW in simult flows
      selftests: mptcp: allow changing subtests prefix
      selftests: mptcp: join: stop transfer when check is done

 tools/testing/selftests/net/mptcp/config          |  3 +++
 tools/testing/selftests/net/mptcp/mptcp_join.sh   | 27 +++++++++--------------
 tools/testing/selftests/net/mptcp/mptcp_lib.sh    |  2 +-
 tools/testing/selftests/net/mptcp/settings        |  2 +-
 tools/testing/selftests/net/mptcp/simult_flows.sh |  8 +++----
 5 files changed, 20 insertions(+), 22 deletions(-)
---
base-commit: 387cadd951df0629cc1098546acb3da13e8f6ebf
change-id: 20240110-selftests-mptcp-simult-slack-qemu-no-kvm-da67d9055c04

Best regards,
-- 
Matthieu Baerts (NGI0) <matttbe@kernel.org>

[PATCH mptcp-next 1/7] selftests: mptcp: add missing kconfig for NF Filter

[Matthieu Baerts (NGI0)]

Since the commit mentioned below, 'mptcp_join' selftests is using
IPTables to add rules to the Filter table.

It is then required to have IP_NF_FILTER KConfig.

This KConfig is usually enabled by default in many defconfig, but we
recently noticed that some CI were running our selftests without them
enabled.

Fixes: 8d014eaa9254 ("selftests: mptcp: add ADD_ADDR timeout test case")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
index e317c2e44dae..2a00bf4acdfa 100644
--- a/tools/testing/selftests/net/mptcp/config
+++ b/tools/testing/selftests/net/mptcp/config
@@ -22,6 +22,7 @@ CONFIG_NFT_TPROXY=m
 CONFIG_NFT_SOCKET=m
 CONFIG_IP_ADVANCED_ROUTER=y
 CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_NF_FILTER=m
 CONFIG_IP_NF_TARGET_REJECT=m
 CONFIG_IPV6_MULTIPLE_TABLES=y
 CONFIG_NET_ACT_CSUM=m

-- 
2.43.0

[PATCH mptcp-next 2/7] selftests: mptcp: add missing kconfig for NF Filter in v6

[Matthieu Baerts (NGI0)]

Since the commit mentioned below, 'mptcp_join' selftests is using
IPTables to add rules to the Filter table for IPv6.

It is then required to have IP6_NF_FILTER KConfig.

This KConfig is usually enabled by default in many defconfig, but we
recently noticed that some CI were running our selftests without them
enabled.

Fixes: 523514ed0a99 ("selftests: mptcp: add ADD_ADDR IPv6 test cases")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
index 2a00bf4acdfa..ec70b27288fc 100644
--- a/tools/testing/selftests/net/mptcp/config
+++ b/tools/testing/selftests/net/mptcp/config
@@ -20,6 +20,7 @@ CONFIG_NETFILTER_XT_TARGET_MARK=m
 CONFIG_NF_TABLES_INET=y
 CONFIG_NFT_TPROXY=m
 CONFIG_NFT_SOCKET=m
+CONFIG_IP6_NF_FILTER=m
 CONFIG_IP_ADVANCED_ROUTER=y
 CONFIG_IP_MULTIPLE_TABLES=y
 CONFIG_IP_NF_FILTER=m

-- 
2.43.0

Re: [PATCH mptcp-next 2/7] selftests: mptcp: add missing kconfig for NF Filter in v6

[Geliang Tang]

> Since the commit mentioned below, 'mptcp_join' selftests is using
> IPTables to add rules to the Filter table for IPv6.
>
> It is then required to have IP6_NF_FILTER KConfig.
>
> This KConfig is usually enabled by default in many defconfig, but we
> recently noticed that some CI were running our selftests without them
> enabled.
>
> Fixes: 523514ed0a99 ("selftests: mptcp: add ADD_ADDR IPv6 test cases")
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> ---
>  tools/testing/selftests/net/mptcp/config | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
> index 2a00bf4acdfa..ec70b27288fc 100644
> --- a/tools/testing/selftests/net/mptcp/config
> +++ b/tools/testing/selftests/net/mptcp/config
> @@ -20,6 +20,7 @@ CONFIG_NETFILTER_XT_TARGET_MARK=m
>  CONFIG_NF_TABLES_INET=y
>  CONFIG_NFT_TPROXY=m
>  CONFIG_NFT_SOCKET=m
> +CONFIG_IP6_NF_FILTER=m

How about adding this config right after CONFIG_IP_NF_TARGET_REJECT=m?

Since "IPv6: Netfilter Configuration" section in kconfig is always
after "IP: Netfilter Configuration" one.

Otherwise, the first three patches LGTM!

Reviewed-by: Geliang Tang <geliang@kernel.org>

-Geliang




>  CONFIG_IP_ADVANCED_ROUTER=y
>  CONFIG_IP_MULTIPLE_TABLES=y
>  CONFIG_IP_NF_FILTER=m
>
> --
> 2.43.0
>
>

Re: [PATCH mptcp-next 2/7] selftests: mptcp: add missing kconfig for NF Filter in v6

[Matthieu Baerts]

Hi Geliang,

Thank you for the review!

Hi Geliang,

Thank you for the review!

On 22/01/2024 02:47, Geliang Tang wrote:
>> Since the commit mentioned below, 'mptcp_join' selftests is using
>> IPTables to add rules to the Filter table for IPv6.
>>
>> It is then required to have IP6_NF_FILTER KConfig.
>>
>> This KConfig is usually enabled by default in many defconfig, but we
>> recently noticed that some CI were running our selftests without them
>> enabled.
>>
>> Fixes: 523514ed0a99 ("selftests: mptcp: add ADD_ADDR IPv6 test cases")
>> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
>> ---
>>  tools/testing/selftests/net/mptcp/config | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
>> index 2a00bf4acdfa..ec70b27288fc 100644
>> --- a/tools/testing/selftests/net/mptcp/config
>> +++ b/tools/testing/selftests/net/mptcp/config
>> @@ -20,6 +20,7 @@ CONFIG_NETFILTER_XT_TARGET_MARK=m
>>  CONFIG_NF_TABLES_INET=y
>>  CONFIG_NFT_TPROXY=m
>>  CONFIG_NFT_SOCKET=m
>> +CONFIG_IP6_NF_FILTER=m
> 
> How about adding this config right after CONFIG_IP_NF_TARGET_REJECT=m?
> 
> Since "IPv6: Netfilter Configuration" section in kconfig is always
> after "IP: Netfilter Configuration" one.

My intension was to place this new entry there to keep the file sorted
by alphabetical order. It is often recommended to do that for that kind
of files to ease the backports. So I put it before "IP_ADVANCED_ROUTER".

But now, I realise this file is not fully sorted by alphabetical order :)

I don't think the order matter, but I can place it after
"IPV6_MULTIPLE_TABLES". I can do that when applying the patches: I don't
think I need to send a v2 just for that.

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

[PATCH mptcp-next 3/7] selftests: mptcp: add missing kconfig for NF Mangle

[Matthieu Baerts (NGI0)]

Since the commit mentioned below, 'mptcp_join' selftests is using
IPTables to add rules to the Mangle table, only in IPv4.

This KConfig is usually enabled by default in many defconfig, but we
recently noticed that some CI were running our selftests without them
enabled.

Fixes: b6e074e171bc ("selftests: mptcp: add infinite map testcase")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
index ec70b27288fc..dfc8bdd0a11e 100644
--- a/tools/testing/selftests/net/mptcp/config
+++ b/tools/testing/selftests/net/mptcp/config
@@ -24,6 +24,7 @@ CONFIG_IP6_NF_FILTER=m
 CONFIG_IP_ADVANCED_ROUTER=y
 CONFIG_IP_MULTIPLE_TABLES=y
 CONFIG_IP_NF_FILTER=m
+CONFIG_IP_NF_MANGLE=m
 CONFIG_IP_NF_TARGET_REJECT=m
 CONFIG_IPV6_MULTIPLE_TABLES=y
 CONFIG_NET_ACT_CSUM=m

-- 
2.43.0

[PATCH mptcp-next 4/7] selftests: mptcp: increase timeout to 30 min

[Matthieu Baerts (NGI0)]

On very slow environments -- e.g. when QEmu is used without KVM --,
mptcp_join.sh selftest can take a bit more than 20 minutes. Bump the
default timeout by 50% as it seems normal to take that long on some
environments.

When a debug kernel config is used, this selftest will take even longer,
but that's certainly not a common test env to consider for the timeout.

The Fixes tag that has been picked here is there simply to help having
this patch backported to older stable versions. It is difficult to point
to the exact commit that made some env reaching the timeout from time to
time.

Fixes: d17b968b9876 ("selftests: mptcp: increase timeout to 20 minutes")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/settings | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/settings b/tools/testing/selftests/net/mptcp/settings
index 79b65bdf05db..abc5648b59ab 100644
--- a/tools/testing/selftests/net/mptcp/settings
+++ b/tools/testing/selftests/net/mptcp/settings
@@ -1 +1 @@
-timeout=1200
+timeout=1800

-- 
2.43.0

Re: [PATCH mptcp-next 4/7] selftests: mptcp: increase timeout to 30 min

[Paolo Abeni]

On Thu, 2024-01-18 at 19:27 +0100, Matthieu Baerts (NGI0) wrote:
> On very slow environments -- e.g. when QEmu is used without KVM --,
> mptcp_join.sh selftest can take a bit more than 20 minutes. Bump the
> default timeout by 50% as it seems normal to take that long on some
> environments.
> 
> When a debug kernel config is used, this selftest will take even longer,
> but that's certainly not a common test env to consider for the timeout.
> 
> The Fixes tag that has been picked here is there simply to help having
> this patch backported to older stable versions. It is difficult to point
> to the exact commit that made some env reaching the timeout from time to
> time.
> 
> Fixes: d17b968b9876 ("selftests: mptcp: increase timeout to 20 minutes")
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> ---
>  tools/testing/selftests/net/mptcp/settings | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/net/mptcp/settings b/tools/testing/selftests/net/mptcp/settings
> index 79b65bdf05db..abc5648b59ab 100644
> --- a/tools/testing/selftests/net/mptcp/settings
> +++ b/tools/testing/selftests/net/mptcp/settings
> @@ -1 +1 @@
> -timeout=1200
> +timeout=1800

Nice! the upstream CI is hitting this limit, too, and 1800 should avoit
that timeout.

Acked-by: Paolo Abeni <pabeni@redhat.com>

[PATCH mptcp-next 5/7] selftests: mptcp: decrease BW in simult flows

[Matthieu Baerts (NGI0)]

When running the simult_flow selftest in slow environments -- e.g. QEmu
without KVM support --, the results can be unstable. This selftest
checks if the aggregated bandwidth is (almost) fully used as expected.

To help improving the stability while still keeping the same validation
in place, the BW and the delay are reduced to lower the pressure on the
CPU.

Fixes: 1a418cb8e888 ("mptcp: simult flow self-tests")
Fixes: 219d04992b68 ("mptcp: push pending frames when subflow has free space")
Suggested-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/simult_flows.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh
index ae8ad5d6fb9d..0cc964e6f2c1 100755
--- a/tools/testing/selftests/net/mptcp/simult_flows.sh
+++ b/tools/testing/selftests/net/mptcp/simult_flows.sh
@@ -284,12 +284,12 @@ done
 
 setup
 run_test 10 10 0 0 "balanced bwidth"
-run_test 10 10 1 50 "balanced bwidth with unbalanced delay"
+run_test 10 10 1 25 "balanced bwidth with unbalanced delay"
 
 # we still need some additional infrastructure to pass the following test-cases
-run_test 30 10 0 0 "unbalanced bwidth"
-run_test 30 10 1 50 "unbalanced bwidth with unbalanced delay"
-run_test 30 10 50 1 "unbalanced bwidth with opposed, unbalanced delay"
+run_test 10 3 0 0 "unbalanced bwidth"
+run_test 10 3 1 25 "unbalanced bwidth with unbalanced delay"
+run_test 10 3 25 1 "unbalanced bwidth with opposed, unbalanced delay"
 
 mptcp_lib_result_print_all_tap
 exit $ret

-- 
2.43.0

[PATCH mptcp-next 6/7] selftests: mptcp: allow changing subtests prefix

[Matthieu Baerts (NGI0)]

If a CI executes the same selftest multiple times with different
options, all results from the same subtests will have the same title,
which confuse the CI. With the same title printed in TAP, the tests are
considered as the same ones.

Now, it is possible to override this prefix by using MPTCP_LIB_KSFT_TEST
env var, and have a different title.

While at it, use 'basename' to remove the suffix as well instead of
using an extra 'sed'.

Fixes: c4192967e62f ("selftests: mptcp: lib: format subtests results in TAP")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/mptcp_lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
index 022262a2cfe0..3a2abae5993e 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
@@ -6,7 +6,7 @@ readonly KSFT_FAIL=1
 readonly KSFT_SKIP=4
 
 # shellcheck disable=SC2155 # declare and assign separately
-readonly KSFT_TEST=$(basename "${0}" | sed 's/\.sh$//g')
+readonly KSFT_TEST="${MPTCP_LIB_KSFT_TEST:-$(basename "${0}" .sh)}"
 
 MPTCP_LIB_SUBTESTS=()
 

-- 
2.43.0

[PATCH mptcp-next 7/7] selftests: mptcp: join: stop transfer when check is done

[Matthieu Baerts (NGI0)]

Since the mentioned commit, the "userspace pm" tests of mptcp_join
selftests are launching the whole transfer in the background, do the
required checks, then wait for the end of transfer.

There is no need to wait longer, especially because the checks at the
end of the transfer are ignored (which is fine). This saves quite a few
seconds on slow environments.

While at it, use 'mptcp_lib_kill_wait()' helper everywhere, instead of
on a specific one with 'kill_tests_wait()'. Note that old versions will
need commit bdbef0a6ff10 ("selftests: mptcp: add mptcp_lib_kill_wait")
as well.

Fixes: 4369c198e599 ("selftests: mptcp: test userspace pm out of transfer")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 27 ++++++++++---------------
 1 file changed, 11 insertions(+), 16 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index 3a5b63026191..c07386e21e0a 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -643,13 +643,6 @@ kill_events_pids()
 	mptcp_lib_kill_wait $evts_ns2_pid
 }
 
-kill_tests_wait()
-{
-	#shellcheck disable=SC2046
-	kill -SIGUSR1 $(ip netns pids $ns2) $(ip netns pids $ns1)
-	wait
-}
-
 pm_nl_set_limits()
 {
 	local ns=$1
@@ -3453,7 +3446,7 @@ userspace_tests()
 		chk_mptcp_info subflows 0 subflows 0
 		chk_subflows_total 1 1
 		kill_events_pids
-		wait $tests_pid
+		mptcp_lib_kill_wait $tests_pid
 	fi
 
 	# userspace pm create destroy subflow
@@ -3475,7 +3468,7 @@ userspace_tests()
 		chk_mptcp_info subflows 0 subflows 0
 		chk_subflows_total 1 1
 		kill_events_pids
-		wait $tests_pid
+		mptcp_lib_kill_wait $tests_pid
 	fi
 
 	# userspace pm create id 0 subflow
@@ -3494,7 +3487,7 @@ userspace_tests()
 		chk_mptcp_info subflows 1 subflows 1
 		chk_subflows_total 2 2
 		kill_events_pids
-		wait $tests_pid
+		mptcp_lib_kill_wait $tests_pid
 	fi
 
 	# userspace pm remove initial subflow
@@ -3518,7 +3511,7 @@ userspace_tests()
 		chk_mptcp_info subflows 1 subflows 1
 		chk_subflows_total 1 1
 		kill_events_pids
-		wait $tests_pid
+		mptcp_lib_kill_wait $tests_pid
 	fi
 
 	# userspace pm send RM_ADDR for ID 0
@@ -3544,7 +3537,7 @@ userspace_tests()
 		chk_mptcp_info subflows 1 subflows 1
 		chk_subflows_total 1 1
 		kill_events_pids
-		wait $tests_pid
+		mptcp_lib_kill_wait $tests_pid
 	fi
 }
 
@@ -3558,7 +3551,8 @@ endpoint_tests()
 		pm_nl_set_limits $ns2 2 2
 		pm_nl_add_endpoint $ns1 10.0.2.1 flags signal
 		speed=slow \
-			run_tests $ns1 $ns2 10.0.1.1 2>/dev/null &
+			run_tests $ns1 $ns2 10.0.1.1 &
+		local tests_pid=$!
 
 		wait_mpj $ns1
 		pm_nl_check_endpoint "creation" \
@@ -3573,7 +3567,7 @@ endpoint_tests()
 		pm_nl_add_endpoint $ns2 10.0.2.2 flags signal
 		pm_nl_check_endpoint "modif is allowed" \
 			$ns2 10.0.2.2 id 1 flags signal
-		kill_tests_wait
+		mptcp_lib_kill_wait $tests_pid
 	fi
 
 	if reset "delete and re-add" &&
@@ -3582,7 +3576,8 @@ endpoint_tests()
 		pm_nl_set_limits $ns2 1 1
 		pm_nl_add_endpoint $ns2 10.0.2.2 id 2 dev ns2eth2 flags subflow
 		test_linkfail=4 speed=20 \
-			run_tests $ns1 $ns2 10.0.1.1 2>/dev/null &
+			run_tests $ns1 $ns2 10.0.1.1 &
+		local tests_pid=$!
 
 		wait_mpj $ns2
 		chk_subflow_nr "before delete" 2
@@ -3597,7 +3592,7 @@ endpoint_tests()
 		wait_mpj $ns2
 		chk_subflow_nr "after re-add" 2
 		chk_mptcp_info subflows 1 subflows 1
-		kill_tests_wait
+		mptcp_lib_kill_wait $tests_pid
 	fi
 }
 

-- 
2.43.0

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (GitHub Action) did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 1 failed test(s): selftest_mptcp_join 🔴:
  - Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/7574570879

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/2af4c20dfe1e


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (Cirrus) did some validations with a debug kernel and here is its report:

- KVM Validation: debug (except selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/4510682075365376
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4510682075365376/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Unstable: 1 failed test(s): selftest_mptcp_join 🔴:
  - Task: https://cirrus-ci.com/task/5636581982208000
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5636581982208000/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/2af4c20dfe1e


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: [PATCH mptcp-next 7/7] selftests: mptcp: join: stop transfer when check is done

[Geliang Tang]

>
> Since the mentioned commit, the "userspace pm" tests of mptcp_join
> selftests are launching the whole transfer in the background, do the
> required checks, then wait for the end of transfer.
>
> There is no need to wait longer, especially because the checks at the
> end of the transfer are ignored (which is fine). This saves quite a few
> seconds on slow environments.
>
> While at it, use 'mptcp_lib_kill_wait()' helper everywhere, instead of
> on a specific one with 'kill_tests_wait()'. Note that old versions will
> need commit bdbef0a6ff10 ("selftests: mptcp: add mptcp_lib_kill_wait")
> as well.

Great! This patch looks good.

Reviewed-and-Tested-by: Geliang Tang <geliang@kernel.org>

>
> Fixes: 4369c198e599 ("selftests: mptcp: test userspace pm out of transfer")
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> ---
>  tools/testing/selftests/net/mptcp/mptcp_join.sh | 27 ++++++++++---------------
>  1 file changed, 11 insertions(+), 16 deletions(-)
>
> diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> index 3a5b63026191..c07386e21e0a 100755
> --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
> +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> @@ -643,13 +643,6 @@ kill_events_pids()
>         mptcp_lib_kill_wait $evts_ns2_pid
>  }
>
> -kill_tests_wait()
> -{
> -       #shellcheck disable=SC2046
> -       kill -SIGUSR1 $(ip netns pids $ns2) $(ip netns pids $ns1)
> -       wait
> -}
> -
>  pm_nl_set_limits()
>  {
>         local ns=$1
> @@ -3453,7 +3446,7 @@ userspace_tests()
>                 chk_mptcp_info subflows 0 subflows 0
>                 chk_subflows_total 1 1
>                 kill_events_pids
> -               wait $tests_pid
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>
>         # userspace pm create destroy subflow
> @@ -3475,7 +3468,7 @@ userspace_tests()
>                 chk_mptcp_info subflows 0 subflows 0
>                 chk_subflows_total 1 1
>                 kill_events_pids
> -               wait $tests_pid
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>
>         # userspace pm create id 0 subflow
> @@ -3494,7 +3487,7 @@ userspace_tests()
>                 chk_mptcp_info subflows 1 subflows 1
>                 chk_subflows_total 2 2
>                 kill_events_pids
> -               wait $tests_pid
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>
>         # userspace pm remove initial subflow
> @@ -3518,7 +3511,7 @@ userspace_tests()
>                 chk_mptcp_info subflows 1 subflows 1
>                 chk_subflows_total 1 1
>                 kill_events_pids
> -               wait $tests_pid
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>
>         # userspace pm send RM_ADDR for ID 0
> @@ -3544,7 +3537,7 @@ userspace_tests()
>                 chk_mptcp_info subflows 1 subflows 1
>                 chk_subflows_total 1 1
>                 kill_events_pids
> -               wait $tests_pid
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>  }
>
> @@ -3558,7 +3551,8 @@ endpoint_tests()
>                 pm_nl_set_limits $ns2 2 2
>                 pm_nl_add_endpoint $ns1 10.0.2.1 flags signal
>                 speed=slow \
> -                       run_tests $ns1 $ns2 10.0.1.1 2>/dev/null &
> +                       run_tests $ns1 $ns2 10.0.1.1 &
> +               local tests_pid=$!
>
>                 wait_mpj $ns1
>                 pm_nl_check_endpoint "creation" \
> @@ -3573,7 +3567,7 @@ endpoint_tests()
>                 pm_nl_add_endpoint $ns2 10.0.2.2 flags signal
>                 pm_nl_check_endpoint "modif is allowed" \
>                         $ns2 10.0.2.2 id 1 flags signal
> -               kill_tests_wait
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>
>         if reset "delete and re-add" &&
> @@ -3582,7 +3576,8 @@ endpoint_tests()
>                 pm_nl_set_limits $ns2 1 1
>                 pm_nl_add_endpoint $ns2 10.0.2.2 id 2 dev ns2eth2 flags subflow
>                 test_linkfail=4 speed=20 \
> -                       run_tests $ns1 $ns2 10.0.1.1 2>/dev/null &
> +                       run_tests $ns1 $ns2 10.0.1.1 &
> +               local tests_pid=$!
>
>                 wait_mpj $ns2
>                 chk_subflow_nr "before delete" 2
> @@ -3597,7 +3592,7 @@ endpoint_tests()
>                 wait_mpj $ns2
>                 chk_subflow_nr "after re-add" 2
>                 chk_mptcp_info subflows 1 subflows 1
> -               kill_tests_wait
> +               mptcp_lib_kill_wait $tests_pid
>         fi
>  }
>
>
> --
> 2.43.0
>
>

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (GitHub Action) did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 3 failed test(s): packetdrill_regressions packetdrill_sockopts selftest_mptcp_join 🔴:
  - Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/7605531103

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a54e692a5628


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (Cirrus) did some validations with a debug kernel and here is its report:

- KVM Validation: debug (except selftest_mptcp_join):
  - Unstable: 3 failed test(s): packetdrill_mp_join packetdrill_sockopts selftest_diag 🔴:
  - Task: https://cirrus-ci.com/task/4673273665945600
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4673273665945600/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5799173572788224
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5799173572788224/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a54e692a5628


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (GitHub Action) did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 1 failed test(s): selftest_mptcp_join 🔴:
  - Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/7631904533

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/6c26191a31c7


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: selftests: mptcp: join: stop transfer when check is done: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI (Cirrus) did some validations with a debug kernel and here is its report:

- KVM Validation: debug (except selftest_mptcp_join):
  - Critical: 1 Call Trace(s) ❌:
  - Task: https://cirrus-ci.com/task/6177496371036160
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6177496371036160/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Critical: 1 Call Trace(s) ❌:
  - Task: https://cirrus-ci.com/task/5895199780503552
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5895199780503552/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/6c26191a31c7


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: [PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs

[Matthieu Baerts]

Hi Geliang, Paolo,

On 18/01/2024 19:27, Matthieu Baerts (NGI0) wrote:
> The initial goal of this series is to improve the results when slow test
> environments, e.g. when using QEmu without KVM.
> 
> I looked at the results from LKFT [1] and noticed that some mptcp_join
> subtests were skipped because it was not possible to add IPTables rules.
> It looks like some KConfig were missing. The 3 first patches should fix
> that.
> 
> I also noticed that mptcp_join selftest stopped after a timeout of 20
> minutes. We keep adding tests there, it is normal to increase this
> timeout at some points, especially because we forced many subtests to
> have a longer connections, to have enough time for some actions to
> happen. Patch 4 fixes that.
> 
> Patch 5 decreases the BW in simult flows, hopefully reducing the load on
> the CPU, and improving the stability.
> 
> Patch 6 is needed because our CI run MPTCP Connect twice: the second
> time with '-m mmap'. When parsing the results, the tests from the two
> runs have the same title, hence the same "fingerprint". Instead of
> having a post-processing step modifying the results for this specific
> case, it is now possible to set a prefix.
> 
> When looking at the results of mptcp_join, I also realised that the
> 'userspace' tests were blocking after having displayed the results.
> That's because they wait for the end of the transfer, while they don't
> need to: tests are the end are ignored because this is done in the
> background (but that's fine, other tests check that). Patch 7 improves
> the situation there.

Thank you for the reviews!

Now in our tree (fix for -net) with the modification demanded by Geliang
on patch 3/7:

New patches for t/upstream-net and t/upstream:
- a773245de129: selftests: mptcp: add missing kconfig for NF Filter
- 616af1a08349: selftests: mptcp: add missing kconfig for NF Filter in v6
- 2d4ccf16a1ac: selftests: mptcp: add missing kconfig for NF Mangle
- ac8595b0b31c: selftests: mptcp: increase timeout to 30 min
- ba7e09d80ffe: selftests: mptcp: decrease BW in simult flows
- e8a61a14a58f: selftests: mptcp: allow changing subtests prefix
- eb5f4f5f0a80: selftests: mptcp: join: stop transfer when check is done
- Results: 6d56aa24e733..b4671e59d316 (export-net)
- Results: cdd4fe00f4c5..81c0f68b0316 (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export-net/20240130T121223
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20240130T121223

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.