[PATCH mptcp-net] selftests: mptcp: gives slow test-case more time

[PATCH mptcp-net] selftests: mptcp: gives slow test-case more time

[Paolo Abeni]

On slow or busy VM, some test-cases still fail because the
data transfer completes before the endpoint manipulation
actually took effect.

Address the issue by artificially increasing the runtime for
the relevant test-cases.

Fixes: ef360019db40 ("selftests: mptcp: signal addresses testcases")
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/309
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index f3dd5f2a0272..2eeaf4aca644 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -2152,7 +2152,7 @@ remove_tests()
 		pm_nl_set_limits $ns2 1 3
 		pm_nl_add_endpoint $ns2 10.0.3.2 flags subflow
 		pm_nl_add_endpoint $ns2 10.0.4.2 flags subflow
-		run_tests $ns1 $ns2 10.0.1.1 0 -1 -2 slow
+		run_tests $ns1 $ns2 10.0.1.1 0 -1 -2 speed_10
 		chk_join_nr 3 3 3
 		chk_add_nr 1 1
 		chk_rm_nr 2 2
@@ -2165,7 +2165,7 @@ remove_tests()
 		pm_nl_add_endpoint $ns1 10.0.3.1 flags signal
 		pm_nl_add_endpoint $ns1 10.0.4.1 flags signal
 		pm_nl_set_limits $ns2 3 3
-		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 slow
+		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 speed_10
 		chk_join_nr 3 3 3
 		chk_add_nr 3 3
 		chk_rm_nr 3 3 invert
@@ -2178,7 +2178,7 @@ remove_tests()
 		pm_nl_add_endpoint $ns1 10.0.3.1 flags signal
 		pm_nl_add_endpoint $ns1 10.0.14.1 flags signal
 		pm_nl_set_limits $ns2 3 3
-		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 slow
+		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 speed_10
 		chk_join_nr 1 1 1
 		chk_add_nr 3 3
 		chk_rm_nr 3 1 invert
-- 
2.37.3

Re: selftests: mptcp: gives slow test-case more time: Tests Results

[MPTCP CI]

Hi Paolo,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5922635212128256
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5922635212128256/summary/summary.txt

- {"code":404,"message":
  - "Can't find artifacts containing file conclusion.txt"}:
  - Task: https://cirrus-ci.com/task/5359685258706944
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5359685258706944/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/b3fe6bfab13a


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-net] selftests: mptcp: gives slow test-case more time

[Mat Martineau]

On Mon, 24 Oct 2022, Paolo Abeni wrote:

> On slow or busy VM, some test-cases still fail because the
> data transfer completes before the endpoint manipulation
> actually took effect.
>
> Address the issue by artificially increasing the runtime for
> the relevant test-cases.
>
> Fixes: ef360019db40 ("selftests: mptcp: signal addresses testcases")
> Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/309
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> tools/testing/selftests/net/mptcp/mptcp_join.sh | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> index f3dd5f2a0272..2eeaf4aca644 100755
> --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
> +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> @@ -2152,7 +2152,7 @@ remove_tests()
> 		pm_nl_set_limits $ns2 1 3
> 		pm_nl_add_endpoint $ns2 10.0.3.2 flags subflow
> 		pm_nl_add_endpoint $ns2 10.0.4.2 flags subflow
> -		run_tests $ns1 $ns2 10.0.1.1 0 -1 -2 slow
> +		run_tests $ns1 $ns2 10.0.1.1 0 -1 -2 speed_10
> 		chk_join_nr 3 3 3
> 		chk_add_nr 1 1
> 		chk_rm_nr 2 2
> @@ -2165,7 +2165,7 @@ remove_tests()
> 		pm_nl_add_endpoint $ns1 10.0.3.1 flags signal
> 		pm_nl_add_endpoint $ns1 10.0.4.1 flags signal
> 		pm_nl_set_limits $ns2 3 3
> -		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 slow
> +		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 speed_10
> 		chk_join_nr 3 3 3
> 		chk_add_nr 3 3
> 		chk_rm_nr 3 3 invert
> @@ -2178,7 +2178,7 @@ remove_tests()
> 		pm_nl_add_endpoint $ns1 10.0.3.1 flags signal
> 		pm_nl_add_endpoint $ns1 10.0.14.1 flags signal
> 		pm_nl_set_limits $ns2 3 3
> -		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 slow
> +		run_tests $ns1 $ns2 10.0.1.1 0 -3 0 speed_10
> 		chk_join_nr 1 1 1
> 		chk_add_nr 3 3
> 		chk_rm_nr 3 1 invert
> -- 
> 2.37.3

Looks good to me:

Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>

Do you want me to get this in to the export branch soon, or should we 
leave it queued for Matthieu?


--
Mat Martineau
Intel

Re: selftests: mptcp: gives slow test-case more time: Tests Results

[MPTCP CI]

Hi Paolo,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 1 failed test(s): packetdrill_add_addr 🔴:
  - Task: https://cirrus-ci.com/task/5863926364635136
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5863926364635136/summary/summary.txt

- KVM Validation: debug:
  - Critical: Global Timeout ❌:
  - Task: https://cirrus-ci.com/task/4839132940730368
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4839132940730368/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/337ba64c69c2


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-net] selftests: mptcp: gives slow test-case more time

[Matthieu Baerts]

Hi Paolo, Mat,

On 24/10/2022 18:27, Paolo Abeni wrote:
> On slow or busy VM, some test-cases still fail because the
> data transfer completes before the endpoint manipulation
> actually took effect.
> 
> Address the issue by artificially increasing the runtime for
> the relevant test-cases.

Thank you for the patch and the review! Sorry for the delay, I was
having a very good relax time in Italy ;)

Now in our tree (fix for -net) with Mat's RvB tag:

New patches for t/upstream-net:
- 46fa4797d3cc: selftests: mptcp: gives slow test-case more time
- Results: e29d415f8a2f..b05afbfd1a6e (export-net)

New patches for t/upstream:
- 46fa4797d3cc: selftests: mptcp: gives slow test-case more time
- Results: c2faaafd29b3..9bcf275fb027 (export)


Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export-net/20221031T144932
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221031T144932


Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net