[PATCH mptcp-next] selftests: mptcp: tweak simult_flows for debug kernels.

[PATCH mptcp-next] selftests: mptcp: tweak simult_flows for debug kernels.

[Paolo Abeni]

The mentioned test measures the transfer run-time to verify
that the user-space program is able to use the full aggregate B/W.

Even on (virtual) link-speed-bound tests, debug kernel can slow
down the transfer enough to cause sporadic test failures.

Instead of unconditionally raising the maximum allowed run-time,
tweak when the running kernel is a debug one, and use some simple/
rough heuristic to guess such scenarios.

Note: this intentionally avoids looking for /boot/config-<version> as
the latter file is not always available in our reference CI
environments.

Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
 tools/testing/selftests/net/mptcp/simult_flows.sh | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh
index f441ff7904fc..141fcf0d40d1 100755
--- a/tools/testing/selftests/net/mptcp/simult_flows.sh
+++ b/tools/testing/selftests/net/mptcp/simult_flows.sh
@@ -12,6 +12,7 @@ timeout_test=$((timeout_poll * 2 + 1))
 test_cnt=1
 ret=0
 bail=0
+slack=50
 
 usage() {
 	echo "Usage: $0 [ -b ] [ -c ] [ -d ]"
@@ -52,6 +53,7 @@ setup()
 	cout=$(mktemp)
 	capout=$(mktemp)
 	size=$((2 * 2048 * 4096))
+
 	dd if=/dev/zero of=$small bs=4096 count=20 >/dev/null 2>&1
 	dd if=/dev/zero of=$large bs=4096 count=$((size / 4096)) >/dev/null 2>&1
 
@@ -104,6 +106,13 @@ setup()
 	ip -net "$ns3" route add default via dead:beef:3::2
 
 	ip netns exec "$ns3" ./pm_nl_ctl limits 1 1
+
+	# debug build can slow down measurably the test program
+	# we use quite tight time limit on the run-time, to ensure
+	# maximum B/W usage.
+	# Use the kmemleak file presence as a rough estimate for this being
+	# a debug kernel and increase the maximum run-time accordingly
+	[ -f /sys/kernel/debug/kmemleak ] && slack=$((slack+200))
 }
 
 # $1: ns, $2: port
@@ -241,7 +250,7 @@ run_test()
 
 	# mptcp_connect will do some sleeps to allow the mp_join handshake
 	# completion (see mptcp_connect): 200ms on each side, add some slack
-	time=$((time + 450))
+	time=$((time + 400 + $slack))
 
 	printf "%-60s" "$msg"
 	do_transfer $small $large $time
-- 
2.35.3

Re: selftests: mptcp: tweak simult_flows for debug kernels.: Tests Results

[MPTCP CI]

Hi Paolo,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5708948505362432
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5708948505362432/summary/summary.txt

- KVM Validation: debug:
  - Unstable: 3 failed test(s): packetdrill_add_addr selftest_diag selftest_mptcp_join 🔴:
  - Task: https://cirrus-ci.com/task/5145998551941120
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5145998551941120/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/727243b29682


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: several messages

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 3986 bytes --]

On Thu, 16 Jun 2022, Paolo Abeni wrote:

> The mentioned test measures the transfer run-time to verify
> that the user-space program is able to use the full aggregate B/W.
>
> Even on (virtual) link-speed-bound tests, debug kernel can slow
> down the transfer enough to cause sporadic test failures.
>
> Instead of unconditionally raising the maximum allowed run-time,
> tweak when the running kernel is a debug one, and use some simple/
> rough heuristic to guess such scenarios.
>
> Note: this intentionally avoids looking for /boot/config-<version> as
> the latter file is not always available in our reference CI
> environments.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>

Looks good, runs fine in my vm with debug kernel config:

Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>

> ---
> tools/testing/selftests/net/mptcp/simult_flows.sh | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh
> index f441ff7904fc..141fcf0d40d1 100755
> --- a/tools/testing/selftests/net/mptcp/simult_flows.sh
> +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh
> @@ -12,6 +12,7 @@ timeout_test=$((timeout_poll * 2 + 1))
> test_cnt=1
> ret=0
> bail=0
> +slack=50
>
> usage() {
> 	echo "Usage: $0 [ -b ] [ -c ] [ -d ]"
> @@ -52,6 +53,7 @@ setup()
> 	cout=$(mktemp)
> 	capout=$(mktemp)
> 	size=$((2 * 2048 * 4096))
> +
> 	dd if=/dev/zero of=$small bs=4096 count=20 >/dev/null 2>&1
> 	dd if=/dev/zero of=$large bs=4096 count=$((size / 4096)) >/dev/null 2>&1
>
> @@ -104,6 +106,13 @@ setup()
> 	ip -net "$ns3" route add default via dead:beef:3::2
>
> 	ip netns exec "$ns3" ./pm_nl_ctl limits 1 1
> +
> +	# debug build can slow down measurably the test program
> +	# we use quite tight time limit on the run-time, to ensure
> +	# maximum B/W usage.
> +	# Use the kmemleak file presence as a rough estimate for this being
> +	# a debug kernel and increase the maximum run-time accordingly
> +	[ -f /sys/kernel/debug/kmemleak ] && slack=$((slack+200))
> }
>
> # $1: ns, $2: port
> @@ -241,7 +250,7 @@ run_test()
>
> 	# mptcp_connect will do some sleeps to allow the mp_join handshake
> 	# completion (see mptcp_connect): 200ms on each side, add some slack
> -	time=$((time + 450))
> +	time=$((time + 400 + $slack))
>
> 	printf "%-60s" "$msg"
> 	do_transfer $small $large $time
> -- 
> 2.35.3
>
>
>

On Thu, 16 Jun 2022, MPTCP CI wrote:

> Hi Paolo,
>
> Thank you for your modifications, that's great!
>
> Our CI did some validations and here is its report:
>
> - KVM Validation: normal:
>  - Success! ✅:
>  - Task: https://cirrus-ci.com/task/5708948505362432
>  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5708948505362432/summary/summary.txt
>
> - KVM Validation: debug:
>  - Unstable: 3 failed test(s): packetdrill_add_addr selftest_diag selftest_mptcp_join 🔴:
>  - Task: https://cirrus-ci.com/task/5145998551941120
>  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5145998551941120/summary/summary.txt
>
> Initiator: Patchew Applier
> Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/727243b29682
>
>
> If there are some issues, you can reproduce them using the same environment as
> the one used by the CI thanks to a docker image, e.g.:
>
>    $ cd [kernel source code]
>    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
>        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
>        auto-debug
>
> For more details:
>
>    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
>
>
> Please note that despite all the efforts that have been already done to have a
> stable tests suite when executed on a public CI like here, it is possible some
> reported issues are not due to your modifications. Still, do not hesitate to
> help us improve that ;-)
>
> Cheers,
> MPTCP GH Action bot
> Bot operated by Matthieu Baerts (Tessares)
>
>

--
Mat Martineau
Intel

Re: [PATCH mptcp-next] selftests: mptcp: tweak simult_flows for debug kernels.

[Matthieu Baerts]

Hi Paolo, Mat,

On 16/06/2022 15:55, Paolo Abeni wrote:
> The mentioned test measures the transfer run-time to verify
> that the user-space program is able to use the full aggregate B/W.
> 
> Even on (virtual) link-speed-bound tests, debug kernel can slow
> down the transfer enough to cause sporadic test failures.
> 
> Instead of unconditionally raising the maximum allowed run-time,
> tweak when the running kernel is a debug one, and use some simple/
> rough heuristic to guess such scenarios.
> 
> Note: this intentionally avoids looking for /boot/config-<version> as
> the latter file is not always available in our reference CI
> environments.

Thank you for the patch and review!

Now in our tree (feat. for net-next but it could also go in 'fixes for
-net', no?) with Mat's RvB tag and a small diff, see below:

New patches for t/upstream:
- 87eefff9e9e8: selftests: mptcp: tweak simult_flows for debug kernels
- Results: 1445c5fa4907..065936dc9f58 (export)


Builds and tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20220620T163537
https://github.com/multipath-tcp/mptcp_net-next/actions/workflows/build-validation.yml?query=branch:export

> diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh
> index f441ff7904fc..141fcf0d40d1 100755
> --- a/tools/testing/selftests/net/mptcp/simult_flows.sh
> +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh

(...)

> @@ -241,7 +250,7 @@ run_test()
>  
>  	# mptcp_connect will do some sleeps to allow the mp_join handshake
>  	# completion (see mptcp_connect): 200ms on each side, add some slack
> -	time=$((time + 450))
> +	time=$((time + 400 + $slack))

(detail: I removed the extra '$' before 'slack' just for the uniformity :) )

Cheers,
Matt

>  
>  	printf "%-60s" "$msg"
>  	do_transfer $small $large $time

-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net