[PATCH net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop

[PATCH net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop

[Jakub Kicinski]

Looking at timestamped output of netdev CI reveals that
most of the time in forwarding tests for custom route
hashing is spent on a single case, namely the test which
uses ping (mausezahn does not support flow labels).

On a non-debug kernel we spend 714 of 730 total test
runtime (97%) on this test case. While having flow label
support in a traffic gen tool / mausezahn would be best,
we can significantly speed up the loop by putting ip vrf exec
outside of the iteration.

In a test of 1000 pings using a normal loop takes 50 seconds
to finish. While using:

  ip vrf exec $vrf sh -c "$loop-body"

takes 12 seconds (1/4 of the time).

Some of the slowness is likely due to our inefficient virtualization
setup, but even on my laptop running "ip link help" 16k times takes
25-30 seconds, so I think it's worth optimizing even for fastest
setups.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: shuah@kernel.org
CC: idosch@nvidia.com
CC: linux-kselftest@vger.kernel.org
---
 .../selftests/net/forwarding/custom_multipath_hash.sh     | 8 ++++----
 .../selftests/net/forwarding/gre_custom_multipath_hash.sh | 8 ++++----
 .../net/forwarding/ip6gre_custom_multipath_hash.sh        | 8 ++++----
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
index 1783c10215e5..7d531f7091e6 100755
--- a/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
+++ b/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
@@ -224,10 +224,10 @@ send_dst_ipv6()
 send_flowlabel()
 {
 	# Generate 16384 echo requests, each with a random flow label.
-	for _ in $(seq 1 16384); do
-		ip vrf exec v$h1 \
-			$PING6 2001:db8:4::2 -F 0 -c 1 -q >/dev/null 2>&1
-	done
+	ip vrf exec v$h1 sh -c \
+		"for _ in {1..16384}; do \
+			$PING6 2001:db8:4::2 -F 0 -c 1 -q >/dev/null 2>&1; \
+		done"
 }
 
 send_src_udp6()
diff --git a/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
index 9788bd0f6e8b..dda11a4a9450 100755
--- a/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
+++ b/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
@@ -319,10 +319,10 @@ send_dst_ipv6()
 send_flowlabel()
 {
 	# Generate 16384 echo requests, each with a random flow label.
-	for _ in $(seq 1 16384); do
-		ip vrf exec v$h1 \
-			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1
-	done
+	ip vrf exec v$h1 sh -c \
+		"for _ in {1..16384}; do \
+			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1; \
+		done"
 }
 
 send_src_udp6()
diff --git a/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
index 2ab9eaaa5532..e28b4a079e52 100755
--- a/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
+++ b/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
@@ -321,10 +321,10 @@ send_dst_ipv6()
 send_flowlabel()
 {
 	# Generate 16384 echo requests, each with a random flow label.
-	for _ in $(seq 1 16384); do
-		ip vrf exec v$h1 \
-			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1
-	done
+	ip vrf exec v$h1 sh -c \
+		"for _ in {1..16384}; do \
+			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1; \
+		done"
 }
 
 send_src_udp6()
-- 
2.46.0

Re: [PATCH net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop

[Ido Schimmel]

On Sat, Aug 17, 2024 at 01:36:59PM -0700, Jakub Kicinski wrote:
> Looking at timestamped output of netdev CI reveals that
> most of the time in forwarding tests for custom route
> hashing is spent on a single case, namely the test which
> uses ping (mausezahn does not support flow labels).
> 
> On a non-debug kernel we spend 714 of 730 total test
> runtime (97%) on this test case. While having flow label
> support in a traffic gen tool / mausezahn would be best,
> we can significantly speed up the loop by putting ip vrf exec
> outside of the iteration.
> 
> In a test of 1000 pings using a normal loop takes 50 seconds
> to finish. While using:
> 
>   ip vrf exec $vrf sh -c "$loop-body"
> 
> takes 12 seconds (1/4 of the time).
> 
> Some of the slowness is likely due to our inefficient virtualization
> setup, but even on my laptop running "ip link help" 16k times takes
> 25-30 seconds, so I think it's worth optimizing even for fastest
> setups.
> 
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Tested-by: Ido Schimmel <idosch@nvidia.com>

Thanks!

Re: [PATCH net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop

[Hangbin Liu]

On Sat, Aug 17, 2024 at 01:36:59PM -0700, Jakub Kicinski wrote:
> Looking at timestamped output of netdev CI reveals that
> most of the time in forwarding tests for custom route
> hashing is spent on a single case, namely the test which
> uses ping (mausezahn does not support flow labels).
> 
> On a non-debug kernel we spend 714 of 730 total test
> runtime (97%) on this test case. While having flow label
> support in a traffic gen tool / mausezahn would be best,
> we can significantly speed up the loop by putting ip vrf exec
> outside of the iteration.
> 
> In a test of 1000 pings using a normal loop takes 50 seconds
> to finish. While using:
> 
>   ip vrf exec $vrf sh -c "$loop-body"
> 
> takes 12 seconds (1/4 of the time).
> 
> Some of the slowness is likely due to our inefficient virtualization
> setup, but even on my laptop running "ip link help" 16k times takes
> 25-30 seconds, so I think it's worth optimizing even for fastest
> setups.
> 
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> CC: shuah@kernel.org
> CC: idosch@nvidia.com
> CC: linux-kselftest@vger.kernel.org
> ---
>  .../selftests/net/forwarding/custom_multipath_hash.sh     | 8 ++++----
>  .../selftests/net/forwarding/gre_custom_multipath_hash.sh | 8 ++++----
>  .../net/forwarding/ip6gre_custom_multipath_hash.sh        | 8 ++++----
>  3 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
> index 1783c10215e5..7d531f7091e6 100755
> --- a/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
> +++ b/tools/testing/selftests/net/forwarding/custom_multipath_hash.sh
> @@ -224,10 +224,10 @@ send_dst_ipv6()
>  send_flowlabel()
>  {
>  	# Generate 16384 echo requests, each with a random flow label.
> -	for _ in $(seq 1 16384); do
> -		ip vrf exec v$h1 \
> -			$PING6 2001:db8:4::2 -F 0 -c 1 -q >/dev/null 2>&1
> -	done
> +	ip vrf exec v$h1 sh -c \
> +		"for _ in {1..16384}; do \
> +			$PING6 2001:db8:4::2 -F 0 -c 1 -q >/dev/null 2>&1; \
> +		done"
>  }
>  
>  send_src_udp6()
> diff --git a/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
> index 9788bd0f6e8b..dda11a4a9450 100755
> --- a/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
> +++ b/tools/testing/selftests/net/forwarding/gre_custom_multipath_hash.sh
> @@ -319,10 +319,10 @@ send_dst_ipv6()
>  send_flowlabel()
>  {
>  	# Generate 16384 echo requests, each with a random flow label.
> -	for _ in $(seq 1 16384); do
> -		ip vrf exec v$h1 \
> -			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1
> -	done
> +	ip vrf exec v$h1 sh -c \
> +		"for _ in {1..16384}; do \
> +			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1; \
> +		done"
>  }
>  
>  send_src_udp6()
> diff --git a/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh b/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
> index 2ab9eaaa5532..e28b4a079e52 100755
> --- a/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
> +++ b/tools/testing/selftests/net/forwarding/ip6gre_custom_multipath_hash.sh
> @@ -321,10 +321,10 @@ send_dst_ipv6()
>  send_flowlabel()
>  {
>  	# Generate 16384 echo requests, each with a random flow label.
> -	for _ in $(seq 1 16384); do
> -		ip vrf exec v$h1 \
> -			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1
> -	done
> +	ip vrf exec v$h1 sh -c \
> +		"for _ in {1..16384}; do \
> +			$PING6 2001:db8:2::2 -F 0 -c 1 -q >/dev/null 2>&1; \
> +		done"
>  }
>  
>  send_src_udp6()
> -- 
> 2.46.0
> 

Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>

Re: [PATCH net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Sat, 17 Aug 2024 13:36:59 -0700 you wrote:
> Looking at timestamped output of netdev CI reveals that
> most of the time in forwarding tests for custom route
> hashing is spent on a single case, namely the test which
> uses ping (mausezahn does not support flow labels).
> 
> On a non-debug kernel we spend 714 of 730 total test
> runtime (97%) on this test case. While having flow label
> support in a traffic gen tool / mausezahn would be best,
> we can significantly speed up the loop by putting ip vrf exec
> outside of the iteration.
> 
> [...]

Here is the summary with links:
  - [net-next] selftests: net/forwarding: spawn sh inside vrf to speed up ping loop
    https://git.kernel.org/netdev/net-next/c/555e5531635a

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html