Linux Kernel Selftest development
 help / color / mirror / Atom feed
From: Petr Machata <petrm@nvidia.com>
To: "David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	<netdev@vger.kernel.org>
Cc: Shuah Khan <shuah@kernel.org>,
	Nikolay Aleksandrov <razor@blackwall.org>,
	Hangbin Liu <liuhangbin@gmail.com>,
	Vladimir Oltean <vladimir.oltean@nxp.com>,
	Benjamin Poirier <bpoirier@nvidia.com>,
	"Ido Schimmel" <idosch@nvidia.com>, Jiri Pirko <jiri@nvidia.com>,
	<linux-kselftest@vger.kernel.org>,
	Petr Machata <petrm@nvidia.com>, <mlxsw@nvidia.com>
Subject: [PATCH net-next 11/14] selftests: forwarding: Support for performance sensitive tests
Date: Tue, 26 Mar 2024 17:54:38 +0100	[thread overview]
Message-ID: <99a376a2d2ffdaeee7752b1910cb0c3ea5d80fbe.1711464583.git.petrm@nvidia.com> (raw)
In-Reply-To: <cover.1711464583.git.petrm@nvidia.com>

Several tests in the suite use large amounts of traffic to e.g. cause
congestion and evaluate RED or shaper performance. These tests will not run
well on a slow machine, be it one with heavy debug kernel, or a VM, or e.g.
a single-board computer. Allow users to specify an environment variable,
KSFT_MACHINE_SLOW=yes, to indicate that the tests are being run on one such
machine.

Performance sensitive tests can then use a new helper, xfail_on_slow(), to
mark parts of the test that are sensitive to low-performance machines.
The helper can be used to just mark the whole suite, like so:

	xfail_on_slow tests_run

... or, on the other side of the granularity spectrum, to override
individual checks:

	xfail_on_slow check_err $? "Expected much, got little."

Signed-off-by: Petr Machata <petrm@nvidia.com>
---
 tools/testing/selftests/net/forwarding/lib.sh | 23 ++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
index 370fc377249b..58df57855bff 100644
--- a/tools/testing/selftests/net/forwarding/lib.sh
+++ b/tools/testing/selftests/net/forwarding/lib.sh
@@ -79,6 +79,11 @@ declare -A NETIFS=(
 # Flags for TC filters.
 : "${TC_FLAG:=skip_hw}"
 
+# Whether the machine is "slow" -- i.e. might be incapable of running tests
+# involving heavy traffic. This might be the case on a debug kernel, a VM, or
+# e.g. a low-power board.
+: "${KSFT_MACHINE_SLOW:=no}"
+
 net_forwarding_dir=$(dirname "$(readlink -e "${BASH_SOURCE[0]}")")
 
 if [[ -f $net_forwarding_dir/forwarding.config ]]; then
@@ -407,13 +412,20 @@ ret_set_ksft_status()
 	fi
 }
 
+# Whether FAILs should be interpreted as XFAILs. Internal.
+FAIL_TO_XFAIL=
+
 check_err()
 {
 	local err=$1
 	local msg=$2
 
 	if ((err)); then
-		ret_set_ksft_status $ksft_fail "$msg"
+		if [[ $FAIL_TO_XFAIL = yes ]]; then
+			ret_set_ksft_status $ksft_xfail "$msg"
+		else
+			ret_set_ksft_status $ksft_fail "$msg"
+		fi
 	fi
 }
 
@@ -438,6 +450,15 @@ check_err_fail()
 	fi
 }
 
+xfail_on_slow()
+{
+	if [[ $KSFT_MACHINE_SLOW = yes ]]; then
+		FAIL_TO_XFAIL=yes "$@"
+	else
+		"$@"
+	fi
+}
+
 log_test_result()
 {
 	local test_name=$1; shift
-- 
2.43.0


  parent reply	other threads:[~2024-03-26 17:01 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-26 16:54 [PATCH net-next 00/14] selftests: Fixes for kernel CI Petr Machata
2024-03-26 16:54 ` [PATCH net-next 01/14] selftests: net: libs: Change variable fallback syntax Petr Machata
2024-03-26 16:54 ` [PATCH net-next 02/14] selftests: forwarding.config.sample: Move overrides to lib.sh Petr Machata
2024-03-26 16:54 ` [PATCH net-next 03/14] selftests: forwarding: README: Document customization Petr Machata
2024-03-26 16:54 ` [PATCH net-next 04/14] selftests: forwarding: ipip_lib: Do not import lib.sh Petr Machata
2024-03-26 16:54 ` [PATCH net-next 05/14] selftests: forwarding: Move several selftests Petr Machata
2024-03-26 16:54 ` [PATCH net-next 06/14] selftests: forwarding: Ditch skip_on_veth() Petr Machata
2024-03-26 16:54 ` [PATCH net-next 07/14] selftests: forwarding: Change inappropriate log_test_skip() calls Petr Machata
2024-03-26 16:54 ` [PATCH net-next 08/14] selftests: lib: Define more kselftest exit codes Petr Machata
2024-03-26 16:54 ` [PATCH net-next 09/14] selftests: forwarding: Have RET track kselftest framework constants Petr Machata
2024-03-26 16:54 ` [PATCH net-next 10/14] selftests: forwarding: Convert log_test() to recognize RET values Petr Machata
2024-03-26 16:54 ` Petr Machata [this message]
2024-03-26 16:54 ` [PATCH net-next 12/14] selftests: forwarding: Mark performance-sensitive tests Petr Machata
2024-03-26 16:54 ` [PATCH net-next 13/14] selftests: forwarding: router_mpath_nh_lib: Don't skip, xfail on veth Petr Machata
2024-03-26 16:54 ` [PATCH net-next 14/14] selftests: forwarding: Add a test for testing lib.sh functionality Petr Machata
2024-03-29  1:11 ` [PATCH net-next 00/14] selftests: Fixes for kernel CI patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=99a376a2d2ffdaeee7752b1910cb0c3ea5d80fbe.1711464583.git.petrm@nvidia.com \
    --to=petrm@nvidia.com \
    --cc=bpoirier@nvidia.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=idosch@nvidia.com \
    --cc=jiri@nvidia.com \
    --cc=kuba@kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=liuhangbin@gmail.com \
    --cc=mlxsw@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=razor@blackwall.org \
    --cc=shuah@kernel.org \
    --cc=vladimir.oltean@nxp.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox