netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Haller <thaller@redhat.com>
To: NetFilter <netfilter-devel@vger.kernel.org>
Cc: Thomas Haller <thaller@redhat.com>
Subject: [PATCH nft v3 06/11] tests/shell: interpret an exit code of 77 from scripts as "skipped"
Date: Mon,  4 Sep 2023 15:48:08 +0200	[thread overview]
Message-ID: <20230904135135.1568180-7-thaller@redhat.com> (raw)
In-Reply-To: <20230904135135.1568180-1-thaller@redhat.com>

Allow scripts to indicate that a test could not run by exiting 77.

"77" is chosen as exit code from automake's testsuites ([1]). Compare to
git-bisect which chooses 125 to indicate skipped.

[1] https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html

Signed-off-by: Thomas Haller <thaller@redhat.com>
---
 tests/shell/helpers/test-wrapper.sh |  2 ++
 tests/shell/run-tests.sh            | 11 ++++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh
index 99546f060e26..8f0dc685e1fe 100755
--- a/tests/shell/helpers/test-wrapper.sh
+++ b/tests/shell/helpers/test-wrapper.sh
@@ -12,6 +12,8 @@ rc_test=0
 
 if [ "$rc_test" -eq 0 ] ; then
 	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-ok"
+elif [ "$rc_test" -eq 77 ] ; then
+	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-skipped"
 else
 	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-failed"
 fi
diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh
index 3d0ef6fa8499..f3b58e1e200b 100755
--- a/tests/shell/run-tests.sh
+++ b/tests/shell/run-tests.sh
@@ -271,6 +271,7 @@ fi
 
 echo ""
 ok=0
+skipped=0
 failed=0
 taint=0
 
@@ -366,6 +367,14 @@ for testfile in "${TESTS[@]}" ; do
 				msg_warn "[DUMP FAIL]	$testfile"
 			fi
 		fi
+	elif [ "$rc_got" -eq 77 ] ; then
+		((skipped++))
+		if [ "$VERBOSE" == "y" ] ; then
+			msg_warn "[SKIPPED]	$testfile"
+			[ ! -z "$test_output" ] && echo "$test_output"
+		else
+			msg_warn "[SKIPPED]	$testfile"
+		fi
 	else
 		((failed++))
 		if [ "$VERBOSE" == "y" ] ; then
@@ -390,7 +399,7 @@ echo ""
 kmemleak_found=0
 check_kmemleak_force
 
-msg_info "results: [OK] $ok [FAILED] $failed [TOTAL] $((ok+failed))"
+msg_info "results: [OK] $ok [SKIPPED] $skipped [FAILED] $failed [TOTAL] $((ok+skipped+failed))"
 
 kernel_cleanup
 
-- 
2.41.0


  parent reply	other threads:[~2023-09-04 13:53 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-04 13:48 [PATCH nft v3 00/11] tests/shell: allow running tests as Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 01/11] tests/shell: rework command line parsing in "run-tests.sh" Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 02/11] tests/shell: rework finding tests and add "--list-tests" option Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 03/11] tests/shell: check test names before start and support directories Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 04/11] tests/shell: export NFT_TEST_BASEDIR and NFT_TEST_TMPDIR for tests Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 05/11] tests/shell: run each test in separate namespace and allow rootless Thomas Haller
2023-09-04 13:48 ` Thomas Haller [this message]
2023-09-04 13:48 ` [PATCH nft v3 07/11] tests/shell: support --keep-logs option (NFT_TEST_KEEP_LOGS=y) to preserve test output Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 08/11] tests/shell: move the dump diff handling inside "test-wrapper.sh" Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 09/11] tests/shell: rework printing of test results Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 10/11] tests/shell: move taint check to "test-wrapper.sh" Thomas Haller
2023-09-04 13:48 ` [PATCH nft v3 11/11] tests/shell: support running tests in parallel Thomas Haller
2023-09-05 11:09 ` [PATCH nft v3 00/11] tests/shell: allow running tests as Florian Westphal
2023-09-05 12:03   ` Thomas Haller
2023-09-05 13:48     ` Florian Westphal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230904135135.1568180-7-thaller@redhat.com \
    --to=thaller@redhat.com \
    --cc=netfilter-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).