netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Haller <thaller@redhat.com>
To: NetFilter <netfilter-devel@vger.kernel.org>
Cc: Thomas Haller <thaller@redhat.com>
Subject: [PATCH nft v4 08/17] tests/shell: interpret an exit code of 77 from scripts as "skipped"
Date: Tue,  5 Sep 2023 13:58:37 +0200	[thread overview]
Message-ID: <20230905115936.607599-9-thaller@redhat.com> (raw)
In-Reply-To: <20230905115936.607599-1-thaller@redhat.com>

Allow scripts to indicate that a test could not run by exiting 77.

"77" is chosen as exit code from automake's testsuites ([1]). Compare to
git-bisect which chooses 125 to indicate skipped.

[1] https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html

Signed-off-by: Thomas Haller <thaller@redhat.com>
---
 tests/shell/helpers/test-wrapper.sh |  2 ++
 tests/shell/run-tests.sh            | 11 ++++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh
index f811b44aab0d..0cf37f408003 100755
--- a/tests/shell/helpers/test-wrapper.sh
+++ b/tests/shell/helpers/test-wrapper.sh
@@ -14,6 +14,8 @@ rc_test=0
 
 if [ "$rc_test" -eq 0 ] ; then
 	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-ok"
+elif [ "$rc_test" -eq 77 ] ; then
+	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-skipped"
 else
 	echo "$rc_test" > "$NFT_TEST_TESTTMPDIR/rc_test-failed"
 fi
diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh
index 66da48ab893e..394073024f5f 100755
--- a/tests/shell/run-tests.sh
+++ b/tests/shell/run-tests.sh
@@ -330,6 +330,7 @@ fi
 
 echo ""
 ok=0
+skipped=0
 failed=0
 taint=0
 
@@ -430,6 +431,14 @@ for testfile in "${TESTS[@]}" ; do
 				msg_warn "[DUMP FAIL]	$testfile"
 			fi
 		fi
+	elif [ "$rc_got" -eq 77 ] ; then
+		((skipped++))
+		if [ "$VERBOSE" == "y" ] ; then
+			msg_warn "[SKIPPED]	$testfile"
+			[ ! -z "$test_output" ] && echo "$test_output"
+		else
+			msg_warn "[SKIPPED]	$testfile"
+		fi
 	else
 		((failed++))
 		if [ "$VERBOSE" == "y" ] ; then
@@ -454,7 +463,7 @@ echo ""
 kmemleak_found=0
 check_kmemleak_force
 
-msg_info "results: [OK] $ok [FAILED] $failed [TOTAL] $((ok+failed))"
+msg_info "results: [OK] $ok [SKIPPED] $skipped [FAILED] $failed [TOTAL] $((ok+skipped+failed))"
 
 kernel_cleanup
 
-- 
2.41.0


  parent reply	other threads:[~2023-09-05 16:26 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-05 11:58 [PATCH nft v4 00/17] tests/shell: allow running tests as Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 01/17] tests/shell: rework command line parsing in "run-tests.sh" Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 02/17] tests/shell: rework finding tests and add "--list-tests" option Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 03/17] tests/shell: check test names before start and support directories Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 04/17] tests/shell: export NFT_TEST_BASEDIR and NFT_TEST_TMPDIR for tests Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 05/17] tests/shell: normalize boolean configuration in environment variables Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 06/17] tests/shell: print test configuration Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 07/17] tests/shell: run each test in separate namespace and allow rootless Thomas Haller
2023-09-05 11:58 ` Thomas Haller [this message]
2023-09-05 11:58 ` [PATCH nft v4 09/17] tests/shell: support --keep-logs option (NFT_TEST_KEEP_LOGS=y) to preserve test output Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 10/17] tests/shell: move the dump diff handling inside "test-wrapper.sh" Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 11/17] tests/shell: rework printing of test results Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 12/17] tests/shell: move taint check to "test-wrapper.sh" Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 13/17] tests/shell: support running tests in parallel Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 14/17] tests/shell: bind mount private /var/run/netns in test container Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 15/17] tests/shell: skip test in rootless that hit socket buffer size limit Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 16/17] tests/shell: record the test duration for investigation Thomas Haller
2023-09-05 11:58 ` [PATCH nft v4 17/17] tests/shell: set TMPDIR for tests in "test-wrapper.sh" Thomas Haller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230905115936.607599-9-thaller@redhat.com \
    --to=thaller@redhat.com \
    --cc=netfilter-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).