[PATCH dwarves 0/4] tests: unify behaviour, pass through fail

[PATCH dwarves 0/4] tests: unify behaviour, pass through fail

[Alan Maguire]

The "tests" script has recently been added to support running tests;
unify behaviour across all tests to

1. return 2 for cases where the test has to be skipped (versus 1
   for an actual failure)
2. for tests that require vmlinux, allow it to be speficied via
   environment variable; this allows the tests script to be run
   with a specific vmlinux not discoverable via pahole

Patches 1-3 handle updating individual tests; patch 4 ensures that
if we hit a failure, the tests script will return 1 to indicate
that a test failed.

Alan Maguire (4):
  tests/reproducible_builds: support envvar-specified vmlinux
  tests/btf_functions: return 2 to indicate test had to be skipped
  tests/reproducible_build: return 2 (skipped) if no vmlinux available
  tests: distinguish between failed/skipped tests

 tests/btf_functions.sh      |  4 ++--
 tests/reproducible_build.sh |  6 +++---
 tests/tests                 | 13 +++++++++++++
 3 files changed, 18 insertions(+), 5 deletions(-)

-- 
2.43.5

[PATCH dwarves 1/4] tests/reproducible_builds: support envvar-specified vmlinux

[Alan Maguire]

for running "tests" script on systems on which vmlinux is not
locatable via pahole, it is useful to be able to specify a
path to vmlinux that works across tests like this:

$ vmlinux=/path2/vmlinux bash ./tests

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tests/reproducible_build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
index e3c5c23..8a151a9 100755
--- a/tests/reproducible_build.sh
+++ b/tests/reproducible_build.sh
@@ -4,7 +4,7 @@
 # Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding
 # Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
 
-vmlinux=$1
+vmlinux=${vmlinux:-1}
 
 if [ -z "$vmlinux" ] ; then
 	vmlinux=$(pahole --running_kernel_vmlinux)
-- 
2.43.5

[PATCH dwarves 2/4] tests/btf_functions: return 2 to indicate test had to be skipped

[Alan Maguire]

When vmlinux is not available, return 2 to indicate test is skipped.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tests/btf_functions.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/btf_functions.sh b/tests/btf_functions.sh
index b8645cc..fbe68f9 100755
--- a/tests/btf_functions.sh
+++ b/tests/btf_functions.sh
@@ -32,13 +32,13 @@ if [ -z "$vmlinux" ] ; then
 	vmlinux=$(pahole --running_kernel_vmlinux)
 	if [ -z "$vmlinux" ] ; then
 		echo "Please specify a vmlinux file to operate on"
-		exit 1
+		exit 2
 	fi
 fi
 
 if [ ! -f "$vmlinux" ] ; then
 	echo "$vmlinux file not available, please specify another"
-	exit 1
+	exit 2
 fi
 
 outdir=$(mktemp -d /tmp/btf_functions.sh.XXXXXX)
-- 
2.43.5

[PATCH dwarves 3/4] tests/reproducible_build: return 2 (skipped) if no vmlinux available

[Alan Maguire]

Distinguish between test skipping (2) and test failure (1).

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tests/reproducible_build.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
index 8a151a9..a360379 100755
--- a/tests/reproducible_build.sh
+++ b/tests/reproducible_build.sh
@@ -10,13 +10,13 @@ if [ -z "$vmlinux" ] ; then
 	vmlinux=$(pahole --running_kernel_vmlinux)
 	if [ -z "$vmlinux" ] ; then
 		echo "Please specify a vmlinux file to operate on"
-		exit 1
+		exit 2
 	fi
 fi
 
 if [ ! -f "$vmlinux" ] ; then
 	echo "$vmlinux file not available, please specify another"
-	exit 1
+	exit 2
 fi
 
 outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX)
-- 
2.43.5

[PATCH dwarves 4/4] tests: distinguish between failed/skipped tests

[Alan Maguire]

When a test returns 2, it indicates it had to be skipped;
distingiush between this and other non-zero failures; the latter
should result in tests returning 1 to indicate something failed.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tests/tests | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/tests/tests b/tests/tests
index 38bb975..11921ad 100755
--- a/tests/tests
+++ b/tests/tests
@@ -5,11 +5,24 @@
 tests_dir=$(dirname $0)
 cd $tests_dir
 
+let status=0
 let nr=1
 for test in *.sh ; do
 	printf "%3d: " $nr
 	./$test
+	case $? in
+	0)
+		;;
+	2)
+		echo "skipping..."
+		;;
+	*)
+		status=1
+		;;
+	esac
 	let nr+=1
 done
 
 cd -
+
+exit $status
-- 
2.43.5

Re: [PATCH dwarves 0/4] tests: unify behaviour, pass through fail

[Arnaldo Carvalho de Melo]

On Tue, Oct 01, 2024 at 12:46:25PM +0100, Alan Maguire wrote:
> The "tests" script has recently been added to support running tests;
> unify behaviour across all tests to
> 
> 1. return 2 for cases where the test has to be skipped (versus 1
>    for an actual failure)
> 2. for tests that require vmlinux, allow it to be speficied via
>    environment variable; this allows the tests script to be run
>    with a specific vmlinux not discoverable via pahole
> 
> Patches 1-3 handle updating individual tests; patch 4 ensures that
> if we hit a failure, the tests script will return 1 to indicate
> that a test failed.
> 
> Alan Maguire (4):
>   tests/reproducible_builds: support envvar-specified vmlinux
>   tests/btf_functions: return 2 to indicate test had to be skipped
>   tests/reproducible_build: return 2 (skipped) if no vmlinux available
>   tests: distinguish between failed/skipped tests

Thanks, applied.

- Arnaldo