[PATCH net-next] selftests/bpf: make correct use of exit codes in bpf selftests

[PATCH net-next] selftests/bpf: make correct use of exit codes in bpf selftests

[Jesper Dangaard Brouer]

The selftests depend on using the shell exit code as a mean of
detecting the success or failure of test-binary executed.  The
appropiate output "[PASS]" or "[FAIL]" in generated by
tools/testing/selftests/lib.mk.

Notice that the exit code is masked with 255. Thus, be careful if
using the number of errors as the exits code, as 256 errors would be
seen as a success.

There are two standard defined exit(3) codes:
 /usr/include/stdlib.h
 #define EXIT_FAILURE    1       /* Failing exit status.  */
 #define EXIT_SUCCESS    0       /* Successful exit status.  */

Fix test_verifier.c to not use the negative value of variable
"results", but instead return EXIT_FAILURE.

Fix test_align.c and test_progs.c to actually use exit codes, before
they were always indicating success regardless of results.

Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
---
 tools/testing/selftests/bpf/test_align.c    |    2 +-
 tools/testing/selftests/bpf/test_progs.c    |    2 +-
 tools/testing/selftests/bpf/test_verifier.c |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_align.c b/tools/testing/selftests/bpf/test_align.c
index 1426594fdf6b..bccebd935907 100644
--- a/tools/testing/selftests/bpf/test_align.c
+++ b/tools/testing/selftests/bpf/test_align.c
@@ -428,7 +428,7 @@ static int do_test(unsigned int from, unsigned int to)
 	}
 	printf("Results: %d pass %d fail\n",
 	       all_pass, all_fail);
-	return 0;
+	return all_fail ? EXIT_FAILURE : EXIT_SUCCESS;
 }
 
 int main(int argc, char **argv)
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index fec13ab84fca..f10493d4c37c 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -497,5 +497,5 @@ int main(void)
 	test_bpf_obj_id();
 
 	printf("Summary: %d PASSED, %d FAILED\n", pass_cnt, error_cnt);
-	return 0;
+	return error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
 }
diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index cabb19b1e371..4ee4708b0d60 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -5418,7 +5418,7 @@ static int do_test(bool unpriv, unsigned int from, unsigned int to)
 	}
 
 	printf("Summary: %d PASSED, %d FAILED\n", passes, errors);
-	return errors ? -errors : 0;
+	return errors ? EXIT_FAILURE : EXIT_SUCCESS;
 }
 
 int main(int argc, char **argv)

Re: [PATCH net-next] selftests/bpf: make correct use of exit codes in bpf selftests

[Daniel Borkmann]

On 06/13/2017 03:17 PM, Jesper Dangaard Brouer wrote:
> The selftests depend on using the shell exit code as a mean of
> detecting the success or failure of test-binary executed.  The
> appropiate output "[PASS]" or "[FAIL]" in generated by
> tools/testing/selftests/lib.mk.
>
> Notice that the exit code is masked with 255. Thus, be careful if
> using the number of errors as the exits code, as 256 errors would be
> seen as a success.
>
> There are two standard defined exit(3) codes:
>   /usr/include/stdlib.h
>   #define EXIT_FAILURE    1       /* Failing exit status.  */
>   #define EXIT_SUCCESS    0       /* Successful exit status.  */
>
> Fix test_verifier.c to not use the negative value of variable
> "results", but instead return EXIT_FAILURE.
>
> Fix test_align.c and test_progs.c to actually use exit codes, before
> they were always indicating success regardless of results.
>
> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

Re: [PATCH net-next] selftests/bpf: make correct use of exit codes in bpf selftests

[Fengguang Wu]

On Tue, Jun 13, 2017 at 03:17:19PM +0200, Jesper Dangaard Brouer wrote:
>The selftests depend on using the shell exit code as a mean of
>detecting the success or failure of test-binary executed.  The
>appropiate output "[PASS]" or "[FAIL]" in generated by
>tools/testing/selftests/lib.mk.
>
>Notice that the exit code is masked with 255. Thus, be careful if
>using the number of errors as the exits code, as 256 errors would be

nit pick:

s/exits/exit/

> 	printf("Summary: %d PASSED, %d FAILED\n", passes, errors);
>-	return errors ? -errors : 0;
>+	return errors ? EXIT_FAILURE : EXIT_SUCCESS;

Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>

Thanks,
Fengguang

Re: [PATCH net-next] selftests/bpf: make correct use of exit codes in bpf selftests

[David Miller]

From: Jesper Dangaard Brouer <brouer@redhat.com>
Date: Tue, 13 Jun 2017 15:17:19 +0200

> The selftests depend on using the shell exit code as a mean of
> detecting the success or failure of test-binary executed.  The
> appropiate output "[PASS]" or "[FAIL]" in generated by
> tools/testing/selftests/lib.mk.
> 
> Notice that the exit code is masked with 255. Thus, be careful if
> using the number of errors as the exits code, as 256 errors would be
> seen as a success.
> 
> There are two standard defined exit(3) codes:
>  /usr/include/stdlib.h
>  #define EXIT_FAILURE    1       /* Failing exit status.  */
>  #define EXIT_SUCCESS    0       /* Successful exit status.  */
> 
> Fix test_verifier.c to not use the negative value of variable
> "results", but instead return EXIT_FAILURE.
> 
> Fix test_align.c and test_progs.c to actually use exit codes, before
> they were always indicating success regardless of results.
> 
> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>

Applied with commit log message corrected, thanks.