public inbox for linux-kselftest@vger.kernel.org
 help / color / mirror / Atom feed
From: Tony Ambardar <tony.ambardar@gmail.com>
To: bpf@vger.kernel.org, linux-kselftest@vger.kernel.org
Cc: Tony Ambardar <Tony.Ambardar@gmail.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Shuah Khan <shuah@kernel.org>
Subject: [PATCH bpf-next v1] bpf/selftests: improve arg parsing in test_verifier
Date: Mon, 25 Sep 2023 16:37:02 -0700	[thread overview]
Message-ID: <20230925233702.19466-1-Tony.Ambardar@gmail.com> (raw)

Current test_verifier provides little feedback or argument validation,
instead silently falling back to running all tests in case of user error
or even expected use cases. Trying to do manual exploratory testing,
switching between kernel versions (e.g. with varying tests), or working
around problematic tests (e.g. kernel hangs/crashes) can be a frustrating
experience.

Rework argument parsing to be more robust and strict, and provide basic
help on errors. Clamp test ranges to valid values and add an option to
list available built-in tests ("-l"). Default "test_verifier" behaviour
(run all tests) is unchanged and backwards-compatible. Updated examples:

     $ test_verifier die die die     # previously ran all tests
     Usage: test_verifier -l | [-v|-vv] [<tst_lo> [<tst_hi>]]

     $ test_verifier 700 9999        # runs test subset from 700 to end

Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com>
---
 tools/testing/selftests/bpf/test_verifier.c | 54 ++++++++++++---------
 1 file changed, 30 insertions(+), 24 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index 98107e0452d3..3712b5363f60 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -10,9 +10,11 @@
 #include <endian.h>
 #include <asm/types.h>
 #include <linux/types.h>
+#include <linux/minmax.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <ctype.h>
 #include <unistd.h>
 #include <errno.h>
 #include <string.h>
@@ -1848,36 +1850,40 @@ int main(int argc, char **argv)
 {
 	unsigned int from = 0, to = ARRAY_SIZE(tests);
 	bool unpriv = !is_admin();
-	int arg = 1;
-
-	if (argc > 1 && strcmp(argv[1], "-v") == 0) {
+	int i, arg = 1;
+
+	while (argc > 1 && *argv[arg] == '-') {
+		if (strcmp(argv[arg], "-l") == 0) {
+			for (i = from; i < to; i++)
+				printf("#%d %s\n", i, tests[i].descr);
+			return EXIT_SUCCESS;
+		} else if (strcmp(argv[arg], "-v") == 0) {
+			verbose = true;
+			verif_log_level = 1;
+		} else if (strcmp(argv[arg], "-vv") == 0) {
+			verbose = true;
+			verif_log_level = 2;
+		} else
+			goto out_help;
 		arg++;
-		verbose = true;
-		verif_log_level = 1;
 		argc--;
 	}
-	if (argc > 1 && strcmp(argv[1], "-vv") == 0) {
-		arg++;
-		verbose = true;
-		verif_log_level = 2;
-		argc--;
-	}
-
-	if (argc == 3) {
-		unsigned int l = atoi(argv[arg]);
-		unsigned int u = atoi(argv[arg + 1]);
 
-		if (l < to && u < to) {
-			from = l;
-			to   = u + 1;
-		}
-	} else if (argc == 2) {
-		unsigned int t = atoi(argv[arg]);
+	for (i = 1; i <= 2 && argc > 1; i++, arg++, argc--) {
+		unsigned int t = min(atoi(argv[arg]), ARRAY_SIZE(tests) - 1);
 
-		if (t < to) {
+		if (!isdigit(*argv[arg]))
+			goto out_help;
+		if (i == 1)
 			from = t;
-			to   = t + 1;
-		}
+		to = t + 1;
+	}
+
+	if (argc > 1) {
+out_help:
+		printf("Usage: %s -l | [-v|-vv] [<tst_lo> [<tst_hi>]]\n",
+		       argv[0]);
+		return EXIT_FAILURE;
 	}
 
 	unpriv_disabled = get_unpriv_disabled();
-- 
2.34.1


             reply	other threads:[~2023-09-25 23:37 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-25 23:37 Tony Ambardar [this message]
2023-09-26 12:40 ` [PATCH bpf-next v1] bpf/selftests: improve arg parsing in test_verifier Jiri Olsa
2023-09-26 12:54 ` Jiri Olsa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230925233702.19466-1-Tony.Ambardar@gmail.com \
    --to=tony.ambardar@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=shuah@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox