[Intel-gfx] [PATCH i-g-t v2 00/17] Fix IGT Kunit implementation issues

[Intel-gfx] [PATCH i-g-t v2 00/17] Fix IGT Kunit implementation issues

[Janusz Krzysztofik]

v2: Fix incorrect and missing includes in the test source file,
  - add license and copyright clauses to the test source file.

Janusz Krzysztofik (17):
  lib/kunit: Drop unused file stream
  lib/kunit: Stop loading kunit module explicitly
  lib/kunit: Fix struct kmod_module kunit_kmod not freed
  lib/kunit: Optimize calls to igt_success/skip/fail()
  lib/kunit: Fix illegal igt_fail() calls inside subtest body
  lib/ktap: Make sure we fail on premature cancel
  lib/ktap: Don't ignore interrupt signals
  lib/kunit: Cancel KTP parser on module load failure
  lib/ktap: Drop is_running flag
  lib/ktap: Read /dev/kmsg in blocking mode
  lib/kunit: Fail / skip on kernel taint
  lib/ktap: Use IGT linked lists for storing KTAP results
  lib/ktap: Reimplement KTAP parser
  lib/kunit: Load test modules in background
  lib/kunit: Parse KTAP report from the main process thread
  lib/kunit: Strip "_test" or "_kunit" suffix from subtest names
  lib/kunit: Omit suite name prefix if the same as subtest name

 lib/igt_kmod.c              | 284 ++++++++----
 lib/igt_ktap.c              | 833 ++++++++++++------------------------
 lib/igt_ktap.h              |  28 +-
 lib/tests/igt_ktap_parser.c | 246 +++++++++++
 lib/tests/meson.build       |   1 +
 tests/drm_mm.c              |  42 +-
 6 files changed, 739 insertions(+), 695 deletions(-)
 create mode 100644 lib/tests/igt_ktap_parser.c

-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 01/17] lib/kunit: Drop unused file stream

[Janusz Krzysztofik]

In the process of reviewing patches that introduced kunit support, I asked
once if we could use line buffered input instead of explicitly looking for
newlines in KTAP data.  While my idea was wrong because reading raw data
from /dev/kmsg already returns full log records that always end with a
newline, conversion of /dev/kmsg file descriptor to a file stream with
freopen() was added to the code.  However, that file stream has never been
used for line buffered input.  While the file stream is passed to
functions that actually read the data, there it is converted back to a
file descriptor with fileno() and the data is read with read().

Drop the unnecessary conversions and teach functions to accept and process
just the file descriptor of /dev/kmsg.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 12 +---------
 lib/igt_ktap.c | 62 +++++++++++++++++++++++---------------------------
 lib/igt_ktap.h |  2 +-
 3 files changed, 31 insertions(+), 45 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 6205871791..97667a896f 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -758,7 +758,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
 {
 	struct igt_ktest tst;
 	struct kmod_module *kunit_kmod;
-	FILE *f;
 	bool is_builtin;
 	int ret;
 	struct ktap_test_results *results;
@@ -774,7 +773,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
 
 	if (igt_ktest_begin(&tst) != 0) {
 		igt_warn("Unable to begin ktest for %s\n", module_name);
-
 		igt_ktest_fini(&tst);
 		igt_fail(IGT_EXIT_ABORT);
 	}
@@ -791,14 +789,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
 		goto unload;
 	}
 
-	f = fdopen(tst.kmsg, "r");
-
-	if (f == NULL) {
-		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer\n");
-		fail = true;
-		goto unload;
-	}
-
 	/* The KUnit module is required for running any KUnit tests */
 	ret = igt_kmod_load("kunit", NULL);
 	if (ret) {
@@ -814,7 +804,7 @@ static void __igt_kunit(const char *module_name, const char *opts)
 
 	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
 
-	results = ktap_parser_start(f, is_builtin);
+	results = ktap_parser_start(tst.kmsg, is_builtin);
 
 	ret = igt_kmod_load(module_name, opts);
 	if (ret) {
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index ecdcb8d83d..123a40d183 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -16,7 +16,7 @@
 #define DELIMITER "-"
 
 struct ktap_parser_args {
-	FILE *fp;
+	int fd;
 	bool is_builtin;
 	volatile bool is_running;
 	int ret;
@@ -24,7 +24,7 @@ struct ktap_parser_args {
 
 static struct ktap_test_results results;
 
-static int log_to_end(enum igt_log_level level, FILE *f,
+static int log_to_end(enum igt_log_level level, int fd,
 		      char *record, const char *format, ...) __attribute__((format(printf, 4, 5)));
 
 /**
@@ -39,12 +39,11 @@ static int log_to_end(enum igt_log_level level, FILE *f,
  *
  * Returns: 0 for success, or -2 if there's an error reading from the file
  */
-static int log_to_end(enum igt_log_level level, FILE *f,
+static int log_to_end(enum igt_log_level level, int fd,
 		      char *record, const char *format, ...)
 {
 	va_list args;
 	const char *lend;
-	int f_fd = fileno(f);
 
 	/* Cutoff after newline character, in order to not display garbage */
 	char *cutoff = strchr(record, '\n');
@@ -61,7 +60,7 @@ static int log_to_end(enum igt_log_level level, FILE *f,
 	while (*lend == '\0') {
 		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
 
-		while (read(f_fd, record, BUF_LEN) < 0) {
+		while (read(fd, record, BUF_LEN) < 0) {
 			if (!READ_ONCE(ktap_args.is_running)) {
 				igt_warn("ktap parser stopped\n");
 				return -2;
@@ -157,8 +156,8 @@ static int tap_version_present(char* record, bool print_info)
 
 /**
  * find_next_tap_subtest:
- * @fp: FILE pointer
- * @record: buffer used to read fp
+ * @fd: file descriptor
+ * @record: buffer used to read fd
  * @is_builtin: whether KUnit is built-in or not
  *
  * Returns:
@@ -167,11 +166,10 @@ static int tap_version_present(char* record, bool print_info)
  * -2 if there are problems while reading the file.
  * any other value corresponds to the amount of cases of the next (sub)test
  */
-static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool is_builtin)
+static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_builtin)
 {
 	const char *test_lookup_str, *subtest_lookup_str, *name_rptr;
 	long test_count;
-	int fp_fd = fileno(fp);
 	char *cutoff;
 
 	test_name[0] = '\0';
@@ -184,7 +182,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
 		return -1;
 
 	if (is_builtin) {
-		while (read(fp_fd, record, BUF_LEN) < 0) {
+		while (read(fd, record, BUF_LEN) < 0) {
 			if (!READ_ONCE(ktap_args.is_running)) {
 				igt_warn("ktap parser stopped\n");
 				return -2;
@@ -228,7 +226,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
 		if (cutoff)
 			cutoff[0] = '\0';
 
-		while (read(fp_fd, record, BUF_LEN) < 0) {
+		while (read(fd, record, BUF_LEN) < 0) {
 			if (!READ_ONCE(ktap_args.is_running)) {
 				igt_warn("ktap parser stopped\n");
 				return -2;
@@ -265,7 +263,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
 		igt_info("Missing test count\n");
 		if (test_name[0] == '\0')
 			return 0;
-		if (log_to_end(IGT_LOG_INFO, fp, record,
+		if (log_to_end(IGT_LOG_INFO, fd, record,
 				"Running some tests in: %s\n",
 				test_name) < 0)
 			return -2;
@@ -275,7 +273,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
 		return 0;
 	}
 
-	if (log_to_end(IGT_LOG_INFO, fp, record,
+	if (log_to_end(IGT_LOG_INFO, fd, record,
 			"Executing %ld tests in: %s\n",
 			test_count, test_name) < 0)
 		return -2;
@@ -285,8 +283,8 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
 
 /**
  * parse_kmsg_for_tap:
- * @fp: FILE pointer
- * @record: buffer used to read fp
+ * @fd: file descriptor
+ * @record: buffer used to read fd
  * @test_name: buffer to store the test name
  *
  * Returns:
@@ -295,7 +293,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
  * -1 if a test failed
  * -2 if there are problems reading the file
  */
-static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
+static int parse_kmsg_for_tap(int fd, char *record, char *test_name)
 {
 	const char *lstart, *ok_lookup_str, *nok_lookup_str,
 	      *ok_rptr, *nok_rptr, *comment_start, *value_parse_start;
@@ -324,7 +322,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
 		while (!isspace(*test_name_end))
 			test_name_end++;
 		*test_name_end = '\0';
-		if (log_to_end(IGT_LOG_WARN, fp, record,
+		if (log_to_end(IGT_LOG_WARN, fd, record,
 			       "%s", lstart) < 0)
 			return -2;
 		return -1;
@@ -338,7 +336,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
 		value_parse_start = comment_start;
 
 		if (lookup_value(value_parse_start, "fail: ") > 0) {
-			if (log_to_end(IGT_LOG_WARN, fp, record,
+			if (log_to_end(IGT_LOG_WARN, fd, record,
 				       "%s", lstart) < 0)
 				return -2;
 			return -1;
@@ -362,7 +360,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
 
 /**
  * parse_tap_level:
- * @fp: FILE pointer
+ * @fd: file descriptor
  * @base_test_name: test_name from upper recursion level
  * @test_count: test_count of this level
  * @failed_tests: top level failed_tests pointer
@@ -373,10 +371,9 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
  * 0 if succeded
  * -1 if error occurred
  */
-static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool *failed_tests,
+static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *failed_tests,
 			   bool *found_tests, bool is_builtin)
 {
-	int fp_fd = fileno(fp);
 	char record[BUF_LEN + 1];
 	struct ktap_test_results_element *r, *temp;
 	int internal_test_count;
@@ -384,7 +381,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
 	char base_test_name_for_next_level[BUF_LEN + 1];
 
 	for (int i = 0; i < test_count; i++) {
-		while (read(fp_fd, record, BUF_LEN) < 0) {
+		while (read(fd, record, BUF_LEN) < 0) {
 			if (!READ_ONCE(ktap_args.is_running)) {
 				igt_warn("ktap parser stopped\n");
 				return -1;
@@ -409,7 +406,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
 		/* Sublevel found */
 		if (tap_version_present(record, false))
 		{
-			internal_test_count = find_next_tap_subtest(fp, record, test_name,
+			internal_test_count = find_next_tap_subtest(fd, record, test_name,
 								    is_builtin);
 			switch (internal_test_count) {
 			case -2:
@@ -433,7 +430,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
 				memcpy(base_test_name_for_next_level + strlen(base_test_name_for_next_level),
 				       test_name, BUF_LEN - strlen(base_test_name_for_next_level));
 
-				if (parse_tap_level(fp, base_test_name_for_next_level,
+				if (parse_tap_level(fd, base_test_name_for_next_level,
 						    internal_test_count, failed_tests, found_tests,
 						    is_builtin) == -1)
 					return -1;
@@ -441,7 +438,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
 			}
 		}
 
-		switch (parse_kmsg_for_tap(fp, record, test_name)) {
+		switch (parse_kmsg_for_tap(fd, record, test_name)) {
 		case -2:
 			return -1;
 		case -1:
@@ -516,8 +513,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
  */
 void *igt_ktap_parser(void *unused)
 {
-	FILE *fp = ktap_args.fp;
-	int fp_fd = fileno(fp);
+	int fd = ktap_args.fd;
 	char record[BUF_LEN + 1];
 	bool is_builtin = ktap_args.is_builtin;
 	char test_name[BUF_LEN + 1];
@@ -534,7 +530,7 @@ igt_ktap_parser_start:
 	test_name[0] = '\0';
 	test_name[BUF_LEN] = '\0';
 
-	while (read(fp_fd, record, BUF_LEN) < 0) {
+	while (read(fd, record, BUF_LEN) < 0) {
 		if (!READ_ONCE(ktap_args.is_running)) {
 			igt_warn("ktap parser stopped\n");
 			goto igt_ktap_parser_end;
@@ -553,7 +549,7 @@ igt_ktap_parser_start:
 		}
 	}
 
-	test_count = find_next_tap_subtest(fp, record, test_name, is_builtin);
+	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);
 
 	switch (test_count) {
 	case -2:
@@ -569,7 +565,7 @@ igt_ktap_parser_start:
 	default:
 		found_tests = true;
 
-		if (parse_tap_level(fp, test_name, test_count, &failed_tests, &found_tests,
+		if (parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests,
 				    is_builtin) == -1)
 			goto igt_ktap_parser_end;
 
@@ -578,7 +574,7 @@ igt_ktap_parser_start:
 
 	/* Parse topmost level */
 	test_name[0] = '\0';
-	parse_tap_level(fp, test_name, test_count, &failed_tests, &found_tests, is_builtin);
+	parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests, is_builtin);
 
 igt_ktap_parser_end:
 	results.still_running = false;
@@ -593,13 +589,13 @@ igt_ktap_parser_end:
 
 static pthread_t ktap_parser_thread;
 
-struct ktap_test_results *ktap_parser_start(FILE *fp, bool is_builtin)
+struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 {
 	results.head = NULL;
 	pthread_mutex_init(&results.mutex, NULL);
 	results.still_running = true;
 
-	ktap_args.fp = fp;
+	ktap_args.fd = fd;
 	ktap_args.is_builtin = is_builtin;
 	ktap_args.is_running = true;
 	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
index 34fe095720..ea57c2bb9b 100644
--- a/lib/igt_ktap.h
+++ b/lib/igt_ktap.h
@@ -44,7 +44,7 @@ struct ktap_test_results {
 
 
 
-struct ktap_test_results *ktap_parser_start(FILE *fp, bool is_builtin);
+struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin);
 int ktap_parser_stop(void);
 
 #endif /* IGT_KTAP_H */
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 02/17] lib/kunit: Stop loading kunit module explicitly

[Janusz Krzysztofik]

Since kmod functions we use for module loading can process module
dependencies, there is no need to load the "kunit" module explicitly
before loading a kunit test module.  For the same reason we already don't
unload the "kunit" module explicitly on cleanup.  Drop the unnecessary
operation.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 97667a896f..faf31afabc 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -789,12 +789,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
 		goto unload;
 	}
 
-	/* The KUnit module is required for running any KUnit tests */
-	ret = igt_kmod_load("kunit", NULL);
-	if (ret) {
-		skip = ret;
-		goto unload;
-	}
 	ret = kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod);
 	if (ret) {
 		igt_warn("Unable to load KUnit\n");
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 03/17] lib/kunit: Fix struct kmod_module kunit_kmod not freed

[Janusz Krzysztofik]

We obtain a kmod_module structure for kunit module in order to check if
it is modular or built-in, then we never release that structure.  Fix it.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index faf31afabc..34ddec3fad 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -797,6 +797,7 @@ static void __igt_kunit(const char *module_name, const char *opts)
 	}
 
 	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
+	kmod_module_unref(kunit_kmod);
 
 	results = ktap_parser_start(tst.kmsg, is_builtin);
 
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 04/17] lib/kunit: Optimize calls to igt_success/skip/fail()

[Janusz Krzysztofik]

Calling igt_success() explicitly at the end of subtest body is not needed.
Other calls to igt_success() can be usually avoided by inverting test
result checks.  Optimize the code that now calls igt_success().

Moreover, using more advanced variants of igt_skip() and igt_fail() where
applicable makes the code more compact.  Go for it.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 34ddec3fad..1d1cd51170 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -814,12 +814,8 @@ static void __igt_kunit(const char *module_name, const char *opts)
 		if (READ_ONCE(results->head) != NULL) {
 			pthread_mutex_lock(&results->mutex);
 
-			igt_dynamic(results->head->test_name) {
-				if (READ_ONCE(results->head->passed))
-					igt_success();
-				else
-					igt_fail(IGT_EXIT_FAILURE);
-			}
+			igt_dynamic(results->head->test_name)
+				igt_assert(READ_ONCE(results->head->passed));
 
 			temp = results->head;
 			results->head = results->head->next;
@@ -834,8 +830,7 @@ unload:
 
 	igt_ktest_fini(&tst);
 
-	if (skip)
-		igt_skip("Skipping test, as probing KUnit module returned %d", skip);
+	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
 
 	if (fail)
 		igt_fail(IGT_EXIT_ABORT);
@@ -844,9 +839,6 @@ unload:
 
 	if (ret != 0)
 		igt_fail(IGT_EXIT_ABORT);
-
-	if (ret == 0)
-		igt_success();
 }
 
 void igt_kunit(const char *module_name, const char *name, const char *opts)
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 05/17] lib/kunit: Fix illegal igt_fail() calls inside subtest body

[Janusz Krzysztofik]

In a body of a subtest with dynamic sub-subtests, it is illegal to call
igt_fail() and its variants from outside of a dynamic sub-subtest body.
On the other hand, it is perfectly legal to call either igt_skip() and
friends or __igt_abort() or its variant from there.

In the current implementation of igt_kunit(), there are several places
where igt_fail() is called despite being illegal.  Moreover, it is called
with IGT_EXIT_ABORT as an argument with no good reason for using such
aggressive method that forces CI to trigger system reboot (in most cases
igt_runner can decide if abort is required).

Follow igt_kselftests() pattern more closely, where similar setup and
cleanup operations are performed but their potential errors are processed
in a more friendly way.  Move common cleanup and their corresponding setup
steps out of the subtest body.  Place the latter as requirements in a
preceding igt_fixture section.  Replace remaining illegal igt_fail() calls
with more friendly skips.  Let igt_runner decide if abort is needed.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 75 +++++++++++++++-----------------------------------
 1 file changed, 22 insertions(+), 53 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 1d1cd51170..78b8eb8f53 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -754,59 +754,27 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
  *
  * Returns: IGT default codes
  */
-static void __igt_kunit(const char *module_name, const char *opts)
+static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 {
-	struct igt_ktest tst;
 	struct kmod_module *kunit_kmod;
 	bool is_builtin;
 	int ret;
 	struct ktap_test_results *results;
 	struct ktap_test_results_element *temp;
-	int skip = 0;
-	bool fail = false;
-
-	/* get normalized module name */
-	if (igt_ktest_init(&tst, module_name) != 0) {
-		igt_warn("Unable to initialize ktest for %s\n", module_name);
-		igt_fail(IGT_EXIT_ABORT);
-	}
-
-	if (igt_ktest_begin(&tst) != 0) {
-		igt_warn("Unable to begin ktest for %s\n", module_name);
-		igt_ktest_fini(&tst);
-		igt_fail(IGT_EXIT_ABORT);
-	}
 
-	if (tst.kmsg < 0) {
-		igt_warn("Could not open /dev/kmsg\n");
-		fail = true;
-		goto unload;
-	}
+	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
 
-	if (lseek(tst.kmsg, 0, SEEK_END)) {
-		igt_warn("Could not seek the end of /dev/kmsg\n");
-		fail = true;
-		goto unload;
-	}
-
-	ret = kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod);
-	if (ret) {
-		igt_warn("Unable to load KUnit\n");
-		skip = ret;
-		goto unload;
-	}
+	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
 
+	igt_skip_on(kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod));
 	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
 	kmod_module_unref(kunit_kmod);
 
-	results = ktap_parser_start(tst.kmsg, is_builtin);
+	results = ktap_parser_start(tst->kmsg, is_builtin);
 
-	ret = igt_kmod_load(module_name, opts);
-	if (ret) {
-		skip = ret;
-		igt_warn("Unable to load %s module\n", module_name);
-		ret = ktap_parser_stop();
-		goto unload;
+	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
+		igt_ignore_warn(ktap_parser_stop());
+		igt_skip("Unable to load %s module\n", tst->module_name);
 	}
 
 	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
@@ -825,24 +793,21 @@ static void __igt_kunit(const char *module_name, const char *opts)
 		}
 	}
 
-unload:
-	igt_ktest_end(&tst);
-
-	igt_ktest_fini(&tst);
-
-	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
-
-	if (fail)
-		igt_fail(IGT_EXIT_ABORT);
-
 	ret = ktap_parser_stop();
 
-	if (ret != 0)
-		igt_fail(IGT_EXIT_ABORT);
+	igt_skip_on_f(ret, "KTAP parser failed\n");
 }
 
 void igt_kunit(const char *module_name, const char *name, const char *opts)
 {
+	struct igt_ktest tst;
+
+	if (igt_ktest_init(&tst, module_name) != 0)
+		return;
+
+	igt_fixture
+		igt_require(igt_ktest_begin(&tst) == 0);
+
 	/*
 	 * We need to use igt_subtest here, as otherwise it may crash with:
 	 *  skipping is allowed only in fixtures, subtests or igt_simple_main
@@ -854,7 +819,11 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
 		name = module_name;
 
 	igt_subtest_with_dynamic(name)
-		__igt_kunit(module_name, opts);
+		__igt_kunit(&tst, opts);
+
+	igt_ktest_end(&tst);
+
+	igt_ktest_fini(&tst);
 }
 
 static int open_parameters(const char *module_name)
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 06/17] lib/ktap: Make sure we fail on premature cancel

[Janusz Krzysztofik]

After loading a kunit test module that executes some kunit test cases, we
evaluate overall result of an IGT subtest that corresponds to that module
based on an error code returned by kunit_parser_stop() helper, obtained
from a .ret field of a ktap_args structure.  That code is now assigned to
that structure field right before completion of the KTAP parser thread
start routine.  If the thread is canceled for some reason then the return
code will be undefined.

Initialize the return code on KTAP parser startup with a value that
indicates a failure, then change it to success when so indicated by result
of KTAP parsing.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_ktap.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 123a40d183..84fb13218f 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -579,9 +579,7 @@ igt_ktap_parser_start:
 igt_ktap_parser_end:
 	results.still_running = false;
 
-	if (failed_tests || !found_tests)
-		ktap_args.ret = IGT_EXIT_FAILURE;
-	else
+	if (found_tests && !failed_tests)
 		ktap_args.ret = IGT_EXIT_SUCCESS;
 
 	return NULL;
@@ -598,6 +596,7 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 	ktap_args.fd = fd;
 	ktap_args.is_builtin = is_builtin;
 	ktap_args.is_running = true;
+	ktap_args.ret = IGT_EXIT_FAILURE;
 	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
 
 	return &results;
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 07/17] lib/ktap: Don't ignore interrupt signals

[Janusz Krzysztofik]

While reading KTAP data from /dev/kmsg we now ignore interrupt signals
that may occur during read() and we continue reading the data.  No
explanation has been provided on what that could be needed for.

Always return with an error code to the caller when read() fails with
errno == EINTR, so igt_runner has no problems with killing us promptly
on timeout.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_ktap.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 84fb13218f..3cfb55ec97 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -67,7 +67,7 @@ static int log_to_end(enum igt_log_level level, int fd,
 			}
 
 			if (errno == EINTR)
-				continue;
+				return -2;
 
 			if (errno == EPIPE) {
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
@@ -189,7 +189,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 			}
 
 			if (errno == EINTR)
-				continue;
+				return -2;
 
 			if (errno == EPIPE) {
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
@@ -233,7 +233,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 			}
 
 			if (errno == EINTR)
-				continue;
+				return -2;
 
 			if (errno == EPIPE) {
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
@@ -388,7 +388,7 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 			}
 
 			if (errno == EINTR)
-				continue;
+				return -1;
 
 			if (errno == EAGAIN)
 				/* No records available */
@@ -541,7 +541,7 @@ igt_ktap_parser_start:
 			continue;
 
 		if (errno == EINTR)
-			continue;
+			goto igt_ktap_parser_end;
 
 		if (errno == EPIPE) {
 			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 08/17] lib/kunit: Cancel KTP parser on module load failure

[Janusz Krzysztofik]

For our KTAP parser to be running in parallel with kunit test module
loading, we now start it in a separate thread before we load the module.
If the module loading fails then we join the KTAP parser thread right
after that failure.  If the KTAP thread sleeps for some reason then we
can fail to break the test immediately.

Cancel the KTAP parser thread right after module load error and before
joining it.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 1 +
 lib/igt_ktap.c | 6 ++++++
 lib/igt_ktap.h | 1 +
 3 files changed, 8 insertions(+)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 78b8eb8f53..fb0bd21ee5 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -773,6 +773,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 	results = ktap_parser_start(tst->kmsg, is_builtin);
 
 	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
+		ktap_parser_cancel();
 		igt_ignore_warn(ktap_parser_stop());
 		igt_skip("Unable to load %s module\n", tst->module_name);
 	}
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 3cfb55ec97..1e75b2ec23 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -602,6 +602,12 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 	return &results;
 }
 
+void ktap_parser_cancel(void)
+{
+	ktap_args.is_running = false;
+	pthread_cancel(ktap_parser_thread);
+}
+
 int ktap_parser_stop(void)
 {
 	ktap_args.is_running = false;
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
index ea57c2bb9b..991800e912 100644
--- a/lib/igt_ktap.h
+++ b/lib/igt_ktap.h
@@ -45,6 +45,7 @@ struct ktap_test_results {
 
 
 struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin);
+void ktap_parser_cancel(void);
 int ktap_parser_stop(void);
 
 #endif /* IGT_KTAP_H */
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 09/17] lib/ktap: Drop is_running flag

[Janusz Krzysztofik]

Since we now call pthread_cancel() when we want to stop KTAP parser before
it completes, and we take care of returning failure in that case as a
result of KTAP parsing, we no longer need to check a flag that indicates
whether we should continue parsing or return a failure.  Drop that flag.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_ktap.c | 32 --------------------------------
 1 file changed, 32 deletions(-)

diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 1e75b2ec23..fe77b62680 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -18,7 +18,6 @@
 struct ktap_parser_args {
 	int fd;
 	bool is_builtin;
-	volatile bool is_running;
 	int ret;
 } ktap_args;
 
@@ -61,11 +60,6 @@ static int log_to_end(enum igt_log_level level, int fd,
 		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
 
 		while (read(fd, record, BUF_LEN) < 0) {
-			if (!READ_ONCE(ktap_args.is_running)) {
-				igt_warn("ktap parser stopped\n");
-				return -2;
-			}
-
 			if (errno == EINTR)
 				return -2;
 
@@ -183,11 +177,6 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 
 	if (is_builtin) {
 		while (read(fd, record, BUF_LEN) < 0) {
-			if (!READ_ONCE(ktap_args.is_running)) {
-				igt_warn("ktap parser stopped\n");
-				return -2;
-			}
-
 			if (errno == EINTR)
 				return -2;
 
@@ -227,11 +216,6 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 			cutoff[0] = '\0';
 
 		while (read(fd, record, BUF_LEN) < 0) {
-			if (!READ_ONCE(ktap_args.is_running)) {
-				igt_warn("ktap parser stopped\n");
-				return -2;
-			}
-
 			if (errno == EINTR)
 				return -2;
 
@@ -382,11 +366,6 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 
 	for (int i = 0; i < test_count; i++) {
 		while (read(fd, record, BUF_LEN) < 0) {
-			if (!READ_ONCE(ktap_args.is_running)) {
-				igt_warn("ktap parser stopped\n");
-				return -1;
-			}
-
 			if (errno == EINTR)
 				return -1;
 
@@ -523,19 +502,11 @@ void *igt_ktap_parser(void *unused)
 	failed_tests = false;
 	found_tests = false;
 
-	if (!READ_ONCE(ktap_args.is_running))
-		goto igt_ktap_parser_end;
-
 igt_ktap_parser_start:
 	test_name[0] = '\0';
 	test_name[BUF_LEN] = '\0';
 
 	while (read(fd, record, BUF_LEN) < 0) {
-		if (!READ_ONCE(ktap_args.is_running)) {
-			igt_warn("ktap parser stopped\n");
-			goto igt_ktap_parser_end;
-		}
-
 		if (errno == EAGAIN)
 			/* No records available */
 			continue;
@@ -595,7 +566,6 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 
 	ktap_args.fd = fd;
 	ktap_args.is_builtin = is_builtin;
-	ktap_args.is_running = true;
 	ktap_args.ret = IGT_EXIT_FAILURE;
 	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
 
@@ -604,13 +574,11 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 
 void ktap_parser_cancel(void)
 {
-	ktap_args.is_running = false;
 	pthread_cancel(ktap_parser_thread);
 }
 
 int ktap_parser_stop(void)
 {
-	ktap_args.is_running = false;
 	pthread_join(ktap_parser_thread, NULL);
 	return ktap_args.ret;
 }
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 10/17] lib/ktap: Read /dev/kmsg in blocking mode

[Janusz Krzysztofik]

We obtain KTAP report from /dev/kmsg.  That file is now opened from
igt_ktest_begin(), a function originally designed for i915 selftests and
now reused with kunit tests.  The original intention of opening that file
was to dump kernel messages to stderr on selftest error.  For that
purpose, the file is now opened in non-blocking mode so we don't end up
waiting for more kernel messages than already available.  Since our ktap
parser code reuses the file descriptor, we now have to loop over
EAGAIN responses, waiting for more KTAP data.  Since we have no sleeps
inside those loops, extremely high CPU usage can be observed.

Simplify reading KTAP reports by first switching the file descriptor back
to blocking mode.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c |  7 ++++-
 lib/igt_ktap.c | 81 ++++++++++++++------------------------------------
 2 files changed, 28 insertions(+), 60 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index fb0bd21ee5..020df286b8 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -24,6 +24,7 @@
 #include <ctype.h>
 #include <signal.h>
 #include <errno.h>
+#include <fcntl.h>
 #include <sys/utsname.h>
 
 #include "igt_aux.h"
@@ -758,12 +759,16 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 {
 	struct kmod_module *kunit_kmod;
 	bool is_builtin;
-	int ret;
 	struct ktap_test_results *results;
 	struct ktap_test_results_element *temp;
+	int flags, ret;
 
 	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
 
+	flags = fcntl(tst->kmsg, F_GETFL, 0) & ~O_NONBLOCK;
+	igt_skip_on_f(fcntl(tst->kmsg, F_SETFL, flags) == -1,
+		      "Could not set /dev/kmsg to blocking mode\n");
+
 	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
 
 	igt_skip_on(kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod));
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index fe77b62680..165f6b2cce 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -59,20 +59,12 @@ static int log_to_end(enum igt_log_level level, int fd,
 	while (*lend == '\0') {
 		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
 
-		while (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EINTR)
-				return -2;
-
-			if (errno == EPIPE) {
+		if (read(fd, record, BUF_LEN) < 0) {
+			if (errno == EPIPE)
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-				return -2;
-			}
+			else if (errno != EINTR)
+				igt_warn("an error occurred while reading kmsg: %m\n");
 
-			if (errno == EAGAIN)
-				/* No records available */
-				continue;
-
-			igt_warn("kmsg truncated: unknown error (%m)\n");
 			return -2;
 		}
 
@@ -176,20 +168,12 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 		return -1;
 
 	if (is_builtin) {
-		while (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EINTR)
-				return -2;
-
-			if (errno == EPIPE) {
+		if (read(fd, record, BUF_LEN) < 0) {
+			if (errno == EPIPE)
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-				return -2;
-			}
+			else if (errno != EINTR)
+				igt_warn("an error occurred while reading kmsg: %m\n");
 
-			if (errno == EAGAIN)
-				/* No records available */
-				continue;
-
-			igt_warn("kmsg truncated: unknown error (%m)\n");
 			return -2;
 		}
 	}
@@ -215,20 +199,12 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
 		if (cutoff)
 			cutoff[0] = '\0';
 
-		while (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EINTR)
-				return -2;
-
-			if (errno == EPIPE) {
+		if (read(fd, record, BUF_LEN) < 0) {
+			if (errno == EPIPE)
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-				return -2;
-			}
+			else if (errno != EINTR)
+				igt_warn("unknown error reading kmsg (%m)\n");
 
-			if (errno == EAGAIN)
-				/* No records available */
-				continue;
-
-			igt_warn("kmsg truncated: unknown error (%m)\n");
 			return -2;
 		}
 
@@ -365,20 +341,12 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 	char base_test_name_for_next_level[BUF_LEN + 1];
 
 	for (int i = 0; i < test_count; i++) {
-		while (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EINTR)
-				return -1;
-
-			if (errno == EAGAIN)
-				/* No records available */
-				continue;
-
-			if (errno == EPIPE) {
+		if (read(fd, record, BUF_LEN) < 0) {
+			if (errno == EPIPE)
 				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-				return -1;
-			}
+			else if (errno != EINTR)
+				igt_warn("error reading kmsg (%m)\n");
 
-			igt_warn("kmsg truncated: unknown error (%m)\n");
 			return -1;
 		}
 
@@ -506,18 +474,13 @@ igt_ktap_parser_start:
 	test_name[0] = '\0';
 	test_name[BUF_LEN] = '\0';
 
-	while (read(fd, record, BUF_LEN) < 0) {
-		if (errno == EAGAIN)
-			/* No records available */
-			continue;
-
-		if (errno == EINTR)
-			goto igt_ktap_parser_end;
-
-		if (errno == EPIPE) {
+	if (read(fd, record, BUF_LEN) < 0) {
+		if (errno == EPIPE)
 			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-			goto igt_ktap_parser_end;
-		}
+		else if (errno != EINTR)
+			igt_warn("error reading kmsg (%m)\n");
+
+		goto igt_ktap_parser_end;
 	}
 
 	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 11/17] lib/kunit: Fail / skip on kernel taint

[Janusz Krzysztofik]

Similar to how igt_kselftest() handles kernel taints, fail current dynamic
sub-subtest and skip remaining ones when a kernel taint is detected during
execution of kunit test cases.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 020df286b8..988ac164cb 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -761,6 +761,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 	bool is_builtin;
 	struct ktap_test_results *results;
 	struct ktap_test_results_element *temp;
+	unsigned long taints;
 	int flags, ret;
 
 	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
@@ -785,12 +786,20 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 
 	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
 	{
+		if (igt_kernel_tainted(&taints)) {
+			ktap_parser_cancel();
+			break;
+		}
+
 		if (READ_ONCE(results->head) != NULL) {
 			pthread_mutex_lock(&results->mutex);
 
-			igt_dynamic(results->head->test_name)
+			igt_dynamic(results->head->test_name) {
 				igt_assert(READ_ONCE(results->head->passed));
 
+				igt_fail_on(igt_kernel_tainted(&taints));
+			}
+
 			temp = results->head;
 			results->head = results->head->next;
 			free(temp);
@@ -801,6 +810,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 
 	ret = ktap_parser_stop();
 
+	igt_skip_on(igt_kernel_tainted(&taints));
 	igt_skip_on_f(ret, "KTAP parser failed\n");
 }
 
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 12/17] lib/ktap: Use IGT linked lists for storing KTAP results

[Janusz Krzysztofik]

For code simplicity and clarity, use existing IGT linked lists library
instead of open coding a custom implementation of a list of KTAP results.

While being at it, flatten the code by inverting a check for pending
results.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 28 ++++++++++++++++------------
 lib/igt_ktap.c | 25 +++++--------------------
 lib/igt_ktap.h |  6 ++++--
 3 files changed, 25 insertions(+), 34 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 988ac164cb..c692954911 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -760,7 +760,6 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 	struct kmod_module *kunit_kmod;
 	bool is_builtin;
 	struct ktap_test_results *results;
-	struct ktap_test_results_element *temp;
 	unsigned long taints;
 	int flags, ret;
 
@@ -784,28 +783,33 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 		igt_skip("Unable to load %s module\n", tst->module_name);
 	}
 
-	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
+	while (READ_ONCE(results->still_running) || !igt_list_empty(&results->list))
 	{
+		struct ktap_test_results_element *result;
+
 		if (igt_kernel_tainted(&taints)) {
 			ktap_parser_cancel();
 			break;
 		}
 
-		if (READ_ONCE(results->head) != NULL) {
-			pthread_mutex_lock(&results->mutex);
+		pthread_mutex_lock(&results->mutex);
+		if (igt_list_empty(&results->list)) {
+			pthread_mutex_unlock(&results->mutex);
+			continue;
+		}
 
-			igt_dynamic(results->head->test_name) {
-				igt_assert(READ_ONCE(results->head->passed));
+		result = igt_list_first_entry(&results->list, result, link);
 
-				igt_fail_on(igt_kernel_tainted(&taints));
-			}
+		igt_list_del(&result->link);
+		pthread_mutex_unlock(&results->mutex);
 
-			temp = results->head;
-			results->head = results->head->next;
-			free(temp);
+		igt_dynamic(result->test_name) {
+			igt_assert(READ_ONCE(result->passed));
 
-			pthread_mutex_unlock(&results->mutex);
+			igt_fail_on(igt_kernel_tainted(&taints));
 		}
+
+		free(result);
 	}
 
 	ret = ktap_parser_stop();
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 165f6b2cce..5e9967f980 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -12,6 +12,7 @@
 #include "igt_aux.h"
 #include "igt_core.h"
 #include "igt_ktap.h"
+#include "igt_list.h"
 
 #define DELIMITER "-"
 
@@ -335,7 +336,7 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 			   bool *found_tests, bool is_builtin)
 {
 	char record[BUF_LEN + 1];
-	struct ktap_test_results_element *r, *temp;
+	struct ktap_test_results_element *r;
 	int internal_test_count;
 	char test_name[BUF_LEN + 1];
 	char base_test_name_for_next_level[BUF_LEN + 1];
@@ -403,17 +404,9 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 			r->test_name[BUF_LEN] = '\0';
 
 			r->passed = false;
-			r->next = NULL;
 
 			pthread_mutex_lock(&results.mutex);
-			if (results.head == NULL) {
-				results.head = r;
-			} else {
-				temp = results.head;
-				while (temp->next != NULL)
-					temp = temp->next;
-				temp->next = r;
-			}
+			igt_list_add_tail(&r->link, &results.list);
 			pthread_mutex_unlock(&results.mutex);
 
 			test_name[0] = '\0';
@@ -431,17 +424,9 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
 			r->test_name[BUF_LEN] = '\0';
 
 			r->passed = true;
-			r->next = NULL;
 
 			pthread_mutex_lock(&results.mutex);
-			if (results.head == NULL) {
-				results.head = r;
-			} else {
-				temp = results.head;
-				while (temp->next != NULL)
-					temp = temp->next;
-				temp->next = r;
-			}
+			igt_list_add_tail(&r->link, &results.list);
 			pthread_mutex_unlock(&results.mutex);
 
 			test_name[0] = '\0';
@@ -523,7 +508,7 @@ static pthread_t ktap_parser_thread;
 
 struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
 {
-	results.head = NULL;
+	IGT_INIT_LIST_HEAD(&results.list);
 	pthread_mutex_init(&results.mutex, NULL);
 	results.still_running = true;
 
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
index 991800e912..b4d7a6dbc7 100644
--- a/lib/igt_ktap.h
+++ b/lib/igt_ktap.h
@@ -28,16 +28,18 @@
 
 #include <pthread.h>
 
+#include "igt_list.h"
+
 void *igt_ktap_parser(void *unused);
 
 typedef struct ktap_test_results_element {
 	char test_name[BUF_LEN + 1];
 	bool passed;
-	struct ktap_test_results_element *next;
+	struct igt_list_head link;
 } ktap_test_results_element;
 
 struct ktap_test_results {
-	ktap_test_results_element *head;
+	struct igt_list_head list;
 	pthread_mutex_t mutex;
 	bool still_running;
 };
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 13/17] lib/ktap: Reimplement KTAP parser

[Janusz Krzysztofik]

Current implementation of KTAP parser suffers from several issues:
- in most cases, kernel messages that are not part of KTAP output but
  happen to appear in between break the parser,
- results from parametrized test cases, not preceded with a "1..N" test
  plan, break the parser,
- skips are not supported, reported as success,
- IGT results from all 3 kunit test nesting levels, i.e., from
  parametrized subtests (should those were fixed to work correctly), test
  cases and test suites, are reported individually as if all those items
  were executed sequentially, all at the same level of nesting, which can
  be confusing to igt_runner,
- the parser is not only parsing the data, but also handles data input
  from a /dev/kmsg like source, which is integrated into it, making it
  hard if not impossible to feed KTAP data from different sources,
  including mock sources,
- since the parser has been designed for running it in a separate thread,
  it's not possible to use igt_skip() nor igt_fail() and friends
  immediately when a result is available, only pass it to the main thread
  over a buffer.  As a consequence, it is virtually impossible to
  synchronize IGT output with KTAP output.

Fixing the existing parser occurred more complicated than re-implementing
it from scratch.  This patch provides a new implementation.

Only results from kunit test cases are reported as results of IGT dynamic
sub-subtests.  Results from individual parametrized subtests have been
considered problematic since many of them provide multi-word descriptions
in place of single-word subtest names.  If a parametrized test case fails
then full KTAP output from its subtests, potentially mixed with
accompanying kernel messages, is available in dmesg for analysis so users
can still find out which individual subtests succeeded and which failed.

Results from test suites level are also omitted in faith that IGT can
handle aggregation of results from individual kunit test cases reported as
IGT dynamic sub-subtests and report those aggregated results correctly as
results from an IGT dynamic subtest.  That 1:1 mapping of kunit test
suites to IGT dynamic subtests now works perfectly for modules that
provide only one test suite, which is the case for all kunit test modules
now existing under drivers/gpu/drm, and the most common case among all
kunit test modules in the whole kernel tree.

New igt_ktap functions can be called directly from igt_kunit subtest body,
but for this patch, the old igt_ktap_parser() function that runs in a
separate thread has been preserved, only modified to use the new
implementation and translate results from those new functions to legacy
format.  Unlike the former implementation, translation of kunit names to
IGT names is handled outside the parser itself, though for now it is still
performed inside the legacy igt_ktap_parser() function.

For better readability of the patch, no longer used functions have been
left untouched, only tagged with __maybe_unused to shut up compiler
warnings / errors.  Kunit library functions will be modified to use the
new igt_ktap interface, and those old ktap functions removed by follow-
up patches.

A test with example subtests that feed igt_ktap_parse() function with some
example data and verifies correctness of their parsing is also provided.

v2: Fix incorrect and missing includes in the test source file,
  - add license and copyright clauses to the test source file.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_ktap.c              | 422 ++++++++++++++++++++++++++++++++----
 lib/igt_ktap.h              |  15 ++
 lib/tests/igt_ktap_parser.c | 246 +++++++++++++++++++++
 lib/tests/meson.build       |   1 +
 4 files changed, 645 insertions(+), 39 deletions(-)
 create mode 100644 lib/tests/igt_ktap_parser.c

diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index 5e9967f980..d46a2433e5 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: MIT
 /*
  * Copyright © 2023 Isabella Basso do Amaral <isabbasso@riseup.net>
+ * Copyright © 2023 Intel Corporation
  */
 
 #include <ctype.h>
@@ -8,12 +9,310 @@
 #include <libkmod.h>
 #include <pthread.h>
 #include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
 
 #include "igt_aux.h"
 #include "igt_core.h"
 #include "igt_ktap.h"
 #include "igt_list.h"
 
+enum ktap_phase {
+	KTAP_START,
+	SUITE_COUNT,
+	SUITE_START,
+	SUITE_NAME,
+	CASE_COUNT,
+	CASE_NAME,
+	SUB_RESULT,
+	CASE_RESULT,
+	SUITE_RESULT,
+};
+
+struct igt_ktap_results {
+	enum ktap_phase expect;
+	unsigned int suite_count;
+	unsigned int suite_last;
+	char *suite_name;
+	unsigned int case_count;
+	unsigned int case_last;
+	char *case_name;
+	unsigned int sub_last;
+	struct igt_list_head *results;
+};
+
+/**
+ * igt_ktap_parse:
+ *
+ * This function parses a line of text for KTAP report data
+ * and passes results back to IGT kunit layer.
+ */
+int igt_ktap_parse(const char *buf, struct igt_ktap_results *ktap)
+{
+	char *suite_name = NULL, *case_name = NULL, *msg = NULL;
+	struct igt_ktap_result *result;
+	int code = IGT_EXIT_INVALID;
+	unsigned int n, len;
+	char s[2];
+
+	/* KTAP report header */
+	if (igt_debug_on(sscanf(buf, "KTAP%*[ ]version%*[ ]%u %n",
+				&n, &len) == 1 && len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != KTAP_START))
+			return -EPROTO;
+
+		ktap->suite_count = 0;
+		ktap->expect = SUITE_COUNT;
+
+	/* malformed TAP test plan? */
+	} else if (len = 0,
+		   igt_debug_on(sscanf(buf, " 1..%1[ ]", s) == 1)) {
+		return -EINPROGRESS;
+
+	/* valid test plan of a KTAP report */
+	} else if (igt_debug_on(sscanf(buf, "1..%u %n", &n, &len) == 1 &&
+				len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != SUITE_COUNT))
+			return -EPROTO;
+
+		if (!n)
+			return 0;
+
+		ktap->suite_count = n;
+		ktap->suite_last = 0;
+		ktap->suite_name = NULL;
+		ktap->expect = SUITE_START;
+
+	/* KTAP test suite header */
+	} else if (len = 0,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]KTAP%*[ ]version%*[ ]%u %n",
+				       &n, &len) == 1 && len == strlen(buf))) {
+		/*
+		 * TODO: drop the following workaround as soon as
+		 * kernel side issue of missing lines with top level
+		 * KTAP version and test suite plan is fixed.
+		 */
+		if (ktap->expect == KTAP_START) {
+			ktap->suite_count = 1;
+			ktap->suite_last = 0;
+			ktap->suite_name = NULL;
+			ktap->expect = SUITE_START;
+		}
+
+		if (igt_debug_on(ktap->expect != SUITE_START))
+			return -EPROTO;
+
+		ktap->expect = SUITE_NAME;
+
+	/* KTAP test suite name */
+	} else if (len = 0,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]#%*[ ]Subtest:%*[ ]%ms %n",
+				       &suite_name, &len) == 1 && len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != SUITE_NAME))
+			return -EPROTO;
+
+		ktap->suite_name = suite_name;
+		suite_name = NULL;
+		ktap->case_count = 0;
+		ktap->expect = CASE_COUNT;
+
+	/* valid test plan of a KTAP test suite */
+	} else if (len = 0, free(suite_name), suite_name = NULL,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]1..%u %n",
+				       &n, &len) == 1 && len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != CASE_COUNT))
+			return -EPROTO;
+
+		if (n) {
+			ktap->case_count = n;
+			ktap->case_last = 0;
+			ktap->case_name = NULL;
+			ktap->expect = CASE_RESULT;
+		} else {
+			ktap->expect = SUITE_RESULT;
+		}
+
+	/* KTAP parametrized test case header */
+	} else if (len = 0,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]KTAP%*[ ]version%*[ ]%u %n",
+				       &n, &len) == 1 && len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != CASE_RESULT))
+			return -EPROTO;
+
+		ktap->sub_last = 0;
+		ktap->expect = CASE_NAME;
+
+	/* KTAP parametrized test case name */
+	} else if (len = 0,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]#%*[ ]Subtest:%*[ ]%ms %n",
+				       &case_name, &len) == 1 && len == strlen(buf))) {
+		if (igt_debug_on(ktap->expect != CASE_NAME))
+			return -EPROTO;
+
+		n = ktap->case_last + 1;
+		ktap->expect = SUB_RESULT;
+
+	/* KTAP parametrized subtest result */
+	} else if (len = 0, free(case_name), case_name = NULL,
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%*[^#\n]%1[#\n]",
+				       &n, s) == 2) ||
+		   igt_debug_on(sscanf(buf,
+				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%*[^#\n]%1[#\n]",
+				       &n, s) == 2)) {
+		/* at lease one result of a parametrised subtest expected */
+		if (igt_debug_on(ktap->expect == SUB_RESULT &&
+				 ktap->sub_last == 0))
+			ktap->expect = CASE_RESULT;
+
+		if (igt_debug_on(ktap->expect != CASE_RESULT) ||
+		    igt_debug_on(n != ++ktap->sub_last))
+			return -EPROTO;
+
+	/* KTAP test case skip result */
+	} else if ((igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]SKIP %n",
+					&n, &case_name, &len) == 2 &&
+				 len == strlen(buf))) ||
+		   (len = 0, free(case_name), case_name = NULL,
+		    igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]SKIP%*[ ]%m[^\n]",
+					&n, &case_name, &msg) == 3))) {
+		code = IGT_EXIT_SKIP;
+
+	/* KTAP test case pass result */
+	} else if ((free(case_name), case_name = NULL, free(msg), msg = NULL,
+		    igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms %n",
+					&n, &case_name, &len) == 2 &&
+				 len == strlen(buf))) ||
+		   (len = 0, free(case_name), case_name = NULL,
+		    igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]%m[^\n]",
+					&n, &case_name, &msg) == 3))) {
+		code = IGT_EXIT_SUCCESS;
+
+	/* KTAP test case fail result */
+	} else if ((free(case_name), case_name = NULL, free(msg), msg = NULL,
+		    igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%ms %n",
+					&n, &case_name, &len) == 2 &&
+				 len == strlen(buf))) ||
+		   (len = 0, free(case_name), case_name = NULL,
+		    igt_debug_on(sscanf(buf,
+					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]%m[^\n]",
+					&n, &case_name, &msg) == 3))) {
+		code = IGT_EXIT_FAILURE;
+
+	/* KTAP test suite result */
+	} else if ((free(case_name), free(msg),
+		    igt_debug_on(sscanf(buf, "ok%*[ ]%u%*[ ]%ms %n",
+					&n, &suite_name, &len) == 2 &&
+				 len == strlen(buf))) ||
+		   (len = 0, free(suite_name), suite_name = NULL,
+		    igt_debug_on(sscanf(buf, "ok%*[ ]%u%*[ ]%ms%*[ ]%1[#]",
+					&n, &suite_name, s) == 3)) ||
+		   (free(suite_name), suite_name = NULL,
+		    igt_debug_on(sscanf(buf,
+					"not%*[ ]ok%*[ ]%u%*[ ]%ms %n",
+					&n, &suite_name, &len) == 2 &&
+				 len == strlen(buf))) ||
+		   (len = 0, free(suite_name), suite_name = NULL,
+		    igt_debug_on(sscanf(buf,
+					"not%*[ ]ok%*[ ]%u%*[ ]%ms%*[ ]%1[#]",
+					&n, &suite_name, s) == 3))) {
+		if (igt_debug_on(ktap->expect != SUITE_RESULT) ||
+		    igt_debug_on(strcmp(suite_name, ktap->suite_name)) ||
+		    igt_debug_on(n != ++ktap->suite_last) ||
+		    igt_debug_on(n > ktap->suite_count)) {
+			free(suite_name);
+			return -EPROTO;
+		}
+		free(suite_name);
+
+		/* last test suite? */
+		if (igt_debug_on(n == ktap->suite_count))
+			return 0;
+
+		ktap->suite_name = NULL;
+		ktap->expect = SUITE_START;
+
+	} else {
+		return -EINPROGRESS;
+	}
+
+	/* neither a test case name nor result */
+	if (ktap->expect != SUB_RESULT && code == IGT_EXIT_INVALID)
+		return -EINPROGRESS;
+
+	if (igt_debug_on(ktap->expect == SUB_RESULT &&
+			 code != IGT_EXIT_INVALID) ||
+	    igt_debug_on(code != IGT_EXIT_INVALID &&
+			 ktap->expect != CASE_RESULT) ||
+	    igt_debug_on(!ktap->suite_name) || igt_debug_on(!case_name) ||
+	    igt_debug_on(ktap->expect == CASE_RESULT && ktap->case_name &&
+			 strcmp(case_name, ktap->case_name)) ||
+	    igt_debug_on(n > ktap->case_count) ||
+	    igt_debug_on(n != (ktap->expect == SUB_RESULT ?
+			       ktap->case_last + 1: ++ktap->case_last))) {
+		free(case_name);
+		free(msg);
+		return -EPROTO;
+	}
+
+	if (ktap->expect == SUB_RESULT) {
+		/* KTAP parametrized test case name */
+		ktap->case_name = case_name;
+
+	} else {
+		/* KTAP test case result */
+		ktap->case_name = NULL;
+
+		/* last test case in a suite */
+		if (n == ktap->case_count)
+			ktap->expect = SUITE_RESULT;
+	}
+
+	if (igt_debug_on((result = calloc(1, sizeof(*result)), !result))) {
+		free(case_name);
+		free(msg);
+		return -ENOMEM;
+	}
+
+	result->suite_name = ktap->suite_name;
+	result->case_name = case_name;
+	result->code = code;
+	result->msg = msg;
+	igt_list_add_tail(&result->link, ktap->results);
+
+	return -EINPROGRESS;
+}
+
+struct igt_ktap_results *igt_ktap_alloc(struct igt_list_head *results)
+{
+	struct igt_ktap_results *ktap = calloc(1, sizeof(*ktap));
+
+	if (!ktap)
+		return NULL;
+
+	ktap->expect = KTAP_START;
+	ktap->results = results;
+
+	return ktap;
+}
+
+void igt_ktap_free(struct igt_ktap_results *ktap)
+{
+	free(ktap);
+}
+
 #define DELIMITER "-"
 
 struct ktap_parser_args {
@@ -332,6 +631,7 @@ static int parse_kmsg_for_tap(int fd, char *record, char *test_name)
  * 0 if succeded
  * -1 if error occurred
  */
+__maybe_unused
 static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *failed_tests,
 			   bool *found_tests, bool is_builtin)
 {
@@ -445,62 +745,106 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
  */
 void *igt_ktap_parser(void *unused)
 {
+	char record[BUF_LEN + 1], *buf, *suite_name = NULL, *case_name = NULL;
+	struct igt_ktap_results *ktap = NULL;
 	int fd = ktap_args.fd;
-	char record[BUF_LEN + 1];
-	bool is_builtin = ktap_args.is_builtin;
-	char test_name[BUF_LEN + 1];
-	bool failed_tests, found_tests;
-	int test_count;
+	IGT_LIST_HEAD(list);
+	int err;
 
-	failed_tests = false;
-	found_tests = false;
+	ktap = igt_ktap_alloc(&list);
+	if (igt_debug_on(!ktap))
+		goto igt_ktap_parser_end;
 
-igt_ktap_parser_start:
-	test_name[0] = '\0';
-	test_name[BUF_LEN] = '\0';
+	while (err = read(fd, record, BUF_LEN), err > 0) {
+		struct igt_ktap_result *r, *rn;
 
-	if (read(fd, record, BUF_LEN) < 0) {
-		if (errno == EPIPE)
-			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-		else if (errno != EINTR)
-			igt_warn("error reading kmsg (%m)\n");
+		/* skip kmsg continuation lines */
+		if (igt_debug_on(*record == ' '))
+			continue;
 
-		goto igt_ktap_parser_end;
-	}
+		/* NULL-terminate the record */
+		record[err] = '\0';
 
-	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);
+		/* detect start of log message, continue if not found */
+		buf = strchrnul(record, ';');
+		if (igt_debug_on(*buf == '\0'))
+			continue;
+		buf++;
 
-	switch (test_count) {
-	case -2:
-		/* Problems while reading the file */
-		goto igt_ktap_parser_end;
-	case -1:
-		/* No test found */
-		goto igt_ktap_parser_start;
-	case 0:
-		/* Tests found, but they're missing info */
-		found_tests = true;
-		goto igt_ktap_parser_end;
-	default:
-		found_tests = true;
+		err = igt_ktap_parse(buf, ktap);
 
-		if (parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests,
-				    is_builtin) == -1)
+		/* parsing error */
+		if (err && err != -EINPROGRESS)
 			goto igt_ktap_parser_end;
 
-		break;
+		igt_list_for_each_entry_safe(r, rn, &list, link) {
+			struct ktap_test_results_element *result = NULL;
+			int code = r->code;
+
+			if (code != IGT_EXIT_INVALID)
+				result = calloc(1, sizeof(*result));
+
+			if (result) {
+				snprintf(result->test_name, sizeof(result->test_name),
+					 "%s-%s", r->suite_name, r->case_name);
+
+				if (code == IGT_EXIT_SUCCESS)
+					result->passed = true;
+			}
+
+			igt_list_del(&r->link);
+			if (r->suite_name != suite_name) {
+				free(suite_name);
+				suite_name = r->suite_name;
+			}
+			if (r->case_name != case_name) {
+				free(case_name);
+				case_name = r->case_name;
+			}
+			free(r->msg);
+			free(r);
+
+			/*
+			 * no extra result record expected on start
+			 * of parametrized test case -- skip it
+			 */
+			if (code == IGT_EXIT_INVALID)
+				continue;
+
+			if (!result) {
+				err = -ENOMEM;
+				goto igt_ktap_parser_end;
+			}
+
+			pthread_mutex_lock(&results.mutex);
+			igt_list_add_tail(&result->link, &results.list);
+			pthread_mutex_unlock(&results.mutex);
+		}
+
+		/* end of KTAP report */
+		if (!err)
+			goto igt_ktap_parser_end;
 	}
 
-	/* Parse topmost level */
-	test_name[0] = '\0';
-	parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests, is_builtin);
+	if (err < 0) {
+		if (errno == EPIPE)
+			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
+		else if (errno != EINTR)
+			igt_warn("error reading kmsg (%m)\n");
+	}
 
 igt_ktap_parser_end:
-	results.still_running = false;
+	free(suite_name);
+	free(case_name);
 
-	if (found_tests && !failed_tests)
+	if (!err)
 		ktap_args.ret = IGT_EXIT_SUCCESS;
 
+	results.still_running = false;
+
+	if (ktap)
+		igt_ktap_free(ktap);
+
 	return NULL;
 }
 
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
index b4d7a6dbc7..6f8da3eab6 100644
--- a/lib/igt_ktap.h
+++ b/lib/igt_ktap.h
@@ -1,5 +1,6 @@
 /*
  * Copyright © 2022 Isabella Basso do Amaral <isabbasso@riseup.net>
+ * Copyright © 2023 Intel Corporation
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
@@ -30,6 +31,20 @@
 
 #include "igt_list.h"
 
+struct igt_ktap_result {
+	struct igt_list_head link;
+	char *suite_name;
+	char *case_name;
+	char *msg;
+	int code;
+};
+
+struct igt_ktap_results;
+
+struct igt_ktap_results *igt_ktap_alloc(struct igt_list_head *results);
+int igt_ktap_parse(const char *buf, struct igt_ktap_results *ktap);
+void igt_ktap_free(struct igt_ktap_results *ktap);
+
 void *igt_ktap_parser(void *unused);
 
 typedef struct ktap_test_results_element {
diff --git a/lib/tests/igt_ktap_parser.c b/lib/tests/igt_ktap_parser.c
new file mode 100644
index 0000000000..6357bdf6a5
--- /dev/null
+++ b/lib/tests/igt_ktap_parser.c
@@ -0,0 +1,246 @@
+// SPDX-License-Identifier: MIT
+/*
+* Copyright © 2023 Intel Corporation
+*/
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "igt_core.h"
+#include "igt_ktap.h"
+#include "igt_list.h"
+
+static void ktap_list(void)
+{
+	struct igt_ktap_result *result, *rn;
+	struct igt_ktap_results *ktap;
+	int suite = 1, test = 1;
+	IGT_LIST_HEAD(results);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+
+	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("1..3\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    1..3\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 2 test_case_2 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 3 test_case_3 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("ok 1 test_suite_1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_2\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("ok 2 test_suite_2\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_3\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    1..4\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 2 test_case_2 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 3 test_case_3 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 4 test_case_4 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("ok 3 test_suite_3\n", ktap), 0);
+
+	igt_ktap_free(ktap);
+
+	igt_assert_eq(igt_list_length(&results), 8);
+
+	igt_list_for_each_entry_safe(result, rn, &results, link) {
+		char *case_name, *suite_name;
+
+		igt_list_del(&result->link);
+
+		igt_assert_lt(0, asprintf(&case_name, "test_case_%u", test));
+		igt_assert_lt(0, asprintf(&suite_name, "test_suite_%u", suite));
+
+		igt_assert(result->case_name);
+		igt_assert_eq(strcmp(result->case_name, case_name), 0);
+		free(result->case_name);
+		free(case_name);
+
+		igt_assert(result->suite_name);
+		igt_assert_eq(strcmp(result->suite_name, suite_name), 0);
+		free(suite_name);
+
+		igt_assert(!result->msg);
+		igt_assert_eq(result->code, IGT_EXIT_SKIP);
+
+		if ((suite == 1 && test < 3) || (suite == 3 && test < 4)) {
+			test++;
+		} else {
+			free(result->suite_name);
+			suite++;
+			test = 1;
+		}
+
+		free(result);
+	}
+}
+
+static void ktap_results(void)
+{
+	struct igt_ktap_result *result;
+	struct igt_ktap_results *ktap;
+	char *suite_name, *case_name;
+	IGT_LIST_HEAD(results);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+
+	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        ok 1 parameter 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        ok 2 parameter 2 # a comment\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        ok 3 parameter 3 # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        ok 4 parameter 4 # SKIP with a message\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        not ok 5 parameter 5\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        not ok 6 parameter 6 # failure message\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("not ok 1 test_suite\n", ktap), 0);
+
+	igt_ktap_free(ktap);
+
+	igt_assert_eq(igt_list_length(&results), 2);
+
+	result = igt_list_first_entry(&results, result, link);
+	igt_list_del(&result->link);
+	igt_assert_eq(strcmp(result->suite_name, "test_suite"), 0);
+	igt_assert_eq(strcmp(result->case_name, "test_case"), 0);
+	igt_assert_eq(result->code, IGT_EXIT_INVALID);
+	igt_assert(!result->msg);
+	free(result->msg);
+	suite_name = result->suite_name;
+	case_name = result->case_name;
+	free(result);
+
+	result = igt_list_first_entry(&results, result, link);
+	igt_list_del(&result->link);
+	igt_assert_eq(strcmp(result->suite_name, suite_name), 0);
+	igt_assert_eq(strcmp(result->case_name, case_name), 0);
+	igt_assert_neq(result->code, IGT_EXIT_INVALID);
+	free(result->msg);
+	free(suite_name);
+	free(case_name);
+	free(result);
+}
+
+static void ktap_success(void)
+{
+	struct igt_ktap_result *result;
+	struct igt_ktap_results *ktap;
+	IGT_LIST_HEAD(results);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+
+	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_assert(igt_list_empty(&results));
+
+	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_list_length(&results), 1);
+
+	igt_assert_eq(igt_ktap_parse("        ok 1 parameter # SKIP\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_list_length(&results), 1);
+
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EINPROGRESS);
+	igt_assert_eq(igt_list_length(&results), 2);
+
+	igt_assert_eq(igt_ktap_parse("not ok 1 test_suite\n", ktap), 0);
+	igt_assert_eq(igt_list_length(&results), 2);
+
+	igt_ktap_free(ktap);
+
+	result = igt_list_last_entry(&results, result, link);
+	igt_list_del(&result->link);
+	igt_assert_eq(result->code, IGT_EXIT_SUCCESS);
+	free(result->msg);
+	free(result);
+
+	result = igt_list_last_entry(&results, result, link);
+	igt_list_del(&result->link);
+	free(result->suite_name);
+	free(result->case_name);
+	free(result->msg);
+	free(result);
+}
+
+static void ktap_top_version(void)
+{
+	struct igt_ktap_results *ktap;
+	IGT_LIST_HEAD(results);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	/* TODO: change to -EPROTO as soon as related workaround is dropped */
+	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("        ok 1 parameter 1\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
+	igt_assert_eq(igt_ktap_parse("ok 1 test_suite\n", ktap), -EPROTO);
+	igt_ktap_free(ktap);
+}
+
+igt_main
+{
+	igt_subtest("list")
+		ktap_list();
+
+	igt_subtest("results")
+		ktap_results();
+
+	igt_subtest("success")
+		ktap_success();
+
+	igt_subtest("top-ktap-version")
+		ktap_top_version();
+}
diff --git a/lib/tests/meson.build b/lib/tests/meson.build
index 7a52a7876e..fa3d81de6c 100644
--- a/lib/tests/meson.build
+++ b/lib/tests/meson.build
@@ -10,6 +10,7 @@ lib_tests = [
 	'igt_exit_handler',
 	'igt_fork',
 	'igt_fork_helper',
+        'igt_ktap_parser',
 	'igt_list_only',
 	'igt_invalid_subtest_name',
 	'igt_nesting',
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 14/17] lib/kunit: Load test modules in background

[Janusz Krzysztofik]

For igt_runner to be able to correlate a stream of IGT results from
dynamic sub-subtests that correspond to individual kunit test cases, read
by the igt_runner from stdout / stderr of the test process, with a stream
of kernel messages read from /dev/kmsg, we need both result streams being
fed with data in parallel.  While our KTAP parser is currently started in
the background and reads KTAP results from /dev/kmsg in parallel with
execution of kunit tests performed by the kernel while we are loading a
kunit test module, results from the parser are then only stored as
intermediate data and not processed any further until the module loading
completes.  As a consequence, there is no synchronization between the two
streams.

Call the function that loads the kunit test module from a separate thread
and process the intermediate results immediately, as soon as available
from the background parser, without waiting for completion of module
loading.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index c692954911..bbde3929f2 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -25,6 +25,7 @@
 #include <signal.h>
 #include <errno.h>
 #include <fcntl.h>
+#include <pthread.h>
 #include <sys/utsname.h>
 
 #include "igt_aux.h"
@@ -746,6 +747,21 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
 	kmod_module_info_free_list(pre);
 }
 
+struct modprobe_data {
+	struct kmod_module *kmod;
+	const char *opts;
+	int err;
+};
+
+static void *modprobe_task(void *arg)
+{
+	struct modprobe_data *data = arg;
+
+	data->err = modprobe(data->kmod, data->opts);
+
+	return NULL;
+}
+
 /**
  * igt_kunit:
  * @module_name: the name of the module
@@ -757,9 +773,11 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
  */
 static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 {
+	struct modprobe_data modprobe = { tst->kmod, opts, 0, };
 	struct kmod_module *kunit_kmod;
 	bool is_builtin;
 	struct ktap_test_results *results;
+	pthread_t modprobe_thread;
 	unsigned long taints;
 	int flags, ret;
 
@@ -777,18 +795,25 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 
 	results = ktap_parser_start(tst->kmsg, is_builtin);
 
-	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
+	if (igt_debug_on(pthread_create(&modprobe_thread, NULL,
+					modprobe_task, &modprobe))) {
 		ktap_parser_cancel();
 		igt_ignore_warn(ktap_parser_stop());
-		igt_skip("Unable to load %s module\n", tst->module_name);
+		igt_skip("Failed to create a modprobe thread\n");
 	}
 
 	while (READ_ONCE(results->still_running) || !igt_list_empty(&results->list))
 	{
 		struct ktap_test_results_element *result;
 
+		if (!pthread_tryjoin_np(modprobe_thread, NULL) && modprobe.err) {
+			ktap_parser_cancel();
+			break;
+		}
+
 		if (igt_kernel_tainted(&taints)) {
 			ktap_parser_cancel();
+			pthread_cancel(modprobe_thread);
 			break;
 		}
 
@@ -806,14 +831,20 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 		igt_dynamic(result->test_name) {
 			igt_assert(READ_ONCE(result->passed));
 
+			if (!pthread_tryjoin_np(modprobe_thread, NULL))
+				igt_assert_eq(modprobe.err, 0);
+
 			igt_fail_on(igt_kernel_tainted(&taints));
 		}
 
 		free(result);
 	}
 
+	pthread_join(modprobe_thread, NULL);
+
 	ret = ktap_parser_stop();
 
+	igt_skip_on(modprobe.err);
 	igt_skip_on(igt_kernel_tainted(&taints));
 	igt_skip_on_f(ret, "KTAP parser failed\n");
 }
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 15/17] lib/kunit: Parse KTAP report from the main process thread

[Janusz Krzysztofik]

There was an attempt to parse KTAP reports in the background while a kunit
test module is loading.  However, since dynamic sub-subtests can be
executed only from the main thread, that attempt was not quite successful,
as IGT results from all executed kunit test cases were generated only
after loading of kunit test module completed.

Now that the parser maintains its state and we can call it separately for
each input line of a KTAP report, it is perfectly possible to call the
parser from the main thread while the module is loading in the background,
and convert results from kunit test cases immediately to results of IGT
dynamic sub-subtests by running an igt_dynamic() section for each result
as soon as returned by the parser.

Drop igt_ktap_parser() thread and execute igt_dynamic() for each kunit
result obtained from igt_ktap_parse() called from the main thread.

Also, drop no longer needed functions from igt_ktap soruces.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 158 +++++++++++---
 lib/igt_ktap.c | 568 -------------------------------------------------
 lib/igt_ktap.h |  22 --
 3 files changed, 123 insertions(+), 625 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index bbde3929f2..46a6f81e73 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -1,5 +1,5 @@
 /*
- * Copyright © 2016 Intel Corporation
+ * Copyright © 2016-2023 Intel Corporation
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
@@ -26,7 +26,10 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <pthread.h>
+#include <stdlib.h>
+#include <string.h>
 #include <sys/utsname.h>
+#include <unistd.h>
 
 #include "igt_aux.h"
 #include "igt_core.h"
@@ -762,6 +765,54 @@ static void *modprobe_task(void *arg)
 	return NULL;
 }
 
+static int kunit_kmsg_result_get(struct igt_list_head *results,
+				 int fd, struct igt_ktap_results *ktap)
+{
+	char record[BUF_LEN + 1], *buf;
+	int ret;
+
+	while (igt_list_empty(results) &&
+	       (ret = read(fd, record, BUF_LEN), ret > 0)) {
+		/* skip kmsg continuation lines */
+		if (igt_debug_on(*record == ' '))
+			continue;
+
+		/* NULL-terminate the record */
+		record[ret] = '\0';
+
+		/* detect start of log message, continue if not found */
+		buf = strchrnul(record, ';');
+		if (igt_debug_on(*buf == '\0'))
+			continue;
+		buf++;
+
+		ret = igt_ktap_parse(buf, ktap);
+		if (ret != -EINPROGRESS)
+			break;
+	}
+
+	return ret;
+}
+
+static void kunit_result_free(struct igt_ktap_result *r,
+			      char **suite_name, char **case_name)
+{
+	igt_list_del(&r->link);
+
+	if (r->suite_name != *suite_name) {
+		free(*suite_name);
+		*suite_name = r->suite_name;
+	}
+
+	if (r->case_name != *case_name) {
+		free(*case_name);
+		*case_name = r->case_name;
+	}
+
+	free(r->msg);
+	free(r);
+}
+
 /**
  * igt_kunit:
  * @module_name: the name of the module
@@ -774,10 +825,11 @@ static void *modprobe_task(void *arg)
 static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 {
 	struct modprobe_data modprobe = { tst->kmod, opts, 0, };
-	struct kmod_module *kunit_kmod;
-	bool is_builtin;
-	struct ktap_test_results *results;
+	char *suite_name = NULL, *case_name = NULL;
+	struct igt_ktap_result *r, *rn;
+	struct igt_ktap_results *ktap;
 	pthread_t modprobe_thread;
+	IGT_LIST_HEAD(results);
 	unsigned long taints;
 	int flags, ret;
 
@@ -787,49 +839,76 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 	igt_skip_on_f(fcntl(tst->kmsg, F_SETFL, flags) == -1,
 		      "Could not set /dev/kmsg to blocking mode\n");
 
-	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
-
-	igt_skip_on(kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod));
-	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
-	kmod_module_unref(kunit_kmod);
+	ktap = igt_ktap_alloc(&results);
+	igt_require(ktap);
 
-	results = ktap_parser_start(tst->kmsg, is_builtin);
+	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
 
 	if (igt_debug_on(pthread_create(&modprobe_thread, NULL,
 					modprobe_task, &modprobe))) {
-		ktap_parser_cancel();
-		igt_ignore_warn(ktap_parser_stop());
+		igt_ktap_free(ktap);
 		igt_skip("Failed to create a modprobe thread\n");
 	}
 
-	while (READ_ONCE(results->still_running) || !igt_list_empty(&results->list))
-	{
-		struct ktap_test_results_element *result;
+	do {
+		if (!pthread_tryjoin_np(modprobe_thread, NULL) && modprobe.err)
+			break;
 
-		if (!pthread_tryjoin_np(modprobe_thread, NULL) && modprobe.err) {
-			ktap_parser_cancel();
+		if (igt_kernel_tainted(&taints))
 			break;
-		}
 
-		if (igt_kernel_tainted(&taints)) {
-			ktap_parser_cancel();
-			pthread_cancel(modprobe_thread);
+		ret = kunit_kmsg_result_get(&results, tst->kmsg, ktap);
+		if (igt_debug_on(ret && ret != -EINPROGRESS))
 			break;
-		}
 
-		pthread_mutex_lock(&results->mutex);
-		if (igt_list_empty(&results->list)) {
-			pthread_mutex_unlock(&results->mutex);
-			continue;
-		}
+		if (igt_debug_on(igt_list_empty(&results)))
+			break;
+
+		r = igt_list_first_entry(&results, r, link);
+
+		igt_dynamic_f("%s-%s", r->suite_name, r->case_name) {
+			if (r->code == IGT_EXIT_INVALID) {
+				/* parametrized test case, get actual result */
+				kunit_result_free(r, &suite_name, &case_name);
+				r = NULL;
 
-		result = igt_list_first_entry(&results->list, result, link);
+				igt_assert(igt_list_empty(&results));
 
-		igt_list_del(&result->link);
-		pthread_mutex_unlock(&results->mutex);
+				ret = kunit_kmsg_result_get(&results,
+							    tst->kmsg, ktap);
+				if (ret != -EINPROGRESS)
+					igt_fail_on(ret);
 
-		igt_dynamic(result->test_name) {
-			igt_assert(READ_ONCE(result->passed));
+				igt_fail_on(igt_list_empty(&results));
+
+				r = igt_list_first_entry(&results, r, link);
+
+				igt_fail_on_f(strcmp(r->suite_name, suite_name),
+					      "suite_name expected: %s, got: %s\n",
+					      suite_name, r->suite_name);
+				igt_fail_on_f(strcmp(r->case_name, case_name),
+					      "case_name expected: %s, got: %s\n",
+					      case_name, r->case_name);
+			}
+
+			igt_assert_neq(r->code, IGT_EXIT_INVALID);
+
+			if (r->msg && *r->msg) {
+				igt_skip_on_f(r->code == IGT_EXIT_SKIP,
+					      "%s", r->msg);
+				igt_fail_on_f(r->code == IGT_EXIT_FAILURE,
+					      "%s", r->msg);
+				igt_abort_on_f(r->code == IGT_EXIT_ABORT,
+					      "%s", r->msg);
+			} else {
+				igt_skip_on(r->code == IGT_EXIT_SKIP);
+				igt_fail_on(r->code == IGT_EXIT_FAILURE);
+				if (igt_debug_on(r->code != IGT_EXIT_SUCCESS))
+					igt_fail(r->code);
+			}
+
+			if (ret != -EINPROGRESS)
+				igt_assert_eq(ret, 0);
 
 			if (!pthread_tryjoin_np(modprobe_thread, NULL))
 				igt_assert_eq(modprobe.err, 0);
@@ -837,12 +916,21 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 			igt_fail_on(igt_kernel_tainted(&taints));
 		}
 
-		free(result);
-	}
+		kunit_result_free(r, &suite_name, &case_name);
+
+	} while (ret == -EINPROGRESS);
+
+	igt_list_for_each_entry_safe(r, rn, &results, link)
+		kunit_result_free(r, &suite_name, &case_name);
 
+	free(case_name);
+	free(suite_name);
+
+	if (ret)
+		pthread_cancel(modprobe_thread);
 	pthread_join(modprobe_thread, NULL);
 
-	ret = ktap_parser_stop();
+	igt_ktap_free(ktap);
 
 	igt_skip_on(modprobe.err);
 	igt_skip_on(igt_kernel_tainted(&taints));
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
index d46a2433e5..53a6c63288 100644
--- a/lib/igt_ktap.c
+++ b/lib/igt_ktap.c
@@ -4,17 +4,11 @@
  * Copyright © 2023 Intel Corporation
  */
 
-#include <ctype.h>
-#include <limits.h>
-#include <libkmod.h>
-#include <pthread.h>
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
-#include <unistd.h>
 
-#include "igt_aux.h"
 #include "igt_core.h"
 #include "igt_ktap.h"
 #include "igt_list.h"
@@ -312,565 +306,3 @@ void igt_ktap_free(struct igt_ktap_results *ktap)
 {
 	free(ktap);
 }
-
-#define DELIMITER "-"
-
-struct ktap_parser_args {
-	int fd;
-	bool is_builtin;
-	int ret;
-} ktap_args;
-
-static struct ktap_test_results results;
-
-static int log_to_end(enum igt_log_level level, int fd,
-		      char *record, const char *format, ...) __attribute__((format(printf, 4, 5)));
-
-/**
- * log_to_end:
- * @level: #igt_log_level
- * @record: record to store the read data
- * @format: format string
- * @...: optional arguments used in the format string
- *
- * This is an altered version of the generic structured logging helper function
- * igt_log capable of reading to the end of a given line.
- *
- * Returns: 0 for success, or -2 if there's an error reading from the file
- */
-static int log_to_end(enum igt_log_level level, int fd,
-		      char *record, const char *format, ...)
-{
-	va_list args;
-	const char *lend;
-
-	/* Cutoff after newline character, in order to not display garbage */
-	char *cutoff = strchr(record, '\n');
-	if (cutoff) {
-		if (cutoff - record < BUF_LEN)
-			cutoff[1] = '\0';
-	}
-
-	va_start(args, format);
-	igt_vlog(IGT_LOG_DOMAIN, level, format, args);
-	va_end(args);
-
-	lend = strchrnul(record, '\n');
-	while (*lend == '\0') {
-		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
-
-		if (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EPIPE)
-				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-			else if (errno != EINTR)
-				igt_warn("an error occurred while reading kmsg: %m\n");
-
-			return -2;
-		}
-
-		lend = strchrnul(record, '\n');
-	}
-	return 0;
-}
-
-/**
- * lookup_value:
- * @haystack: the string to search in
- * @needle: the string to search for
- *
- * Returns: the value of the needle in the haystack, or -1 if not found.
- */
-static long lookup_value(const char *haystack, const char *needle)
-{
-	const char *needle_rptr;
-	char *needle_end;
-	long num;
-
-	needle_rptr = strcasestr(haystack, needle);
-
-	if (needle_rptr == NULL)
-		return -1;
-
-	/* Skip search string and whitespaces after it */
-	needle_rptr += strlen(needle);
-
-	num = strtol(needle_rptr, &needle_end, 10);
-
-	if (needle_rptr == needle_end)
-		return -1;
-
-	if (num == LONG_MIN || num == LONG_MAX)
-		return 0;
-
-	return num > 0 ? num : 0;
-}
-
-/**
- * tap_version_present:
- * @record: buffer with tap data
- * @print_info: whether tap version should be printed or not
- *
- * Returns:
- * 0 if not found
- * 1 if found
- */
-static int tap_version_present(char* record, bool print_info)
-{
-	/*
-	 * "(K)TAP version XX" should be the first line on all (sub)tests as per
-	 * https://kernel.org/doc/html/latest/dev-tools/ktap.html#version-lines
-	 *
-	 * but actually isn't, as it currently depends on the KUnit module
-	 * being built-in, so we can't rely on it every time
-	 */
-	const char *version_rptr = strcasestr(record, "TAP version ");
-	char *cutoff;
-
-	if (version_rptr == NULL)
-		return 0;
-
-	/* Cutoff after newline character, in order to not display garbage */
-	cutoff = strchr(version_rptr, '\n');
-	if (cutoff)
-		cutoff[0] = '\0';
-
-	if (print_info)
-		igt_info("%s\n", version_rptr);
-
-	return 1;
-}
-
-/**
- * find_next_tap_subtest:
- * @fd: file descriptor
- * @record: buffer used to read fd
- * @is_builtin: whether KUnit is built-in or not
- *
- * Returns:
- * 0 if there's missing information
- * -1 if not found
- * -2 if there are problems while reading the file.
- * any other value corresponds to the amount of cases of the next (sub)test
- */
-static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_builtin)
-{
-	const char *test_lookup_str, *subtest_lookup_str, *name_rptr;
-	long test_count;
-	char *cutoff;
-
-	test_name[0] = '\0';
-	test_name[BUF_LEN] = '\0';
-
-	test_lookup_str = " subtest: ";
-	subtest_lookup_str = " test: ";
-
-	if (!tap_version_present(record, true))
-		return -1;
-
-	if (is_builtin) {
-		if (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EPIPE)
-				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-			else if (errno != EINTR)
-				igt_warn("an error occurred while reading kmsg: %m\n");
-
-			return -2;
-		}
-	}
-
-	name_rptr = strcasestr(record, test_lookup_str);
-	if (name_rptr != NULL) {
-		name_rptr += strlen(test_lookup_str);
-	} else {
-		name_rptr = strcasestr(record, subtest_lookup_str);
-		if (name_rptr != NULL)
-			name_rptr += strlen(subtest_lookup_str);
-	}
-
-	if (name_rptr == NULL) {
-		if (!is_builtin)
-			/* We've probably found nothing */
-			return -1;
-		igt_info("Missing test name\n");
-	} else {
-		strncpy(test_name, name_rptr, BUF_LEN);
-		/* Cutoff after newline character, in order to not display garbage */
-		cutoff = strchr(test_name, '\n');
-		if (cutoff)
-			cutoff[0] = '\0';
-
-		if (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EPIPE)
-				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-			else if (errno != EINTR)
-				igt_warn("unknown error reading kmsg (%m)\n");
-
-			return -2;
-		}
-
-		/* Now we can be sure we found tests */
-		if (!is_builtin)
-			igt_info("KUnit is not built-in, skipping version check...\n");
-	}
-
-	/*
-	 * Total test count will almost always appear as 0..N at the beginning
-	 * of a run, so we use it to reliably identify a new run
-	 */
-	test_count = lookup_value(record, "..");
-
-	if (test_count <= 0) {
-		igt_info("Missing test count\n");
-		if (test_name[0] == '\0')
-			return 0;
-		if (log_to_end(IGT_LOG_INFO, fd, record,
-				"Running some tests in: %s\n",
-				test_name) < 0)
-			return -2;
-		return 0;
-	} else if (test_name[0] == '\0') {
-		igt_info("Running %ld tests...\n", test_count);
-		return 0;
-	}
-
-	if (log_to_end(IGT_LOG_INFO, fd, record,
-			"Executing %ld tests in: %s\n",
-			test_count, test_name) < 0)
-		return -2;
-
-	return test_count;
-}
-
-/**
- * parse_kmsg_for_tap:
- * @fd: file descriptor
- * @record: buffer used to read fd
- * @test_name: buffer to store the test name
- *
- * Returns:
- * 1 if no results were found
- * 0 if a test succeded
- * -1 if a test failed
- * -2 if there are problems reading the file
- */
-static int parse_kmsg_for_tap(int fd, char *record, char *test_name)
-{
-	const char *lstart, *ok_lookup_str, *nok_lookup_str,
-	      *ok_rptr, *nok_rptr, *comment_start, *value_parse_start;
-	char *test_name_end;
-
-	ok_lookup_str = "ok ";
-	nok_lookup_str = "not ok ";
-
-	lstart = strchrnul(record, ';');
-
-	if (*lstart == '\0') {
-		igt_warn("kmsg truncated: output malformed (%m)\n");
-		return -2;
-	}
-
-	lstart++;
-	while (isspace(*lstart))
-		lstart++;
-
-	nok_rptr = strstr(lstart, nok_lookup_str);
-	if (nok_rptr != NULL) {
-		nok_rptr += strlen(nok_lookup_str);
-		while (isdigit(*nok_rptr) || isspace(*nok_rptr) || *nok_rptr == '-')
-			nok_rptr++;
-		test_name_end = strncpy(test_name, nok_rptr, BUF_LEN);
-		while (!isspace(*test_name_end))
-			test_name_end++;
-		*test_name_end = '\0';
-		if (log_to_end(IGT_LOG_WARN, fd, record,
-			       "%s", lstart) < 0)
-			return -2;
-		return -1;
-	}
-
-	comment_start = strchrnul(lstart, '#');
-
-	/* Check if we're still in a subtest */
-	if (*comment_start != '\0') {
-		comment_start++;
-		value_parse_start = comment_start;
-
-		if (lookup_value(value_parse_start, "fail: ") > 0) {
-			if (log_to_end(IGT_LOG_WARN, fd, record,
-				       "%s", lstart) < 0)
-				return -2;
-			return -1;
-		}
-	}
-
-	ok_rptr = strstr(lstart, ok_lookup_str);
-	if (ok_rptr != NULL) {
-		ok_rptr += strlen(ok_lookup_str);
-		while (isdigit(*ok_rptr) || isspace(*ok_rptr) || *ok_rptr == '-')
-			ok_rptr++;
-		test_name_end = strncpy(test_name, ok_rptr, BUF_LEN);
-		while (!isspace(*test_name_end))
-			test_name_end++;
-		*test_name_end = '\0';
-		return 0;
-	}
-
-	return 1;
-}
-
-/**
- * parse_tap_level:
- * @fd: file descriptor
- * @base_test_name: test_name from upper recursion level
- * @test_count: test_count of this level
- * @failed_tests: top level failed_tests pointer
- * @found_tests: top level found_tests pointer
- * @is_builtin: whether the KUnit module is built-in or not
- *
- * Returns:
- * 0 if succeded
- * -1 if error occurred
- */
-__maybe_unused
-static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *failed_tests,
-			   bool *found_tests, bool is_builtin)
-{
-	char record[BUF_LEN + 1];
-	struct ktap_test_results_element *r;
-	int internal_test_count;
-	char test_name[BUF_LEN + 1];
-	char base_test_name_for_next_level[BUF_LEN + 1];
-
-	for (int i = 0; i < test_count; i++) {
-		if (read(fd, record, BUF_LEN) < 0) {
-			if (errno == EPIPE)
-				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-			else if (errno != EINTR)
-				igt_warn("error reading kmsg (%m)\n");
-
-			return -1;
-		}
-
-		/* Sublevel found */
-		if (tap_version_present(record, false))
-		{
-			internal_test_count = find_next_tap_subtest(fd, record, test_name,
-								    is_builtin);
-			switch (internal_test_count) {
-			case -2:
-				/* No more data to read */
-				return -1;
-			case -1:
-				/* No test found */
-				return -1;
-			case 0:
-				/* Tests found, but they're missing info */
-				*found_tests = true;
-				return -1;
-			default:
-				*found_tests = true;
-
-				memcpy(base_test_name_for_next_level, base_test_name, BUF_LEN);
-				if (strlen(base_test_name_for_next_level) < BUF_LEN - 1 &&
-				    base_test_name_for_next_level[0])
-					strncat(base_test_name_for_next_level, DELIMITER,
-						BUF_LEN - strlen(base_test_name_for_next_level));
-				memcpy(base_test_name_for_next_level + strlen(base_test_name_for_next_level),
-				       test_name, BUF_LEN - strlen(base_test_name_for_next_level));
-
-				if (parse_tap_level(fd, base_test_name_for_next_level,
-						    internal_test_count, failed_tests, found_tests,
-						    is_builtin) == -1)
-					return -1;
-				break;
-			}
-		}
-
-		switch (parse_kmsg_for_tap(fd, record, test_name)) {
-		case -2:
-			return -1;
-		case -1:
-			*failed_tests = true;
-
-			r = malloc(sizeof(*r));
-
-			memcpy(r->test_name, base_test_name, BUF_LEN);
-			if (strlen(r->test_name) < BUF_LEN - 1)
-				if (r->test_name[0])
-					strncat(r->test_name, DELIMITER,
-						BUF_LEN - strlen(r->test_name));
-			memcpy(r->test_name + strlen(r->test_name), test_name,
-			       BUF_LEN - strlen(r->test_name));
-			r->test_name[BUF_LEN] = '\0';
-
-			r->passed = false;
-
-			pthread_mutex_lock(&results.mutex);
-			igt_list_add_tail(&r->link, &results.list);
-			pthread_mutex_unlock(&results.mutex);
-
-			test_name[0] = '\0';
-			break;
-		case 0:
-			r = malloc(sizeof(*r));
-
-			memcpy(r->test_name, base_test_name, BUF_LEN);
-			if (strlen(r->test_name) < BUF_LEN - 1)
-				if (r->test_name[0])
-					strncat(r->test_name, DELIMITER,
-						BUF_LEN - strlen(r->test_name));
-			memcpy(r->test_name + strlen(r->test_name), test_name,
-			       BUF_LEN - strlen(r->test_name));
-			r->test_name[BUF_LEN] = '\0';
-
-			r->passed = true;
-
-			pthread_mutex_lock(&results.mutex);
-			igt_list_add_tail(&r->link, &results.list);
-			pthread_mutex_unlock(&results.mutex);
-
-			test_name[0] = '\0';
-			break;
-		default:
-			break;
-		}
-	}
-	return 0;
-}
-
-/**
- * igt_ktap_parser:
- *
- * This function parses the output of a ktap script and passes it to main thread.
- */
-void *igt_ktap_parser(void *unused)
-{
-	char record[BUF_LEN + 1], *buf, *suite_name = NULL, *case_name = NULL;
-	struct igt_ktap_results *ktap = NULL;
-	int fd = ktap_args.fd;
-	IGT_LIST_HEAD(list);
-	int err;
-
-	ktap = igt_ktap_alloc(&list);
-	if (igt_debug_on(!ktap))
-		goto igt_ktap_parser_end;
-
-	while (err = read(fd, record, BUF_LEN), err > 0) {
-		struct igt_ktap_result *r, *rn;
-
-		/* skip kmsg continuation lines */
-		if (igt_debug_on(*record == ' '))
-			continue;
-
-		/* NULL-terminate the record */
-		record[err] = '\0';
-
-		/* detect start of log message, continue if not found */
-		buf = strchrnul(record, ';');
-		if (igt_debug_on(*buf == '\0'))
-			continue;
-		buf++;
-
-		err = igt_ktap_parse(buf, ktap);
-
-		/* parsing error */
-		if (err && err != -EINPROGRESS)
-			goto igt_ktap_parser_end;
-
-		igt_list_for_each_entry_safe(r, rn, &list, link) {
-			struct ktap_test_results_element *result = NULL;
-			int code = r->code;
-
-			if (code != IGT_EXIT_INVALID)
-				result = calloc(1, sizeof(*result));
-
-			if (result) {
-				snprintf(result->test_name, sizeof(result->test_name),
-					 "%s-%s", r->suite_name, r->case_name);
-
-				if (code == IGT_EXIT_SUCCESS)
-					result->passed = true;
-			}
-
-			igt_list_del(&r->link);
-			if (r->suite_name != suite_name) {
-				free(suite_name);
-				suite_name = r->suite_name;
-			}
-			if (r->case_name != case_name) {
-				free(case_name);
-				case_name = r->case_name;
-			}
-			free(r->msg);
-			free(r);
-
-			/*
-			 * no extra result record expected on start
-			 * of parametrized test case -- skip it
-			 */
-			if (code == IGT_EXIT_INVALID)
-				continue;
-
-			if (!result) {
-				err = -ENOMEM;
-				goto igt_ktap_parser_end;
-			}
-
-			pthread_mutex_lock(&results.mutex);
-			igt_list_add_tail(&result->link, &results.list);
-			pthread_mutex_unlock(&results.mutex);
-		}
-
-		/* end of KTAP report */
-		if (!err)
-			goto igt_ktap_parser_end;
-	}
-
-	if (err < 0) {
-		if (errno == EPIPE)
-			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
-		else if (errno != EINTR)
-			igt_warn("error reading kmsg (%m)\n");
-	}
-
-igt_ktap_parser_end:
-	free(suite_name);
-	free(case_name);
-
-	if (!err)
-		ktap_args.ret = IGT_EXIT_SUCCESS;
-
-	results.still_running = false;
-
-	if (ktap)
-		igt_ktap_free(ktap);
-
-	return NULL;
-}
-
-static pthread_t ktap_parser_thread;
-
-struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
-{
-	IGT_INIT_LIST_HEAD(&results.list);
-	pthread_mutex_init(&results.mutex, NULL);
-	results.still_running = true;
-
-	ktap_args.fd = fd;
-	ktap_args.is_builtin = is_builtin;
-	ktap_args.ret = IGT_EXIT_FAILURE;
-	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
-
-	return &results;
-}
-
-void ktap_parser_cancel(void)
-{
-	pthread_cancel(ktap_parser_thread);
-}
-
-int ktap_parser_stop(void)
-{
-	pthread_join(ktap_parser_thread, NULL);
-	return ktap_args.ret;
-}
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
index 6f8da3eab6..c422636bfc 100644
--- a/lib/igt_ktap.h
+++ b/lib/igt_ktap.h
@@ -27,8 +27,6 @@
 
 #define BUF_LEN 4096
 
-#include <pthread.h>
-
 #include "igt_list.h"
 
 struct igt_ktap_result {
@@ -45,24 +43,4 @@ struct igt_ktap_results *igt_ktap_alloc(struct igt_list_head *results);
 int igt_ktap_parse(const char *buf, struct igt_ktap_results *ktap);
 void igt_ktap_free(struct igt_ktap_results *ktap);
 
-void *igt_ktap_parser(void *unused);
-
-typedef struct ktap_test_results_element {
-	char test_name[BUF_LEN + 1];
-	bool passed;
-	struct igt_list_head link;
-} ktap_test_results_element;
-
-struct ktap_test_results {
-	struct igt_list_head list;
-	pthread_mutex_t mutex;
-	bool still_running;
-};
-
-
-
-struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin);
-void ktap_parser_cancel(void);
-int ktap_parser_stop(void);
-
 #endif /* IGT_KTAP_H */
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 16/17] lib/kunit: Strip "_test" or "_kunit" suffix from subtest names

[Janusz Krzysztofik]

If a user (an IGT test) doesn't provide a subtest name when calling
igt_kunit() then we now use the requested kernel module name as IGT
subtest name.  Since names of kunit test modules usually end with a
"_test" or "_kunit" suffix, those parts of the names don't carry any
useful information.  Strip those suffixes from IGT subtest names.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 26 ++++++++++++++++++++++----
 tests/drm_mm.c | 42 +++++++++++++++++++++---------------------
 2 files changed, 43 insertions(+), 25 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 46a6f81e73..ddd5499f5e 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -944,8 +944,29 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
 	if (igt_ktest_init(&tst, module_name) != 0)
 		return;
 
-	igt_fixture
+	/*
+	 * If the caller (an IGT test) provides no subtest name then we
+	 * take the module name, drop the trailing "_test" or "_kunit"
+	 * suffix, if any, and use the result as our IGT subtest name.
+	 */
+	if (!name) {
+		name = strdup(module_name);
+		if (name) {
+			char *suffix = strstr(name, "_test");
+
+			if (!suffix)
+				suffix = strstr(name, "_kunit");
+
+			if (suffix)
+				*suffix = '\0';
+		}
+	}
+
+	igt_fixture {
+		igt_require(name);
+
 		igt_require(igt_ktest_begin(&tst) == 0);
+	}
 
 	/*
 	 * We need to use igt_subtest here, as otherwise it may crash with:
@@ -954,9 +975,6 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
 	 * proper namespace for dynamic subtests, with is required for CI
 	 * and for documentation.
 	 */
-	if (name == NULL)
-		name = module_name;
-
 	igt_subtest_with_dynamic(name)
 		__igt_kunit(&tst, opts);
 
diff --git a/tests/drm_mm.c b/tests/drm_mm.c
index 9a8b3f3fcb..e6ba224745 100644
--- a/tests/drm_mm.c
+++ b/tests/drm_mm.c
@@ -29,123 +29,123 @@
  * Feature: mapping
  * Run type: FULL
  *
- * SUBTEST: drm_mm_test
+ * SUBTEST: drm_mm
  *
- * SUBTEST: drm_mm_test@align
+ * SUBTEST: drm_mm@align
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@align32
+ * SUBTEST: drm_mm@align32
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@align64
+ * SUBTEST: drm_mm@align64
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@bottomup
+ * SUBTEST: drm_mm@bottomup
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@color
+ * SUBTEST: drm_mm@color
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@color_evict
+ * SUBTEST: drm_mm@color_evict
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@color_evict_range
+ * SUBTEST: drm_mm@color_evict_range
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@debug
+ * SUBTEST: drm_mm@debug
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@evict
+ * SUBTEST: drm_mm@evict
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@evict_range
+ * SUBTEST: drm_mm@evict_range
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@frag
+ * SUBTEST: drm_mm@frag
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@highest
+ * SUBTEST: drm_mm@highest
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@init
+ * SUBTEST: drm_mm@init
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@insert
+ * SUBTEST: drm_mm@insert
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@insert_range
+ * SUBTEST: drm_mm@insert_range
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@lowest
+ * SUBTEST: drm_mm@lowest
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@replace
+ * SUBTEST: drm_mm@replace
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@reserve
+ * SUBTEST: drm_mm@reserve
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@sanitycheck
+ * SUBTEST: drm_mm@sanitycheck
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
  * Test category: GEM_Legacy
  *
- * SUBTEST: drm_mm_test@topdown
+ * SUBTEST: drm_mm@topdown
  * Category: Infrastructure
  * Description: drm_mm range manager SW validation
  * Functionality: DRM memory mangemnt
-- 
2.41.0

[Intel-gfx] [PATCH i-g-t v2 17/17] lib/kunit: Omit suite name prefix if the same as subtest name

[Janusz Krzysztofik]

Kunit test modules usually contain one test suite, named after the module
name with the trailing "_test" or "_kunit" suffix omitted.  Since we
follow the same convention when we derive subtest names from module names,
there is a great chance that those two names match.  Take this into
account when composing names for IGT dynamic sub-subtest names and drop
the leading test suite name component when it is the same as subtest name.

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
---
 lib/igt_kmod.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index ddd5499f5e..bd4305482b 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -822,7 +822,8 @@ static void kunit_result_free(struct igt_ktap_result *r,
  *
  * Returns: IGT default codes
  */
-static void __igt_kunit(struct igt_ktest *tst, const char *opts)
+static void
+__igt_kunit(struct igt_ktest *tst, const char *name, const char *opts)
 {
 	struct modprobe_data modprobe = { tst->kmod, opts, 0, };
 	char *suite_name = NULL, *case_name = NULL;
@@ -866,7 +867,11 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
 
 		r = igt_list_first_entry(&results, r, link);
 
-		igt_dynamic_f("%s-%s", r->suite_name, r->case_name) {
+		igt_dynamic_f("%s%s%s",
+			      strcmp(r->suite_name, name) ?  r->suite_name : "",
+			      strcmp(r->suite_name, name) ? "-" : "",
+			      r->case_name) {
+
 			if (r->code == IGT_EXIT_INVALID) {
 				/* parametrized test case, get actual result */
 				kunit_result_free(r, &suite_name, &case_name);
@@ -976,7 +981,7 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
 	 * and for documentation.
 	 */
 	igt_subtest_with_dynamic(name)
-		__igt_kunit(&tst, opts);
+		__igt_kunit(&tst, name, opts);
 
 	igt_ktest_end(&tst);
 
-- 
2.41.0

Re: [Intel-gfx] [PATCH i-g-t v2 04/17] lib/kunit: Optimize calls to igt_success/skip/fail()

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:38 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Calling igt_success() explicitly at the end of subtest body is not needed.
> Other calls to igt_success() can be usually avoided by inverting test
> result checks.  Optimize the code that now calls igt_success().
> 
> Moreover, using more advanced variants of igt_skip() and igt_fail() where
> applicable makes the code more compact.  Go for it.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 34ddec3fad..1d1cd51170 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -814,12 +814,8 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  		if (READ_ONCE(results->head) != NULL) {
>  			pthread_mutex_lock(&results->mutex);
>  
> -			igt_dynamic(results->head->test_name) {
> -				if (READ_ONCE(results->head->passed))
> -					igt_success();
> -				else
> -					igt_fail(IGT_EXIT_FAILURE);
> -			}
> +			igt_dynamic(results->head->test_name)
> +				igt_assert(READ_ONCE(results->head->passed));
>  
>  			temp = results->head;
>  			results->head = results->head->next;
> @@ -834,8 +830,7 @@ unload:
>  
>  	igt_ktest_fini(&tst);
>  
> -	if (skip)
> -		igt_skip("Skipping test, as probing KUnit module returned %d", skip);
> +	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
>  
>  	if (fail)
>  		igt_fail(IGT_EXIT_ABORT);
> @@ -844,9 +839,6 @@ unload:
>  
>  	if (ret != 0)
>  		igt_fail(IGT_EXIT_ABORT);
> -
> -	if (ret == 0)
> -		igt_success();
>  }
>  
>  void igt_kunit(const char *module_name, const char *name, const char *opts)

Re: [Intel-gfx] [PATCH i-g-t v2 05/17] lib/kunit: Fix illegal igt_fail() calls inside subtest body

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:39 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> In a body of a subtest with dynamic sub-subtests, it is illegal to call
> igt_fail() and its variants from outside of a dynamic sub-subtest body.
> On the other hand, it is perfectly legal to call either igt_skip() and
> friends or __igt_abort() or its variant from there.
> 
> In the current implementation of igt_kunit(), there are several places
> where igt_fail() is called despite being illegal.  Moreover, it is called
> with IGT_EXIT_ABORT as an argument with no good reason for using such
> aggressive method that forces CI to trigger system reboot (in most cases
> igt_runner can decide if abort is required).
> 
> Follow igt_kselftests() pattern more closely, where similar setup and
> cleanup operations are performed but their potential errors are processed
> in a more friendly way.  Move common cleanup and their corresponding setup
> steps out of the subtest body.  Place the latter as requirements in a
> preceding igt_fixture section.  Replace remaining illegal igt_fail() calls
> with more friendly skips.  Let igt_runner decide if abort is needed.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 75 +++++++++++++++-----------------------------------
>  1 file changed, 22 insertions(+), 53 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 1d1cd51170..78b8eb8f53 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -754,59 +754,27 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
>   *
>   * Returns: IGT default codes
>   */
> -static void __igt_kunit(const char *module_name, const char *opts)
> +static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  {
> -	struct igt_ktest tst;
>  	struct kmod_module *kunit_kmod;
>  	bool is_builtin;
>  	int ret;
>  	struct ktap_test_results *results;
>  	struct ktap_test_results_element *temp;
> -	int skip = 0;
> -	bool fail = false;
> -
> -	/* get normalized module name */
> -	if (igt_ktest_init(&tst, module_name) != 0) {
> -		igt_warn("Unable to initialize ktest for %s\n", module_name);
> -		igt_fail(IGT_EXIT_ABORT);
> -	}
> -
> -	if (igt_ktest_begin(&tst) != 0) {
> -		igt_warn("Unable to begin ktest for %s\n", module_name);
> -		igt_ktest_fini(&tst);
> -		igt_fail(IGT_EXIT_ABORT);
> -	}
>  
> -	if (tst.kmsg < 0) {
> -		igt_warn("Could not open /dev/kmsg\n");
> -		fail = true;
> -		goto unload;
> -	}
> +	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
>  
> -	if (lseek(tst.kmsg, 0, SEEK_END)) {
> -		igt_warn("Could not seek the end of /dev/kmsg\n");
> -		fail = true;
> -		goto unload;
> -	}
> -
> -	ret = kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod);
> -	if (ret) {
> -		igt_warn("Unable to load KUnit\n");
> -		skip = ret;
> -		goto unload;
> -	}
> +	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
>  
> +	igt_skip_on(kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod));
>  	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
>  	kmod_module_unref(kunit_kmod);
>  
> -	results = ktap_parser_start(tst.kmsg, is_builtin);
> +	results = ktap_parser_start(tst->kmsg, is_builtin);
>  
> -	ret = igt_kmod_load(module_name, opts);
> -	if (ret) {
> -		skip = ret;
> -		igt_warn("Unable to load %s module\n", module_name);
> -		ret = ktap_parser_stop();
> -		goto unload;
> +	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
> +		igt_ignore_warn(ktap_parser_stop());
> +		igt_skip("Unable to load %s module\n", tst->module_name);
>  	}
>  
>  	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
> @@ -825,24 +793,21 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  		}
>  	}
>  
> -unload:
> -	igt_ktest_end(&tst);
> -
> -	igt_ktest_fini(&tst);
> -
> -	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
> -
> -	if (fail)
> -		igt_fail(IGT_EXIT_ABORT);
> -
>  	ret = ktap_parser_stop();
>  
> -	if (ret != 0)
> -		igt_fail(IGT_EXIT_ABORT);
> +	igt_skip_on_f(ret, "KTAP parser failed\n");
>  }
>  
>  void igt_kunit(const char *module_name, const char *name, const char *opts)
>  {
> +	struct igt_ktest tst;
> +
> +	if (igt_ktest_init(&tst, module_name) != 0)
> +		return;

Shouldn't it be calling igt_skip() here too?

> +
> +	igt_fixture
> +		igt_require(igt_ktest_begin(&tst) == 0);
> +
>  	/*
>  	 * We need to use igt_subtest here, as otherwise it may crash with:
>  	 *  skipping is allowed only in fixtures, subtests or igt_simple_main
> @@ -854,7 +819,11 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
>  		name = module_name;
>  
>  	igt_subtest_with_dynamic(name)
> -		__igt_kunit(module_name, opts);
> +		__igt_kunit(&tst, opts);
> +
> +	igt_ktest_end(&tst);
> +
> +	igt_ktest_fini(&tst);
>  }
>  
>  static int open_parameters(const char *module_name)

Re: [Intel-gfx] [PATCH i-g-t v2 06/17] lib/ktap: Make sure we fail on premature cancel

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:40 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> After loading a kunit test module that executes some kunit test cases, we
> evaluate overall result of an IGT subtest that corresponds to that module
> based on an error code returned by kunit_parser_stop() helper, obtained
> from a .ret field of a ktap_args structure.  That code is now assigned to
> that structure field right before completion of the KTAP parser thread
> start routine.  If the thread is canceled for some reason then the return
> code will be undefined.
> 
> Initialize the return code on KTAP parser startup with a value that
> indicates a failure, then change it to success when so indicated by result
> of KTAP parsing.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> ---
>  lib/igt_ktap.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 123a40d183..84fb13218f 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -579,9 +579,7 @@ igt_ktap_parser_start:
>  igt_ktap_parser_end:
>  	results.still_running = false;
>  
> -	if (failed_tests || !found_tests)
> -		ktap_args.ret = IGT_EXIT_FAILURE;
> -	else
> +	if (found_tests && !failed_tests)
>  		ktap_args.ret = IGT_EXIT_SUCCESS;
>  
>  	return NULL;
> @@ -598,6 +596,7 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  	ktap_args.fd = fd;
>  	ktap_args.is_builtin = is_builtin;
>  	ktap_args.is_running = true;
> +	ktap_args.ret = IGT_EXIT_FAILURE;
>  	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
>  
>  	return &results;

Re: [Intel-gfx] [PATCH i-g-t v2 07/17] lib/ktap: Don't ignore interrupt signals

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:41 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> While reading KTAP data from /dev/kmsg we now ignore interrupt signals
> that may occur during read() and we continue reading the data.  No
> explanation has been provided on what that could be needed for.

The reason is that kunit module load takes seconds, as it only finishes
loading after all tests are executed.

So, interrupting IGT won't interrupt the tests, and kmsg will still
be filled by test results.

IMO, the right thing to do here is to ignore interrupts, as otherwise
the logs for the next test will be polluted by the KTAP messages and
the Kernel will be kept on an unstable state, as running tests while
kUnit tests are running is not supported.

> 
> Always return with an error code to the caller when read() fails with
> errno == EINTR, so igt_runner has no problems with killing us promptly
> on timeout.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_ktap.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 84fb13218f..3cfb55ec97 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -67,7 +67,7 @@ static int log_to_end(enum igt_log_level level, int fd,
>  			}
>  
>  			if (errno == EINTR)
> -				continue;
> +				return -2;
>  
>  			if (errno == EPIPE) {
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> @@ -189,7 +189,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  			}
>  
>  			if (errno == EINTR)
> -				continue;
> +				return -2;
>  
>  			if (errno == EPIPE) {
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> @@ -233,7 +233,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  			}
>  
>  			if (errno == EINTR)
> -				continue;
> +				return -2;
>  
>  			if (errno == EPIPE) {
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> @@ -388,7 +388,7 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  			}
>  
>  			if (errno == EINTR)
> -				continue;
> +				return -1;
>  
>  			if (errno == EAGAIN)
>  				/* No records available */
> @@ -541,7 +541,7 @@ igt_ktap_parser_start:
>  			continue;
>  
>  		if (errno == EINTR)
> -			continue;
> +			goto igt_ktap_parser_end;
>  
>  		if (errno == EPIPE) {
>  			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");

Re: [Intel-gfx] [PATCH i-g-t v2 08/17] lib/kunit: Cancel KTP parser on module load failure

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:42 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> For our KTAP parser to be running in parallel with kunit test module
> loading, we now start it in a separate thread before we load the module.
> If the module loading fails then we join the KTAP parser thread right
> after that failure.  If the KTAP thread sleeps for some reason then we
> can fail to break the test immediately.
> 
> Cancel the KTAP parser thread right after module load error and before
> joining it.

LGTM.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 1 +
>  lib/igt_ktap.c | 6 ++++++
>  lib/igt_ktap.h | 1 +
>  3 files changed, 8 insertions(+)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 78b8eb8f53..fb0bd21ee5 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -773,6 +773,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  	results = ktap_parser_start(tst->kmsg, is_builtin);
>  
>  	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
> +		ktap_parser_cancel();
>  		igt_ignore_warn(ktap_parser_stop());
>  		igt_skip("Unable to load %s module\n", tst->module_name);
>  	}
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 3cfb55ec97..1e75b2ec23 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -602,6 +602,12 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  	return &results;
>  }
>  
> +void ktap_parser_cancel(void)
> +{
> +	ktap_args.is_running = false;
> +	pthread_cancel(ktap_parser_thread);
> +}
> +
>  int ktap_parser_stop(void)
>  {
>  	ktap_args.is_running = false;
> diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> index ea57c2bb9b..991800e912 100644
> --- a/lib/igt_ktap.h
> +++ b/lib/igt_ktap.h
> @@ -45,6 +45,7 @@ struct ktap_test_results {
>  
>  
>  struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin);
> +void ktap_parser_cancel(void);
>  int ktap_parser_stop(void);
>  
>  #endif /* IGT_KTAP_H */

Re: [Intel-gfx] [PATCH i-g-t v2 09/17] lib/ktap: Drop is_running flag

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:43 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Since we now call pthread_cancel() when we want to stop KTAP parser before
> it completes, and we take care of returning failure in that case as a
> result of KTAP parsing, we no longer need to check a flag that indicates
> whether we should continue parsing or return a failure.  Drop that flag.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>

LGTM.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>


> ---
>  lib/igt_ktap.c | 32 --------------------------------
>  1 file changed, 32 deletions(-)
> 
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 1e75b2ec23..fe77b62680 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -18,7 +18,6 @@
>  struct ktap_parser_args {
>  	int fd;
>  	bool is_builtin;
> -	volatile bool is_running;
>  	int ret;
>  } ktap_args;
>  
> @@ -61,11 +60,6 @@ static int log_to_end(enum igt_log_level level, int fd,
>  		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
>  
>  		while (read(fd, record, BUF_LEN) < 0) {
> -			if (!READ_ONCE(ktap_args.is_running)) {
> -				igt_warn("ktap parser stopped\n");
> -				return -2;
> -			}
> -
>  			if (errno == EINTR)
>  				return -2;
>  
> @@ -183,11 +177,6 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  
>  	if (is_builtin) {
>  		while (read(fd, record, BUF_LEN) < 0) {
> -			if (!READ_ONCE(ktap_args.is_running)) {
> -				igt_warn("ktap parser stopped\n");
> -				return -2;
> -			}
> -
>  			if (errno == EINTR)
>  				return -2;
>  
> @@ -227,11 +216,6 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  			cutoff[0] = '\0';
>  
>  		while (read(fd, record, BUF_LEN) < 0) {
> -			if (!READ_ONCE(ktap_args.is_running)) {
> -				igt_warn("ktap parser stopped\n");
> -				return -2;
> -			}
> -
>  			if (errno == EINTR)
>  				return -2;
>  
> @@ -382,11 +366,6 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  
>  	for (int i = 0; i < test_count; i++) {
>  		while (read(fd, record, BUF_LEN) < 0) {
> -			if (!READ_ONCE(ktap_args.is_running)) {
> -				igt_warn("ktap parser stopped\n");
> -				return -1;
> -			}
> -
>  			if (errno == EINTR)
>  				return -1;
>  
> @@ -523,19 +502,11 @@ void *igt_ktap_parser(void *unused)
>  	failed_tests = false;
>  	found_tests = false;
>  
> -	if (!READ_ONCE(ktap_args.is_running))
> -		goto igt_ktap_parser_end;
> -
>  igt_ktap_parser_start:
>  	test_name[0] = '\0';
>  	test_name[BUF_LEN] = '\0';
>  
>  	while (read(fd, record, BUF_LEN) < 0) {
> -		if (!READ_ONCE(ktap_args.is_running)) {
> -			igt_warn("ktap parser stopped\n");
> -			goto igt_ktap_parser_end;
> -		}
> -
>  		if (errno == EAGAIN)
>  			/* No records available */
>  			continue;
> @@ -595,7 +566,6 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  
>  	ktap_args.fd = fd;
>  	ktap_args.is_builtin = is_builtin;
> -	ktap_args.is_running = true;
>  	ktap_args.ret = IGT_EXIT_FAILURE;
>  	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
>  
> @@ -604,13 +574,11 @@ struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  
>  void ktap_parser_cancel(void)
>  {
> -	ktap_args.is_running = false;
>  	pthread_cancel(ktap_parser_thread);
>  }
>  
>  int ktap_parser_stop(void)
>  {
> -	ktap_args.is_running = false;
>  	pthread_join(ktap_parser_thread, NULL);
>  	return ktap_args.ret;
>  }

Re: [Intel-gfx] [PATCH i-g-t v2 05/17] lib/kunit: Fix illegal igt_fail() calls inside subtest body

[Janusz Krzysztofik]

Hi Mauro,

Thanks for review.

On Monday, 11 September 2023 10:52:51 CEST Mauro Carvalho Chehab wrote:
> On Fri,  8 Sep 2023 14:32:39 +0200
> Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> 
> > In a body of a subtest with dynamic sub-subtests, it is illegal to call
> > igt_fail() and its variants from outside of a dynamic sub-subtest body.
> > On the other hand, it is perfectly legal to call either igt_skip() and
> > friends or __igt_abort() or its variant from there.
> > 
> > In the current implementation of igt_kunit(), there are several places
> > where igt_fail() is called despite being illegal.  Moreover, it is called
> > with IGT_EXIT_ABORT as an argument with no good reason for using such
> > aggressive method that forces CI to trigger system reboot (in most cases
> > igt_runner can decide if abort is required).
> > 
> > Follow igt_kselftests() pattern more closely, where similar setup and
> > cleanup operations are performed but their potential errors are processed
> > in a more friendly way.  Move common cleanup and their corresponding setup
> > steps out of the subtest body.  Place the latter as requirements in a
> > preceding igt_fixture section.  Replace remaining illegal igt_fail() calls
> > with more friendly skips.  Let igt_runner decide if abort is needed.
> > 
> > Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> > ---
> >  lib/igt_kmod.c | 75 +++++++++++++++-----------------------------------
> >  1 file changed, 22 insertions(+), 53 deletions(-)
> > 
> > diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> > index 1d1cd51170..78b8eb8f53 100644
> > --- a/lib/igt_kmod.c
> > +++ b/lib/igt_kmod.c
...
> > @@ -825,24 +793,21 @@ static void __igt_kunit(const char *module_name, const char *opts)
> >  		}
> >  	}
> >  
> > -unload:
> > -	igt_ktest_end(&tst);
> > -
> > -	igt_ktest_fini(&tst);
> > -
> > -	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
> > -
> > -	if (fail)
> > -		igt_fail(IGT_EXIT_ABORT);
> > -
> >  	ret = ktap_parser_stop();
> >  
> > -	if (ret != 0)
> > -		igt_fail(IGT_EXIT_ABORT);
> > +	igt_skip_on_f(ret, "KTAP parser failed\n");
> >  }
> >  
> >  void igt_kunit(const char *module_name, const char *name, const char *opts)
> >  {
> > +	struct igt_ktest tst;
> > +
> > +	if (igt_ktest_init(&tst, module_name) != 0)
> > +		return;
> 
> Shouldn't it be calling igt_skip() here too?

Maybe yes.  I've chosen to follow the algorithm used in igt_kselftest.  There 
was an igt_skip() variant there initially but in 2017 that was converted to 
the current return only by Peter with IGT commit 9f92893b11e8 ("lib/igt_kmod: 
Don't call igt_assert or igt_require without a fixture").  However, 
justification for dropping igt_require() instead of calling it from an 
igt_fixture section may not apply to kunit modules:

"If kmod_module_new_from_name fails, ... return normally from igt_kselftest, 
matching behaviour when the module loading is successful but it doesn't 
contain selftests."

While i915 could be built with no selftests included, a kunit module without 
any tests doesn't make sense, then silent return may be not what we need.

Thanks,
Janusz

> 
> > +
> > +	igt_fixture
> > +		igt_require(igt_ktest_begin(&tst) == 0);
> > +
> >  	/*
> >  	 * We need to use igt_subtest here, as otherwise it may crash with:
> >  	 *  skipping is allowed only in fixtures, subtests or igt_simple_main
> > @@ -854,7 +819,11 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
> >  		name = module_name;
> >  
> >  	igt_subtest_with_dynamic(name)
> > -		__igt_kunit(module_name, opts);
> > +		__igt_kunit(&tst, opts);
> > +
> > +	igt_ktest_end(&tst);
> > +
> > +	igt_ktest_fini(&tst);
> >  }
> >  
> >  static int open_parameters(const char *module_name)
> 

Re: [Intel-gfx] [PATCH i-g-t v2 05/17] lib/kunit: Fix illegal igt_fail() calls inside subtest body

[Mauro Carvalho Chehab]

On Mon, 11 Sep 2023 11:28:32 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Hi Mauro,
> 
> Thanks for review.
> 
> On Monday, 11 September 2023 10:52:51 CEST Mauro Carvalho Chehab wrote:
> > On Fri,  8 Sep 2023 14:32:39 +0200
> > Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> >   
> > > In a body of a subtest with dynamic sub-subtests, it is illegal to call
> > > igt_fail() and its variants from outside of a dynamic sub-subtest body.
> > > On the other hand, it is perfectly legal to call either igt_skip() and
> > > friends or __igt_abort() or its variant from there.
> > > 
> > > In the current implementation of igt_kunit(), there are several places
> > > where igt_fail() is called despite being illegal.  Moreover, it is called
> > > with IGT_EXIT_ABORT as an argument with no good reason for using such
> > > aggressive method that forces CI to trigger system reboot (in most cases
> > > igt_runner can decide if abort is required).
> > > 
> > > Follow igt_kselftests() pattern more closely, where similar setup and
> > > cleanup operations are performed but their potential errors are processed
> > > in a more friendly way.  Move common cleanup and their corresponding setup
> > > steps out of the subtest body.  Place the latter as requirements in a
> > > preceding igt_fixture section.  Replace remaining illegal igt_fail() calls
> > > with more friendly skips.  Let igt_runner decide if abort is needed.
> > > 
> > > Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> > > ---
> > >  lib/igt_kmod.c | 75 +++++++++++++++-----------------------------------
> > >  1 file changed, 22 insertions(+), 53 deletions(-)
> > > 
> > > diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> > > index 1d1cd51170..78b8eb8f53 100644
> > > --- a/lib/igt_kmod.c
> > > +++ b/lib/igt_kmod.c  
> ...
> > > @@ -825,24 +793,21 @@ static void __igt_kunit(const char *module_name, const char *opts)
> > >  		}
> > >  	}
> > >  
> > > -unload:
> > > -	igt_ktest_end(&tst);
> > > -
> > > -	igt_ktest_fini(&tst);
> > > -
> > > -	igt_skip_on_f(skip, "Skipping test, as probing KUnit module failed\n");
> > > -
> > > -	if (fail)
> > > -		igt_fail(IGT_EXIT_ABORT);
> > > -
> > >  	ret = ktap_parser_stop();
> > >  
> > > -	if (ret != 0)
> > > -		igt_fail(IGT_EXIT_ABORT);
> > > +	igt_skip_on_f(ret, "KTAP parser failed\n");
> > >  }
> > >  
> > >  void igt_kunit(const char *module_name, const char *name, const char *opts)
> > >  {
> > > +	struct igt_ktest tst;
> > > +
> > > +	if (igt_ktest_init(&tst, module_name) != 0)
> > > +		return;  
> > 
> > Shouldn't it be calling igt_skip() here too?  
> 
> Maybe yes.  I've chosen to follow the algorithm used in igt_kselftest.  There 
> was an igt_skip() variant there initially but in 2017 that was converted to 
> the current return only by Peter with IGT commit 9f92893b11e8 ("lib/igt_kmod: 
> Don't call igt_assert or igt_require without a fixture").  However, 
> justification for dropping igt_require() instead of calling it from an 
> igt_fixture section may not apply to kunit modules:
> 
> "If kmod_module_new_from_name fails, ... return normally from igt_kselftest, 
> matching behaviour when the module loading is successful but it doesn't 
> contain selftests."
> 
> While i915 could be built with no selftests included, a kunit module without 
> any tests doesn't make sense, then silent return may be not what we need.

Yeah, selftests are handled on a different way with regards to module
probe, so I guess we need the igt_skip there if modprobe fails.

Well, you can probably simulate it by renaming a Kunit module
and see how IGT will handle that with the current code and with
igt_skip().

(Btw, I intend to review the other patches on this series, but need
some time to do tests, as some changes here are not trivial)

Regards,
Mauro

Re: [Intel-gfx] [PATCH i-g-t v2 05/17] lib/kunit: Fix illegal igt_fail() calls inside subtest body

[Janusz Krzysztofik]

On Monday, 11 September 2023 13:57:29 CEST Mauro Carvalho Chehab wrote:
> On Mon, 11 Sep 2023 11:28:32 +0200
> Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> 
> > Hi Mauro,
> > 
> > Thanks for review.
> > 
> > On Monday, 11 September 2023 10:52:51 CEST Mauro Carvalho Chehab wrote:
> > > On Fri,  8 Sep 2023 14:32:39 +0200
> > > Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> > >   
> > > > In a body of a subtest with dynamic sub-subtests, it is illegal to 
call
> > > > igt_fail() and its variants from outside of a dynamic sub-subtest 
body.
> > > > On the other hand, it is perfectly legal to call either igt_skip() and
> > > > friends or __igt_abort() or its variant from there.
> > > > 
> > > > In the current implementation of igt_kunit(), there are several places
> > > > where igt_fail() is called despite being illegal.  Moreover, it is 
called
> > > > with IGT_EXIT_ABORT as an argument with no good reason for using such
> > > > aggressive method that forces CI to trigger system reboot (in most 
cases
> > > > igt_runner can decide if abort is required).
> > > > 
> > > > Follow igt_kselftests() pattern more closely, where similar setup and
> > > > cleanup operations are performed but their potential errors are 
processed
> > > > in a more friendly way.  Move common cleanup and their corresponding 
setup
> > > > steps out of the subtest body.  Place the latter as requirements in a
> > > > preceding igt_fixture section.  Replace remaining illegal igt_fail() 
calls
> > > > with more friendly skips.  Let igt_runner decide if abort is needed.
> > > > 
> > > > Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> > > > ---
> > > >  lib/igt_kmod.c | 75 ++++++++++++++
+-----------------------------------
> > > >  1 file changed, 22 insertions(+), 53 deletions(-)
> > > > 
> > > > diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> > > > index 1d1cd51170..78b8eb8f53 100644
> > > > --- a/lib/igt_kmod.c
> > > > +++ b/lib/igt_kmod.c  
> > ...
> > > > @@ -825,24 +793,21 @@ static void __igt_kunit(const char *module_name, 
const char *opts)
> > > >  		}
> > > >  	}
> > > >  
> > > > -unload:
> > > > -	igt_ktest_end(&tst);
> > > > -
> > > > -	igt_ktest_fini(&tst);
> > > > -
> > > > -	igt_skip_on_f(skip, "Skipping test, as probing KUnit module 
failed\n");
> > > > -
> > > > -	if (fail)
> > > > -		igt_fail(IGT_EXIT_ABORT);
> > > > -
> > > >  	ret = ktap_parser_stop();
> > > >  
> > > > -	if (ret != 0)
> > > > -		igt_fail(IGT_EXIT_ABORT);
> > > > +	igt_skip_on_f(ret, "KTAP parser failed\n");
> > > >  }
> > > >  
> > > >  void igt_kunit(const char *module_name, const char *name, const char 
*opts)
> > > >  {
> > > > +	struct igt_ktest tst;
> > > > +
> > > > +	if (igt_ktest_init(&tst, module_name) != 0)
> > > > +		return;  
> > > 
> > > Shouldn't it be calling igt_skip() here too?  
> > 
> > Maybe yes.  I've chosen to follow the algorithm used in igt_kselftest.  
There 
> > was an igt_skip() variant there initially but in 2017 that was converted 
to 
> > the current return only by Peter with IGT commit 9f92893b11e8 ("lib/
igt_kmod: 
> > Don't call igt_assert or igt_require without a fixture").  However, 
> > justification for dropping igt_require() instead of calling it from an 
> > igt_fixture section may not apply to kunit modules:
> > 
> > "If kmod_module_new_from_name fails, ... return normally from 
igt_kselftest, 
> > matching behaviour when the module loading is successful but it doesn't 
> > contain selftests."
> > 
> > While i915 could be built with no selftests included, a kunit module 
without 
> > any tests doesn't make sense, then silent return may be not what we need.
> 
> Yeah, selftests are handled on a different way with regards to module
> probe, so I guess we need the igt_skip there if modprobe fails.

After having a closer look at it, I think that igt_ktest_init() has nothing to 
do with actual modprobe, and it can fail only on either no memory or if 
module_name == NULL.  Anyway, I'll make the subtest skip if it fails.

> Well, you can probably simulate it by renaming a Kunit module
> and see how IGT will handle that with the current code and with
> igt_skip().

Yes, I've tired, and my results have confirmed my conclusions from code 
review.  But more important, I've found an issue in patch 15/17, "Parse KTAP 
report from the main process thread", that can cause first read() to wait 
infinitely, unless interrupted, if modprobe fails.  I've already developed a 
working fix that interrupts that read() on modprobe failure, and I'll include 
it in next version of the series.

Thanks,
Janusz

> 
> (Btw, I intend to review the other patches on this series, but need
> some time to do tests, as some changes here are not trivial)
> 
> Regards,
> Mauro
> 

Re: [Intel-gfx] [PATCH i-g-t v2 07/17] lib/ktap: Don't ignore interrupt signals

[Janusz Krzysztofik]

On Monday, 11 September 2023 11:01:42 CEST Mauro Carvalho Chehab wrote:
> On Fri,  8 Sep 2023 14:32:41 +0200
> Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> 
> > While reading KTAP data from /dev/kmsg we now ignore interrupt signals
> > that may occur during read() and we continue reading the data.  No
> > explanation has been provided on what that could be needed for.
> 
> The reason is that kunit module load takes seconds, as it only finishes
> loading after all tests are executed.
> 
> So, interrupting IGT won't interrupt the tests, and kmsg will still
> be filled by test results.
> 
> IMO, the right thing to do here is to ignore interrupts, as otherwise
> the logs for the next test will be polluted by the KTAP messages and
> the Kernel will be kept on an unstable state, as running tests while
> kUnit tests are running is not supported.

Well, not really.  Please have a look at the following two log excerpts.  The 
first one is from a complete, not interrupted execution of drm_mm test:

Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: jkrzyszt : TTY=pts/14 ; PWD=/home/jkrzyszt/build/igt ; USER=root ; COMMAND=./tests/drm_mm
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0x5b01fc53 max_iterations=8192 max_prime=128
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_reserve
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_reserve, SUCCESS
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert
Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_replace
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_replace, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert_range
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert_range, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: bottom-up fragmented insert of 10000 and 20000 insertions took 6009658 and 13648333 nsecs
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: top-down fragmented insert of 10000 and 20000 insertions took 6518544 and 13824246 nsecs
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_frag
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_frag, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align32
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align32, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align64
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align64, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict_range
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict_range, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_topdown
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_topdown, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_bottomup
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_bottomup, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_lowest
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_lowest, SUCCESS
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_highest
Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_highest, SUCCESS
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color, SUCCESS
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict, SUCCESS
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict_range
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict_range, SUCCESS
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm_test, SUCCESS
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session closed for user root
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: exiting, ret=0
Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75

You can see it took 17 seconds for the test to complete.  Now the same test 
but interrupted after 2 seconds:

Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0xadd9af85 max_iterations=8192 max_prime=128
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session closed for user root

When you compare timestamps, you can see that kunit layer reported an OK 
result from test case 3 drm_test_mm_reserve two seconds after a result from 
test case 2 drm_test_mm_debug, presumably as soon as user interrupt occurred, 
while it took ca. 7 seconds before.  We can also see that the process tried to 
exit, but a still running despite interrupted modprobe thread prevented it 
from exiting before modprobe completion.  Moreover, we can see an OK result 
from test case 4 drm_test_mm_insert now appeared after 7 seconds, presumably 
after test case 3 actually completed, and OK results from remaining test cases 
appeared immediately thereafter, and module loading completed within those ca. 
7 seconds.  All those OK results that appeared after the test was interrupted 
were then wrong, however, they were not converted to any IGT results, which is 
fine.

Other than that, while fixing the issue of infinite first blocking read() on 
modprobe failure, introduced by patch 15/17, "Parse KTAP report from the main 
process thread", I've understood that the code that tried to handle EINTR this 
way or another was anyway a dead code.  For it to function as expected, we 
would have to disable default signal handler for SIGINT so we actually get 
that return code from read() instead of the process being killed, while we 
don't do that.  Then, unless we capture SIGINT, only dropping that code 
completely instead of modifying it makes sense, I believe, and commit 
description must be updated.

Thanks,
Janusz

> 
> > 
> > Always return with an error code to the caller when read() fails with
> > errno == EINTR, so igt_runner has no problems with killing us promptly
> > on timeout.
> > 
> > Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> > ---
> >  lib/igt_ktap.c | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> > 
> > diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> > index 84fb13218f..3cfb55ec97 100644
> > --- a/lib/igt_ktap.c
> > +++ b/lib/igt_ktap.c
> > @@ -67,7 +67,7 @@ static int log_to_end(enum igt_log_level level, int fd,
> >  			}
> >  
> >  			if (errno == EINTR)
> > -				continue;
> > +				return -2;
> >  
> >  			if (errno == EPIPE) {
> >  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> > @@ -189,7 +189,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
> >  			}
> >  
> >  			if (errno == EINTR)
> > -				continue;
> > +				return -2;
> >  
> >  			if (errno == EPIPE) {
> >  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> > @@ -233,7 +233,7 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
> >  			}
> >  
> >  			if (errno == EINTR)
> > -				continue;
> > +				return -2;
> >  
> >  			if (errno == EPIPE) {
> >  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> > @@ -388,7 +388,7 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
> >  			}
> >  
> >  			if (errno == EINTR)
> > -				continue;
> > +				return -1;
> >  
> >  			if (errno == EAGAIN)
> >  				/* No records available */
> > @@ -541,7 +541,7 @@ igt_ktap_parser_start:
> >  			continue;
> >  
> >  		if (errno == EINTR)
> > -			continue;
> > +			goto igt_ktap_parser_end;
> >  
> >  		if (errno == EPIPE) {
> >  			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> 

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2 01/17] lib/kunit: Drop unused file stream

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:35 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> In the process of reviewing patches that introduced kunit support, I asked
> once if we could use line buffered input instead of explicitly looking for
> newlines in KTAP data.  While my idea was wrong because reading raw data
> from /dev/kmsg already returns full log records that always end with a
> newline, conversion of /dev/kmsg file descriptor to a file stream with
> freopen() was added to the code.  However, that file stream has never been
> used for line buffered input.  While the file stream is passed to
> functions that actually read the data, there it is converted back to a
> file descriptor with fileno() and the data is read with read().
> 
> Drop the unnecessary conversions and teach functions to accept and process
> just the file descriptor of /dev/kmsg.

LGTM.

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 12 +---------
>  lib/igt_ktap.c | 62 +++++++++++++++++++++++---------------------------
>  lib/igt_ktap.h |  2 +-
>  3 files changed, 31 insertions(+), 45 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 6205871791..97667a896f 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -758,7 +758,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  {
>  	struct igt_ktest tst;
>  	struct kmod_module *kunit_kmod;
> -	FILE *f;
>  	bool is_builtin;
>  	int ret;
>  	struct ktap_test_results *results;
> @@ -774,7 +773,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  
>  	if (igt_ktest_begin(&tst) != 0) {
>  		igt_warn("Unable to begin ktest for %s\n", module_name);
> -
>  		igt_ktest_fini(&tst);
>  		igt_fail(IGT_EXIT_ABORT);
>  	}
> @@ -791,14 +789,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  		goto unload;
>  	}
>  
> -	f = fdopen(tst.kmsg, "r");
> -
> -	if (f == NULL) {
> -		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer\n");
> -		fail = true;
> -		goto unload;
> -	}
> -
>  	/* The KUnit module is required for running any KUnit tests */
>  	ret = igt_kmod_load("kunit", NULL);
>  	if (ret) {
> @@ -814,7 +804,7 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  
>  	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
>  
> -	results = ktap_parser_start(f, is_builtin);
> +	results = ktap_parser_start(tst.kmsg, is_builtin);
>  
>  	ret = igt_kmod_load(module_name, opts);
>  	if (ret) {
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index ecdcb8d83d..123a40d183 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -16,7 +16,7 @@
>  #define DELIMITER "-"
>  
>  struct ktap_parser_args {
> -	FILE *fp;
> +	int fd;
>  	bool is_builtin;
>  	volatile bool is_running;
>  	int ret;
> @@ -24,7 +24,7 @@ struct ktap_parser_args {
>  
>  static struct ktap_test_results results;
>  
> -static int log_to_end(enum igt_log_level level, FILE *f,
> +static int log_to_end(enum igt_log_level level, int fd,
>  		      char *record, const char *format, ...) __attribute__((format(printf, 4, 5)));
>  
>  /**
> @@ -39,12 +39,11 @@ static int log_to_end(enum igt_log_level level, FILE *f,
>   *
>   * Returns: 0 for success, or -2 if there's an error reading from the file
>   */
> -static int log_to_end(enum igt_log_level level, FILE *f,
> +static int log_to_end(enum igt_log_level level, int fd,
>  		      char *record, const char *format, ...)
>  {
>  	va_list args;
>  	const char *lend;
> -	int f_fd = fileno(f);
>  
>  	/* Cutoff after newline character, in order to not display garbage */
>  	char *cutoff = strchr(record, '\n');
> @@ -61,7 +60,7 @@ static int log_to_end(enum igt_log_level level, FILE *f,
>  	while (*lend == '\0') {
>  		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
>  
> -		while (read(f_fd, record, BUF_LEN) < 0) {
> +		while (read(fd, record, BUF_LEN) < 0) {
>  			if (!READ_ONCE(ktap_args.is_running)) {
>  				igt_warn("ktap parser stopped\n");
>  				return -2;
> @@ -157,8 +156,8 @@ static int tap_version_present(char* record, bool print_info)
>  
>  /**
>   * find_next_tap_subtest:
> - * @fp: FILE pointer
> - * @record: buffer used to read fp
> + * @fd: file descriptor
> + * @record: buffer used to read fd
>   * @is_builtin: whether KUnit is built-in or not
>   *
>   * Returns:
> @@ -167,11 +166,10 @@ static int tap_version_present(char* record, bool print_info)
>   * -2 if there are problems while reading the file.
>   * any other value corresponds to the amount of cases of the next (sub)test
>   */
> -static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool is_builtin)
> +static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_builtin)
>  {
>  	const char *test_lookup_str, *subtest_lookup_str, *name_rptr;
>  	long test_count;
> -	int fp_fd = fileno(fp);
>  	char *cutoff;
>  
>  	test_name[0] = '\0';
> @@ -184,7 +182,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>  		return -1;
>  
>  	if (is_builtin) {
> -		while (read(fp_fd, record, BUF_LEN) < 0) {
> +		while (read(fd, record, BUF_LEN) < 0) {
>  			if (!READ_ONCE(ktap_args.is_running)) {
>  				igt_warn("ktap parser stopped\n");
>  				return -2;
> @@ -228,7 +226,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>  		if (cutoff)
>  			cutoff[0] = '\0';
>  
> -		while (read(fp_fd, record, BUF_LEN) < 0) {
> +		while (read(fd, record, BUF_LEN) < 0) {
>  			if (!READ_ONCE(ktap_args.is_running)) {
>  				igt_warn("ktap parser stopped\n");
>  				return -2;
> @@ -265,7 +263,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>  		igt_info("Missing test count\n");
>  		if (test_name[0] == '\0')
>  			return 0;
> -		if (log_to_end(IGT_LOG_INFO, fp, record,
> +		if (log_to_end(IGT_LOG_INFO, fd, record,
>  				"Running some tests in: %s\n",
>  				test_name) < 0)
>  			return -2;
> @@ -275,7 +273,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>  		return 0;
>  	}
>  
> -	if (log_to_end(IGT_LOG_INFO, fp, record,
> +	if (log_to_end(IGT_LOG_INFO, fd, record,
>  			"Executing %ld tests in: %s\n",
>  			test_count, test_name) < 0)
>  		return -2;
> @@ -285,8 +283,8 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>  
>  /**
>   * parse_kmsg_for_tap:
> - * @fp: FILE pointer
> - * @record: buffer used to read fp
> + * @fd: file descriptor
> + * @record: buffer used to read fd
>   * @test_name: buffer to store the test name
>   *
>   * Returns:
> @@ -295,7 +293,7 @@ static int find_next_tap_subtest(FILE *fp, char *record, char *test_name, bool i
>   * -1 if a test failed
>   * -2 if there are problems reading the file
>   */
> -static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
> +static int parse_kmsg_for_tap(int fd, char *record, char *test_name)
>  {
>  	const char *lstart, *ok_lookup_str, *nok_lookup_str,
>  	      *ok_rptr, *nok_rptr, *comment_start, *value_parse_start;
> @@ -324,7 +322,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
>  		while (!isspace(*test_name_end))
>  			test_name_end++;
>  		*test_name_end = '\0';
> -		if (log_to_end(IGT_LOG_WARN, fp, record,
> +		if (log_to_end(IGT_LOG_WARN, fd, record,
>  			       "%s", lstart) < 0)
>  			return -2;
>  		return -1;
> @@ -338,7 +336,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
>  		value_parse_start = comment_start;
>  
>  		if (lookup_value(value_parse_start, "fail: ") > 0) {
> -			if (log_to_end(IGT_LOG_WARN, fp, record,
> +			if (log_to_end(IGT_LOG_WARN, fd, record,
>  				       "%s", lstart) < 0)
>  				return -2;
>  			return -1;
> @@ -362,7 +360,7 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
>  
>  /**
>   * parse_tap_level:
> - * @fp: FILE pointer
> + * @fd: file descriptor
>   * @base_test_name: test_name from upper recursion level
>   * @test_count: test_count of this level
>   * @failed_tests: top level failed_tests pointer
> @@ -373,10 +371,9 @@ static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
>   * 0 if succeded
>   * -1 if error occurred
>   */
> -static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool *failed_tests,
> +static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *failed_tests,
>  			   bool *found_tests, bool is_builtin)
>  {
> -	int fp_fd = fileno(fp);
>  	char record[BUF_LEN + 1];
>  	struct ktap_test_results_element *r, *temp;
>  	int internal_test_count;
> @@ -384,7 +381,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
>  	char base_test_name_for_next_level[BUF_LEN + 1];
>  
>  	for (int i = 0; i < test_count; i++) {
> -		while (read(fp_fd, record, BUF_LEN) < 0) {
> +		while (read(fd, record, BUF_LEN) < 0) {
>  			if (!READ_ONCE(ktap_args.is_running)) {
>  				igt_warn("ktap parser stopped\n");
>  				return -1;
> @@ -409,7 +406,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
>  		/* Sublevel found */
>  		if (tap_version_present(record, false))
>  		{
> -			internal_test_count = find_next_tap_subtest(fp, record, test_name,
> +			internal_test_count = find_next_tap_subtest(fd, record, test_name,
>  								    is_builtin);
>  			switch (internal_test_count) {
>  			case -2:
> @@ -433,7 +430,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
>  				memcpy(base_test_name_for_next_level + strlen(base_test_name_for_next_level),
>  				       test_name, BUF_LEN - strlen(base_test_name_for_next_level));
>  
> -				if (parse_tap_level(fp, base_test_name_for_next_level,
> +				if (parse_tap_level(fd, base_test_name_for_next_level,
>  						    internal_test_count, failed_tests, found_tests,
>  						    is_builtin) == -1)
>  					return -1;
> @@ -441,7 +438,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
>  			}
>  		}
>  
> -		switch (parse_kmsg_for_tap(fp, record, test_name)) {
> +		switch (parse_kmsg_for_tap(fd, record, test_name)) {
>  		case -2:
>  			return -1;
>  		case -1:
> @@ -516,8 +513,7 @@ static int parse_tap_level(FILE *fp, char *base_test_name, int test_count, bool
>   */
>  void *igt_ktap_parser(void *unused)
>  {
> -	FILE *fp = ktap_args.fp;
> -	int fp_fd = fileno(fp);
> +	int fd = ktap_args.fd;
>  	char record[BUF_LEN + 1];
>  	bool is_builtin = ktap_args.is_builtin;
>  	char test_name[BUF_LEN + 1];
> @@ -534,7 +530,7 @@ igt_ktap_parser_start:
>  	test_name[0] = '\0';
>  	test_name[BUF_LEN] = '\0';
>  
> -	while (read(fp_fd, record, BUF_LEN) < 0) {
> +	while (read(fd, record, BUF_LEN) < 0) {
>  		if (!READ_ONCE(ktap_args.is_running)) {
>  			igt_warn("ktap parser stopped\n");
>  			goto igt_ktap_parser_end;
> @@ -553,7 +549,7 @@ igt_ktap_parser_start:
>  		}
>  	}
>  
> -	test_count = find_next_tap_subtest(fp, record, test_name, is_builtin);
> +	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);
>  
>  	switch (test_count) {
>  	case -2:
> @@ -569,7 +565,7 @@ igt_ktap_parser_start:
>  	default:
>  		found_tests = true;
>  
> -		if (parse_tap_level(fp, test_name, test_count, &failed_tests, &found_tests,
> +		if (parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests,
>  				    is_builtin) == -1)
>  			goto igt_ktap_parser_end;
>  
> @@ -578,7 +574,7 @@ igt_ktap_parser_start:
>  
>  	/* Parse topmost level */
>  	test_name[0] = '\0';
> -	parse_tap_level(fp, test_name, test_count, &failed_tests, &found_tests, is_builtin);
> +	parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests, is_builtin);
>  
>  igt_ktap_parser_end:
>  	results.still_running = false;
> @@ -593,13 +589,13 @@ igt_ktap_parser_end:
>  
>  static pthread_t ktap_parser_thread;
>  
> -struct ktap_test_results *ktap_parser_start(FILE *fp, bool is_builtin)
> +struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  {
>  	results.head = NULL;
>  	pthread_mutex_init(&results.mutex, NULL);
>  	results.still_running = true;
>  
> -	ktap_args.fp = fp;
> +	ktap_args.fd = fd;
>  	ktap_args.is_builtin = is_builtin;
>  	ktap_args.is_running = true;
>  	pthread_create(&ktap_parser_thread, NULL, igt_ktap_parser, NULL);
> diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> index 34fe095720..ea57c2bb9b 100644
> --- a/lib/igt_ktap.h
> +++ b/lib/igt_ktap.h
> @@ -44,7 +44,7 @@ struct ktap_test_results {
>  
>  
>  
> -struct ktap_test_results *ktap_parser_start(FILE *fp, bool is_builtin);
> +struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin);
>  int ktap_parser_stop(void);
>  
>  #endif /* IGT_KTAP_H */

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2 02/17] lib/kunit: Stop loading kunit module explicitly

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:36 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Since kmod functions we use for module loading can process module
> dependencies, there is no need to load the "kunit" module explicitly
> before loading a kunit test module.  For the same reason we already don't
> unload the "kunit" module explicitly on cleanup.  Drop the unnecessary
> operation.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>

LGTM.

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> ---
>  lib/igt_kmod.c | 6 ------
>  1 file changed, 6 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 97667a896f..faf31afabc 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -789,12 +789,6 @@ static void __igt_kunit(const char *module_name, const char *opts)
>  		goto unload;
>  	}
>  
> -	/* The KUnit module is required for running any KUnit tests */
> -	ret = igt_kmod_load("kunit", NULL);
> -	if (ret) {
> -		skip = ret;
> -		goto unload;
> -	}
>  	ret = kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod);
>  	if (ret) {
>  		igt_warn("Unable to load KUnit\n");

Re: [Intel-gfx] [PATCH i-g-t v2 10/17] lib/ktap: Read /dev/kmsg in blocking mode

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:44 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> We obtain KTAP report from /dev/kmsg.  That file is now opened from
> igt_ktest_begin(), a function originally designed for i915 selftests and
> now reused with kunit tests.  The original intention of opening that file
> was to dump kernel messages to stderr on selftest error.  For that
> purpose, the file is now opened in non-blocking mode so we don't end up
> waiting for more kernel messages than already available.  Since our ktap
> parser code reuses the file descriptor, we now have to loop over
> EAGAIN responses, waiting for more KTAP data.  Since we have no sleeps
> inside those loops, extremely high CPU usage can be observed.
> 
> Simplify reading KTAP reports by first switching the file descriptor back
> to blocking mode.

LGTM.

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c |  7 ++++-
>  lib/igt_ktap.c | 81 ++++++++++++++------------------------------------
>  2 files changed, 28 insertions(+), 60 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index fb0bd21ee5..020df286b8 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -24,6 +24,7 @@
>  #include <ctype.h>
>  #include <signal.h>
>  #include <errno.h>
> +#include <fcntl.h>
>  #include <sys/utsname.h>
>  
>  #include "igt_aux.h"
> @@ -758,12 +759,16 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  {
>  	struct kmod_module *kunit_kmod;
>  	bool is_builtin;
> -	int ret;
>  	struct ktap_test_results *results;
>  	struct ktap_test_results_element *temp;
> +	int flags, ret;
>  
>  	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
>  
> +	flags = fcntl(tst->kmsg, F_GETFL, 0) & ~O_NONBLOCK;
> +	igt_skip_on_f(fcntl(tst->kmsg, F_SETFL, flags) == -1,
> +		      "Could not set /dev/kmsg to blocking mode\n");
> +
>  	igt_skip_on(lseek(tst->kmsg, 0, SEEK_END) < 0);
>  
>  	igt_skip_on(kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod));
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index fe77b62680..165f6b2cce 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -59,20 +59,12 @@ static int log_to_end(enum igt_log_level level, int fd,
>  	while (*lend == '\0') {
>  		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
>  
> -		while (read(fd, record, BUF_LEN) < 0) {
> -			if (errno == EINTR)
> -				return -2;
> -
> -			if (errno == EPIPE) {
> +		if (read(fd, record, BUF_LEN) < 0) {
> +			if (errno == EPIPE)
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -				return -2;
> -			}
> +			else if (errno != EINTR)
> +				igt_warn("an error occurred while reading kmsg: %m\n");
>  
> -			if (errno == EAGAIN)
> -				/* No records available */
> -				continue;
> -
> -			igt_warn("kmsg truncated: unknown error (%m)\n");
>  			return -2;
>  		}
>  
> @@ -176,20 +168,12 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  		return -1;
>  
>  	if (is_builtin) {
> -		while (read(fd, record, BUF_LEN) < 0) {
> -			if (errno == EINTR)
> -				return -2;
> -
> -			if (errno == EPIPE) {
> +		if (read(fd, record, BUF_LEN) < 0) {
> +			if (errno == EPIPE)
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -				return -2;
> -			}
> +			else if (errno != EINTR)
> +				igt_warn("an error occurred while reading kmsg: %m\n");
>  
> -			if (errno == EAGAIN)
> -				/* No records available */
> -				continue;
> -
> -			igt_warn("kmsg truncated: unknown error (%m)\n");
>  			return -2;
>  		}
>  	}
> @@ -215,20 +199,12 @@ static int find_next_tap_subtest(int fd, char *record, char *test_name, bool is_
>  		if (cutoff)
>  			cutoff[0] = '\0';
>  
> -		while (read(fd, record, BUF_LEN) < 0) {
> -			if (errno == EINTR)
> -				return -2;
> -
> -			if (errno == EPIPE) {
> +		if (read(fd, record, BUF_LEN) < 0) {
> +			if (errno == EPIPE)
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -				return -2;
> -			}
> +			else if (errno != EINTR)
> +				igt_warn("unknown error reading kmsg (%m)\n");
>  
> -			if (errno == EAGAIN)
> -				/* No records available */
> -				continue;
> -
> -			igt_warn("kmsg truncated: unknown error (%m)\n");
>  			return -2;
>  		}
>  
> @@ -365,20 +341,12 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  	char base_test_name_for_next_level[BUF_LEN + 1];
>  
>  	for (int i = 0; i < test_count; i++) {
> -		while (read(fd, record, BUF_LEN) < 0) {
> -			if (errno == EINTR)
> -				return -1;
> -
> -			if (errno == EAGAIN)
> -				/* No records available */
> -				continue;
> -
> -			if (errno == EPIPE) {
> +		if (read(fd, record, BUF_LEN) < 0) {
> +			if (errno == EPIPE)
>  				igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -				return -1;
> -			}
> +			else if (errno != EINTR)
> +				igt_warn("error reading kmsg (%m)\n");
>  
> -			igt_warn("kmsg truncated: unknown error (%m)\n");
>  			return -1;
>  		}
>  
> @@ -506,18 +474,13 @@ igt_ktap_parser_start:
>  	test_name[0] = '\0';
>  	test_name[BUF_LEN] = '\0';
>  
> -	while (read(fd, record, BUF_LEN) < 0) {
> -		if (errno == EAGAIN)
> -			/* No records available */
> -			continue;
> -
> -		if (errno == EINTR)
> -			goto igt_ktap_parser_end;
> -
> -		if (errno == EPIPE) {
> +	if (read(fd, record, BUF_LEN) < 0) {
> +		if (errno == EPIPE)
>  			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -			goto igt_ktap_parser_end;
> -		}
> +		else if (errno != EINTR)
> +			igt_warn("error reading kmsg (%m)\n");
> +
> +		goto igt_ktap_parser_end;
>  	}
>  
>  	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);

Re: [Intel-gfx] [PATCH i-g-t v2 11/17] lib/kunit: Fail / skip on kernel taint

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:45 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Similar to how igt_kselftest() handles kernel taints, fail current dynamic
> sub-subtest and skip remaining ones when a kernel taint is detected during
> execution of kunit test cases.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>

Yeah, makes sense to me.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> ---
>  lib/igt_kmod.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 020df286b8..988ac164cb 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -761,6 +761,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  	bool is_builtin;
>  	struct ktap_test_results *results;
>  	struct ktap_test_results_element *temp;
> +	unsigned long taints;
>  	int flags, ret;
>  
>  	igt_skip_on_f(tst->kmsg < 0, "Could not open /dev/kmsg\n");
> @@ -785,12 +786,20 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  
>  	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
>  	{
> +		if (igt_kernel_tainted(&taints)) {
> +			ktap_parser_cancel();
> +			break;
> +		}
> +
>  		if (READ_ONCE(results->head) != NULL) {
>  			pthread_mutex_lock(&results->mutex);
>  
> -			igt_dynamic(results->head->test_name)
> +			igt_dynamic(results->head->test_name) {
>  				igt_assert(READ_ONCE(results->head->passed));
>  
> +				igt_fail_on(igt_kernel_tainted(&taints));
> +			}
> +
>  			temp = results->head;
>  			results->head = results->head->next;
>  			free(temp);
> @@ -801,6 +810,7 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  
>  	ret = ktap_parser_stop();
>  
> +	igt_skip_on(igt_kernel_tainted(&taints));
>  	igt_skip_on_f(ret, "KTAP parser failed\n");
>  }
>  

Re: [Intel-gfx] [PATCH i-g-t v2 12/17] lib/ktap: Use IGT linked lists for storing KTAP results

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:46 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> For code simplicity and clarity, use existing IGT linked lists library
> instead of open coding a custom implementation of a list of KTAP results.
> 
> While being at it, flatten the code by inverting a check for pending
> results.

LGTM.

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 28 ++++++++++++++++------------
>  lib/igt_ktap.c | 25 +++++--------------------
>  lib/igt_ktap.h |  6 ++++--
>  3 files changed, 25 insertions(+), 34 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 988ac164cb..c692954911 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -760,7 +760,6 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  	struct kmod_module *kunit_kmod;
>  	bool is_builtin;
>  	struct ktap_test_results *results;
> -	struct ktap_test_results_element *temp;
>  	unsigned long taints;
>  	int flags, ret;
>  
> @@ -784,28 +783,33 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  		igt_skip("Unable to load %s module\n", tst->module_name);
>  	}
>  
> -	while (READ_ONCE(results->still_running) || READ_ONCE(results->head) != NULL)
> +	while (READ_ONCE(results->still_running) || !igt_list_empty(&results->list))
>  	{
> +		struct ktap_test_results_element *result;
> +
>  		if (igt_kernel_tainted(&taints)) {
>  			ktap_parser_cancel();
>  			break;
>  		}
>  
> -		if (READ_ONCE(results->head) != NULL) {
> -			pthread_mutex_lock(&results->mutex);
> +		pthread_mutex_lock(&results->mutex);
> +		if (igt_list_empty(&results->list)) {
> +			pthread_mutex_unlock(&results->mutex);
> +			continue;
> +		}
>  
> -			igt_dynamic(results->head->test_name) {
> -				igt_assert(READ_ONCE(results->head->passed));
> +		result = igt_list_first_entry(&results->list, result, link);
>  
> -				igt_fail_on(igt_kernel_tainted(&taints));
> -			}
> +		igt_list_del(&result->link);
> +		pthread_mutex_unlock(&results->mutex);
>  
> -			temp = results->head;
> -			results->head = results->head->next;
> -			free(temp);
> +		igt_dynamic(result->test_name) {
> +			igt_assert(READ_ONCE(result->passed));
>  
> -			pthread_mutex_unlock(&results->mutex);
> +			igt_fail_on(igt_kernel_tainted(&taints));
>  		}
> +
> +		free(result);
>  	}
>  
>  	ret = ktap_parser_stop();
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 165f6b2cce..5e9967f980 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -12,6 +12,7 @@
>  #include "igt_aux.h"
>  #include "igt_core.h"
>  #include "igt_ktap.h"
> +#include "igt_list.h"
>  
>  #define DELIMITER "-"
>  
> @@ -335,7 +336,7 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  			   bool *found_tests, bool is_builtin)
>  {
>  	char record[BUF_LEN + 1];
> -	struct ktap_test_results_element *r, *temp;
> +	struct ktap_test_results_element *r;
>  	int internal_test_count;
>  	char test_name[BUF_LEN + 1];
>  	char base_test_name_for_next_level[BUF_LEN + 1];
> @@ -403,17 +404,9 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  			r->test_name[BUF_LEN] = '\0';
>  
>  			r->passed = false;
> -			r->next = NULL;
>  
>  			pthread_mutex_lock(&results.mutex);
> -			if (results.head == NULL) {
> -				results.head = r;
> -			} else {
> -				temp = results.head;
> -				while (temp->next != NULL)
> -					temp = temp->next;
> -				temp->next = r;
> -			}
> +			igt_list_add_tail(&r->link, &results.list);
>  			pthread_mutex_unlock(&results.mutex);
>  
>  			test_name[0] = '\0';
> @@ -431,17 +424,9 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>  			r->test_name[BUF_LEN] = '\0';
>  
>  			r->passed = true;
> -			r->next = NULL;
>  
>  			pthread_mutex_lock(&results.mutex);
> -			if (results.head == NULL) {
> -				results.head = r;
> -			} else {
> -				temp = results.head;
> -				while (temp->next != NULL)
> -					temp = temp->next;
> -				temp->next = r;
> -			}
> +			igt_list_add_tail(&r->link, &results.list);
>  			pthread_mutex_unlock(&results.mutex);
>  
>  			test_name[0] = '\0';
> @@ -523,7 +508,7 @@ static pthread_t ktap_parser_thread;
>  
>  struct ktap_test_results *ktap_parser_start(int fd, bool is_builtin)
>  {
> -	results.head = NULL;
> +	IGT_INIT_LIST_HEAD(&results.list);
>  	pthread_mutex_init(&results.mutex, NULL);
>  	results.still_running = true;
>  
> diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> index 991800e912..b4d7a6dbc7 100644
> --- a/lib/igt_ktap.h
> +++ b/lib/igt_ktap.h
> @@ -28,16 +28,18 @@
>  
>  #include <pthread.h>
>  
> +#include "igt_list.h"
> +
>  void *igt_ktap_parser(void *unused);
>  
>  typedef struct ktap_test_results_element {
>  	char test_name[BUF_LEN + 1];
>  	bool passed;
> -	struct ktap_test_results_element *next;
> +	struct igt_list_head link;
>  } ktap_test_results_element;
>  
>  struct ktap_test_results {
> -	ktap_test_results_element *head;
> +	struct igt_list_head list;
>  	pthread_mutex_t mutex;
>  	bool still_running;
>  };

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2 16/17] lib/kunit: Strip "_test" or "_kunit" suffix from subtest names

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:50 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> If a user (an IGT test) doesn't provide a subtest name when calling
> igt_kunit() then we now use the requested kernel module name as IGT
> subtest name.  Since names of kunit test modules usually end with a
> "_test" or "_kunit" suffix, those parts of the names don't carry any
> useful information.  Strip those suffixes from IGT subtest names.

LGTM.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 26 ++++++++++++++++++++++----
>  tests/drm_mm.c | 42 +++++++++++++++++++++---------------------
>  2 files changed, 43 insertions(+), 25 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 46a6f81e73..ddd5499f5e 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -944,8 +944,29 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
>  	if (igt_ktest_init(&tst, module_name) != 0)
>  		return;
>  
> -	igt_fixture
> +	/*
> +	 * If the caller (an IGT test) provides no subtest name then we
> +	 * take the module name, drop the trailing "_test" or "_kunit"
> +	 * suffix, if any, and use the result as our IGT subtest name.
> +	 */
> +	if (!name) {
> +		name = strdup(module_name);
> +		if (name) {
> +			char *suffix = strstr(name, "_test");
> +
> +			if (!suffix)
> +				suffix = strstr(name, "_kunit");
> +
> +			if (suffix)
> +				*suffix = '\0';
> +		}
> +	}
> +
> +	igt_fixture {
> +		igt_require(name);
> +
>  		igt_require(igt_ktest_begin(&tst) == 0);
> +	}
>  
>  	/*
>  	 * We need to use igt_subtest here, as otherwise it may crash with:
> @@ -954,9 +975,6 @@ void igt_kunit(const char *module_name, const char *name, const char *opts)
>  	 * proper namespace for dynamic subtests, with is required for CI
>  	 * and for documentation.
>  	 */
> -	if (name == NULL)
> -		name = module_name;
> -
>  	igt_subtest_with_dynamic(name)
>  		__igt_kunit(&tst, opts);
>  
> diff --git a/tests/drm_mm.c b/tests/drm_mm.c
> index 9a8b3f3fcb..e6ba224745 100644
> --- a/tests/drm_mm.c
> +++ b/tests/drm_mm.c
> @@ -29,123 +29,123 @@
>   * Feature: mapping
>   * Run type: FULL
>   *
> - * SUBTEST: drm_mm_test
> + * SUBTEST: drm_mm
>   *
> - * SUBTEST: drm_mm_test@align
> + * SUBTEST: drm_mm@align
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@align32
> + * SUBTEST: drm_mm@align32
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@align64
> + * SUBTEST: drm_mm@align64
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@bottomup
> + * SUBTEST: drm_mm@bottomup
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@color
> + * SUBTEST: drm_mm@color
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@color_evict
> + * SUBTEST: drm_mm@color_evict
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@color_evict_range
> + * SUBTEST: drm_mm@color_evict_range
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@debug
> + * SUBTEST: drm_mm@debug
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@evict
> + * SUBTEST: drm_mm@evict
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@evict_range
> + * SUBTEST: drm_mm@evict_range
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@frag
> + * SUBTEST: drm_mm@frag
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@highest
> + * SUBTEST: drm_mm@highest
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@init
> + * SUBTEST: drm_mm@init
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@insert
> + * SUBTEST: drm_mm@insert
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@insert_range
> + * SUBTEST: drm_mm@insert_range
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@lowest
> + * SUBTEST: drm_mm@lowest
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@replace
> + * SUBTEST: drm_mm@replace
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@reserve
> + * SUBTEST: drm_mm@reserve
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@sanitycheck
> + * SUBTEST: drm_mm@sanitycheck
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt
>   * Test category: GEM_Legacy
>   *
> - * SUBTEST: drm_mm_test@topdown
> + * SUBTEST: drm_mm@topdown
>   * Category: Infrastructure
>   * Description: drm_mm range manager SW validation
>   * Functionality: DRM memory mangemnt

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2 13/17] lib/ktap: Reimplement KTAP parser

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:47 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> Current implementation of KTAP parser suffers from several issues:
> - in most cases, kernel messages that are not part of KTAP output but
>   happen to appear in between break the parser,
> - results from parametrized test cases, not preceded with a "1..N" test
>   plan, break the parser,
> - skips are not supported, reported as success,
> - IGT results from all 3 kunit test nesting levels, i.e., from
>   parametrized subtests (should those were fixed to work correctly), test
>   cases and test suites, are reported individually as if all those items
>   were executed sequentially, all at the same level of nesting, which can
>   be confusing to igt_runner,
> - the parser is not only parsing the data, but also handles data input
>   from a /dev/kmsg like source, which is integrated into it, making it
>   hard if not impossible to feed KTAP data from different sources,
>   including mock sources,
> - since the parser has been designed for running it in a separate thread,
>   it's not possible to use igt_skip() nor igt_fail() and friends
>   immediately when a result is available, only pass it to the main thread
>   over a buffer.  As a consequence, it is virtually impossible to
>   synchronize IGT output with KTAP output.
> 
> Fixing the existing parser occurred more complicated than re-implementing
> it from scratch.  This patch provides a new implementation.
> 
> Only results from kunit test cases are reported as results of IGT dynamic
> sub-subtests.  Results from individual parametrized subtests have been
> considered problematic since many of them provide multi-word descriptions
> in place of single-word subtest names.  If a parametrized test case fails
> then full KTAP output from its subtests, potentially mixed with
> accompanying kernel messages, is available in dmesg for analysis so users
> can still find out which individual subtests succeeded and which failed.
> 
> Results from test suites level are also omitted in faith that IGT can
> handle aggregation of results from individual kunit test cases reported as
> IGT dynamic sub-subtests and report those aggregated results correctly as
> results from an IGT dynamic subtest.  That 1:1 mapping of kunit test
> suites to IGT dynamic subtests now works perfectly for modules that
> provide only one test suite, which is the case for all kunit test modules
> now existing under drivers/gpu/drm, and the most common case among all
> kunit test modules in the whole kernel tree.
> 
> New igt_ktap functions can be called directly from igt_kunit subtest body,
> but for this patch, the old igt_ktap_parser() function that runs in a
> separate thread has been preserved, only modified to use the new
> implementation and translate results from those new functions to legacy
> format.  Unlike the former implementation, translation of kunit names to
> IGT names is handled outside the parser itself, though for now it is still
> performed inside the legacy igt_ktap_parser() function.
> 
> For better readability of the patch, no longer used functions have been
> left untouched, only tagged with __maybe_unused to shut up compiler
> warnings / errors.  Kunit library functions will be modified to use the
> new igt_ktap interface, and those old ktap functions removed by follow-
> up patches.
> 
> A test with example subtests that feed igt_ktap_parse() function with some
> example data and verifies correctness of their parsing is also provided.
> 
> v2: Fix incorrect and missing includes in the test source file,
>   - add license and copyright clauses to the test source file.
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> ---
>  lib/igt_ktap.c              | 422 ++++++++++++++++++++++++++++++++----
>  lib/igt_ktap.h              |  15 ++
>  lib/tests/igt_ktap_parser.c | 246 +++++++++++++++++++++
>  lib/tests/meson.build       |   1 +
>  4 files changed, 645 insertions(+), 39 deletions(-)
>  create mode 100644 lib/tests/igt_ktap_parser.c
> 
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> index 5e9967f980..d46a2433e5 100644
> --- a/lib/igt_ktap.c
> +++ b/lib/igt_ktap.c
> @@ -1,6 +1,7 @@
>  // SPDX-License-Identifier: MIT
>  /*
>   * Copyright © 2023 Isabella Basso do Amaral <isabbasso@riseup.net>
> + * Copyright © 2023 Intel Corporation
>   */
>  
>  #include <ctype.h>
> @@ -8,12 +9,310 @@
>  #include <libkmod.h>
>  #include <pthread.h>
>  #include <errno.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
>  
>  #include "igt_aux.h"
>  #include "igt_core.h"
>  #include "igt_ktap.h"
>  #include "igt_list.h"
>  
> +enum ktap_phase {
> +	KTAP_START,
> +	SUITE_COUNT,
> +	SUITE_START,
> +	SUITE_NAME,
> +	CASE_COUNT,
> +	CASE_NAME,
> +	SUB_RESULT,
> +	CASE_RESULT,
> +	SUITE_RESULT,
> +};
> +
> +struct igt_ktap_results {
> +	enum ktap_phase expect;
> +	unsigned int suite_count;
> +	unsigned int suite_last;
> +	char *suite_name;
> +	unsigned int case_count;
> +	unsigned int case_last;
> +	char *case_name;
> +	unsigned int sub_last;
> +	struct igt_list_head *results;
> +};
> +
> +/**
> + * igt_ktap_parse:
> + *
> + * This function parses a line of text for KTAP report data
> + * and passes results back to IGT kunit layer.
> + */
> +int igt_ktap_parse(const char *buf, struct igt_ktap_results *ktap)
> +{
> +	char *suite_name = NULL, *case_name = NULL, *msg = NULL;
> +	struct igt_ktap_result *result;
> +	int code = IGT_EXIT_INVALID;
> +	unsigned int n, len;
> +	char s[2];
> +
> +	/* KTAP report header */
> +	if (igt_debug_on(sscanf(buf, "KTAP%*[ ]version%*[ ]%u %n",
> +				&n, &len) == 1 && len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != KTAP_START))
> +			return -EPROTO;
> +
> +		ktap->suite_count = 0;
> +		ktap->expect = SUITE_COUNT;
> +
> +	/* malformed TAP test plan? */
> +	} else if (len = 0,
> +		   igt_debug_on(sscanf(buf, " 1..%1[ ]", s) == 1)) {
> +		return -EINPROGRESS;
> +
> +	/* valid test plan of a KTAP report */
> +	} else if (igt_debug_on(sscanf(buf, "1..%u %n", &n, &len) == 1 &&
> +				len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != SUITE_COUNT))
> +			return -EPROTO;
> +
> +		if (!n)
> +			return 0;
> +
> +		ktap->suite_count = n;
> +		ktap->suite_last = 0;
> +		ktap->suite_name = NULL;
> +		ktap->expect = SUITE_START;
> +
> +	/* KTAP test suite header */
> +	} else if (len = 0,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]KTAP%*[ ]version%*[ ]%u %n",
> +				       &n, &len) == 1 && len == strlen(buf))) {
> +		/*
> +		 * TODO: drop the following workaround as soon as
> +		 * kernel side issue of missing lines with top level
> +		 * KTAP version and test suite plan is fixed.
> +		 */
> +		if (ktap->expect == KTAP_START) {
> +			ktap->suite_count = 1;
> +			ktap->suite_last = 0;
> +			ktap->suite_name = NULL;
> +			ktap->expect = SUITE_START;
> +		}
> +
> +		if (igt_debug_on(ktap->expect != SUITE_START))
> +			return -EPROTO;
> +
> +		ktap->expect = SUITE_NAME;
> +
> +	/* KTAP test suite name */
> +	} else if (len = 0,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]#%*[ ]Subtest:%*[ ]%ms %n",
> +				       &suite_name, &len) == 1 && len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != SUITE_NAME))
> +			return -EPROTO;
> +
> +		ktap->suite_name = suite_name;
> +		suite_name = NULL;
> +		ktap->case_count = 0;
> +		ktap->expect = CASE_COUNT;
> +
> +	/* valid test plan of a KTAP test suite */
> +	} else if (len = 0, free(suite_name), suite_name = NULL,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]1..%u %n",
> +				       &n, &len) == 1 && len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != CASE_COUNT))
> +			return -EPROTO;
> +
> +		if (n) {
> +			ktap->case_count = n;
> +			ktap->case_last = 0;
> +			ktap->case_name = NULL;
> +			ktap->expect = CASE_RESULT;
> +		} else {
> +			ktap->expect = SUITE_RESULT;
> +		}
> +
> +	/* KTAP parametrized test case header */
> +	} else if (len = 0,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]KTAP%*[ ]version%*[ ]%u %n",
> +				       &n, &len) == 1 && len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != CASE_RESULT))
> +			return -EPROTO;
> +
> +		ktap->sub_last = 0;
> +		ktap->expect = CASE_NAME;
> +
> +	/* KTAP parametrized test case name */
> +	} else if (len = 0,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]#%*[ ]Subtest:%*[ ]%ms %n",
> +				       &case_name, &len) == 1 && len == strlen(buf))) {
> +		if (igt_debug_on(ktap->expect != CASE_NAME))
> +			return -EPROTO;
> +
> +		n = ktap->case_last + 1;
> +		ktap->expect = SUB_RESULT;
> +
> +	/* KTAP parametrized subtest result */
> +	} else if (len = 0, free(case_name), case_name = NULL,
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%*[^#\n]%1[#\n]",
> +				       &n, s) == 2) ||
> +		   igt_debug_on(sscanf(buf,
> +				       "%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%*[^#\n]%1[#\n]",
> +				       &n, s) == 2)) {
> +		/* at lease one result of a parametrised subtest expected */
> +		if (igt_debug_on(ktap->expect == SUB_RESULT &&
> +				 ktap->sub_last == 0))
> +			ktap->expect = CASE_RESULT;
> +
> +		if (igt_debug_on(ktap->expect != CASE_RESULT) ||
> +		    igt_debug_on(n != ++ktap->sub_last))
> +			return -EPROTO;
> +
> +	/* KTAP test case skip result */
> +	} else if ((igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]SKIP %n",
> +					&n, &case_name, &len) == 2 &&
> +				 len == strlen(buf))) ||
> +		   (len = 0, free(case_name), case_name = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]SKIP%*[ ]%m[^\n]",
> +					&n, &case_name, &msg) == 3))) {
> +		code = IGT_EXIT_SKIP;
> +
> +	/* KTAP test case pass result */
> +	} else if ((free(case_name), case_name = NULL, free(msg), msg = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms %n",
> +					&n, &case_name, &len) == 2 &&
> +				 len == strlen(buf))) ||
> +		   (len = 0, free(case_name), case_name = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]%m[^\n]",
> +					&n, &case_name, &msg) == 3))) {
> +		code = IGT_EXIT_SUCCESS;
> +
> +	/* KTAP test case fail result */
> +	} else if ((free(case_name), case_name = NULL, free(msg), msg = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%ms %n",
> +					&n, &case_name, &len) == 2 &&
> +				 len == strlen(buf))) ||
> +		   (len = 0, free(case_name), case_name = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"%*1[ ]%*1[ ]%*1[ ]%*1[ ]not%*1[ ]ok%*[ ]%u%*[ ]%ms%*[ ]#%*[ ]%m[^\n]",
> +					&n, &case_name, &msg) == 3))) {
> +		code = IGT_EXIT_FAILURE;
> +
> +	/* KTAP test suite result */
> +	} else if ((free(case_name), free(msg),
> +		    igt_debug_on(sscanf(buf, "ok%*[ ]%u%*[ ]%ms %n",
> +					&n, &suite_name, &len) == 2 &&
> +				 len == strlen(buf))) ||
> +		   (len = 0, free(suite_name), suite_name = NULL,
> +		    igt_debug_on(sscanf(buf, "ok%*[ ]%u%*[ ]%ms%*[ ]%1[#]",
> +					&n, &suite_name, s) == 3)) ||
> +		   (free(suite_name), suite_name = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"not%*[ ]ok%*[ ]%u%*[ ]%ms %n",
> +					&n, &suite_name, &len) == 2 &&
> +				 len == strlen(buf))) ||
> +		   (len = 0, free(suite_name), suite_name = NULL,
> +		    igt_debug_on(sscanf(buf,
> +					"not%*[ ]ok%*[ ]%u%*[ ]%ms%*[ ]%1[#]",
> +					&n, &suite_name, s) == 3))) {
> +		if (igt_debug_on(ktap->expect != SUITE_RESULT) ||
> +		    igt_debug_on(strcmp(suite_name, ktap->suite_name)) ||
> +		    igt_debug_on(n != ++ktap->suite_last) ||
> +		    igt_debug_on(n > ktap->suite_count)) {
> +			free(suite_name);
> +			return -EPROTO;
> +		}
> +		free(suite_name);
> +
> +		/* last test suite? */
> +		if (igt_debug_on(n == ktap->suite_count))
> +			return 0;
> +
> +		ktap->suite_name = NULL;
> +		ktap->expect = SUITE_START;
> +
> +	} else {
> +		return -EINPROGRESS;
> +	}
> +
> +	/* neither a test case name nor result */
> +	if (ktap->expect != SUB_RESULT && code == IGT_EXIT_INVALID)
> +		return -EINPROGRESS;
> +
> +	if (igt_debug_on(ktap->expect == SUB_RESULT &&
> +			 code != IGT_EXIT_INVALID) ||
> +	    igt_debug_on(code != IGT_EXIT_INVALID &&
> +			 ktap->expect != CASE_RESULT) ||
> +	    igt_debug_on(!ktap->suite_name) || igt_debug_on(!case_name) ||
> +	    igt_debug_on(ktap->expect == CASE_RESULT && ktap->case_name &&
> +			 strcmp(case_name, ktap->case_name)) ||
> +	    igt_debug_on(n > ktap->case_count) ||
> +	    igt_debug_on(n != (ktap->expect == SUB_RESULT ?
> +			       ktap->case_last + 1: ++ktap->case_last))) {
> +		free(case_name);
> +		free(msg);
> +		return -EPROTO;
> +	}
> +
> +	if (ktap->expect == SUB_RESULT) {
> +		/* KTAP parametrized test case name */
> +		ktap->case_name = case_name;
> +
> +	} else {
> +		/* KTAP test case result */
> +		ktap->case_name = NULL;
> +
> +		/* last test case in a suite */
> +		if (n == ktap->case_count)
> +			ktap->expect = SUITE_RESULT;
> +	}
> +
> +	if (igt_debug_on((result = calloc(1, sizeof(*result)), !result))) {
> +		free(case_name);
> +		free(msg);
> +		return -ENOMEM;
> +	}
> +
> +	result->suite_name = ktap->suite_name;
> +	result->case_name = case_name;
> +	result->code = code;
> +	result->msg = msg;
> +	igt_list_add_tail(&result->link, ktap->results);
> +
> +	return -EINPROGRESS;
> +}
> +
> +struct igt_ktap_results *igt_ktap_alloc(struct igt_list_head *results)
> +{
> +	struct igt_ktap_results *ktap = calloc(1, sizeof(*ktap));
> +
> +	if (!ktap)
> +		return NULL;
> +
> +	ktap->expect = KTAP_START;
> +	ktap->results = results;
> +
> +	return ktap;
> +}
> +
> +void igt_ktap_free(struct igt_ktap_results *ktap)
> +{
> +	free(ktap);
> +}
> +
>  #define DELIMITER "-"
>  
>  struct ktap_parser_args {
> @@ -332,6 +631,7 @@ static int parse_kmsg_for_tap(int fd, char *record, char *test_name)
>   * 0 if succeded
>   * -1 if error occurred
>   */
> +__maybe_unused
>  static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *failed_tests,
>  			   bool *found_tests, bool is_builtin)
>  {
> @@ -445,62 +745,106 @@ static int parse_tap_level(int fd, char *base_test_name, int test_count, bool *f
>   */
>  void *igt_ktap_parser(void *unused)
>  {
> +	char record[BUF_LEN + 1], *buf, *suite_name = NULL, *case_name = NULL;
> +	struct igt_ktap_results *ktap = NULL;
>  	int fd = ktap_args.fd;
> -	char record[BUF_LEN + 1];
> -	bool is_builtin = ktap_args.is_builtin;
> -	char test_name[BUF_LEN + 1];
> -	bool failed_tests, found_tests;
> -	int test_count;
> +	IGT_LIST_HEAD(list);
> +	int err;
>  
> -	failed_tests = false;
> -	found_tests = false;
> +	ktap = igt_ktap_alloc(&list);
> +	if (igt_debug_on(!ktap))
> +		goto igt_ktap_parser_end;
>  
> -igt_ktap_parser_start:
> -	test_name[0] = '\0';
> -	test_name[BUF_LEN] = '\0';
> +	while (err = read(fd, record, BUF_LEN), err > 0) {
> +		struct igt_ktap_result *r, *rn;
>  
> -	if (read(fd, record, BUF_LEN) < 0) {
> -		if (errno == EPIPE)
> -			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> -		else if (errno != EINTR)
> -			igt_warn("error reading kmsg (%m)\n");
> +		/* skip kmsg continuation lines */
> +		if (igt_debug_on(*record == ' '))
> +			continue;
>  
> -		goto igt_ktap_parser_end;
> -	}
> +		/* NULL-terminate the record */
> +		record[err] = '\0';
>  
> -	test_count = find_next_tap_subtest(fd, record, test_name, is_builtin);
> +		/* detect start of log message, continue if not found */
> +		buf = strchrnul(record, ';');
> +		if (igt_debug_on(*buf == '\0'))
> +			continue;
> +		buf++;
>  
> -	switch (test_count) {
> -	case -2:
> -		/* Problems while reading the file */
> -		goto igt_ktap_parser_end;
> -	case -1:
> -		/* No test found */
> -		goto igt_ktap_parser_start;
> -	case 0:
> -		/* Tests found, but they're missing info */
> -		found_tests = true;
> -		goto igt_ktap_parser_end;
> -	default:
> -		found_tests = true;
> +		err = igt_ktap_parse(buf, ktap);
>  
> -		if (parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests,
> -				    is_builtin) == -1)
> +		/* parsing error */
> +		if (err && err != -EINPROGRESS)
>  			goto igt_ktap_parser_end;
>  
> -		break;
> +		igt_list_for_each_entry_safe(r, rn, &list, link) {
> +			struct ktap_test_results_element *result = NULL;
> +			int code = r->code;
> +
> +			if (code != IGT_EXIT_INVALID)
> +				result = calloc(1, sizeof(*result));
> +
> +			if (result) {
> +				snprintf(result->test_name, sizeof(result->test_name),
> +					 "%s-%s", r->suite_name, r->case_name);
> +
> +				if (code == IGT_EXIT_SUCCESS)
> +					result->passed = true;
> +			}
> +
> +			igt_list_del(&r->link);
> +			if (r->suite_name != suite_name) {
> +				free(suite_name);
> +				suite_name = r->suite_name;
> +			}
> +			if (r->case_name != case_name) {
> +				free(case_name);
> +				case_name = r->case_name;
> +			}
> +			free(r->msg);
> +			free(r);
> +
> +			/*
> +			 * no extra result record expected on start
> +			 * of parametrized test case -- skip it
> +			 */
> +			if (code == IGT_EXIT_INVALID)
> +				continue;
> +
> +			if (!result) {
> +				err = -ENOMEM;
> +				goto igt_ktap_parser_end;
> +			}
> +
> +			pthread_mutex_lock(&results.mutex);
> +			igt_list_add_tail(&result->link, &results.list);
> +			pthread_mutex_unlock(&results.mutex);
> +		}
> +
> +		/* end of KTAP report */
> +		if (!err)
> +			goto igt_ktap_parser_end;
>  	}
>  
> -	/* Parse topmost level */
> -	test_name[0] = '\0';
> -	parse_tap_level(fd, test_name, test_count, &failed_tests, &found_tests, is_builtin);
> +	if (err < 0) {
> +		if (errno == EPIPE)
> +			igt_warn("kmsg truncated: too many messages. You may want to increase log_buf_len in kmcdline\n");
> +		else if (errno != EINTR)
> +			igt_warn("error reading kmsg (%m)\n");
> +	}
>  
>  igt_ktap_parser_end:
> -	results.still_running = false;
> +	free(suite_name);
> +	free(case_name);
>  
> -	if (found_tests && !failed_tests)
> +	if (!err)
>  		ktap_args.ret = IGT_EXIT_SUCCESS;
>  
> +	results.still_running = false;
> +
> +	if (ktap)
> +		igt_ktap_free(ktap);
> +
>  	return NULL;
>  }
>  
> diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> index b4d7a6dbc7..6f8da3eab6 100644
> --- a/lib/igt_ktap.h
> +++ b/lib/igt_ktap.h
> @@ -1,5 +1,6 @@
>  /*
>   * Copyright © 2022 Isabella Basso do Amaral <isabbasso@riseup.net>
> + * Copyright © 2023 Intel Corporation
>   *
>   * Permission is hereby granted, free of charge, to any person obtaining a
>   * copy of this software and associated documentation files (the "Software"),
> @@ -30,6 +31,20 @@
>  
>  #include "igt_list.h"
>  
> +struct igt_ktap_result {
> +	struct igt_list_head link;
> +	char *suite_name;
> +	char *case_name;
> +	char *msg;
> +	int code;
> +};
> +
> +struct igt_ktap_results;
> +
> +struct igt_ktap_results *igt_ktap_alloc(struct igt_list_head *results);
> +int igt_ktap_parse(const char *buf, struct igt_ktap_results *ktap);
> +void igt_ktap_free(struct igt_ktap_results *ktap);
> +
>  void *igt_ktap_parser(void *unused);
>  
>  typedef struct ktap_test_results_element {
> diff --git a/lib/tests/igt_ktap_parser.c b/lib/tests/igt_ktap_parser.c
> new file mode 100644
> index 0000000000..6357bdf6a5
> --- /dev/null
> +++ b/lib/tests/igt_ktap_parser.c
> @@ -0,0 +1,246 @@
> +// SPDX-License-Identifier: MIT
> +/*
> +* Copyright © 2023 Intel Corporation
> +*/
> +
> +#include <errno.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +
> +#include "igt_core.h"
> +#include "igt_ktap.h"
> +#include "igt_list.h"
> +
> +static void ktap_list(void)
> +{
> +	struct igt_ktap_result *result, *rn;
> +	struct igt_ktap_results *ktap;
> +	int suite = 1, test = 1;
> +	IGT_LIST_HEAD(results);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +
> +	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("1..3\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    1..3\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 2 test_case_2 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 3 test_case_3 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("ok 1 test_suite_1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_2\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("ok 2 test_suite_2\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite_3\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    1..4\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case_1 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 2 test_case_2 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 3 test_case_3 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 4 test_case_4 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("ok 3 test_suite_3\n", ktap), 0);
> +
> +	igt_ktap_free(ktap);
> +
> +	igt_assert_eq(igt_list_length(&results), 8);
> +
> +	igt_list_for_each_entry_safe(result, rn, &results, link) {
> +		char *case_name, *suite_name;
> +
> +		igt_list_del(&result->link);
> +
> +		igt_assert_lt(0, asprintf(&case_name, "test_case_%u", test));
> +		igt_assert_lt(0, asprintf(&suite_name, "test_suite_%u", suite));
> +
> +		igt_assert(result->case_name);
> +		igt_assert_eq(strcmp(result->case_name, case_name), 0);
> +		free(result->case_name);
> +		free(case_name);
> +
> +		igt_assert(result->suite_name);
> +		igt_assert_eq(strcmp(result->suite_name, suite_name), 0);
> +		free(suite_name);
> +
> +		igt_assert(!result->msg);
> +		igt_assert_eq(result->code, IGT_EXIT_SKIP);
> +
> +		if ((suite == 1 && test < 3) || (suite == 3 && test < 4)) {
> +			test++;
> +		} else {
> +			free(result->suite_name);
> +			suite++;
> +			test = 1;
> +		}
> +
> +		free(result);
> +	}
> +}
> +
> +static void ktap_results(void)
> +{
> +	struct igt_ktap_result *result;
> +	struct igt_ktap_results *ktap;
> +	char *suite_name, *case_name;
> +	IGT_LIST_HEAD(results);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +
> +	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        ok 1 parameter 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        ok 2 parameter 2 # a comment\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        ok 3 parameter 3 # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        ok 4 parameter 4 # SKIP with a message\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        not ok 5 parameter 5\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        not ok 6 parameter 6 # failure message\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("not ok 1 test_suite\n", ktap), 0);
> +
> +	igt_ktap_free(ktap);
> +
> +	igt_assert_eq(igt_list_length(&results), 2);
> +
> +	result = igt_list_first_entry(&results, result, link);
> +	igt_list_del(&result->link);
> +	igt_assert_eq(strcmp(result->suite_name, "test_suite"), 0);
> +	igt_assert_eq(strcmp(result->case_name, "test_case"), 0);
> +	igt_assert_eq(result->code, IGT_EXIT_INVALID);
> +	igt_assert(!result->msg);
> +	free(result->msg);
> +	suite_name = result->suite_name;
> +	case_name = result->case_name;
> +	free(result);
> +
> +	result = igt_list_first_entry(&results, result, link);
> +	igt_list_del(&result->link);
> +	igt_assert_eq(strcmp(result->suite_name, suite_name), 0);
> +	igt_assert_eq(strcmp(result->case_name, case_name), 0);
> +	igt_assert_neq(result->code, IGT_EXIT_INVALID);
> +	free(result->msg);
> +	free(suite_name);
> +	free(case_name);
> +	free(result);
> +}
> +
> +static void ktap_success(void)
> +{
> +	struct igt_ktap_result *result;
> +	struct igt_ktap_results *ktap;
> +	IGT_LIST_HEAD(results);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +
> +	igt_assert_eq(igt_ktap_parse("KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_assert(igt_list_empty(&results));
> +
> +	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_list_length(&results), 1);
> +
> +	igt_assert_eq(igt_ktap_parse("        ok 1 parameter # SKIP\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_list_length(&results), 1);
> +
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EINPROGRESS);
> +	igt_assert_eq(igt_list_length(&results), 2);
> +
> +	igt_assert_eq(igt_ktap_parse("not ok 1 test_suite\n", ktap), 0);
> +	igt_assert_eq(igt_list_length(&results), 2);
> +
> +	igt_ktap_free(ktap);
> +
> +	result = igt_list_last_entry(&results, result, link);
> +	igt_list_del(&result->link);
> +	igt_assert_eq(result->code, IGT_EXIT_SUCCESS);
> +	free(result->msg);
> +	free(result);
> +
> +	result = igt_list_last_entry(&results, result, link);
> +	igt_list_del(&result->link);
> +	free(result->suite_name);
> +	free(result->case_name);
> +	free(result->msg);
> +	free(result);
> +}
> +
> +static void ktap_top_version(void)
> +{
> +	struct igt_ktap_results *ktap;
> +	IGT_LIST_HEAD(results);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("1..1\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	/* TODO: change to -EPROTO as soon as related workaround is dropped */
> +	igt_assert_eq(igt_ktap_parse("    KTAP version 1\n", ktap), -EINPROGRESS);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("    # Subtest: test_suite\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("    1..1\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("        KTAP version 1\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("        # Subtest: test_case\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("        ok 1 parameter 1\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("    ok 1 test_case\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +
> +	ktap = igt_ktap_alloc(&results);
> +	igt_require(ktap);
> +	igt_assert_eq(igt_ktap_parse("ok 1 test_suite\n", ktap), -EPROTO);
> +	igt_ktap_free(ktap);
> +}
> +
> +igt_main
> +{
> +	igt_subtest("list")
> +		ktap_list();
> +
> +	igt_subtest("results")
> +		ktap_results();
> +
> +	igt_subtest("success")
> +		ktap_success();
> +
> +	igt_subtest("top-ktap-version")
> +		ktap_top_version();
> +}
> diff --git a/lib/tests/meson.build b/lib/tests/meson.build
> index 7a52a7876e..fa3d81de6c 100644
> --- a/lib/tests/meson.build
> +++ b/lib/tests/meson.build
> @@ -10,6 +10,7 @@ lib_tests = [
>  	'igt_exit_handler',
>  	'igt_fork',
>  	'igt_fork_helper',
> +        'igt_ktap_parser',
>  	'igt_list_only',
>  	'igt_invalid_subtest_name',
>  	'igt_nesting',

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2 14/17] lib/kunit: Load test modules in background

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:48 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> For igt_runner to be able to correlate a stream of IGT results from
> dynamic sub-subtests that correspond to individual kunit test cases, read
> by the igt_runner from stdout / stderr of the test process, with a stream
> of kernel messages read from /dev/kmsg, we need both result streams being
> fed with data in parallel.  While our KTAP parser is currently started in
> the background and reads KTAP results from /dev/kmsg in parallel with
> execution of kunit tests performed by the kernel while we are loading a
> kunit test module, results from the parser are then only stored as
> intermediate data and not processed any further until the module loading
> completes.  As a consequence, there is no synchronization between the two
> streams.
> 
> Call the function that loads the kunit test module from a separate thread
> and process the intermediate results immediately, as soon as available
> from the background parser, without waiting for completion of module
> loading.

LGTM.

Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>

> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
> ---
>  lib/igt_kmod.c | 35 +++++++++++++++++++++++++++++++++--
>  1 file changed, 33 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index c692954911..bbde3929f2 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -25,6 +25,7 @@
>  #include <signal.h>
>  #include <errno.h>
>  #include <fcntl.h>
> +#include <pthread.h>
>  #include <sys/utsname.h>
>  
>  #include "igt_aux.h"
> @@ -746,6 +747,21 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
>  	kmod_module_info_free_list(pre);
>  }
>  
> +struct modprobe_data {
> +	struct kmod_module *kmod;
> +	const char *opts;
> +	int err;
> +};
> +
> +static void *modprobe_task(void *arg)
> +{
> +	struct modprobe_data *data = arg;
> +
> +	data->err = modprobe(data->kmod, data->opts);
> +
> +	return NULL;
> +}
> +
>  /**
>   * igt_kunit:
>   * @module_name: the name of the module
> @@ -757,9 +773,11 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
>   */
>  static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  {
> +	struct modprobe_data modprobe = { tst->kmod, opts, 0, };
>  	struct kmod_module *kunit_kmod;
>  	bool is_builtin;
>  	struct ktap_test_results *results;
> +	pthread_t modprobe_thread;
>  	unsigned long taints;
>  	int flags, ret;
>  
> @@ -777,18 +795,25 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  
>  	results = ktap_parser_start(tst->kmsg, is_builtin);
>  
> -	if (igt_debug_on(igt_kmod_load(tst->module_name, opts) < 0)) {
> +	if (igt_debug_on(pthread_create(&modprobe_thread, NULL,
> +					modprobe_task, &modprobe))) {
>  		ktap_parser_cancel();
>  		igt_ignore_warn(ktap_parser_stop());
> -		igt_skip("Unable to load %s module\n", tst->module_name);
> +		igt_skip("Failed to create a modprobe thread\n");
>  	}
>  
>  	while (READ_ONCE(results->still_running) || !igt_list_empty(&results->list))
>  	{
>  		struct ktap_test_results_element *result;
>  
> +		if (!pthread_tryjoin_np(modprobe_thread, NULL) && modprobe.err) {
> +			ktap_parser_cancel();
> +			break;
> +		}
> +
>  		if (igt_kernel_tainted(&taints)) {
>  			ktap_parser_cancel();
> +			pthread_cancel(modprobe_thread);
>  			break;
>  		}
>  
> @@ -806,14 +831,20 @@ static void __igt_kunit(struct igt_ktest *tst, const char *opts)
>  		igt_dynamic(result->test_name) {
>  			igt_assert(READ_ONCE(result->passed));
>  
> +			if (!pthread_tryjoin_np(modprobe_thread, NULL))
> +				igt_assert_eq(modprobe.err, 0);
> +
>  			igt_fail_on(igt_kernel_tainted(&taints));
>  		}
>  
>  		free(result);
>  	}
>  
> +	pthread_join(modprobe_thread, NULL);
> +
>  	ret = ktap_parser_stop();
>  
> +	igt_skip_on(modprobe.err);
>  	igt_skip_on(igt_kernel_tainted(&taints));
>  	igt_skip_on_f(ret, "KTAP parser failed\n");
>  }

Re: [Intel-gfx] [PATCH i-g-t v2 07/17] lib/ktap: Don't ignore interrupt signals

[Mauro Carvalho Chehab]

On Wed, 13 Sep 2023 16:04:10 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

> On Monday, 11 September 2023 11:01:42 CEST Mauro Carvalho Chehab wrote:
> > On Fri,  8 Sep 2023 14:32:41 +0200
> > Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> >   
> > > While reading KTAP data from /dev/kmsg we now ignore interrupt signals
> > > that may occur during read() and we continue reading the data.  No
> > > explanation has been provided on what that could be needed for.  
> > 
> > The reason is that kunit module load takes seconds, as it only finishes
> > loading after all tests are executed.
> > 
> > So, interrupting IGT won't interrupt the tests, and kmsg will still
> > be filled by test results.
> > 
> > IMO, the right thing to do here is to ignore interrupts, as otherwise
> > the logs for the next test will be polluted by the KTAP messages and
> > the Kernel will be kept on an unstable state, as running tests while
> > kUnit tests are running is not supported.  
> 
> Well, not really.  Please have a look at the following two log excerpts.  The 
> first one is from a complete, not interrupted execution of drm_mm test:
> 
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: jkrzyszt : TTY=pts/14 ; PWD=/home/jkrzyszt/build/igt ; USER=root ; COMMAND=./tests/drm_mm
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0x5b01fc53 max_iterations=8192 max_prime=128
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
> Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_reserve
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_reserve, SUCCESS
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert
> Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_replace
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_replace, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert_range
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert_range, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: bottom-up fragmented insert of 10000 and 20000 insertions took 6009658 and 13648333 nsecs
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: top-down fragmented insert of 10000 and 20000 insertions took 6518544 and 13824246 nsecs
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_frag
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_frag, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align32
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align32, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align64
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align64, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict_range
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict_range, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_topdown
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_topdown, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_bottomup
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_bottomup, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_lowest
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_lowest, SUCCESS
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_highest
> Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_highest, SUCCESS
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color, SUCCESS
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict, SUCCESS
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict_range
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict_range, SUCCESS
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm_test, SUCCESS
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session closed for user root
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: exiting, ret=0
> Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75
> 
> You can see it took 17 seconds for the test to complete.  Now the same test 
> but interrupted after 2 seconds:
> 
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0xadd9af85 max_iterations=8192 max_prime=128
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
> Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
> Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
> Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
> Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session closed for user root
> 
> When you compare timestamps, you can see that kunit layer reported an OK 
> result from test case 3 drm_test_mm_reserve two seconds after a result from 
> test case 2 drm_test_mm_debug, presumably as soon as user interrupt occurred, 
> while it took ca. 7 seconds before.  We can also see that the process tried to 
> exit, but a still running despite interrupted modprobe thread prevented it 
> from exiting before modprobe completion.  Moreover, we can see an OK result 
> from test case 4 drm_test_mm_insert now appeared after 7 seconds, presumably 
> after test case 3 actually completed, and OK results from remaining test cases 
> appeared immediately thereafter, and module loading completed within those ca. 
> 7 seconds.  All those OK results that appeared after the test was interrupted 
> were then wrong, however, they were not converted to any IGT results, which is 
> fine.

The problem will happen if another test starts afterwards.

E. g. if you call a testset with something like (untested):

	./build/runner/igt_runner --per-test-timeout 2

The test which takes 7 seconds will be interrupted, and the next
test will start running while the "aborted" one will still run.

Regards,
Mauro

Re: [Intel-gfx] [PATCH i-g-t v2 13/17] lib/ktap: Reimplement KTAP parser

[Mauro Carvalho Chehab]

On Fri,  8 Sep 2023 14:32:47 +0200
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:

Forgot to mention on my past review:

> +				       &n, &len) == 1 && len == strlen(buf))) {
> +		/*
> +		 * TODO: drop the following workaround as soon as
> +		 * kernel side issue of missing lines with top level
> +		 * KTAP version and test suite plan is fixed.
> +		 */

Please add a link to lore with the thread discussing the issue upstream,
as it can be helpful while this workaround is here.

Also, I would place the workaround on a separate patch, to make
easier to revert it when upstream fixes it.

Re: [Intel-gfx] [PATCH i-g-t v2 07/17] lib/ktap: Don't ignore interrupt signals

[Janusz Krzysztofik]

On Friday, 15 September 2023 14:25:24 CEST Mauro Carvalho Chehab wrote:
> On Wed, 13 Sep 2023 16:04:10 +0200
> Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> 
> > On Monday, 11 September 2023 11:01:42 CEST Mauro Carvalho Chehab wrote:
> > > On Fri,  8 Sep 2023 14:32:41 +0200
> > > Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> > >   
> > > > While reading KTAP data from /dev/kmsg we now ignore interrupt signals
> > > > that may occur during read() and we continue reading the data.  No
> > > > explanation has been provided on what that could be needed for.  
> > > 
> > > The reason is that kunit module load takes seconds, as it only finishes
> > > loading after all tests are executed.
> > > 
> > > So, interrupting IGT won't interrupt the tests, and kmsg will still
> > > be filled by test results.
> > > 
> > > IMO, the right thing to do here is to ignore interrupts, as otherwise
> > > the logs for the next test will be polluted by the KTAP messages and
> > > the Kernel will be kept on an unstable state, as running tests while
> > > kUnit tests are running is not supported.  
> > 
> > Well, not really.  Please have a look at the following two log excerpts.  The 
> > first one is from a complete, not interrupted execution of drm_mm test:
> > 
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: jkrzyszt : TTY=pts/14 ; PWD=/home/jkrzyszt/build/igt ; USER=root ; COMMAND=./tests/drm_mm
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0x5b01fc53 max_iterations=8192 max_prime=128
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
> > Sep 13 15:14:39 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_reserve
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_reserve, SUCCESS
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert
> > Sep 13 15:14:46 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_replace
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_replace, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_insert_range
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_insert_range, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: bottom-up fragmented insert of 10000 and 20000 insertions took 6009658 and 13648333 nsecs
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_test_mm_frag: top-down fragmented insert of 10000 and 20000 insertions took 6518544 and 13824246 nsecs
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_frag
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_frag, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align32
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align32, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_align64
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_align64, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_evict_range
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_evict_range, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_topdown
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_topdown, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_bottomup
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_bottomup, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_lowest
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_lowest, SUCCESS
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_highest
> > Sep 13 15:14:55 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_highest, SUCCESS
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color, SUCCESS
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict, SUCCESS
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_color_evict_range
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_color_evict_range, SUCCESS
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm_test, SUCCESS
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com sudo[15594]: pam_unix(sudo:session): session closed for user root
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: exiting, ret=0
> > Sep 13 15:14:56 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75
> > 
> > You can see it took 17 seconds for the test to complete.  Now the same test 
> > but interrupted after 2 seconds:
> > 
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session opened for user root(uid=0) by jkrzyszt(uid=1000)
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour dummy device 80x25
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: executing
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting subtest drm_mm_test
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # drm_mm: Testing DRM range manager, with random_seed=0xadd9af85 max_iterations=8192 max_prime=128
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     KTAP version 1
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     # Subtest: drm_mm
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     1..19
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 1 drm_test_mm_init
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 2 drm_test_mm_debug
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_init
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_init, SUCCESS
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: starting dynamic subtest drm_mm-drm_test_mm_debug
> > Sep 13 15:15:15 jkrzyszt-mobl2.ger.corp.intel.com unknown: [IGT] drm_mm: finished subtest drm_mm-drm_test_mm_debug, SUCCESS
> > Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 3 drm_test_mm_reserve
> > Sep 13 15:15:17 jkrzyszt-mobl2.ger.corp.intel.com kernel: Console: switching to colour frame buffer device 240x75
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 4 drm_test_mm_insert
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 5 drm_test_mm_replace
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 6 drm_test_mm_insert_range
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 7 drm_test_mm_frag
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 8 drm_test_mm_align
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 9 drm_test_mm_align32
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 10 drm_test_mm_align64
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 11 drm_test_mm_evict
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 12 drm_test_mm_evict_range
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 13 drm_test_mm_topdown
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 14 drm_test_mm_bottomup
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 15 drm_test_mm_lowest
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 16 drm_test_mm_highest
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 17 drm_test_mm_color
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 18 drm_test_mm_color_evict
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel:     ok 19 drm_test_mm_color_evict_range
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # drm_mm: pass:19 fail:0 skip:0 total:19
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: # Totals: pass:19 fail:0 skip:0 total:19
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com kernel: ok 1 drm_mm
> > Sep 13 15:15:22 jkrzyszt-mobl2.ger.corp.intel.com sudo[15668]: pam_unix(sudo:session): session closed for user root
> > 
> > When you compare timestamps, you can see that kunit layer reported an OK 
> > result from test case 3 drm_test_mm_reserve two seconds after a result from 
> > test case 2 drm_test_mm_debug, presumably as soon as user interrupt occurred, 
> > while it took ca. 7 seconds before.  We can also see that the process tried to 
> > exit, but a still running despite interrupted modprobe thread prevented it 
> > from exiting before modprobe completion.  Moreover, we can see an OK result 
> > from test case 4 drm_test_mm_insert now appeared after 7 seconds, presumably 
> > after test case 3 actually completed, and OK results from remaining test cases 
> > appeared immediately thereafter, and module loading completed within those ca. 
> > 7 seconds.  All those OK results that appeared after the test was interrupted 
> > were then wrong, however, they were not converted to any IGT results, which is 
> > fine.
> 
> The problem will happen if another test starts afterwards.
> 
> E. g. if you call a testset with something like (untested):
> 
> 	./build/runner/igt_runner --per-test-timeout 2
> 
> The test which takes 7 seconds will be interrupted, and the next
> test will start running while the "aborted" one will still run.

As you can see, a message from completion of sudo, designating actual 
completion of the IGT test process called via that sudo, appeared after  
reporting from the kunit test module (then loading the module) had completed. 
Then, no next IGT test would have had a chance to be called from the same 
caller (igt_runner process) before that one completed, I believe.  Situation 
could be different if we used a modprobe sub-process, but here we created a 
thread that loaded the module via libkmod, then the main process thread 
couldn't terminate as longs as the modprobe thread didn't complete yet.

With this patch positioned in this series before we start loading the module 
from a separate thread, that was even more simple since we were calling 
kmod_module_probe_insert_module() from the main process.  The test just waited 
for completion of that function call, and that could happen only after the 
kernel side processing completed.  Then the KTAP parser thread was terminated 
by the main one via dropping is_running flag.

Anyway, speaking of this patch, as long as we use default signal handler 
for SIGINT that terminates the process and all its threads, special handling 
of errno == EINTR on read() failure is pointless, I believe.  I'll propose to 
drop that completely in next version of this patch.

Thanks,
Janusz

> 
> Regards,
> Mauro
> 
> 

Re: [Intel-gfx] [PATCH i-g-t v2 13/17] lib/ktap: Reimplement KTAP parser

[Janusz Krzysztofik]

Hi Mauro,

On Friday, 15 September 2023 14:28:49 CEST Mauro Carvalho Chehab wrote:
> On Fri,  8 Sep 2023 14:32:47 +0200
> Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> 
> Forgot to mention on my past review:
> 
> > +				       &n, &len) == 1 && len == strlen(buf))) {
> > +		/*
> > +		 * TODO: drop the following workaround as soon as
> > +		 * kernel side issue of missing lines with top level
> > +		 * KTAP version and test suite plan is fixed.
> > +		 */
> 
> Please add a link to lore with the thread discussing the issue upstream,
> as it can be helpful while this workaround is here.
> 
> Also, I would place the workaround on a separate patch, to make
> easier to revert it when upstream fixes it.

Thanks for pointing that out.  Since we already have those kernel changes 
included in v6.6-rc1, then in drm-tip, I'll drop the workaround now.

Thanks,
Janusz

Re: [Intel-gfx] [PATCH i-g-t v2 13/17] lib/ktap: Reimplement KTAP parser

[Janusz Krzysztofik]

On Friday, 15 September 2023 15:09:58 CEST Janusz Krzysztofik wrote:
> Hi Mauro,
> 
> On Friday, 15 September 2023 14:28:49 CEST Mauro Carvalho Chehab wrote:
> > On Fri,  8 Sep 2023 14:32:47 +0200
> > Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> wrote:
> > 
> > Forgot to mention on my past review:
> > 
> > > +				       &n, &len) == 1 && len == 
strlen(buf))) {
> > > +		/*
> > > +		 * TODO: drop the following workaround as soon as
> > > +		 * kernel side issue of missing lines with top level
> > > +		 * KTAP version and test suite plan is fixed.
> > > +		 */
> > 
> > Please add a link to lore with the thread discussing the issue upstream,
> > as it can be helpful while this workaround is here.
> > 
> > Also, I would place the workaround on a separate patch, to make
> > easier to revert it when upstream fixes it.
> 
> Thanks for pointing that out.  Since we already have those kernel changes 
> included in v6.6-rc1, then in drm-tip, I'll drop the workaround now.

Or would you like, Mauro, to keep the workaround for CI testing of linux-xe 
not yet updated to v6.6-rc1?

Thanks,
Janusz

> 
> Thanks,
> Janusz
>