[igt-dev] [PATCH i-g-t 0/4] Introduce KUnit

[igt-dev] [PATCH i-g-t 0/4] Introduce KUnit

[Dominik Karol Piatkowski]

This series is a continuation of Isabella's work on introducing
KUnit to IGT.

Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
Cc: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>

Isabella Basso (4):
  lib/igt_kmod: rename kselftest functions to ktest
  lib/igt_kmod.c: check if module is builtin before attempting to unload
    it
  lib/igt_kmod: add compatibility for KUnit
  tests: DRM selftests: switch to KUnit

 lib/igt_kmod.c       | 109 ++++++++++++--
 lib/igt_kmod.h       |  14 +-
 lib/igt_ktap.c       | 334 +++++++++++++++++++++++++++++++++++++++++++
 lib/igt_ktap.h       |  31 ++++
 lib/meson.build      |   1 +
 tests/drm_buddy.c    |   4 +-
 tests/drm_mm.c       |   4 +-
 tests/kms_selftest.c |  17 ++-
 8 files changed, 494 insertions(+), 20 deletions(-)
 create mode 100644 lib/igt_ktap.c
 create mode 100644 lib/igt_ktap.h

-- 
2.34.1

[igt-dev] [PATCH i-g-t 1/4] lib/igt_kmod: rename kselftest functions to ktest

[Dominik Karol Piatkowski]

From: Isabella Basso <isabbasso@riseup.net>

This aims at making IGT's structure more general to different kernel
testing frameworks such as KUnit, as they use a lot of the same
functionality.

Reviewed-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Signed-off-by: Isabella Basso <isabbasso@riseup.net>
---
 lib/igt_kmod.c | 22 +++++++++++-----------
 lib/igt_kmod.h | 12 ++++++------
 2 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 6d6ecf01..0b714ce3 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -749,8 +749,8 @@ static int open_parameters(const char *module_name)
 	return open(path, O_RDONLY);
 }
 
-int igt_kselftest_init(struct igt_kselftest *tst,
-		       const char *module_name)
+int igt_ktest_init(struct igt_ktest *tst,
+		   const char *module_name)
 {
 	int err;
 
@@ -769,7 +769,7 @@ int igt_kselftest_init(struct igt_kselftest *tst,
 	return 0;
 }
 
-int igt_kselftest_begin(struct igt_kselftest *tst)
+int igt_ktest_begin(struct igt_ktest *tst)
 {
 	int err;
 
@@ -784,7 +784,7 @@ int igt_kselftest_begin(struct igt_kselftest *tst)
 	return 0;
 }
 
-int igt_kselftest_execute(struct igt_kselftest *tst,
+int igt_kselftest_execute(struct igt_ktest *tst,
 			  struct igt_kselftest_list *tl,
 			  const char *options,
 			  const char *result)
@@ -822,13 +822,13 @@ int igt_kselftest_execute(struct igt_kselftest *tst,
 	return err;
 }
 
-void igt_kselftest_end(struct igt_kselftest *tst)
+void igt_ktest_end(struct igt_ktest *tst)
 {
 	kmod_module_remove_module(tst->kmod, KMOD_REMOVE_FORCE);
 	close(tst->kmsg);
 }
 
-void igt_kselftest_fini(struct igt_kselftest *tst)
+void igt_ktest_fini(struct igt_ktest *tst)
 {
 	free(tst->module_name);
 	kmod_module_unref(tst->kmod);
@@ -851,15 +851,15 @@ void igt_kselftests(const char *module_name,
 		    const char *result,
 		    const char *filter)
 {
-	struct igt_kselftest tst;
+	struct igt_ktest tst;
 	IGT_LIST_HEAD(tests);
 	struct igt_kselftest_list *tl, *tn;
 
-	if (igt_kselftest_init(&tst, module_name) != 0)
+	if (igt_ktest_init(&tst, module_name) != 0)
 		return;
 
 	igt_fixture
-		igt_require(igt_kselftest_begin(&tst) == 0);
+		igt_require(igt_ktest_begin(&tst) == 0);
 
 	igt_kselftest_get_tests(tst.kmod, filter, &tests);
 	igt_subtest_with_dynamic(filter ?: "all-tests") {
@@ -878,9 +878,9 @@ void igt_kselftests(const char *module_name,
 	}
 
 	igt_fixture {
-		igt_kselftest_end(&tst);
+		igt_ktest_end(&tst);
 		igt_require(!igt_list_empty(&tests));
 	}
 
-	igt_kselftest_fini(&tst);
+	igt_ktest_fini(&tst);
 }
diff --git a/lib/igt_kmod.h b/lib/igt_kmod.h
index f98dd29f..ceb10cd0 100644
--- a/lib/igt_kmod.h
+++ b/lib/igt_kmod.h
@@ -50,7 +50,7 @@ void igt_kselftests(const char *module_name,
 		    const char *result_option,
 		    const char *filter);
 
-struct igt_kselftest {
+struct igt_ktest {
 	struct kmod_module *kmod;
 	char *module_name;
 	int kmsg;
@@ -63,19 +63,19 @@ struct igt_kselftest_list {
 	char param[];
 };
 
-int igt_kselftest_init(struct igt_kselftest *tst,
+int igt_ktest_init(struct igt_ktest *tst,
 		       const char *module_name);
-int igt_kselftest_begin(struct igt_kselftest *tst);
+int igt_ktest_begin(struct igt_ktest *tst);
 
 void igt_kselftest_get_tests(struct kmod_module *kmod,
 			     const char *filter,
 			     struct igt_list_head *tests);
-int igt_kselftest_execute(struct igt_kselftest *tst,
+int igt_kselftest_execute(struct igt_ktest *tst,
 			  struct igt_kselftest_list *tl,
 			  const char *module_options,
 			  const char *result);
 
-void igt_kselftest_end(struct igt_kselftest *tst);
-void igt_kselftest_fini(struct igt_kselftest *tst);
+void igt_ktest_end(struct igt_ktest *tst);
+void igt_ktest_fini(struct igt_ktest *tst);
 
 #endif /* IGT_KMOD_H */
-- 
2.34.1

[igt-dev] [PATCH i-g-t 2/4] lib/igt_kmod.c: check if module is builtin before attempting to unload it

[Dominik Karol Piatkowski]

From: Isabella Basso <isabbasso@riseup.net>

This change makes `igt_kmod_unload_r` safer as it checks whether the
module can be unloaded before attempting it.

v2 -> v3:
- Fix commit message
- Make return value clearer

Acked-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Signed-off-by: Isabella Basso <isabbasso@riseup.net>
---
 lib/igt_kmod.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index 0b714ce3..fbda1aa6 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -259,6 +259,9 @@ static int igt_kmod_unload_r(struct kmod_module *kmod, unsigned int flags)
 	int err, tries;
 	const char *mod_name = kmod_module_get_name(kmod);
 
+	if (kmod_module_get_initstate(kmod) == KMOD_MODULE_BUILTIN)
+		return 0;
+
 	holders = kmod_module_get_holders(kmod);
 	kmod_list_foreach(pos, holders) {
 		struct kmod_module *it = kmod_module_get_module(pos);
-- 
2.34.1

[igt-dev] [PATCH i-g-t 3/4] lib/igt_kmod: add compatibility for KUnit

[Dominik Karol Piatkowski]

From: Isabella Basso <isabbasso@riseup.net>

This adds functions for both executing the tests as well as parsing (K)TAP
kmsg output, as per the KTAP spec [1].

[1] https://www.kernel.org/doc/html/latest/dev-tools/ktap.html

v1 -> v2:
- refactor igt_kunit function and ktap parser so that we have only one
  parser that we call only once (code size is now less than half the
  size as v1)
- add lookup_value helper
- fix parsing problems
v2 -> v3:
- move ktap parsing functions to own file
- rename to ktap_parser
- get rid of unneeded pointers in igt_kunit
- change return values to allow for subsequent call to igt_kselftests if
  needed
- add docs to parsing functions and helpers
- switch to line buffering
- add line buffering logging helper
- fix kunit module handling
- fix parsing of version lines
- use igt_subtest blocks to improve output handling on the CI
- fix output handling during crashes

Signed-off-by: Isabella Basso <isabbasso@riseup.net>

v3 -> v4:
- handle igt_ktap_parser fail with IGT_EXIT_ABORT code

Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
---
 lib/igt_kmod.c  |  84 ++++++++++++
 lib/igt_kmod.h  |   2 +
 lib/igt_ktap.c  | 334 ++++++++++++++++++++++++++++++++++++++++++++++++
 lib/igt_ktap.h  |  31 +++++
 lib/meson.build |   1 +
 5 files changed, 452 insertions(+)
 create mode 100644 lib/igt_ktap.c
 create mode 100644 lib/igt_ktap.h

diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
index fbda1aa6..a156e43d 100644
--- a/lib/igt_kmod.c
+++ b/lib/igt_kmod.c
@@ -29,6 +29,7 @@
 #include "igt_aux.h"
 #include "igt_core.h"
 #include "igt_kmod.h"
+#include "igt_ktap.h"
 #include "igt_sysfs.h"
 #include "igt_taints.h"
 
@@ -744,6 +745,89 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
 	kmod_module_info_free_list(pre);
 }
 
+/**
+ * igt_kunit:
+ * @module_name: the name of the module
+ * @opts: options to load the module
+ *
+ * Loads the test module, parses its (k)tap dmesg output, then unloads it
+ *
+ * Returns: IGT default codes
+ */
+int igt_kunit(const char *module_name, const char *opts)
+{
+	struct igt_ktest tst;
+	struct kmod_module *kunit_kmod;
+	char record[BUF_LEN + 1];
+	FILE *f;
+	bool is_builtin;
+	int ret;
+
+	ret = IGT_EXIT_INVALID;
+
+	/* get normalized module name */
+	if (igt_ktest_init(&tst, module_name) != 0) {
+		igt_warn("Unable to initialize ktest for %s\n", module_name);
+		return ret;
+	}
+
+	if (igt_ktest_begin(&tst) != 0) {
+		igt_warn("Unable to begin ktest for %s\n", module_name);
+
+		igt_ktest_fini(&tst);
+		return ret;
+	}
+
+	if (tst.kmsg < 0) {
+		igt_warn("Could not open /dev/kmsg");
+		goto unload;
+	}
+
+	if (lseek(tst.kmsg, 0, SEEK_END)) {
+		igt_warn("Could not seek the end of /dev/kmsg");
+		goto unload;
+	}
+
+	f = fdopen(tst.kmsg, "r");
+
+	if (f == NULL) {
+		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer");
+		goto unload;
+	}
+
+	if (setvbuf(f, record, _IOLBF, BUF_LEN)) {
+		igt_warn("Could not set line buffering on /dev/kmsg");
+		goto unload;
+	}
+
+	/* The KUnit module is required for running any KUnit tests */
+	if (igt_kmod_load("kunit", NULL) != 0 ||
+	    kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod) != 0) {
+		igt_warn("Unable to load KUnit\n");
+		igt_fail(IGT_EXIT_FAILURE);
+	}
+
+	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
+
+	if (igt_kmod_load(module_name, opts) != 0) {
+		igt_warn("Unable to load %s module\n", module_name);
+		igt_fail(IGT_EXIT_FAILURE);
+	}
+
+	ret = igt_ktap_parser(f, record, is_builtin);
+	if (ret != 0)
+		ret = IGT_EXIT_ABORT;
+unload:
+	igt_ktest_end(&tst);
+
+	igt_ktest_fini(&tst);
+
+	if (ret == 0)
+		igt_success();
+
+	return ret;
+}
+
 static int open_parameters(const char *module_name)
 {
 	char path[256];
diff --git a/lib/igt_kmod.h b/lib/igt_kmod.h
index ceb10cd0..72f9f445 100644
--- a/lib/igt_kmod.h
+++ b/lib/igt_kmod.h
@@ -45,6 +45,8 @@ int __igt_i915_driver_unload(char **whom);
 int igt_amdgpu_driver_load(const char *opts);
 int igt_amdgpu_driver_unload(void);
 
+int igt_kunit(const char *module_name, const char *opts);
+
 void igt_kselftests(const char *module_name,
 		    const char *module_options,
 		    const char *result_option,
diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
new file mode 100644
index 00000000..117598fa
--- /dev/null
+++ b/lib/igt_ktap.c
@@ -0,0 +1,334 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Isabella Basso do Amaral <isabbasso@riseup.net>
+ */
+
+#include <ctype.h>
+#include <limits.h>
+
+#include "igt_aux.h"
+#include "igt_core.h"
+#include "igt_ktap.h"
+
+static int log_to_end(enum igt_log_level level, FILE *f,
+		      char *record, const char *format, ...) __attribute__((format(printf, 4, 5)));
+
+/**
+ * log_to_end:
+ * @level: #igt_log_level
+ * @record: record to store the read data
+ * @format: format string
+ * @...: optional arguments used in the format string
+ *
+ * This is an altered version of the generic structured logging helper function
+ * igt_log capable of reading to the end of a given line.
+ *
+ * Returns: 0 for success, or -2 if there's an error reading from the file
+ */
+static int log_to_end(enum igt_log_level level, FILE *f,
+		      char *record, const char *format, ...)
+{
+	va_list args;
+	const char *lend;
+
+	va_start(args, format);
+	igt_vlog(IGT_LOG_DOMAIN, level, format, args);
+	va_end(args);
+
+	lend = strchrnul(record, '\n');
+	while (*lend == '\0') {
+		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
+		if (fgets(record, BUF_LEN, f) == NULL) {
+			igt_warn("kmsg truncated: unknown error (%m)\n");
+			return -2;
+		}
+		lend = strchrnul(record, '\n');
+	}
+	return 0;
+}
+
+/**
+ * lookup_value:
+ * @haystack: the string to search in
+ * @needle: the string to search for
+ *
+ * Returns: the value of the needle in the haystack, or -1 if not found.
+ */
+static long lookup_value(const char *haystack, const char *needle)
+{
+	const char *needle_rptr;
+	char *needle_end;
+	long num;
+
+	needle_rptr = strcasestr(haystack, needle);
+
+	if (needle_rptr == NULL)
+		return -1;
+
+	/* skip search string and whitespaces after it */
+	needle_rptr += strlen(needle);
+
+	num = strtol(needle_rptr, &needle_end, 10);
+
+	if (needle_rptr == needle_end)
+		return -1;
+
+	if (num == LONG_MIN || num == LONG_MAX)
+		return 0;
+
+	return num > 0 ? num : 0;
+}
+
+/**
+ * find_next_tap_subtest:
+ * @fp: FILE pointer
+ * @record: buffer used to read fp
+ * @is_builtin: whether KUnit is built-in or not
+ *
+ * Returns:
+ * 0 if there's missing information
+ * -1 if not found
+ * -2 if there are problems while reading the file.
+ * any other value corresponds to the amount of cases of the next (sub)test
+ */
+static int find_next_tap_subtest(FILE *fp, char *record, bool is_builtin)
+{
+	const char *test_lookup_str, *subtest_lookup_str, *name_rptr, *version_rptr;
+	char test_name[BUF_LEN + 1];
+	long test_count;
+
+	test_name[0] = '\0';
+	test_name[BUF_LEN] = '\0';
+
+	test_lookup_str = " subtest: ";
+	subtest_lookup_str = " test: ";
+
+	/*
+	 * "(K)TAP version XX" should be the first line on all (sub)tests as per
+	 * https://kernel.org/doc/html/latest/dev-tools/ktap.html#version-lines
+	 *
+	 * but actually isn't, as it currently depends on the KUnit module
+	 * being built-in, so we can't rely on it every time
+	 */
+	if (is_builtin) {
+		version_rptr = strcasestr(record, "TAP version ");
+		if (version_rptr == NULL)
+			return -1;
+
+		igt_info("%s", version_rptr);
+
+		if (fgets(record, BUF_LEN, fp) == NULL) {
+			igt_warn("kmsg truncated: unknown error (%m)\n");
+			return -2;
+		}
+	}
+
+	name_rptr = strcasestr(record, test_lookup_str);
+	if (name_rptr != NULL) {
+		name_rptr += strlen(test_lookup_str);
+	} else {
+		name_rptr = strcasestr(record, subtest_lookup_str);
+		if (name_rptr != NULL)
+			name_rptr += strlen(subtest_lookup_str);
+	}
+
+	if (name_rptr == NULL) {
+		if (!is_builtin)
+			/* we've probably found nothing */
+			return -1;
+		igt_info("Missing test name\n");
+	} else {
+		strncpy(test_name, name_rptr, BUF_LEN);
+		if (fgets(record, BUF_LEN, fp) == NULL) {
+			igt_warn("kmsg truncated: unknown error (%m)\n");
+			return -2;
+		}
+		/* now we can be sure we found tests */
+		if (!is_builtin)
+			igt_info("KUnit is not built-in, skipping version check...\n");
+	}
+
+	/*
+	 * total test count will almost always appear as 0..N at the beginning
+	 * of a run, so we use it to reliably identify a new run
+	 */
+	test_count = lookup_value(record, "..");
+
+	if (test_count <= 0) {
+		igt_info("Missing test count\n");
+		if (test_name[0] == '\0')
+			return 0;
+		if (log_to_end(IGT_LOG_INFO, fp, record,
+				"Running some tests in: %s",
+				test_name) < 0)
+			return -2;
+		return 0;
+	} else if (test_name[0] == '\0') {
+		igt_info("Running %ld tests...\n", test_count);
+		return 0;
+	}
+
+	if (log_to_end(IGT_LOG_INFO, fp, record,
+			"Executing %ld tests in: %s",
+			test_count, test_name) < 0)
+		return -2;
+
+	return test_count;
+}
+
+/**
+ * find_next_tap_test:
+ * @fp: FILE pointer
+ * @record: buffer used to read fp
+ * @test_name: buffer to store the test name
+ *
+ * Returns:
+ * 1 if no results were found
+ * 0 if a test succeded
+ * -1 if a test failed
+ * -2 if there are problems reading the file
+ */
+static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
+{
+	const char *lstart, *ok_lookup_str, *nok_lookup_str,
+	      *ok_rptr, *nok_rptr, *comment_start, *value_parse_start;
+	char *test_name_end;
+
+	ok_lookup_str = "ok ";
+	nok_lookup_str = "not ok ";
+
+	lstart = strchrnul(record, ';');
+
+	if (*lstart == '\0') {
+		igt_warn("kmsg truncated: output malformed (%m)\n");
+		return -2;
+	}
+
+	lstart++;
+	while (isspace(*lstart))
+		lstart++;
+
+	nok_rptr = strstr(lstart, nok_lookup_str);
+	if (nok_rptr != NULL) {
+		nok_rptr += strlen(nok_lookup_str);
+		while (isdigit(*nok_rptr) || isspace(*nok_rptr) || *nok_rptr == '-')
+			nok_rptr++;
+		test_name_end = strncpy(test_name, nok_rptr, BUF_LEN);
+		while (!isspace(*test_name_end))
+			test_name_end++;
+		*test_name_end = '\0';
+		if (log_to_end(IGT_LOG_WARN, fp, record,
+			       "%s", lstart) < 0)
+			return -2;
+		return -1;
+	}
+
+	comment_start = strchrnul(lstart, '#');
+
+	/* check if we're still in a subtest */
+	if (*comment_start != '\0') {
+		comment_start++;
+		value_parse_start = comment_start;
+
+		if (lookup_value(value_parse_start, "fail: ") > 0) {
+			if (log_to_end(IGT_LOG_WARN, fp, record,
+				       "%s", lstart) < 0)
+				return -2;
+			return -1;
+		}
+	}
+
+	ok_rptr = strstr(lstart, ok_lookup_str);
+	if (ok_rptr != NULL) {
+		ok_rptr += strlen(ok_lookup_str);
+		while (isdigit(*ok_rptr) || isspace(*ok_rptr) || *ok_rptr == '-')
+			ok_rptr++;
+		test_name_end = strncpy(test_name, ok_rptr, BUF_LEN);
+		while (!isspace(*test_name_end))
+			test_name_end++;
+		*test_name_end = '\0';
+		return 0;
+	}
+
+	return 1;
+}
+
+/**
+ * igt_ktap_parser:
+ * @fp: FILE pointer
+ * @record: buffer used to read fp
+ * @is_builtin: whether the KUnit module is built-in or not
+ *
+ * This function parses the output of a ktap script and prints the test results,
+ * as well as any other output to stdout.
+ *
+ * Returns: IGT default codes
+ */
+int igt_ktap_parser(FILE *fp, char *record, bool is_builtin)
+{
+	char test_name[BUF_LEN + 1];
+	bool failed_tests, found_tests;
+	int sublevel = 0;
+
+	test_name[0] = '\0';
+	test_name[BUF_LEN] = '\0';
+
+	failed_tests = false;
+	found_tests = false;
+
+	while (sublevel >= 0) {
+		if (fgets(record, BUF_LEN, fp) == NULL) {
+			if (!found_tests)
+				igt_warn("kmsg truncated: unknown error (%m)\n");
+			break;
+		}
+
+		switch (find_next_tap_subtest(fp, record, is_builtin)) {
+		case -2:
+			/* no more data to read */
+			return IGT_EXIT_FAILURE;
+		case -1:
+			/* no test found, so we keep parsing */
+			break;
+		case 0:
+			/*
+			 * tests found, but they're missing info, so we might
+			 * have read into test output
+			 */
+			found_tests = true;
+			sublevel++;
+			break;
+		default:
+			if (fgets(record, BUF_LEN, fp) == NULL) {
+				igt_warn("kmsg truncated: unknown error (%m)\n");
+				return -2;
+			}
+			found_tests = true;
+			sublevel++;
+			break;
+		}
+
+		switch (parse_kmsg_for_tap(fp, record, test_name)) {
+		case -2:
+			return IGT_EXIT_FAILURE;
+		case -1:
+			sublevel--;
+			failed_tests = true;
+			igt_subtest(test_name)
+				igt_fail(IGT_EXIT_FAILURE);
+			test_name[0] = '\0';
+			break;
+		case 0: /* fallthrough */
+			igt_subtest(test_name)
+				igt_success();
+			test_name[0] = '\0';
+		default:
+			break;
+		}
+	}
+
+	if (failed_tests || !found_tests)
+		return IGT_EXIT_FAILURE;
+
+	return IGT_EXIT_SUCCESS;
+}
diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
new file mode 100644
index 00000000..b2f69df2
--- /dev/null
+++ b/lib/igt_ktap.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright © 2022 Isabella Basso do Amaral <isabbasso@riseup.net>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#ifndef IGT_KTAP_H
+#define IGT_KTAP_H
+
+#define BUF_LEN 4096
+
+int igt_ktap_parser(FILE *fp, char *record, bool is_builtin);
+
+#endif /* IGT_KTAP_H */
diff --git a/lib/meson.build b/lib/meson.build
index c5131d9a..58cd6f44 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -87,6 +87,7 @@ lib_sources = [
 	'igt_store.c',
 	'uwildmat/uwildmat.c',
 	'igt_kmod.c',
+	'igt_ktap.c',
 	'igt_panfrost.c',
 	'igt_v3d.c',
 	'igt_vc4.c',
-- 
2.34.1

Re: [igt-dev] [PATCH i-g-t 3/4] lib/igt_kmod: add compatibility for KUnit

[Petri Latvala]

On Fri, Mar 03, 2023 at 12:07:14PM +0100, Dominik Karol Piatkowski wrote:
> From: Isabella Basso <isabbasso@riseup.net>
> 
> This adds functions for both executing the tests as well as parsing (K)TAP
> kmsg output, as per the KTAP spec [1].
> 
> [1] https://www.kernel.org/doc/html/latest/dev-tools/ktap.html
> 
> v1 -> v2:
> - refactor igt_kunit function and ktap parser so that we have only one
>   parser that we call only once (code size is now less than half the
>   size as v1)
> - add lookup_value helper
> - fix parsing problems
> v2 -> v3:
> - move ktap parsing functions to own file
> - rename to ktap_parser
> - get rid of unneeded pointers in igt_kunit
> - change return values to allow for subsequent call to igt_kselftests if
>   needed
> - add docs to parsing functions and helpers
> - switch to line buffering
> - add line buffering logging helper
> - fix kunit module handling
> - fix parsing of version lines
> - use igt_subtest blocks to improve output handling on the CI
> - fix output handling during crashes
> 
> Signed-off-by: Isabella Basso <isabbasso@riseup.net>
> 
> v3 -> v4:
> - handle igt_ktap_parser fail with IGT_EXIT_ABORT code
> 
> Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
> ---
>  lib/igt_kmod.c  |  84 ++++++++++++
>  lib/igt_kmod.h  |   2 +
>  lib/igt_ktap.c  | 334 ++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/igt_ktap.h  |  31 +++++
>  lib/meson.build |   1 +
>  5 files changed, 452 insertions(+)
>  create mode 100644 lib/igt_ktap.c
>  create mode 100644 lib/igt_ktap.h
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index fbda1aa6..a156e43d 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -29,6 +29,7 @@
>  #include "igt_aux.h"
>  #include "igt_core.h"
>  #include "igt_kmod.h"
> +#include "igt_ktap.h"
>  #include "igt_sysfs.h"
>  #include "igt_taints.h"
>  
> @@ -744,6 +745,89 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
>  	kmod_module_info_free_list(pre);
>  }
>  
> +/**
> + * igt_kunit:
> + * @module_name: the name of the module
> + * @opts: options to load the module
> + *
> + * Loads the test module, parses its (k)tap dmesg output, then unloads it
> + *
> + * Returns: IGT default codes
> + */
> +int igt_kunit(const char *module_name, const char *opts)
> +{
> +	struct igt_ktest tst;
> +	struct kmod_module *kunit_kmod;
> +	char record[BUF_LEN + 1];
> +	FILE *f;
> +	bool is_builtin;
> +	int ret;
> +
> +	ret = IGT_EXIT_INVALID;
> +
> +	/* get normalized module name */
> +	if (igt_ktest_init(&tst, module_name) != 0) {
> +		igt_warn("Unable to initialize ktest for %s\n", module_name);
> +		return ret;
> +	}
> +
> +	if (igt_ktest_begin(&tst) != 0) {
> +		igt_warn("Unable to begin ktest for %s\n", module_name);
> +
> +		igt_ktest_fini(&tst);
> +		return ret;
> +	}
> +
> +	if (tst.kmsg < 0) {
> +		igt_warn("Could not open /dev/kmsg");
> +		goto unload;
> +	}
> +
> +	if (lseek(tst.kmsg, 0, SEEK_END)) {
> +		igt_warn("Could not seek the end of /dev/kmsg");
> +		goto unload;
> +	}
> +
> +	f = fdopen(tst.kmsg, "r");
> +
> +	if (f == NULL) {
> +		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer");
> +		goto unload;
> +	}
> +
> +	if (setvbuf(f, record, _IOLBF, BUF_LEN)) {
> +		igt_warn("Could not set line buffering on /dev/kmsg");
> +		goto unload;
> +	}

Make sure all your igt_warn()s end with a newline.

There seems to be some kind of misunderstanding with how setvbuf
works. Here a buffer is given to setvbuf() so that fgets() in the
parser can read one line at a time, but then that's also reading into
the same buffer. According to setvbuf() docs you're not supposed to
touch the buffer yourself.

I vaguely remember someone suggesting line-buffered IO for reading
/dev/kmsg but can't find that message now. What was the reason for it?
Using read() on /dev/kmsg already gives only one log record at a time.


-- 
Petri Latvala


> +
> +	/* The KUnit module is required for running any KUnit tests */
> +	if (igt_kmod_load("kunit", NULL) != 0 ||
> +	    kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod) != 0) {
> +		igt_warn("Unable to load KUnit\n");
> +		igt_fail(IGT_EXIT_FAILURE);
> +	}
> +
> +	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
> +
> +	if (igt_kmod_load(module_name, opts) != 0) {
> +		igt_warn("Unable to load %s module\n", module_name);
> +		igt_fail(IGT_EXIT_FAILURE);
> +	}
> +
> +	ret = igt_ktap_parser(f, record, is_builtin);
> +	if (ret != 0)
> +		ret = IGT_EXIT_ABORT;
> +unload:
> +	igt_ktest_end(&tst);
> +
> +	igt_ktest_fini(&tst);
> +
> +	if (ret == 0)
> +		igt_success();
> +
> +	return ret;
> +}
> +
>  static int open_parameters(const char *module_name)
>  {
>  	char path[256];
> diff --git a/lib/igt_kmod.h b/lib/igt_kmod.h
> index ceb10cd0..72f9f445 100644
> --- a/lib/igt_kmod.h
> +++ b/lib/igt_kmod.h
> @@ -45,6 +45,8 @@ int __igt_i915_driver_unload(char **whom);
>  int igt_amdgpu_driver_load(const char *opts);
>  int igt_amdgpu_driver_unload(void);
>  
> +int igt_kunit(const char *module_name, const char *opts);
> +
>  void igt_kselftests(const char *module_name,
>  		    const char *module_options,
>  		    const char *result_option,
> diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c
> new file mode 100644
> index 00000000..117598fa
> --- /dev/null
> +++ b/lib/igt_ktap.c
> @@ -0,0 +1,334 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2023 Isabella Basso do Amaral <isabbasso@riseup.net>
> + */
> +
> +#include <ctype.h>
> +#include <limits.h>
> +
> +#include "igt_aux.h"
> +#include "igt_core.h"
> +#include "igt_ktap.h"
> +
> +static int log_to_end(enum igt_log_level level, FILE *f,
> +		      char *record, const char *format, ...) __attribute__((format(printf, 4, 5)));
> +
> +/**
> + * log_to_end:
> + * @level: #igt_log_level
> + * @record: record to store the read data
> + * @format: format string
> + * @...: optional arguments used in the format string
> + *
> + * This is an altered version of the generic structured logging helper function
> + * igt_log capable of reading to the end of a given line.
> + *
> + * Returns: 0 for success, or -2 if there's an error reading from the file
> + */
> +static int log_to_end(enum igt_log_level level, FILE *f,
> +		      char *record, const char *format, ...)
> +{
> +	va_list args;
> +	const char *lend;
> +
> +	va_start(args, format);
> +	igt_vlog(IGT_LOG_DOMAIN, level, format, args);
> +	va_end(args);
> +
> +	lend = strchrnul(record, '\n');
> +	while (*lend == '\0') {
> +		igt_log(IGT_LOG_DOMAIN, level, "%s", record);
> +		if (fgets(record, BUF_LEN, f) == NULL) {
> +			igt_warn("kmsg truncated: unknown error (%m)\n");
> +			return -2;
> +		}
> +		lend = strchrnul(record, '\n');
> +	}
> +	return 0;
> +}
> +
> +/**
> + * lookup_value:
> + * @haystack: the string to search in
> + * @needle: the string to search for
> + *
> + * Returns: the value of the needle in the haystack, or -1 if not found.
> + */
> +static long lookup_value(const char *haystack, const char *needle)
> +{
> +	const char *needle_rptr;
> +	char *needle_end;
> +	long num;
> +
> +	needle_rptr = strcasestr(haystack, needle);
> +
> +	if (needle_rptr == NULL)
> +		return -1;
> +
> +	/* skip search string and whitespaces after it */
> +	needle_rptr += strlen(needle);
> +
> +	num = strtol(needle_rptr, &needle_end, 10);
> +
> +	if (needle_rptr == needle_end)
> +		return -1;
> +
> +	if (num == LONG_MIN || num == LONG_MAX)
> +		return 0;
> +
> +	return num > 0 ? num : 0;
> +}
> +
> +/**
> + * find_next_tap_subtest:
> + * @fp: FILE pointer
> + * @record: buffer used to read fp
> + * @is_builtin: whether KUnit is built-in or not
> + *
> + * Returns:
> + * 0 if there's missing information
> + * -1 if not found
> + * -2 if there are problems while reading the file.
> + * any other value corresponds to the amount of cases of the next (sub)test
> + */
> +static int find_next_tap_subtest(FILE *fp, char *record, bool is_builtin)
> +{
> +	const char *test_lookup_str, *subtest_lookup_str, *name_rptr, *version_rptr;
> +	char test_name[BUF_LEN + 1];
> +	long test_count;
> +
> +	test_name[0] = '\0';
> +	test_name[BUF_LEN] = '\0';
> +
> +	test_lookup_str = " subtest: ";
> +	subtest_lookup_str = " test: ";
> +
> +	/*
> +	 * "(K)TAP version XX" should be the first line on all (sub)tests as per
> +	 * https://kernel.org/doc/html/latest/dev-tools/ktap.html#version-lines
> +	 *
> +	 * but actually isn't, as it currently depends on the KUnit module
> +	 * being built-in, so we can't rely on it every time
> +	 */
> +	if (is_builtin) {
> +		version_rptr = strcasestr(record, "TAP version ");
> +		if (version_rptr == NULL)
> +			return -1;
> +
> +		igt_info("%s", version_rptr);
> +
> +		if (fgets(record, BUF_LEN, fp) == NULL) {
> +			igt_warn("kmsg truncated: unknown error (%m)\n");
> +			return -2;
> +		}
> +	}
> +
> +	name_rptr = strcasestr(record, test_lookup_str);
> +	if (name_rptr != NULL) {
> +		name_rptr += strlen(test_lookup_str);
> +	} else {
> +		name_rptr = strcasestr(record, subtest_lookup_str);
> +		if (name_rptr != NULL)
> +			name_rptr += strlen(subtest_lookup_str);
> +	}
> +
> +	if (name_rptr == NULL) {
> +		if (!is_builtin)
> +			/* we've probably found nothing */
> +			return -1;
> +		igt_info("Missing test name\n");
> +	} else {
> +		strncpy(test_name, name_rptr, BUF_LEN);
> +		if (fgets(record, BUF_LEN, fp) == NULL) {
> +			igt_warn("kmsg truncated: unknown error (%m)\n");
> +			return -2;
> +		}
> +		/* now we can be sure we found tests */
> +		if (!is_builtin)
> +			igt_info("KUnit is not built-in, skipping version check...\n");
> +	}
> +
> +	/*
> +	 * total test count will almost always appear as 0..N at the beginning
> +	 * of a run, so we use it to reliably identify a new run
> +	 */
> +	test_count = lookup_value(record, "..");
> +
> +	if (test_count <= 0) {
> +		igt_info("Missing test count\n");
> +		if (test_name[0] == '\0')
> +			return 0;
> +		if (log_to_end(IGT_LOG_INFO, fp, record,
> +				"Running some tests in: %s",
> +				test_name) < 0)
> +			return -2;
> +		return 0;
> +	} else if (test_name[0] == '\0') {
> +		igt_info("Running %ld tests...\n", test_count);
> +		return 0;
> +	}
> +
> +	if (log_to_end(IGT_LOG_INFO, fp, record,
> +			"Executing %ld tests in: %s",
> +			test_count, test_name) < 0)
> +		return -2;
> +
> +	return test_count;
> +}
> +
> +/**
> + * find_next_tap_test:
> + * @fp: FILE pointer
> + * @record: buffer used to read fp
> + * @test_name: buffer to store the test name
> + *
> + * Returns:
> + * 1 if no results were found
> + * 0 if a test succeded
> + * -1 if a test failed
> + * -2 if there are problems reading the file
> + */
> +static int parse_kmsg_for_tap(FILE *fp, char *record, char *test_name)
> +{
> +	const char *lstart, *ok_lookup_str, *nok_lookup_str,
> +	      *ok_rptr, *nok_rptr, *comment_start, *value_parse_start;
> +	char *test_name_end;
> +
> +	ok_lookup_str = "ok ";
> +	nok_lookup_str = "not ok ";
> +
> +	lstart = strchrnul(record, ';');
> +
> +	if (*lstart == '\0') {
> +		igt_warn("kmsg truncated: output malformed (%m)\n");
> +		return -2;
> +	}
> +
> +	lstart++;
> +	while (isspace(*lstart))
> +		lstart++;
> +
> +	nok_rptr = strstr(lstart, nok_lookup_str);
> +	if (nok_rptr != NULL) {
> +		nok_rptr += strlen(nok_lookup_str);
> +		while (isdigit(*nok_rptr) || isspace(*nok_rptr) || *nok_rptr == '-')
> +			nok_rptr++;
> +		test_name_end = strncpy(test_name, nok_rptr, BUF_LEN);
> +		while (!isspace(*test_name_end))
> +			test_name_end++;
> +		*test_name_end = '\0';
> +		if (log_to_end(IGT_LOG_WARN, fp, record,
> +			       "%s", lstart) < 0)
> +			return -2;
> +		return -1;
> +	}
> +
> +	comment_start = strchrnul(lstart, '#');
> +
> +	/* check if we're still in a subtest */
> +	if (*comment_start != '\0') {
> +		comment_start++;
> +		value_parse_start = comment_start;
> +
> +		if (lookup_value(value_parse_start, "fail: ") > 0) {
> +			if (log_to_end(IGT_LOG_WARN, fp, record,
> +				       "%s", lstart) < 0)
> +				return -2;
> +			return -1;
> +		}
> +	}
> +
> +	ok_rptr = strstr(lstart, ok_lookup_str);
> +	if (ok_rptr != NULL) {
> +		ok_rptr += strlen(ok_lookup_str);
> +		while (isdigit(*ok_rptr) || isspace(*ok_rptr) || *ok_rptr == '-')
> +			ok_rptr++;
> +		test_name_end = strncpy(test_name, ok_rptr, BUF_LEN);
> +		while (!isspace(*test_name_end))
> +			test_name_end++;
> +		*test_name_end = '\0';
> +		return 0;
> +	}
> +
> +	return 1;
> +}
> +
> +/**
> + * igt_ktap_parser:
> + * @fp: FILE pointer
> + * @record: buffer used to read fp
> + * @is_builtin: whether the KUnit module is built-in or not
> + *
> + * This function parses the output of a ktap script and prints the test results,
> + * as well as any other output to stdout.
> + *
> + * Returns: IGT default codes
> + */
> +int igt_ktap_parser(FILE *fp, char *record, bool is_builtin)
> +{
> +	char test_name[BUF_LEN + 1];
> +	bool failed_tests, found_tests;
> +	int sublevel = 0;
> +
> +	test_name[0] = '\0';
> +	test_name[BUF_LEN] = '\0';
> +
> +	failed_tests = false;
> +	found_tests = false;
> +
> +	while (sublevel >= 0) {
> +		if (fgets(record, BUF_LEN, fp) == NULL) {
> +			if (!found_tests)
> +				igt_warn("kmsg truncated: unknown error (%m)\n");
> +			break;
> +		}
> +
> +		switch (find_next_tap_subtest(fp, record, is_builtin)) {
> +		case -2:
> +			/* no more data to read */
> +			return IGT_EXIT_FAILURE;
> +		case -1:
> +			/* no test found, so we keep parsing */
> +			break;
> +		case 0:
> +			/*
> +			 * tests found, but they're missing info, so we might
> +			 * have read into test output
> +			 */
> +			found_tests = true;
> +			sublevel++;
> +			break;
> +		default:
> +			if (fgets(record, BUF_LEN, fp) == NULL) {
> +				igt_warn("kmsg truncated: unknown error (%m)\n");
> +				return -2;
> +			}
> +			found_tests = true;
> +			sublevel++;
> +			break;
> +		}
> +
> +		switch (parse_kmsg_for_tap(fp, record, test_name)) {
> +		case -2:
> +			return IGT_EXIT_FAILURE;
> +		case -1:
> +			sublevel--;
> +			failed_tests = true;
> +			igt_subtest(test_name)
> +				igt_fail(IGT_EXIT_FAILURE);
> +			test_name[0] = '\0';
> +			break;
> +		case 0: /* fallthrough */
> +			igt_subtest(test_name)
> +				igt_success();
> +			test_name[0] = '\0';
> +		default:
> +			break;
> +		}
> +	}
> +
> +	if (failed_tests || !found_tests)
> +		return IGT_EXIT_FAILURE;
> +
> +	return IGT_EXIT_SUCCESS;
> +}
> diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> new file mode 100644
> index 00000000..b2f69df2
> --- /dev/null
> +++ b/lib/igt_ktap.h
> @@ -0,0 +1,31 @@
> +/*
> + * Copyright © 2022 Isabella Basso do Amaral <isabbasso@riseup.net>
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + */
> +
> +#ifndef IGT_KTAP_H
> +#define IGT_KTAP_H
> +
> +#define BUF_LEN 4096
> +
> +int igt_ktap_parser(FILE *fp, char *record, bool is_builtin);
> +
> +#endif /* IGT_KTAP_H */
> diff --git a/lib/meson.build b/lib/meson.build
> index c5131d9a..58cd6f44 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -87,6 +87,7 @@ lib_sources = [
>  	'igt_store.c',
>  	'uwildmat/uwildmat.c',
>  	'igt_kmod.c',
> +	'igt_ktap.c',
>  	'igt_panfrost.c',
>  	'igt_v3d.c',
>  	'igt_vc4.c',
> -- 
> 2.34.1
> 

Re: [igt-dev] [PATCH i-g-t 3/4] lib/igt_kmod: add compatibility for KUnit

[Janusz Krzysztofik]

On Friday, 3 March 2023 13:23:35 CET Petri Latvala wrote:
> On Fri, Mar 03, 2023 at 12:07:14PM +0100, Dominik Karol Piatkowski wrote:
> > From: Isabella Basso <isabbasso@riseup.net>
> > 
> > This adds functions for both executing the tests as well as parsing (K)TAP
> > kmsg output, as per the KTAP spec [1].
> > 
> > [1] https://www.kernel.org/doc/html/latest/dev-tools/ktap.html
> > 
> > v1 -> v2:
> > - refactor igt_kunit function and ktap parser so that we have only one
> >   parser that we call only once (code size is now less than half the
> >   size as v1)
> > - add lookup_value helper
> > - fix parsing problems
> > v2 -> v3:
> > - move ktap parsing functions to own file
> > - rename to ktap_parser
> > - get rid of unneeded pointers in igt_kunit
> > - change return values to allow for subsequent call to igt_kselftests if
> >   needed
> > - add docs to parsing functions and helpers
> > - switch to line buffering
> > - add line buffering logging helper
> > - fix kunit module handling
> > - fix parsing of version lines
> > - use igt_subtest blocks to improve output handling on the CI
> > - fix output handling during crashes
> > 
> > Signed-off-by: Isabella Basso <isabbasso@riseup.net>
> > 
> > v3 -> v4:
> > - handle igt_ktap_parser fail with IGT_EXIT_ABORT code
> > 
> > Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
> > ---
> >  lib/igt_kmod.c  |  84 ++++++++++++
> >  lib/igt_kmod.h  |   2 +
> >  lib/igt_ktap.c  | 334 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  lib/igt_ktap.h  |  31 +++++
> >  lib/meson.build |   1 +
> >  5 files changed, 452 insertions(+)
> >  create mode 100644 lib/igt_ktap.c
> >  create mode 100644 lib/igt_ktap.h
> > 
> > diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> > index fbda1aa6..a156e43d 100644
> > --- a/lib/igt_kmod.c
> > +++ b/lib/igt_kmod.c
> > @@ -29,6 +29,7 @@
> >  #include "igt_aux.h"
> >  #include "igt_core.h"
> >  #include "igt_kmod.h"
> > +#include "igt_ktap.h"
> >  #include "igt_sysfs.h"
> >  #include "igt_taints.h"
> >  
> > @@ -744,6 +745,89 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
> >  	kmod_module_info_free_list(pre);
> >  }
> >  
> > +/**
> > + * igt_kunit:
> > + * @module_name: the name of the module
> > + * @opts: options to load the module
> > + *
> > + * Loads the test module, parses its (k)tap dmesg output, then unloads it
> > + *
> > + * Returns: IGT default codes
> > + */
> > +int igt_kunit(const char *module_name, const char *opts)
> > +{
> > +	struct igt_ktest tst;
> > +	struct kmod_module *kunit_kmod;
> > +	char record[BUF_LEN + 1];
> > +	FILE *f;
> > +	bool is_builtin;
> > +	int ret;
> > +
> > +	ret = IGT_EXIT_INVALID;
> > +
> > +	/* get normalized module name */
> > +	if (igt_ktest_init(&tst, module_name) != 0) {
> > +		igt_warn("Unable to initialize ktest for %s\n", module_name);
> > +		return ret;
> > +	}
> > +
> > +	if (igt_ktest_begin(&tst) != 0) {
> > +		igt_warn("Unable to begin ktest for %s\n", module_name);
> > +
> > +		igt_ktest_fini(&tst);
> > +		return ret;
> > +	}
> > +
> > +	if (tst.kmsg < 0) {
> > +		igt_warn("Could not open /dev/kmsg");
> > +		goto unload;
> > +	}
> > +
> > +	if (lseek(tst.kmsg, 0, SEEK_END)) {
> > +		igt_warn("Could not seek the end of /dev/kmsg");
> > +		goto unload;
> > +	}
> > +
> > +	f = fdopen(tst.kmsg, "r");
> > +
> > +	if (f == NULL) {
> > +		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer");
> > +		goto unload;
> > +	}
> > +
> > +	if (setvbuf(f, record, _IOLBF, BUF_LEN)) {
> > +		igt_warn("Could not set line buffering on /dev/kmsg");
> > +		goto unload;
> > +	}
> 
> Make sure all your igt_warn()s end with a newline.
> 
> There seems to be some kind of misunderstanding with how setvbuf
> works. Here a buffer is given to setvbuf() so that fgets() in the
> parser can read one line at a time, but then that's also reading into
> the same buffer. According to setvbuf() docs you're not supposed to
> touch the buffer yourself.
> 
> I vaguely remember someone suggesting line-buffered IO for reading
> /dev/kmsg but can't find that message now. What was the reason for it?
> Using read() on /dev/kmsg already gives only one log record at a time.

That was me, in reaction to previous implementation, based on read(), but not 
looking good to me, see my comments to
https://patchwork.freedesktop.org/patch/499945/?series=105294&rev=2
I had no idea that read() on /dev/kmsg behaves so friendly.
 
...
> > +
> > +/**
> > + * igt_ktap_parser:
> > + * @fp: FILE pointer
> > + * @record: buffer used to read fp
> > + * @is_builtin: whether the KUnit module is built-in or not
> > + *
> > + * This function parses the output of a ktap script and prints the test results,
> > + * as well as any other output to stdout.
> > + *
> > + * Returns: IGT default codes
> > + */
> > +int igt_ktap_parser(FILE *fp, char *record, bool is_builtin)
> > +{
> > +	char test_name[BUF_LEN + 1];
> > +	bool failed_tests, found_tests;
> > +	int sublevel = 0;
> > +
> > +	test_name[0] = '\0';
> > +	test_name[BUF_LEN] = '\0';
> > +
> > +	failed_tests = false;
> > +	found_tests = false;
> > +
> > +	while (sublevel >= 0) {
> > +		if (fgets(record, BUF_LEN, fp) == NULL) {
> > +			if (!found_tests)
> > +				igt_warn("kmsg truncated: unknown error (%m)\n");
> > +			break;
> > +		}
> > +
> > +		switch (find_next_tap_subtest(fp, record, is_builtin)) {
> > +		case -2:
> > +			/* no more data to read */
> > +			return IGT_EXIT_FAILURE;
> > +		case -1:
> > +			/* no test found, so we keep parsing */
> > +			break;
> > +		case 0:
> > +			/*
> > +			 * tests found, but they're missing info, so we might
> > +			 * have read into test output
> > +			 */
> > +			found_tests = true;
> > +			sublevel++;
> > +			break;
> > +		default:
> > +			if (fgets(record, BUF_LEN, fp) == NULL) {
> > +				igt_warn("kmsg truncated: unknown error (%m)\n");
> > +				return -2;
> > +			}
> > +			found_tests = true;
> > +			sublevel++;
> > +			break;
> > +		}
> > +
> > +		switch (parse_kmsg_for_tap(fp, record, test_name)) {
> > +		case -2:
> > +			return IGT_EXIT_FAILURE;
> > +		case -1:
> > +			sublevel--;
> > +			failed_tests = true;
> > +			igt_subtest(test_name)
> > +				igt_fail(IGT_EXIT_FAILURE);
> > +			test_name[0] = '\0';
> > +			break;
> > +		case 0: /* fallthrough */
> > +			igt_subtest(test_name)
> > +				igt_success();
> > +			test_name[0] = '\0';
> > +		default:
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (failed_tests || !found_tests)
> > +		return IGT_EXIT_FAILURE;
> > +
> > +	return IGT_EXIT_SUCCESS;

I think that instead of returning just one total result, the ktap parser could
better be a background process providing us with results of every subtest it 
sees being executed, which we then read and handle on the fly like if those 
were IGT dynamic sub-subtests we execute, and their results.

Thanks,
Janusz

> > +}
> > diff --git a/lib/igt_ktap.h b/lib/igt_ktap.h
> > new file mode 100644
> > index 00000000..b2f69df2
> > --- /dev/null
> > +++ b/lib/igt_ktap.h
> > @@ -0,0 +1,31 @@
> > +/*
> > + * Copyright © 2022 Isabella Basso do Amaral <isabbasso@riseup.net>
> > + *
> > + * Permission is hereby granted, free of charge, to any person obtaining a
> > + * copy of this software and associated documentation files (the "Software"),
> > + * to deal in the Software without restriction, including without limitation
> > + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> > + * and/or sell copies of the Software, and to permit persons to whom the
> > + * Software is furnished to do so, subject to the following conditions:
> > + *
> > + * The above copyright notice and this permission notice (including the next
> > + * paragraph) shall be included in all copies or substantial portions of the
> > + * Software.
> > + *
> > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> > + * IN THE SOFTWARE.
> > + */
> > +
> > +#ifndef IGT_KTAP_H
> > +#define IGT_KTAP_H
> > +
> > +#define BUF_LEN 4096
> > +
> > +int igt_ktap_parser(FILE *fp, char *record, bool is_builtin);
> > +
> > +#endif /* IGT_KTAP_H */
> > diff --git a/lib/meson.build b/lib/meson.build
> > index c5131d9a..58cd6f44 100644
> > --- a/lib/meson.build
> > +++ b/lib/meson.build
> > @@ -87,6 +87,7 @@ lib_sources = [
> >  	'igt_store.c',
> >  	'uwildmat/uwildmat.c',
> >  	'igt_kmod.c',
> > +	'igt_ktap.c',
> >  	'igt_panfrost.c',
> >  	'igt_v3d.c',
> >  	'igt_vc4.c',
> 

Re: [igt-dev] [PATCH i-g-t 3/4] lib/igt_kmod: add compatibility for KUnit

[Mauro Carvalho Chehab]

On Fri,  3 Mar 2023 12:07:14 +0100
Dominik Karol Piatkowski <dominik.karol.piatkowski@intel.com> wrote:

> From: Isabella Basso <isabbasso@riseup.net>
> 
> This adds functions for both executing the tests as well as parsing (K)TAP
> kmsg output, as per the KTAP spec [1].
> 
> [1] https://www.kernel.org/doc/html/latest/dev-tools/ktap.html
> 
> v1 -> v2:
> - refactor igt_kunit function and ktap parser so that we have only one
>   parser that we call only once (code size is now less than half the
>   size as v1)
> - add lookup_value helper
> - fix parsing problems
> v2 -> v3:
> - move ktap parsing functions to own file
> - rename to ktap_parser
> - get rid of unneeded pointers in igt_kunit
> - change return values to allow for subsequent call to igt_kselftests if
>   needed
> - add docs to parsing functions and helpers
> - switch to line buffering
> - add line buffering logging helper
> - fix kunit module handling
> - fix parsing of version lines
> - use igt_subtest blocks to improve output handling on the CI
> - fix output handling during crashes
> 
> Signed-off-by: Isabella Basso <isabbasso@riseup.net>
> 
> v3 -> v4:
> - handle igt_ktap_parser fail with IGT_EXIT_ABORT code
> 
> Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
> ---
>  lib/igt_kmod.c  |  84 ++++++++++++
>  lib/igt_kmod.h  |   2 +
>  lib/igt_ktap.c  | 334 ++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/igt_ktap.h  |  31 +++++
>  lib/meson.build |   1 +
>  5 files changed, 452 insertions(+)
>  create mode 100644 lib/igt_ktap.c
>  create mode 100644 lib/igt_ktap.h
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index fbda1aa6..a156e43d 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -29,6 +29,7 @@
>  #include "igt_aux.h"
>  #include "igt_core.h"
>  #include "igt_kmod.h"
> +#include "igt_ktap.h"
>  #include "igt_sysfs.h"
>  #include "igt_taints.h"
>  
> @@ -744,6 +745,89 @@ void igt_kselftest_get_tests(struct kmod_module *kmod,
>  	kmod_module_info_free_list(pre);
>  }
>  
> +/**
> + * igt_kunit:
> + * @module_name: the name of the module
> + * @opts: options to load the module
> + *
> + * Loads the test module, parses its (k)tap dmesg output, then unloads it
> + *
> + * Returns: IGT default codes
> + */
> +int igt_kunit(const char *module_name, const char *opts)
> +{
> +	struct igt_ktest tst;
> +	struct kmod_module *kunit_kmod;
> +	char record[BUF_LEN + 1];
> +	FILE *f;
> +	bool is_builtin;
> +	int ret;
> +
> +	ret = IGT_EXIT_INVALID;
> +
> +	/* get normalized module name */
> +	if (igt_ktest_init(&tst, module_name) != 0) {
> +		igt_warn("Unable to initialize ktest for %s\n", module_name);
> +		return ret;
> +	}
> +
> +	if (igt_ktest_begin(&tst) != 0) {
> +		igt_warn("Unable to begin ktest for %s\n", module_name);
> +
> +		igt_ktest_fini(&tst);
> +		return ret;
> +	}
> +
> +	if (tst.kmsg < 0) {
> +		igt_warn("Could not open /dev/kmsg");
> +		goto unload;
> +	}
> +
> +	if (lseek(tst.kmsg, 0, SEEK_END)) {
> +		igt_warn("Could not seek the end of /dev/kmsg");
> +		goto unload;
> +	}
> +
> +	f = fdopen(tst.kmsg, "r");
> +
> +	if (f == NULL) {
> +		igt_warn("Could not turn /dev/kmsg file descriptor into a FILE pointer");
> +		goto unload;
> +	}
> +
> +	if (setvbuf(f, record, _IOLBF, BUF_LEN)) {
> +		igt_warn("Could not set line buffering on /dev/kmsg");
> +		goto unload;
> +	}
> +
> +	/* The KUnit module is required for running any KUnit tests */
> +	if (igt_kmod_load("kunit", NULL) != 0 ||
> +	    kmod_module_new_from_name(kmod_ctx(), "kunit", &kunit_kmod) != 0) {
> +		igt_warn("Unable to load KUnit\n");
> +		igt_fail(IGT_EXIT_FAILURE);
> +	}
> +
> +	is_builtin = kmod_module_get_initstate(kunit_kmod) == KMOD_MODULE_BUILTIN;
> +
> +	if (igt_kmod_load(module_name, opts) != 0) {
> +		igt_warn("Unable to load %s module\n", module_name);
> +		igt_fail(IGT_EXIT_FAILURE);
> +	}
> +
> +	ret = igt_ktap_parser(f, record, is_builtin);
> +	if (ret != 0)
> +		ret = IGT_EXIT_ABORT;


No. you need to actually start monitoring /dev/kmsg before loading
the module. The ktap parser should be listening and parsing the messages
as the tests are executed, and not after the end of module load.

If you don't do that:

1. the test results will only be displayed after the end of all tests;
2. if a crash happens during a test, the log output will be lost.

In order to preserver Isabella's authorship, my suggestion is to
write a patch after this one changing the logic of the ktap parser
for it to run on a thread, started before loading the module, and
finished after the end of the module probe.

Regards,
Mauro

[igt-dev] [PATCH i-g-t 4/4] tests: DRM selftests: switch to KUnit

[Dominik Karol Piatkowski]

From: Isabella Basso <isabbasso@riseup.net>

As the DRM selftests are now using KUnit [1], update IGT tests as well.

[1] - https://lore.kernel.org/all/20220708203052.236290-1-maira.canal@usp.br/

Signed-off-by: Isabella Basso <isabbasso@riseup.net>

v1 -> v2:
- drm_buddy|drm_mm: fallback to igt_kselftests if igt_kunit failed
  with code other than IGT_EXIT_ABORT
- kms_selftest: move igt_kunit tests to separate subtests
- kms_selftest: fallback to igt_kselftests if all subtests failed

Co-authored-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
---
 tests/drm_buddy.c    |  4 +++-
 tests/drm_mm.c       |  4 +++-
 tests/kms_selftest.c | 17 ++++++++++++++++-
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/tests/drm_buddy.c b/tests/drm_buddy.c
index 06876e0c..3261f0d6 100644
--- a/tests/drm_buddy.c
+++ b/tests/drm_buddy.c
@@ -10,5 +10,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's buddy allocator (struct drm_bu
 
 igt_main
 {
-	igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
+	int ret = igt_kunit("drm_buddy_test", NULL);
+	if (ret != 0 && ret != IGT_EXIT_ABORT)
+		igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
 }
diff --git a/tests/drm_mm.c b/tests/drm_mm.c
index 2052b115..46d0142f 100644
--- a/tests/drm_mm.c
+++ b/tests/drm_mm.c
@@ -28,5 +28,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's range manager (struct drm_mm)"
 
 igt_main
 {
-	igt_kselftests("test-drm_mm", NULL, NULL, NULL);
+	int ret = igt_kunit("drm_mm_test", NULL);
+	if (ret != 0 && ret != IGT_EXIT_ABORT)
+		igt_kselftests("test-drm_mm", NULL, NULL, NULL);
 }
diff --git a/tests/kms_selftest.c b/tests/kms_selftest.c
index abc4bfe9..88d48545 100644
--- a/tests/kms_selftest.c
+++ b/tests/kms_selftest.c
@@ -28,5 +28,20 @@ IGT_TEST_DESCRIPTION("Basic sanity check of KMS selftests.");
 
 igt_main
 {
-	igt_kselftests("test-drm_modeset", NULL, NULL, NULL);
+	int ret;
+	int passed = 0;
+	static const char *kunit_subtests[] = { "drm_cmdline_parser_test", "drm_damage_helper_test",
+						"drm_dp_mst_helper_test", "drm_format_helper_test",
+						"drm_format_test", "drm_framebuffer_test",
+						"drm_plane_helper_test", NULL };
+
+	for (int i = 0; kunit_subtests[i] != NULL; i++)
+		igt_subtest(kunit_subtests[i]) {
+			ret = igt_kunit(kunit_subtests[i], NULL);
+			passed += (ret == 0);
+			igt_assert(ret == 0);
+		}
+
+	if (passed == 0)
+		igt_kselftests("test-drm_modeset", NULL, NULL, NULL);
 }
-- 
2.34.1

Re: [igt-dev] [PATCH i-g-t 4/4] tests: DRM selftests: switch to KUnit

[Janusz Krzysztofik]

On Friday, 3 March 2023 12:07:15 CET Dominik Karol Piatkowski wrote:
> From: Isabella Basso <isabbasso@riseup.net>
> 
> As the DRM selftests are now using KUnit [1], update IGT tests as well.
> 
> [1] - https://lore.kernel.org/all/20220708203052.236290-1-maira.canal@usp.br/
> 
> Signed-off-by: Isabella Basso <isabbasso@riseup.net>
> 
> v1 -> v2:
> - drm_buddy|drm_mm: fallback to igt_kselftests if igt_kunit failed
>   with code other than IGT_EXIT_ABORT
> - kms_selftest: move igt_kunit tests to separate subtests
> - kms_selftest: fallback to igt_kselftests if all subtests failed
> 
> Co-authored-by: Dominik Karol Piątkowski 
<dominik.karol.piatkowski@intel.com>
> ---
>  tests/drm_buddy.c    |  4 +++-
>  tests/drm_mm.c       |  4 +++-
>  tests/kms_selftest.c | 17 ++++++++++++++++-
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/drm_buddy.c b/tests/drm_buddy.c
> index 06876e0c..3261f0d6 100644
> --- a/tests/drm_buddy.c
> +++ b/tests/drm_buddy.c
> @@ -10,5 +10,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's buddy 
allocator (struct drm_bu
>  
>  igt_main
>  {
> -	igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
> +	int ret = igt_kunit("drm_buddy_test", NULL);
> +	if (ret != 0 && ret != IGT_EXIT_ABORT)
> +		igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
>  }
> diff --git a/tests/drm_mm.c b/tests/drm_mm.c
> index 2052b115..46d0142f 100644
> --- a/tests/drm_mm.c
> +++ b/tests/drm_mm.c
> @@ -28,5 +28,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's range 
manager (struct drm_mm)"
>  
>  igt_main
>  {
> -	igt_kselftests("test-drm_mm", NULL, NULL, NULL);
> +	int ret = igt_kunit("drm_mm_test", NULL);
> +	if (ret != 0 && ret != IGT_EXIT_ABORT)
> +		igt_kselftests("test-drm_mm", NULL, NULL, NULL);
>  }
> diff --git a/tests/kms_selftest.c b/tests/kms_selftest.c
> index abc4bfe9..88d48545 100644
> --- a/tests/kms_selftest.c
> +++ b/tests/kms_selftest.c
> @@ -28,5 +28,20 @@ IGT_TEST_DESCRIPTION("Basic sanity check of KMS 
selftests.");
>  
>  igt_main
>  {
> -	igt_kselftests("test-drm_modeset", NULL, NULL, NULL);
> +	int ret;
> +	int passed = 0;
> +	static const char *kunit_subtests[] = { "drm_cmdline_parser_test", 
"drm_damage_helper_test",
> +						
"drm_dp_mst_helper_test", "drm_format_helper_test",
> +						
"drm_format_test", "drm_framebuffer_test",
> +						
"drm_plane_helper_test", NULL };
> +
> +	for (int i = 0; kunit_subtests[i] != NULL; i++)
> +		igt_subtest(kunit_subtests[i]) {
> +			ret = igt_kunit(kunit_subtests[i], NULL);
> +			passed += (ret == 0);
> +			igt_assert(ret == 0);
> +		}
> +
> +	if (passed == 0)
> +		igt_kselftests("test-drm_modeset", NULL, NULL, NULL);

I think that's not correct.  Since igt_kselftests() exposes a subtest (with 
dynamic sub-subtests), and CI expects a static list of subtests, that function 
shouldn't be called conditionally, I believe.

I think we have two options:
1) both igt_kselftests() and igt_kunit() expose a subtest which skips if no 
   requested module is found, 
or
2) a wrapper calls either igt_kunit() or igt_kselftests(), based on which 
   modules are present.

Thanks,
Janusz
  
>  }
> 

Re: [igt-dev] [PATCH i-g-t 4/4] tests: DRM selftests: switch to KUnit

[Janusz Krzysztofik]

On Friday, 3 March 2023 16:08:33 CET Janusz Krzysztofik wrote:
> On Friday, 3 March 2023 12:07:15 CET Dominik Karol Piatkowski wrote:
> > From: Isabella Basso <isabbasso@riseup.net>
> > 
> > As the DRM selftests are now using KUnit [1], update IGT tests as well.
> > 
> > [1] - https://lore.kernel.org/all/20220708203052.236290-1-maira.canal@usp.br/
> > 
> > Signed-off-by: Isabella Basso <isabbasso@riseup.net>
> > 
> > v1 -> v2:
> > - drm_buddy|drm_mm: fallback to igt_kselftests if igt_kunit failed
> >   with code other than IGT_EXIT_ABORT
> > - kms_selftest: move igt_kunit tests to separate subtests
> > - kms_selftest: fallback to igt_kselftests if all subtests failed
> > 
> > Co-authored-by: Dominik Karol Piątkowski 
> <dominik.karol.piatkowski@intel.com>
> > ---
> >  tests/drm_buddy.c    |  4 +++-
> >  tests/drm_mm.c       |  4 +++-
> >  tests/kms_selftest.c | 17 ++++++++++++++++-
> >  3 files changed, 22 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tests/drm_buddy.c b/tests/drm_buddy.c
> > index 06876e0c..3261f0d6 100644
> > --- a/tests/drm_buddy.c
> > +++ b/tests/drm_buddy.c
> > @@ -10,5 +10,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's buddy 
> allocator (struct drm_bu
> >  
> >  igt_main
> >  {
> > -	igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
> > +	int ret = igt_kunit("drm_buddy_test", NULL);
> > +	if (ret != 0 && ret != IGT_EXIT_ABORT)
> > +		igt_kselftests("test-drm_buddy", NULL, NULL, NULL);
> >  }
> > diff --git a/tests/drm_mm.c b/tests/drm_mm.c
> > index 2052b115..46d0142f 100644
> > --- a/tests/drm_mm.c
> > +++ b/tests/drm_mm.c
> > @@ -28,5 +28,7 @@ IGT_TEST_DESCRIPTION("Basic sanity check of DRM's range 
> manager (struct drm_mm)"
> >  
> >  igt_main
> >  {
> > -	igt_kselftests("test-drm_mm", NULL, NULL, NULL);
> > +	int ret = igt_kunit("drm_mm_test", NULL);
> > +	if (ret != 0 && ret != IGT_EXIT_ABORT)
> > +		igt_kselftests("test-drm_mm", NULL, NULL, NULL);
> >  }
> > diff --git a/tests/kms_selftest.c b/tests/kms_selftest.c
> > index abc4bfe9..88d48545 100644
> > --- a/tests/kms_selftest.c
> > +++ b/tests/kms_selftest.c
> > @@ -28,5 +28,20 @@ IGT_TEST_DESCRIPTION("Basic sanity check of KMS 
> selftests.");
> >  
> >  igt_main
> >  {
> > -	igt_kselftests("test-drm_modeset", NULL, NULL, NULL);
> > +	int ret;
> > +	int passed = 0;
> > +	static const char *kunit_subtests[] = { "drm_cmdline_parser_test", 
> "drm_damage_helper_test",
> > +						
> "drm_dp_mst_helper_test", "drm_format_helper_test",
> > +						
> "drm_format_test", "drm_framebuffer_test",
> > +						
> "drm_plane_helper_test", NULL };
> > +
> > +	for (int i = 0; kunit_subtests[i] != NULL; i++)
> > +		igt_subtest(kunit_subtests[i]) {
> > +			ret = igt_kunit(kunit_subtests[i], NULL);
> > +			passed += (ret == 0);
> > +			igt_assert(ret == 0);
> > +		}
> > +
> > +	if (passed == 0)
> > +		igt_kselftests("test-drm_modeset", NULL, NULL, NULL);
> 
> I think that's not correct.  Since igt_kselftests() exposes a subtest (with 
> dynamic sub-subtests), and CI expects a static list of subtests, that function 
> shouldn't be called conditionally, I believe.
> 
> I think we have two options:
> 1) both igt_kselftests() and igt_kunit() expose a subtest which skips if no 
>    requested module is found, 
> or
> 2) a wrapper calls either igt_kunit() or igt_kselftests(), based on which 
>    modules are present.

The wrapper provides the subtest, while igt_kunit() and igt_kselftests() 
provide dynamic sub-subtests.  The wrapper likely doesn't need to look for 
modules' presence.

Thanks,
Janusz

> 
> Thanks,
> Janusz
>   
> >  }
> > 
> 
> 

[igt-dev] ✓ Fi.CI.BAT: success for Introduce KUnit

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 4789 bytes --]

== Series Details ==

Series: Introduce KUnit
URL   : https://patchwork.freedesktop.org/series/114612/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12806 -> IGTPW_8558
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html

Participating hosts (39 -> 39)
------------------------------

  No changes in participating hosts

Known issues
------------

  Here are the changes found in IGTPW_8558 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_exec_suspend@basic-s0@smem:
    - bat-jsl-3:          [PASS][1] -> [ABORT][2] ([i915#5122])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-jsl-3/igt@gem_exec_suspend@basic-s0@smem.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-jsl-3/igt@gem_exec_suspend@basic-s0@smem.html

  * igt@i915_selftest@live@reset:
    - bat-rpls-2:         NOTRUN -> [ABORT][3] ([i915#4983] / [i915#7913])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-rpls-2/igt@i915_selftest@live@reset.html

  * igt@i915_suspend@basic-s2idle-without-i915:
    - bat-atsm-1:         [PASS][4] -> [ABORT][5] ([i915#8174])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-atsm-1/igt@i915_suspend@basic-s2idle-without-i915.html
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-atsm-1/igt@i915_suspend@basic-s2idle-without-i915.html

  * igt@i915_suspend@basic-s3-without-i915:
    - bat-jsl-3:          [PASS][6] -> [FAIL][7] ([fdo#103375])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-jsl-3/igt@i915_suspend@basic-s3-without-i915.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-jsl-3/igt@i915_suspend@basic-s3-without-i915.html

  
#### Possible fixes ####

  * igt@i915_selftest@live@gt_heartbeat:
    - fi-apl-guc:         [DMESG-FAIL][8] ([i915#5334]) -> [PASS][9]
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html

  * igt@i915_selftest@live@requests:
    - bat-rpls-2:         [ABORT][10] ([i915#4983] / [i915#7694] / [i915#7913]) -> [PASS][11]
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-rpls-2/igt@i915_selftest@live@requests.html
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-rpls-2/igt@i915_selftest@live@requests.html

  * igt@i915_selftest@live@slpc:
    - bat-rpls-1:         [DMESG-FAIL][12] ([i915#6367] / [i915#6997]) -> [PASS][13]
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-rpls-1/igt@i915_selftest@live@slpc.html
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-rpls-1/igt@i915_selftest@live@slpc.html

  * igt@i915_selftest@live@workarounds:
    - bat-dg2-11:         [DMESG-WARN][14] ([i915#7102]) -> [PASS][15]
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/bat-dg2-11/igt@i915_selftest@live@workarounds.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/bat-dg2-11/igt@i915_selftest@live@workarounds.html

  
  [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
  [i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
  [i915#5122]: https://gitlab.freedesktop.org/drm/intel/issues/5122
  [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
  [i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
  [i915#6997]: https://gitlab.freedesktop.org/drm/intel/issues/6997
  [i915#7102]: https://gitlab.freedesktop.org/drm/intel/issues/7102
  [i915#7694]: https://gitlab.freedesktop.org/drm/intel/issues/7694
  [i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
  [i915#8174]: https://gitlab.freedesktop.org/drm/intel/issues/8174


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7180 -> IGTPW_8558

  CI-20190529: 20190529
  CI_DRM_12806: fc05dcda11c98d438e65280e949332cd86c22522 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8558: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html
  IGT_7180: b6b1ceb4fc4d8f83a9540e4628a6c2648514bb19 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git


Testlist changes
----------------

+igt@kms_selftest@drm_cmdline_parser_test
+igt@kms_selftest@drm_damage_helper_test
+igt@kms_selftest@drm_dp_mst_helper_test
+igt@kms_selftest@drm_format_helper_test
+igt@kms_selftest@drm_format_test
+igt@kms_selftest@drm_framebuffer_test
+igt@kms_selftest@drm_plane_helper_test
-igt@drm_buddy@all-tests
-igt@drm_mm@all-tests

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html

[-- Attachment #2: Type: text/html, Size: 5747 bytes --]

[igt-dev] ✗ GitLab.Pipeline: warning for Introduce KUnit

[Patchwork]

== Series Details ==

Series: Introduce KUnit
URL   : https://patchwork.freedesktop.org/series/114612/
State : warning

== Summary ==

Pipeline status: FAILED.

see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/821058 for the overview.

test:ninja-test has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/37408049):
  348/352 assembler test/rnde-intsrc              OK       0.02 s 
  349/352 assembler test/rndz                     OK       0.03 s 
  350/352 assembler test/lzd                      OK       0.02 s 
  351/352 assembler test/not                      OK       0.03 s 
  352/352 assembler test/immediate                OK       0.02 s 
  
  Ok:                  346
  Expected Fail:         4
  Fail:                  2
  Unexpected Pass:       0
  Skipped:               0
  Timeout:               0
  
  Full log written to /builds/gfx-ci/igt-ci-tags/build/meson-logs/testlog.txt
  section_end:1677844730:step_script
  section_start:1677844730:cleanup_file_variables
  Cleaning up project directory and file based variables
  section_end:1677844730:cleanup_file_variables
  ERROR: Job failed: exit code 1
  

test:ninja-test-clang has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/37408136):
  348/352 assembler test/rnde-intsrc              OK       0.03 s 
  349/352 assembler test/rndz                     OK       0.03 s 
  350/352 assembler test/lzd                      OK       0.03 s 
  351/352 assembler test/not                      OK       0.03 s 
  352/352 assembler test/immediate                OK       0.03 s 
  
  Ok:                  346
  Expected Fail:         4
  Fail:                  2
  Unexpected Pass:       0
  Skipped:               0
  Timeout:               0
  
  Full log written to /builds/gfx-ci/igt-ci-tags/build/meson-logs/testlog.txt
  section_end:1677844949:step_script
  section_start:1677844949:cleanup_file_variables
  Cleaning up project directory and file based variables
  section_end:1677844949:cleanup_file_variables
  ERROR: Job failed: exit code 1

== Logs ==

For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/821058

[igt-dev] ✓ Fi.CI.IGT: success for Introduce KUnit

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 24810 bytes --]

== Series Details ==

Series: Introduce KUnit
URL   : https://patchwork.freedesktop.org/series/114612/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12806_full -> IGTPW_8558_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html

Participating hosts (9 -> 9)
------------------------------

  No changes in participating hosts

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_8558_full:

### IGT changes ###

#### Possible regressions ####

  * {igt@kms_selftest@drm_format_helper_test} (NEW):
    - shard-apl:          NOTRUN -> [FAIL][1] +4 similar issues
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@kms_selftest@drm_format_helper_test.html

  * {igt@kms_selftest@drm_format_test} (NEW):
    - {shard-dg1}:        NOTRUN -> [FAIL][2] +1 similar issue
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-dg1-15/igt@kms_selftest@drm_format_test.html

  * {igt@kms_selftest@drm_framebuffer_test} (NEW):
    - shard-snb:          NOTRUN -> [FAIL][3] +3 similar issues
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-snb6/igt@kms_selftest@drm_framebuffer_test.html
    - {shard-tglu}:       NOTRUN -> [FAIL][4] +1 similar issue
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-tglu-3/igt@kms_selftest@drm_framebuffer_test.html
    - shard-glk:          NOTRUN -> [FAIL][5] +4 similar issues
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk5/igt@kms_selftest@drm_framebuffer_test.html

  * {igt@kms_selftest@drm_plane_helper_test} (NEW):
    - {shard-rkl}:        NOTRUN -> [FAIL][6] +4 similar issues
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-4/igt@kms_selftest@drm_plane_helper_test.html

  
#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * {igt@kms_plane_scaling@i915-max-source-size}:
    - {shard-dg1}:        NOTRUN -> [SKIP][7]
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-dg1-13/igt@kms_plane_scaling@i915-max-source-size.html

  
New tests
---------

  New tests have been introduced between CI_DRM_12806_full and IGTPW_8558_full:

### New IGT tests (7) ###

  * igt@kms_selftest@drm_cmdline_parser_test:
    - Statuses : 4 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_damage_helper_test:
    - Statuses : 4 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_dp_mst_helper_test:
    - Statuses : 3 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_format_helper_test:
    - Statuses : 2 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_format_test:
    - Statuses : 1 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_framebuffer_test:
    - Statuses : 5 fail(s)
    - Exec time: [0.0] s

  * igt@kms_selftest@drm_plane_helper_test:
    - Statuses : 4 fail(s)
    - Exec time: [0.0] s

  

Known issues
------------

  Here are the changes found in IGTPW_8558_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_persistence@legacy-engines-queued:
    - shard-snb:          NOTRUN -> [SKIP][8] ([fdo#109271] / [i915#1099]) +7 similar issues
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-snb4/igt@gem_ctx_persistence@legacy-engines-queued.html

  * igt@gem_exec_fair@basic-none-solo@rcs0:
    - shard-apl:          [PASS][9] -> [FAIL][10] ([i915#2842])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-apl1/igt@gem_exec_fair@basic-none-solo@rcs0.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl1/igt@gem_exec_fair@basic-none-solo@rcs0.html

  * igt@gem_exec_fair@basic-none@rcs0:
    - shard-glk:          NOTRUN -> [FAIL][11] ([i915#2842])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk4/igt@gem_exec_fair@basic-none@rcs0.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-glk:          [PASS][12] -> [FAIL][13] ([i915#2842])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-glk3/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk3/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_lmem_swapping@heavy-multi:
    - shard-apl:          NOTRUN -> [SKIP][14] ([fdo#109271] / [i915#4613]) +3 similar issues
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@gem_lmem_swapping@heavy-multi.html

  * igt@gem_lmem_swapping@heavy-verify-random-ccs:
    - shard-glk:          NOTRUN -> [SKIP][15] ([fdo#109271] / [i915#4613]) +2 similar issues
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk1/igt@gem_lmem_swapping@heavy-verify-random-ccs.html

  * igt@gem_render_copy@x-tiled-to-vebox-yf-tiled:
    - shard-apl:          NOTRUN -> [SKIP][16] ([fdo#109271]) +316 similar issues
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl1/igt@gem_render_copy@x-tiled-to-vebox-yf-tiled.html

  * igt@gem_userptr_blits@vma-merge:
    - shard-apl:          NOTRUN -> [FAIL][17] ([i915#3318])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@gem_userptr_blits@vma-merge.html

  * igt@gem_vm_create@invalid-create:
    - shard-snb:          NOTRUN -> [SKIP][18] ([fdo#109271]) +529 similar issues
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-snb6/igt@gem_vm_create@invalid-create.html

  * igt@gen9_exec_parse@allowed-all:
    - shard-apl:          [PASS][19] -> [ABORT][20] ([i915#5566])
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-apl1/igt@gen9_exec_parse@allowed-all.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl1/igt@gen9_exec_parse@allowed-all.html

  * igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-dp:
    - shard-apl:          NOTRUN -> [SKIP][21] ([fdo#109271] / [i915#1937])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-dp.html

  * igt@kms_big_fb@x-tiled-64bpp-rotate-90:
    - shard-glk:          NOTRUN -> [SKIP][22] ([fdo#109271]) +169 similar issues
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk4/igt@kms_big_fb@x-tiled-64bpp-rotate-90.html

  * igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
    - shard-apl:          NOTRUN -> [SKIP][23] ([fdo#109271] / [i915#3886]) +12 similar issues
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-b-bad-aux-stride-y_tiled_gen12_rc_ccs_cc:
    - shard-glk:          NOTRUN -> [SKIP][24] ([fdo#109271] / [i915#3886]) +7 similar issues
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk3/igt@kms_ccs@pipe-b-bad-aux-stride-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_content_protection@srm@pipe-a-dp-1:
    - shard-apl:          NOTRUN -> [TIMEOUT][25] ([i915#7173]) +2 similar issues
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl1/igt@kms_content_protection@srm@pipe-a-dp-1.html

  * igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2:
    - shard-glk:          [PASS][26] -> [FAIL][27] ([i915#2122])
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-glk1/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk1/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html

  * igt@kms_flip@2x-flip-vs-expired-vblank@ab-hdmi-a1-hdmi-a2:
    - shard-glk:          NOTRUN -> [FAIL][28] ([i915#79])
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk8/igt@kms_flip@2x-flip-vs-expired-vblank@ab-hdmi-a1-hdmi-a2.html

  * igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf:
    - shard-glk:          NOTRUN -> [SKIP][29] ([fdo#109271] / [i915#658]) +3 similar issues
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk5/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf.html

  * igt@kms_psr2_sf@cursor-plane-move-continuous-sf:
    - shard-apl:          NOTRUN -> [SKIP][30] ([fdo#109271] / [i915#658]) +3 similar issues
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl2/igt@kms_psr2_sf@cursor-plane-move-continuous-sf.html

  * igt@kms_writeback@writeback-invalid-parameters:
    - shard-apl:          NOTRUN -> [SKIP][31] ([fdo#109271] / [i915#2437])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl2/igt@kms_writeback@writeback-invalid-parameters.html
    - shard-glk:          NOTRUN -> [SKIP][32] ([fdo#109271] / [i915#2437])
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk8/igt@kms_writeback@writeback-invalid-parameters.html

  
#### Possible fixes ####

  * igt@gem_exec_fair@basic-deadline:
    - {shard-rkl}:        [FAIL][33] ([i915#2846]) -> [PASS][34]
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-rkl-2/igt@gem_exec_fair@basic-deadline.html
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-4/igt@gem_exec_fair@basic-deadline.html

  * igt@gem_exec_reloc@basic-softpin:
    - {shard-rkl}:        [SKIP][35] ([i915#3281]) -> [PASS][36] +1 similar issue
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-rkl-2/igt@gem_exec_reloc@basic-softpin.html
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-5/igt@gem_exec_reloc@basic-softpin.html

  * igt@gem_mmap_gtt@coherency:
    - {shard-rkl}:        [SKIP][37] ([fdo#111656]) -> [PASS][38]
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-rkl-2/igt@gem_mmap_gtt@coherency.html
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-5/igt@gem_mmap_gtt@coherency.html

  * igt@gem_readwrite@read-write:
    - {shard-rkl}:        [SKIP][39] ([i915#3282]) -> [PASS][40] +1 similar issue
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-rkl-2/igt@gem_readwrite@read-write.html
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-5/igt@gem_readwrite@read-write.html

  * igt@gen9_exec_parse@allowed-single:
    - shard-apl:          [ABORT][41] ([i915#5566]) -> [PASS][42]
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-apl1/igt@gen9_exec_parse@allowed-single.html
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl3/igt@gen9_exec_parse@allowed-single.html

  * igt@gen9_exec_parse@unaligned-jump:
    - {shard-rkl}:        [SKIP][43] ([i915#2527]) -> [PASS][44]
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-rkl-2/igt@gen9_exec_parse@unaligned-jump.html
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-rkl-5/igt@gen9_exec_parse@unaligned-jump.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait:
    - {shard-dg1}:        [SKIP][45] ([i915#1397]) -> [PASS][46]
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-dg1-14/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-dg1-15/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html

  * igt@i915_pm_rps@engine-order:
    - shard-apl:          [FAIL][47] ([i915#6537]) -> [PASS][48]
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-apl1/igt@i915_pm_rps@engine-order.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-apl1/igt@i915_pm_rps@engine-order.html

  * igt@i915_suspend@forcewake:
    - {shard-tglu}:       [ABORT][49] ([i915#5122] / [i915#5251]) -> [PASS][50]
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-tglu-3/igt@i915_suspend@forcewake.html
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-tglu-3/igt@i915_suspend@forcewake.html

  * igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip:
    - {shard-tglu}:       [SKIP][51] ([i915#1845] / [i915#7651]) -> [PASS][52] +11 similar issues
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-tglu-6/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-tglu-5/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
    - shard-glk:          [FAIL][53] ([i915#2346]) -> [PASS][54]
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-glk7/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-glk1/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html

  * igt@kms_cursor_legacy@short-flip-before-cursor-atomic-transitions:
    - {shard-tglu}:       [SKIP][55] ([i915#1845]) -> [PASS][56] +1 similar issue
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-tglu-6/igt@kms_cursor_legacy@short-flip-before-cursor-atomic-transitions.html
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-tglu-3/igt@kms_cursor_legacy@short-flip-before-cursor-atomic-transitions.html

  * igt@kms_frontbuffer_tracking@fbc-stridechange:
    - {shard-tglu}:       [SKIP][57] ([i915#1849]) -> [PASS][58]
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12806/shard-tglu-6/igt@kms_frontbuffer_tracking@fbc-stridechange.html
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/shard-tglu-7/igt@kms_frontbuffer_tracking@fbc-stridechange.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [IGT#2]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/2
  [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
  [fdo#109295]: https://bugs.freedesktop.org/show_bug.cgi?id=109295
  [fdo#109302]: https://bugs.freedesktop.org/show_bug.cgi?id=109302
  [fdo#109303]: https://bugs.freedesktop.org/show_bug.cgi?id=109303
  [fdo#109308]: https://bugs.freedesktop.org/show_bug.cgi?id=109308
  [fdo#109314]: https://bugs.freedesktop.org/show_bug.cgi?id=109314
  [fdo#109315]: https://bugs.freedesktop.org/show_bug.cgi?id=109315
  [fdo#109506]: https://bugs.freedesktop.org/show_bug.cgi?id=109506
  [fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#110542]: https://bugs.freedesktop.org/show_bug.cgi?id=110542
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111644]: https://bugs.freedesktop.org/show_bug.cgi?id=111644
  [fdo#111656]: https://bugs.freedesktop.org/show_bug.cgi?id=111656
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [fdo#112054]: https://bugs.freedesktop.org/show_bug.cgi?id=112054
  [fdo#112283]: https://bugs.freedesktop.org/show_bug.cgi?id=112283
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
  [i915#1257]: https://gitlab.freedesktop.org/drm/intel/issues/1257
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1722]: https://gitlab.freedesktop.org/drm/intel/issues/1722
  [i915#1755]: https://gitlab.freedesktop.org/drm/intel/issues/1755
  [i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
  [i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
  [i915#1849]: https://gitlab.freedesktop.org/drm/intel/issues/1849
  [i915#1850]: https://gitlab.freedesktop.org/drm/intel/issues/1850
  [i915#1902]: https://gitlab.freedesktop.org/drm/intel/issues/1902
  [i915#1937]: https://gitlab.freedesktop.org/drm/intel/issues/1937
  [i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2532]: https://gitlab.freedesktop.org/drm/intel/issues/2532
  [i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
  [i915#2582]: https://gitlab.freedesktop.org/drm/intel/issues/2582
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2681]: https://gitlab.freedesktop.org/drm/intel/issues/2681
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
  [i915#284]: https://gitlab.freedesktop.org/drm/intel/issues/284
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#3116]: https://gitlab.freedesktop.org/drm/intel/issues/3116
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
  [i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
  [i915#3318]: https://gitlab.freedesktop.org/drm/intel/issues/3318
  [i915#3323]: https://gitlab.freedesktop.org/drm/intel/issues/3323
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
  [i915#3469]: https://gitlab.freedesktop.org/drm/intel/issues/3469
  [i915#3528]: https://gitlab.freedesktop.org/drm/intel/issues/3528
  [i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
  [i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#3742]: https://gitlab.freedesktop.org/drm/intel/issues/3742
  [i915#3825]: https://gitlab.freedesktop.org/drm/intel/issues/3825
  [i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3952]: https://gitlab.freedesktop.org/drm/intel/issues/3952
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
  [i915#426]: https://gitlab.freedesktop.org/drm/intel/issues/426
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4387]: https://gitlab.freedesktop.org/drm/intel/issues/4387
  [i915#4525]: https://gitlab.freedesktop.org/drm/intel/issues/4525
  [i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
  [i915#4565]: https://gitlab.freedesktop.org/drm/intel/issues/4565
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
  [i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
  [i915#4833]: https://gitlab.freedesktop.org/drm/intel/issues/4833
  [i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
  [i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
  [i915#4880]: https://gitlab.freedesktop.org/drm/intel/issues/4880
  [i915#4881]: https://gitlab.freedesktop.org/drm/intel/issues/4881
  [i915#4884]: https://gitlab.freedesktop.org/drm/intel/issues/4884
  [i915#5122]: https://gitlab.freedesktop.org/drm/intel/issues/5122
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5251]: https://gitlab.freedesktop.org/drm/intel/issues/5251
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5288]: https://gitlab.freedesktop.org/drm/intel/issues/5288
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#5325]: https://gitlab.freedesktop.org/drm/intel/issues/5325
  [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
  [i915#5439]: https://gitlab.freedesktop.org/drm/intel/issues/5439
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5563]: https://gitlab.freedesktop.org/drm/intel/issues/5563
  [i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6227]: https://gitlab.freedesktop.org/drm/intel/issues/6227
  [i915#6245]: https://gitlab.freedesktop.org/drm/intel/issues/6245
  [i915#6247]: https://gitlab.freedesktop.org/drm/intel/issues/6247
  [i915#6248]: https://gitlab.freedesktop.org/drm/intel/issues/6248
  [i915#6268]: https://gitlab.freedesktop.org/drm/intel/issues/6268
  [i915#6301]: https://gitlab.freedesktop.org/drm/intel/issues/6301
  [i915#6334]: https://gitlab.freedesktop.org/drm/intel/issues/6334
  [i915#6335]: https://gitlab.freedesktop.org/drm/intel/issues/6335
  [i915#6344]: https://gitlab.freedesktop.org/drm/intel/issues/6344
  [i915#6433]: https://gitlab.freedesktop.org/drm/intel/issues/6433
  [i915#6493]: https://gitlab.freedesktop.org/drm/intel/issues/6493
  [i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
  [i915#6537]: https://gitlab.freedesktop.org/drm/intel/issues/6537
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6590]: https://gitlab.freedesktop.org/drm/intel/issues/6590
  [i915#6621]: https://gitlab.freedesktop.org/drm/intel/issues/6621
  [i915#6768]: https://gitlab.freedesktop.org/drm/intel/issues/6768
  [i915#6944]: https://gitlab.freedesktop.org/drm/intel/issues/6944
  [i915#7116]: https://gitlab.freedesktop.org/drm/intel/issues/7116
  [i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
  [i915#7173]: https://gitlab.freedesktop.org/drm/intel/issues/7173
  [i915#7276]: https://gitlab.freedesktop.org/drm/intel/issues/7276
  [i915#7561]: https://gitlab.freedesktop.org/drm/intel/issues/7561
  [i915#7651]: https://gitlab.freedesktop.org/drm/intel/issues/7651
  [i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
  [i915#7701]: https://gitlab.freedesktop.org/drm/intel/issues/7701
  [i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
  [i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79
  [i915#7949]: https://gitlab.freedesktop.org/drm/intel/issues/7949
  [i915#7957]: https://gitlab.freedesktop.org/drm/intel/issues/7957
  [i915#8150]: https://gitlab.freedesktop.org/drm/intel/issues/8150
  [i915#8151]: https://gitlab.freedesktop.org/drm/intel/issues/8151
  [i915#8152]: https://gitlab.freedesktop.org/drm/intel/issues/8152
  [i915#8228]: https://gitlab.freedesktop.org/drm/intel/issues/8228


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7180 -> IGTPW_8558
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_12806: fc05dcda11c98d438e65280e949332cd86c22522 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8558: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html
  IGT_7180: b6b1ceb4fc4d8f83a9540e4628a6c2648514bb19 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8558/index.html

[-- Attachment #2: Type: text/html, Size: 19442 bytes --]