[igt-dev] [PATCH i-g-t 0/5] Speedup IGT build time

[igt-dev] [PATCH i-g-t 0/5] Speedup IGT build time

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Currently, even a single change on IGT makes the build time too
slow.

This is because it will re-generate the check testlist on every
single build that touches any file.

Change the logic to generate per-file testlists, with speeds up
build time from 22 seconds to 4 seconds.

It can be improved even further after changing the build logic
to maintain separate test_executables for i915, kms and xe.

As there's a pending patch series touching those tests, moving
them to tests/intel directory, let's not do a conflicting change.
Once such patch series gets merged, I'll write a patch with the
pending optimization.

Mauro Carvalho Chehab (5):
  lib/igt_core: use the macro when checking for list_subtests
  lib/igt_core: add an option to show the testlist
  tests/meson.build: create testlists for tests
  tests/meson.build: add gem_stress to the test_list logic
  scripts/test_list.py: speedup testlist check logic

 lib/igt_core.c       | 26 ++++++++++++++++++++------
 scripts/test_list.py | 30 ++++++------------------------
 tests/meson.build    | 25 ++++++++++++++++++-------
 3 files changed, 44 insertions(+), 37 deletions(-)

-- 
2.40.1

[igt-dev] [PATCH i-g-t 1/5] lib/igt_core: use the macro when checking for list_subtests

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

We're implementing a new argument to display all tests. Before
doing that, make the check for it more generic to prepare for
the changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 lib/igt_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/igt_core.c b/lib/igt_core.c
index 3ee3a01c3668..9a0029d294a8 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -870,7 +870,7 @@ static void print_version(void)
 {
 	struct utsname uts;
 
-	if (list_subtests)
+	if (igt_only_list_subtests())
 		return;
 
 	uname(&uts);
@@ -1198,12 +1198,12 @@ static int common_init(int *argc, char **argv,
 			break;
 		case OPT_RUN_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_subtest = strdup(optarg);
 			break;
 		case OPT_RUN_DYNAMIC_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_dynamic_subtest = strdup(optarg);
 			break;
 		case OPT_DESCRIPTION:
@@ -1265,7 +1265,7 @@ out:
 		/* exit with no error for -h/--help */
 		exit(ret == -1 ? 0 : IGT_EXIT_INVALID);
 
-	if (!list_subtests) {
+	if (!igt_only_list_subtests()) {
 		bind_fbcon(false);
 		igt_kmsg(KMSG_INFO "%s: executing\n", command_str);
 		print_version();
@@ -3098,7 +3098,7 @@ void igt_vlog(const char *domain, enum igt_log_level level, const char *format,
 	if (!thread_id)
 		return;
 
-	if (list_subtests && level <= IGT_LOG_WARN)
+	if (igt_only_list_subtests() && level <= IGT_LOG_WARN)
 		return;
 
 	if (vasprintf(&line, format, args) == -1)
-- 
2.40.1

[igt-dev] [PATCH i-g-t 2/5] lib/igt_core: add an option to show the testlist

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

There are several cases where we need to list the tests the same
way as igt_runner displays them.

That should not return errors on failures and should work also
when igt_simple_main() macro is used.

Ideally, it should also show dynamic subtests, but this is more
complex, specially for Kselftest/KUnit ones. So, for now, let's
not handle dynamic ones.

Add an option to produce such testlist.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 lib/igt_core.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/lib/igt_core.c b/lib/igt_core.c
index 9a0029d294a8..2ae2cb68835e 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -274,6 +274,7 @@ const char *igt_interactive_debug;
 bool igt_skip_crc_compare;
 
 /* subtests helpers */
+static bool show_testlist = false;
 static bool list_subtests = false;
 static bool describe_subtests = false;
 static char *run_single_subtest = NULL;
@@ -326,6 +327,7 @@ enum {
 	 * conflict with core ones
 	 */
 	OPT_LIST_SUBTESTS = 500,
+	OPT_SHOW_TESTLIST,
 	OPT_DESCRIBE_SUBTESTS,
 	OPT_RUN_SUBTEST,
 	OPT_RUN_DYNAMIC_SUBTEST,
@@ -896,6 +898,7 @@ static void print_usage(const char *help_str, bool output_on_stderr)
 
 	fprintf(f, "Usage: %s [OPTIONS]\n", command_str);
 	fprintf(f, "  --list-subtests\n"
+		   "  --show-testlist\n"
 		   "  --run-subtest <pattern>\n"
 		   "  --dynamic-subtest <pattern>\n"
 		   "  --debug[=log-domain]\n"
@@ -1076,6 +1079,7 @@ static int common_init(int *argc, char **argv,
 	int c, option_index = 0, i, x;
 	static struct option long_options[] = {
 		{"list-subtests",     no_argument,       NULL, OPT_LIST_SUBTESTS},
+		{"show-testlist",     no_argument,       NULL, OPT_SHOW_TESTLIST},
 		{"describe",          optional_argument, NULL, OPT_DESCRIBE_SUBTESTS},
 		{"run-subtest",       required_argument, NULL, OPT_RUN_SUBTEST},
 		{"dynamic-subtest",   required_argument, NULL, OPT_RUN_DYNAMIC_SUBTEST},
@@ -1189,6 +1193,9 @@ static int common_init(int *argc, char **argv,
 			if (!run_single_subtest)
 				list_subtests = true;
 			break;
+		case OPT_SHOW_TESTLIST:
+			show_testlist = true;
+			break;
 		case OPT_DESCRIBE_SUBTESTS:
 			if (optarg)
 				run_single_subtest = strdup(optarg);
@@ -1257,6 +1264,10 @@ out:
 			igt_warn("Unknown subtest: %s\n", run_single_subtest);
 			exit(IGT_EXIT_INVALID);
 		}
+		if (show_testlist) {
+			printf("igt@%s\n", igt_test_name());
+			exit(0);
+		}
 		if (list_subtests)
 			exit(IGT_EXIT_INVALID);
 	}
@@ -1449,6 +1460,9 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 		__igt_print_description(subtest_name, file, line);
 		_clear_current_description();
 		return false;
+	} else if (show_testlist) {
+		printf("igt@%s@%s\n", igt_test_name(), subtest_name);
+		return false;
 	} else if (list_subtests) {
 		printf("%s\n", subtest_name);
 		return false;
@@ -1522,7 +1536,7 @@ const char *igt_subtest_name(void)
  */
 bool igt_only_list_subtests(void)
 {
-	return list_subtests;
+	return list_subtests || show_testlist;
 }
 
 
-- 
2.40.1

[igt-dev] [PATCH i-g-t 3/5] tests/meson.build: create testlists for tests

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Let's dynamically create testlists for during build time, as this
can speed up a lot the validation check for testplan.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/meson.build | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/tests/meson.build b/tests/meson.build
index ee066b84900b..246a585aa423 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -545,7 +545,17 @@ test_list_full_target = custom_target('testlist-full',
 	      install_dir : libexecdir)
 
 test_script = find_program('igt_command_line.sh')
-foreach prog : test_list
+foreach testexe : test_executables
+	prog = testexe.name()
+
+	output = prog.split('/').get(-1) + '.testlist'
+	custom_target(output,
+		     build_by_default : true,
+		     command : [ testexe.full_path(), '--show-testlist'],
+		     capture : true,
+		     depends : testexe,
+		     output : output)
+
 	test('testcase check ' + prog, test_script, args : prog)
 endforeach
 
-- 
2.40.1

[igt-dev] [PATCH i-g-t 4/5] tests/meson.build: add gem_stress to the test_list logic

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

This test seems to be somewhat abandoned at the build system,
as it was not used to update the test lists and their dependencies.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/meson.build | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/tests/meson.build b/tests/meson.build
index 246a585aa423..7620bd3598f0 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -516,6 +516,13 @@ test_executables += executable('sw_sync', 'sw_sync.c',
 	   install : true)
 test_list += 'sw_sync'
 
+test_executables += executable('gem_stress', 'i915/gem_stress.c',
+	   install : true,
+	   install_dir : libexecdir,
+	   install_rpath : libexecdir_rpathdir,
+	   dependencies : igt_deps)
+test_list += 'gem_stress'
+
 subdir('amdgpu')
 
 subdir('v3d')
@@ -559,12 +566,6 @@ foreach testexe : test_executables
 	test('testcase check ' + prog, test_script, args : prog)
 endforeach
 
-executable('gem_stress', 'i915/gem_stress.c',
-	   install : true,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   dependencies : igt_deps)
-
 image_files = [
   '1080p-left.png',
   '1080p-right.png',
-- 
2.40.1

[igt-dev] [PATCH i-g-t 5/5] scripts/test_list.py: speedup testlist check logic

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Now that the build system will generate the testlist for us,
just read them when checking for missing symbols.

This should speed up a log the check logic.

After the change, touching on a single file and rebuilding the
testplan with check enabled is a lot faster:

	$ touch tests/xe/xe_compute.c
	$ time make
	ninja -C build
	ninja: Entering directory `build'
	[10/10] Generating docs/testplan/i915_tests.html with a custom command

	real	0m4.510s
	user	0m5.944s
	sys	0m0.217s

This can be improved even further if we split the test_executables
dependencies per driver.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index a55d2977c7c3..2c1a6f230fb5 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -909,35 +909,17 @@ class TestList:
 
         return subtests
 
-    def __get_testlist(self, name):
-        match = re.match(r"(.*/)?(.*)\.c$", name)
-        if not match:
-            return []
-
-        basename = "igt@" + match.group(2)
-
-        fname = os.path.join(self.igt_build_path, "tests", match.group(2))
-        if not os.path.isfile(fname):
-            print(f"Error: file {fname} doesn't exist.")
-            sys.exit(1)
-        try:
-            result = subprocess.run([ fname, "--list-subtests" ],
-                                    check = True,
-                                    stdout = subprocess.PIPE,
-                                    universal_newlines=True)
-            subtests = result.stdout.splitlines()
-
-            return [basename  + "@" + i for i in subtests]
-        except subprocess.CalledProcessError:
-            # Handle it as a test using igt_simple_main
-            return [basename]
-
     def get_testlist(self):
 
         """ Return a list of tests as reported by --list-subtests """
         tests = []
         for name in self.filenames:
-            tests += self.__get_testlist(name)
+            fname = re.sub(r"\.c$", ".testlist", name.split('/')[-1])
+            fname = os.path.join(self.igt_build_path, "tests", fname)
+
+            with open(fname, 'r', encoding='utf8') as handle:
+                for line in handle:
+                    tests.append(line)
 
         return sorted(tests)
 
-- 
2.40.1

[igt-dev] ✗ Fi.CI.BUILD: failure for Speedup IGT build time

[Patchwork]

== Series Details ==

Series: Speedup IGT build time
URL   : https://patchwork.freedesktop.org/series/120233/
State : failure

== Summary ==

IGT patchset build failed on latest successful build
f8d05fd574fad1526cbf5e20672910df87b8839b intel-ci/xe-fast-feedback.testlist: Add xe_exec_store@basic-store subtest in fastfeedback

Compiler for C supports arguments -Werror=sequence-point: YES 
Compiler for C supports arguments -Werror=trigraphs: YES 
Compiler for C supports arguments -Werror=write-strings: YES 
Compiler for C supports arguments -fno-builtin-malloc: YES 
Compiler for C supports arguments -fno-builtin-calloc: YES 
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency libdrm found: YES 2.4.107
Run-time dependency libdrm_intel found: YES 2.4.107
Run-time dependency pciaccess found: YES 0.16
Run-time dependency libkmod found: YES 27
Run-time dependency libprocps found: YES 3.3.16
Found CMake: /usr/bin/cmake (3.16.3)
Run-time dependency libproc2 found: NO (tried pkgconfig and cmake)
Run-time dependency libunwind found: YES 1.21
Run-time dependency libdw found: YES 0.176
Run-time dependency pixman-1 found: YES 0.38.0
Run-time dependency valgrind found: YES 3.15.0
Run-time dependency cairo found: YES 1.17.2
Run-time dependency libudev found: YES 245
Run-time dependency glib-2.0 found: YES 2.64.6
Run-time dependency xmlrpc found: NO (tried pkgconfig and cmake)
Run-time dependency xmlrpc_util found: NO (tried pkgconfig and cmake)
Run-time dependency xmlrpc_client found: NO (tried pkgconfig and cmake)
Program xmlrpc-c-config found: YES (/usr/bin/xmlrpc-c-config)
Run-time dependency gsl found: YES 2.5
Run-time dependency alsa found: YES 1.2.2
Run-time dependency libcurl found: YES 7.68.0
Run-time dependency threads found: YES
Library m found: YES
Library rt found: YES
Library dl found: YES
Library z found: YES
Checking if "built-in atomics" links: YES 
Has header "linux/kd.h" : YES 
Has header "sys/kd.h" : YES 
Has header "libgen.h" : YES 
Has header "sys/io.h" : YES 
Checking if "cpuid.h" links: YES 
Header <unistd.h> has symbol "gettid" : YES 
Checking whether type "struct sysinfo" has member "totalram" : YES 
Checking for function "memfd_create" : YES 
Configuring config.h using configuration
Program python3 found: YES (/usr/bin/python3)
Configuring i915-perf.pc using configuration
Program generate_testlist.sh found: YES (/usr/src/igt-gpu-tools/tests/generate_testlist.sh)
Program igt_command_line.sh found: YES (/usr/src/igt-gpu-tools/tests/igt_command_line.sh)

tests/meson.build:556:1: ERROR: Unknown method "name" in object.

A full log can be found at /opt/igt/build/meson-logs/meson-log.txt