[igt-dev] [PATCH i-g-t v4 0/6] Speedup IGT build time

[igt-dev] [PATCH i-g-t v4 0/6] Speedup IGT build time

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Currently, even a single change on IGT makes the build time too
slow.

This is because it will re-generate the check testlist on every
single build that touches any file.

Change the logic to generate per-file testlists, with speeds up
build time from 22 seconds to 4 seconds.

It can be improved even further after changing the build logic
to maintain separate test_executables for i915, kms and xe.

As there's a pending patch series touching those tests, moving
them to tests/intel directory, let's not do a conflicting change.
Once such patch series gets merged, I'll write a patch with the
pending optimization.

---

v4:
- fix dependency chain for igt-doc.py;
- create an intel-ci.testlist with core tests and tests for
  intel drivers.

v3:
- custom_target that generates testlist is now simpler and
  should work better.

v2:
- make it compatible with Meson 0.47;
- added a patch to simplify tests/meson.build
- dropped a change at gem_stress - maybe there's a real reason
  why this one was excluded from test_list, so let's keep it
  as-is.

Mauro Carvalho Chehab (6):
  tests/meson.build: Simplify builds for core and i915 targets
  lib/igt_core: use the macro when checking for list_subtests
  lib/igt_core: add an option to show the testlist
  tests/meson.build: create testlists for tests
  scripts/test_list.py: speedup testlist check logic
  tests/build: Generate and store an intel-ci.testlist

 docs/testplan/meson.build  |   2 +-
 lib/igt_core.c             |  26 +++-
 scripts/test_list.py       |  30 +---
 tests/intel-ci/meson.build |   8 ++
 tests/meson.build          | 272 +++++++++++++++----------------------
 5 files changed, 141 insertions(+), 197 deletions(-)

-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 1/6] tests/meson.build: Simplify builds for core and i915 targets

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

There are some tests that either require:
	- extra dependencies
	- extra source files

Right now, they have special executable() targets that are handled
outside the normal way. remove those special targets, converting
them into simple ones, and using two extra dictionaries to map
those extra requirements.

This makes the meson.build file a lot simpler, and makes easier
to add other rules when creating test lists.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/meson.build | 223 +++++++++++++---------------------------------
 1 file changed, 61 insertions(+), 162 deletions(-)

diff --git a/tests/meson.build b/tests/meson.build
index ee066b84900b..3116c458d60b 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -3,12 +3,14 @@ test_progs = [
 	'core_getclient',
 	'core_getstats',
 	'core_getversion',
+	'core_hotunplug',
 	'core_setmaster',
 	'core_setmaster_vs_auth',
 	'debugfs_test',
 	'dmabuf',
 	'dmabuf_sync_file',
 	'device_reset',
+	'dumb_buffer',
 	'drm_buddy',
 	'drm_mm',
 	'drm_read',
@@ -21,6 +23,7 @@ test_progs = [
 	'kms_atomic_interruptible',
 	'kms_atomic_transition',
 	'kms_bw',
+	'kms_color',
 	'kms_concurrent',
 	'kms_content_protection',
 	'kms_cursor_crc',
@@ -77,7 +80,9 @@ test_progs = [
 	'syncobj_basic',
 	'syncobj_wait',
 	'syncobj_timeline',
+	'sw_sync',
 	'template',
+	'testdisplay',
 	'tools_test',
 	'vgem_basic',
 	'vgem_slow',
@@ -86,16 +91,19 @@ test_progs = [
 i915_progs = [
 	'api_intel_allocator',
 	'api_intel_bb',
+	'drm_fdinfo',
 	'gen3_mixed_blits',
 	'gen3_render_linear_blits',
 	'gen3_render_mixed_blits',
 	'gen3_render_tiledx_blits',
 	'gen3_render_tiledy_blits',
 	'gem_bad_reloc',
+	'gem_barrier_race',
 	'gem_basic',
 	'gem_blits',
 	'gem_busy',
 	'gem_caching',
+	'gem_create',
 	'gem_ccs',
 	'gem_close',
 	'gem_close_race',
@@ -105,16 +113,20 @@ i915_progs = [
 	'gem_ctx_create',
 	'gem_ctx_engines',
 	'gem_ctx_exec',
+	'gem_ctx_freq',
 	'gem_ctx_isolation',
 	'gem_ctx_param',
 	'gem_ctx_persistence',
 	'gem_ctx_shared',
+	'gem_ctx_sseu',
 	'gem_ctx_switch',
+	'gem_eio',
 	'gem_evict_alignment',
 	'gem_evict_everything',
 	'gem_exec_alignment',
 	'gem_exec_async',
 	'gem_exec_await',
+	'gem_exec_balancer',
 	'gem_exec_basic',
 	'gem_exec_big',
 	'gem_exec_capture',
@@ -157,6 +169,7 @@ i915_progs = [
 	'gem_media_vme',
 	'gem_mmap',
 	'gem_mmap_gtt',
+	'gem_mmap_offset',
 	'gem_mmap_wc',
 	'gem_partial_pwrite_pread',
 	'gem_pipe_control_store_loop',
@@ -200,6 +213,8 @@ i915_progs = [
 	'gem_wait',
 	'gem_watchdog',
 	'gem_workarounds',
+	'perf',
+	'perf_pmu',
 	'i915_fb_tiling',
 	'i915_getparams_basic',
 	'i915_pm_freq_api',
@@ -209,9 +224,11 @@ i915_progs = [
 	'i915_pciid',
 	'i915_pipe_stress',
 	'i915_pm_backlight',
+	'i915_pm_freq_mult',
 	'i915_pm_lpsp',
 	'i915_pm_rpm',
 	'i915_pm_dc',
+	'i915_pm_rc6_residency',
 	'i915_pm_rps',
 	'i915_pm_sseu',
 	'i915_power',
@@ -224,6 +241,7 @@ i915_progs = [
 	'kms_ccs',
 	'kms_cdclk',
 	'kms_draw_crc',
+	'kms_dsc',
 	'kms_fb_coherency',
 	'kms_fbcon_fbt',
 	'kms_fence_pin_leak',
@@ -234,6 +252,7 @@ i915_progs = [
 	'kms_mmap_write_crc',
 	'kms_pipe_b_c_ivb',
 	'kms_psr',
+	'kms_psr2_sf',
 	'kms_psr2_su',
 	'kms_psr_stress_test',
 	'kms_pwrite_crc',
@@ -301,12 +320,41 @@ if libdrm_nouveau.found()
 	test_deps += libdrm_nouveau
 endif
 
+# Extra sources used on core and Intel drivers
+extra_sources = {
+	'core_hotunplug': [ 'core_hotunplug.c' ],
+	'dumb_buffer': ['dumb_buffer.c' ],
+	'testdisplay': [ 'testdisplay_hotplug.c' ],
+	'kms_color':  [ 'kms_color_helper.c' ],
+	'kms_dsc': [ join_paths ('i915', 'kms_dsc_helper.c') ],
+	'kms_psr2_sf':  [ join_paths ('i915', 'kms_dsc_helper.c') ],
+}
+
+# Extra dependencies used on core and Intel drivers
+extra_dependencies = {
+	'core_hotunplug': [ lib_igt_i915_perf ],
+	'drm_fdinfo': [ lib_igt_drm_fdinfo ],
+	'dumb_buffer':  [ libatomic ],
+	'gem_barrier_race': [ lib_igt_i915_perf ],
+	'gem_create': [ libatomic ],
+	'gem_ctx_freq': [ lib_igt_perf ],
+	'gem_ctx_sseu': [ lib_igt_perf ],
+	'gem_eio': [ realtime ],
+	'gem_exec_balancer': [ lib_igt_perf ],
+	'gem_mmap_offset': [ libatomic ],
+	'i915_pm_freq_mult': [ lib_igt_perf ],
+	'i915_pm_rc6_residency': [ lib_igt_perf ],
+	'perf': [ lib_igt_i915_perf ],
+	'perf_pmu':  [ lib_igt_perf ],
+	'sw_sync': [ libatomic ],
+}
+
 test_executables = []
 test_list = []
 
 foreach prog : test_progs
-	test_executables += executable(prog, prog + '.c',
-		   dependencies : test_deps,
+	test_executables += executable(prog, [prog + '.c'] + extra_sources.get(prog, []),
+		   dependencies : test_deps + extra_dependencies.get(prog, []),
 		   install_dir : libexecdir,
 		   install_rpath : libexecdir_rpathdir,
 		   install : true)
@@ -315,8 +363,8 @@ endforeach
 
 foreach prog : i915_progs
 	test_executables += executable(prog,
-		   join_paths('i915', prog + '.c'),
-		   dependencies : test_deps,
+		   [join_paths('i915', prog + '.c')] + extra_sources.get(prog, []),
+		   dependencies : test_deps + extra_dependencies.get(prog, []),
 		   install_dir : libexecdir,
 		   install_rpath : libexecdir_rpathdir,
 		   install : true)
@@ -326,8 +374,8 @@ endforeach
 if build_xe
 	foreach prog : xe_progs
 		test_executables += executable(prog,
-			   join_paths('xe', prog + '.c'),
-			   dependencies : test_deps,
+			   [join_paths('xe', prog + '.c')] + extra_sources.get(prog, []),
+			   dependencies : test_deps + extra_dependencies.get(prog, []),
 			   install_dir : libexecdir,
 			   install_rpath : libexecdir_rpathdir,
 			   install : true)
@@ -357,165 +405,16 @@ if chamelium.found()
 		test_list += prog
 	endforeach
 	test_deps += chamelium
-endif
-
-test_executables += executable('drm_fdinfo',
-	   join_paths('i915', 'drm_fdinfo.c'),
-	   dependencies : test_deps + [ lib_igt_drm_fdinfo ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'drm_fdinfo'
-
-test_executables += executable('dumb_buffer', 'dumb_buffer.c',
-	   dependencies : test_deps + [ libatomic ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'dumb_buffer'
-
-test_executables += executable('gem_create',
-	   join_paths('i915', 'gem_create.c'),
-	   dependencies : test_deps + [ libatomic ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_create'
-
-test_executables += executable('gem_ctx_freq',
-	   join_paths('i915', 'gem_ctx_freq.c'),
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_ctx_freq'
-
-test_executables += executable('gem_ctx_sseu',
-	   join_paths('i915', 'gem_ctx_sseu.c'),
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_ctx_sseu'
-
-test_executables += executable('gem_eio',
-	   join_paths('i915', 'gem_eio.c'),
-	   dependencies : test_deps + [ realtime ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_eio'
-
-test_executables += executable('gem_exec_balancer', 'i915/gem_exec_balancer.c',
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_exec_balancer'
-
-test_executables += executable('gem_mmap_offset',
-	   join_paths('i915', 'gem_mmap_offset.c'),
-	   dependencies : test_deps + [ libatomic ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_mmap_offset'
-
-test_executables += executable('i915_pm_freq_mult',
-	   join_paths('i915', 'i915_pm_freq_mult.c'),
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'i915_pm_freq_mult'
 
-test_executables += executable('i915_pm_rc6_residency',
-	   join_paths('i915', 'i915_pm_rc6_residency.c'),
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'i915_pm_rc6_residency'
-
-test_executables += executable('gem_barrier_race',
-	   join_paths('i915', 'gem_barrier_race.c'),
-	   dependencies : test_deps + [ lib_igt_i915_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'gem_barrier_race'
-
-test_executables += executable('perf_pmu',
-	   join_paths('i915', 'perf_pmu.c'),
-	   dependencies : test_deps + [ lib_igt_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'perf_pmu'
-
-test_executables += executable('perf',
-	   join_paths('i915', 'perf.c'),
-	   dependencies : test_deps + [ lib_igt_i915_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'perf'
-
-test_executables += executable('core_hotunplug', 'core_hotunplug.c',
-	   dependencies : test_deps + [ lib_igt_i915_perf ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'core_hotunplug'
-
-executable('testdisplay', ['testdisplay.c', 'testdisplay_hotplug.c'],
-	   dependencies : test_deps,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'testdisplay'
-
-test_executables += executable('kms_color',
-	   [ 'kms_color.c', 'kms_color_helper.c' ],
-	   dependencies : test_deps,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'kms_color'
-
-test_executables += executable('kms_dsc',
-	   [ join_paths('i915', 'kms_dsc.c'), join_paths ('i915', 'kms_dsc_helper.c')],
-	   dependencies : test_deps,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'kms_dsc'
-
-test_executables += executable('kms_psr2_sf',
-	   [ join_paths('i915', 'kms_psr2_sf.c'), join_paths ('i915', 'kms_dsc_helper.c')],
-	   dependencies : test_deps,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'kms_psr2_sf'
-
-if chamelium.found()
-       test_executables += executable('kms_chamelium_color',
-                             [ 'chamelium/kms_chamelium_color.c', 'kms_color_helper.c' ],
-                             dependencies : test_deps + [ chamelium ],
-                             install_dir : libexecdir,
-                             install_rpath : libexecdir_rpathdir,
-                             install : true)
-       test_list += 'kms_chamelium_color'
+	test_executables += executable('kms_chamelium_color',
+				[ 'chamelium/kms_chamelium_color.c', 'kms_color_helper.c' ],
+				dependencies : test_deps + [ chamelium ],
+				install_dir : libexecdir,
+				install_rpath : libexecdir_rpathdir,
+				install : true)
+	test_list += 'kms_chamelium_color'
 endif
 
-test_executables += executable('sw_sync', 'sw_sync.c',
-	   dependencies : test_deps + [ libatomic ],
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   install : true)
-test_list += 'sw_sync'
-
 subdir('amdgpu')
 
 subdir('v3d')
-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 2/6] lib/igt_core: use the macro when checking for list_subtests

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

We're implementing a new argument to display all tests. Before
doing that, make the check for it more generic to prepare for
the changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 lib/igt_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/igt_core.c b/lib/igt_core.c
index 3ee3a01c3668..9a0029d294a8 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -870,7 +870,7 @@ static void print_version(void)
 {
 	struct utsname uts;
 
-	if (list_subtests)
+	if (igt_only_list_subtests())
 		return;
 
 	uname(&uts);
@@ -1198,12 +1198,12 @@ static int common_init(int *argc, char **argv,
 			break;
 		case OPT_RUN_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_subtest = strdup(optarg);
 			break;
 		case OPT_RUN_DYNAMIC_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_dynamic_subtest = strdup(optarg);
 			break;
 		case OPT_DESCRIPTION:
@@ -1265,7 +1265,7 @@ out:
 		/* exit with no error for -h/--help */
 		exit(ret == -1 ? 0 : IGT_EXIT_INVALID);
 
-	if (!list_subtests) {
+	if (!igt_only_list_subtests()) {
 		bind_fbcon(false);
 		igt_kmsg(KMSG_INFO "%s: executing\n", command_str);
 		print_version();
@@ -3098,7 +3098,7 @@ void igt_vlog(const char *domain, enum igt_log_level level, const char *format,
 	if (!thread_id)
 		return;
 
-	if (list_subtests && level <= IGT_LOG_WARN)
+	if (igt_only_list_subtests() && level <= IGT_LOG_WARN)
 		return;
 
 	if (vasprintf(&line, format, args) == -1)
-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 3/6] lib/igt_core: add an option to show the testlist

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

There are several cases where we need to list the tests the same
way as igt_runner displays them.

That should not return errors on failures and should work also
when igt_simple_main() macro is used.

Ideally, it should also show dynamic subtests, but this is more
complex, specially for Kselftest/KUnit ones. So, for now, let's
not handle dynamic ones.

Add an option to produce such testlist.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 lib/igt_core.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/lib/igt_core.c b/lib/igt_core.c
index 9a0029d294a8..2ae2cb68835e 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -274,6 +274,7 @@ const char *igt_interactive_debug;
 bool igt_skip_crc_compare;
 
 /* subtests helpers */
+static bool show_testlist = false;
 static bool list_subtests = false;
 static bool describe_subtests = false;
 static char *run_single_subtest = NULL;
@@ -326,6 +327,7 @@ enum {
 	 * conflict with core ones
 	 */
 	OPT_LIST_SUBTESTS = 500,
+	OPT_SHOW_TESTLIST,
 	OPT_DESCRIBE_SUBTESTS,
 	OPT_RUN_SUBTEST,
 	OPT_RUN_DYNAMIC_SUBTEST,
@@ -896,6 +898,7 @@ static void print_usage(const char *help_str, bool output_on_stderr)
 
 	fprintf(f, "Usage: %s [OPTIONS]\n", command_str);
 	fprintf(f, "  --list-subtests\n"
+		   "  --show-testlist\n"
 		   "  --run-subtest <pattern>\n"
 		   "  --dynamic-subtest <pattern>\n"
 		   "  --debug[=log-domain]\n"
@@ -1076,6 +1079,7 @@ static int common_init(int *argc, char **argv,
 	int c, option_index = 0, i, x;
 	static struct option long_options[] = {
 		{"list-subtests",     no_argument,       NULL, OPT_LIST_SUBTESTS},
+		{"show-testlist",     no_argument,       NULL, OPT_SHOW_TESTLIST},
 		{"describe",          optional_argument, NULL, OPT_DESCRIBE_SUBTESTS},
 		{"run-subtest",       required_argument, NULL, OPT_RUN_SUBTEST},
 		{"dynamic-subtest",   required_argument, NULL, OPT_RUN_DYNAMIC_SUBTEST},
@@ -1189,6 +1193,9 @@ static int common_init(int *argc, char **argv,
 			if (!run_single_subtest)
 				list_subtests = true;
 			break;
+		case OPT_SHOW_TESTLIST:
+			show_testlist = true;
+			break;
 		case OPT_DESCRIBE_SUBTESTS:
 			if (optarg)
 				run_single_subtest = strdup(optarg);
@@ -1257,6 +1264,10 @@ out:
 			igt_warn("Unknown subtest: %s\n", run_single_subtest);
 			exit(IGT_EXIT_INVALID);
 		}
+		if (show_testlist) {
+			printf("igt@%s\n", igt_test_name());
+			exit(0);
+		}
 		if (list_subtests)
 			exit(IGT_EXIT_INVALID);
 	}
@@ -1449,6 +1460,9 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 		__igt_print_description(subtest_name, file, line);
 		_clear_current_description();
 		return false;
+	} else if (show_testlist) {
+		printf("igt@%s@%s\n", igt_test_name(), subtest_name);
+		return false;
 	} else if (list_subtests) {
 		printf("%s\n", subtest_name);
 		return false;
@@ -1522,7 +1536,7 @@ const char *igt_subtest_name(void)
  */
 bool igt_only_list_subtests(void)
 {
-	return list_subtests;
+	return list_subtests || show_testlist;
 }
 
 
-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 4/6] tests/meson.build: create testlists for tests

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Let's dynamically create testlists for during build time, as this
can speed up a lot the validation check for testplan.

Ideally, all it would need to to that would be to use this
logic at the end of tests/meson.build:

    foreach testexe : test_executables
        prog = testexe.name()

        output = prog.split('/').get(-1) + '.testlist'
        custom_target(output,
                     build_by_default : true,
                     command : [ testexe, '--show-testlist'],
                     capture : true,
                     output : output)

Unfortunately, this requies Meson >= 0.54. So, we need to add
one custom_target per executable() call.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 docs/testplan/meson.build |  2 +-
 tests/meson.build         | 55 ++++++++++++++++++++++++++++++++++-----
 2 files changed, 49 insertions(+), 8 deletions(-)

diff --git a/docs/testplan/meson.build b/docs/testplan/meson.build
index bf73dd32a15f..e838f2eb1540 100644
--- a/docs/testplan/meson.build
+++ b/docs/testplan/meson.build
@@ -12,7 +12,7 @@ i915_test_config = join_paths(source_root, 'tests', 'i915', 'i915_test_config.js
 
 check_testlist = []
 if build_tests
-	doc_dependencies = test_executables
+	doc_dependencies = testlist_files
 	# Check if documentation matches the actual tests and tests can run
 	if not meson.is_cross_build()
 		build_info += 'Will Check if documentation is in sync with testlist'
diff --git a/tests/meson.build b/tests/meson.build
index 3116c458d60b..897f17ab4561 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -351,35 +351,57 @@ extra_dependencies = {
 
 test_executables = []
 test_list = []
+testlist_files = []
 
 foreach prog : test_progs
-	test_executables += executable(prog, [prog + '.c'] + extra_sources.get(prog, []),
+	testexe = executable(prog, [prog + '.c'] + extra_sources.get(prog, []),
 		   dependencies : test_deps + extra_dependencies.get(prog, []),
 		   install_dir : libexecdir,
 		   install_rpath : libexecdir_rpathdir,
 		   install : true)
+	test_executables += testexe
 	test_list += prog
+	name = prog.split('/').get(-1)
+	testlist_files += custom_target(name + '.testlist',
+		     build_by_default : true,
+		     command : [testexe, '--show-testlist'],
+		     capture : true,
+		     output : name + '.testlist')
 endforeach
 
 foreach prog : i915_progs
-	test_executables += executable(prog,
+	testexe = executable(prog,
 		   [join_paths('i915', prog + '.c')] + extra_sources.get(prog, []),
 		   dependencies : test_deps + extra_dependencies.get(prog, []),
 		   install_dir : libexecdir,
 		   install_rpath : libexecdir_rpathdir,
 		   install : true)
+	test_executables += testexe
 	test_list += prog
+	name = prog.split('/').get(-1)
+	testlist_files += custom_target(name + '.testlist',
+		     build_by_default : true,
+		     command : [testexe, '--show-testlist'],
+		     capture : true,
+		     output : name + '.testlist')
 endforeach
 
 if build_xe
 	foreach prog : xe_progs
-		test_executables += executable(prog,
+		testexe = executable(prog,
 			   [join_paths('xe', prog + '.c')] + extra_sources.get(prog, []),
 			   dependencies : test_deps + extra_dependencies.get(prog, []),
 			   install_dir : libexecdir,
 			   install_rpath : libexecdir_rpathdir,
 			   install : true)
 		test_list += prog
+		test_executables += testexe
+		name = prog.split('/').get(-1)
+		testlist_files += custom_target(name + '.testlist',
+			     build_by_default : true,
+			     command : [testexe, '--show-testlist'],
+			     capture : true,
+			     output : name + '.testlist')
 	endforeach
 	build_info += 'Xe **experimental** tests enabled.'
 endif
@@ -395,7 +417,7 @@ endforeach
 
 if chamelium.found()
 	foreach prog : chamelium_progs
-		test_executables += executable(prog,
+		testexe = executable(prog,
 				 [join_paths('chamelium', prog + '.c'),
 					join_paths('chamelium', 'kms_chamelium_helper.c')],
 				 dependencies : test_deps,
@@ -403,16 +425,30 @@ if chamelium.found()
 				 install_rpath : libexecdir_rpathdir,
 				 install : true)
 		test_list += prog
+		test_executables += testexe
+		name = prog.split('/').get(-1)
+		testlist_files += custom_target(name + '.testlist',
+			     build_by_default : true,
+			     command : [testexe, '--show-testlist'],
+			     capture : true,
+			     output : name + '.testlist')
 	endforeach
 	test_deps += chamelium
 
-	test_executables += executable('kms_chamelium_color',
+	name = 'kms_chamelium_color'
+	testexe = executable('kms_chamelium_color',
 				[ 'chamelium/kms_chamelium_color.c', 'kms_color_helper.c' ],
 				dependencies : test_deps + [ chamelium ],
 				install_dir : libexecdir,
 				install_rpath : libexecdir_rpathdir,
 				install : true)
-	test_list += 'kms_chamelium_color'
+	test_list += name
+	test_executables += testexe
+	testlist_files += custom_target(name + '.testlist',
+		     build_by_default : true,
+		     command : [testexe, '--show-testlist'],
+		     capture : true,
+		     output : name + '.testlist')
 endif
 
 subdir('amdgpu')
@@ -448,11 +484,16 @@ foreach prog : test_list
 	test('testcase check ' + prog, test_script, args : prog)
 endforeach
 
-executable('gem_stress', 'i915/gem_stress.c',
+testexe = executable('gem_stress', 'i915/gem_stress.c',
 	   install : true,
 	   install_dir : libexecdir,
 	   install_rpath : libexecdir_rpathdir,
 	   dependencies : igt_deps)
+testlist_files += custom_target('gem_stress.testlist',
+                build_by_default : true,
+                command : [testexe, '--show-testlist'],
+                capture : true,
+                output : 'gem_stress.testlist')
 
 image_files = [
   '1080p-left.png',
-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 5/6] scripts/test_list.py: speedup testlist check logic

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Now that the build system will generate the testlist for us,
just read them when checking for missing symbols.

This should speed up a log the check logic.

After the change, touching on a single file and rebuilding the
testplan with check enabled is a lot faster:

	$ touch tests/xe/xe_compute.c
	$ time make
	ninja -C build
	ninja: Entering directory `build'
	[10/10] Generating docs/testplan/i915_tests.html with a custom command

	real	0m4.510s
	user	0m5.944s
	sys	0m0.217s

This can be improved even further if we split the test_executables
dependencies per driver.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index a55d2977c7c3..2c1a6f230fb5 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -909,35 +909,17 @@ class TestList:
 
         return subtests
 
-    def __get_testlist(self, name):
-        match = re.match(r"(.*/)?(.*)\.c$", name)
-        if not match:
-            return []
-
-        basename = "igt@" + match.group(2)
-
-        fname = os.path.join(self.igt_build_path, "tests", match.group(2))
-        if not os.path.isfile(fname):
-            print(f"Error: file {fname} doesn't exist.")
-            sys.exit(1)
-        try:
-            result = subprocess.run([ fname, "--list-subtests" ],
-                                    check = True,
-                                    stdout = subprocess.PIPE,
-                                    universal_newlines=True)
-            subtests = result.stdout.splitlines()
-
-            return [basename  + "@" + i for i in subtests]
-        except subprocess.CalledProcessError:
-            # Handle it as a test using igt_simple_main
-            return [basename]
-
     def get_testlist(self):
 
         """ Return a list of tests as reported by --list-subtests """
         tests = []
         for name in self.filenames:
-            tests += self.__get_testlist(name)
+            fname = re.sub(r"\.c$", ".testlist", name.split('/')[-1])
+            fname = os.path.join(self.igt_build_path, "tests", fname)
+
+            with open(fname, 'r', encoding='utf8') as handle:
+                for line in handle:
+                    tests.append(line)
 
         return sorted(tests)
 
-- 
2.40.1

[igt-dev] [PATCH i-g-t v4 6/6] tests/build: Generate and store an intel-ci.testlist

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Store a sorted testlist with tests that could be used to test the
DRM core and the Intel drivers in a format that it would be
expected by igt_runner.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/intel-ci/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tests/intel-ci/meson.build b/tests/intel-ci/meson.build
index d5a6e4f47203..a1ec517f8da1 100644
--- a/tests/intel-ci/meson.build
+++ b/tests/intel-ci/meson.build
@@ -9,4 +9,12 @@ intelci_files = [
   'xe.blocklist.txt',
 ]
 
+custom_target('intel-ci.testlist',
+              build_by_default : true,
+              command : ['sort', testlist_files],
+              install : true,
+              install_dir : datadir,
+              capture : true,
+              output : 'intel-ci.testlist')
+
 install_data(sources : intelci_files, install_dir : datadir)
-- 
2.40.1

[igt-dev] ✗ Fi.CI.BAT: failure for Speedup IGT build time (rev5)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 7876 bytes --]

== Series Details ==

Series: Speedup IGT build time (rev5)
URL   : https://patchwork.freedesktop.org/series/120233/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_13360 -> IGTPW_9373
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with IGTPW_9373 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_9373, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/index.html

Participating hosts (40 -> 39)
------------------------------

  Missing    (1): fi-snb-2520m 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_9373:

### IGT changes ###

#### Possible regressions ####

  * igt@i915_module_load@reload:
    - bat-mtlp-6:         [PASS][1] -> [ABORT][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-mtlp-6/igt@i915_module_load@reload.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-6/igt@i915_module_load@reload.html

  
Known issues
------------

  Here are the changes found in IGTPW_9373 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_lmem_swapping@verify-random:
    - bat-mtlp-8:         NOTRUN -> [SKIP][3] ([i915#4613]) +3 similar issues
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@gem_lmem_swapping@verify-random.html

  * igt@i915_pm_rps@basic-api:
    - bat-mtlp-8:         NOTRUN -> [SKIP][4] ([i915#6621])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@i915_pm_rps@basic-api.html

  * igt@i915_selftest@live@execlists:
    - fi-bsw-n3050:       [PASS][5] -> [ABORT][6] ([i915#7911] / [i915#7913])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/fi-bsw-n3050/igt@i915_selftest@live@execlists.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/fi-bsw-n3050/igt@i915_selftest@live@execlists.html

  * igt@i915_selftest@live@hangcheck:
    - bat-rpls-1:         NOTRUN -> [ABORT][7] ([i915#7677])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rpls-1/igt@i915_selftest@live@hangcheck.html

  * igt@i915_suspend@basic-s3-without-i915:
    - bat-mtlp-8:         NOTRUN -> [SKIP][8] ([i915#6645])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@i915_suspend@basic-s3-without-i915.html

  * igt@kms_chamelium_hpd@common-hpd-after-suspend:
    - bat-mtlp-8:         NOTRUN -> [SKIP][9] ([i915#7828])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@kms_chamelium_hpd@common-hpd-after-suspend.html

  * igt@kms_psr@primary_page_flip:
    - bat-rplp-1:         NOTRUN -> [ABORT][10] ([i915#8442] / [i915#8668])
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rplp-1/igt@kms_psr@primary_page_flip.html

  * igt@prime_vgem@basic-fence-mmap:
    - bat-mtlp-8:         NOTRUN -> [SKIP][11] ([i915#3708] / [i915#4077]) +1 similar issue
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@prime_vgem@basic-fence-mmap.html

  * igt@prime_vgem@basic-fence-read:
    - bat-mtlp-8:         NOTRUN -> [SKIP][12] ([i915#3708]) +2 similar issues
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@prime_vgem@basic-fence-read.html

  
#### Possible fixes ####

  * igt@i915_pm_rpm@basic-pci-d3-state:
    - bat-mtlp-8:         [ABORT][13] ([i915#7077] / [i915#7977] / [i915#8668]) -> [PASS][14]
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-mtlp-8/igt@i915_pm_rpm@basic-pci-d3-state.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-mtlp-8/igt@i915_pm_rpm@basic-pci-d3-state.html

  * igt@i915_selftest@live@gt_pm:
    - bat-rpls-2:         [DMESG-FAIL][15] ([i915#4258] / [i915#7913]) -> [PASS][16]
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-rpls-2/igt@i915_selftest@live@gt_pm.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rpls-2/igt@i915_selftest@live@gt_pm.html

  * igt@i915_selftest@live@migrate:
    - bat-dg2-11:         [DMESG-WARN][17] ([i915#7699]) -> [PASS][18]
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-dg2-11/igt@i915_selftest@live@migrate.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-dg2-11/igt@i915_selftest@live@migrate.html

  * igt@i915_selftest@live@requests:
    - bat-rpls-1:         [ABORT][19] ([i915#4983] / [i915#7911] / [i915#7920]) -> [PASS][20]
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-rpls-1/igt@i915_selftest@live@requests.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rpls-1/igt@i915_selftest@live@requests.html

  * igt@i915_selftest@live@slpc:
    - bat-rpls-2:         [DMESG-WARN][21] ([i915#6367]) -> [PASS][22]
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-rpls-2/igt@i915_selftest@live@slpc.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rpls-2/igt@i915_selftest@live@slpc.html

  * igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1:
    - bat-rplp-1:         [ABORT][23] ([i915#8442] / [i915#8668]) -> [PASS][24]
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html

  
#### Warnings ####

  * igt@i915_module_load@load:
    - bat-adlp-11:        [DMESG-WARN][25] ([i915#4423]) -> [ABORT][26] ([i915#4423])
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13360/bat-adlp-11/igt@i915_module_load@load.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/bat-adlp-11/igt@i915_module_load@load.html

  
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4258]: https://gitlab.freedesktop.org/drm/intel/issues/4258
  [i915#4423]: https://gitlab.freedesktop.org/drm/intel/issues/4423
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
  [i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
  [i915#6621]: https://gitlab.freedesktop.org/drm/intel/issues/6621
  [i915#6645]: https://gitlab.freedesktop.org/drm/intel/issues/6645
  [i915#7077]: https://gitlab.freedesktop.org/drm/intel/issues/7077
  [i915#7677]: https://gitlab.freedesktop.org/drm/intel/issues/7677
  [i915#7699]: https://gitlab.freedesktop.org/drm/intel/issues/7699
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#7911]: https://gitlab.freedesktop.org/drm/intel/issues/7911
  [i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
  [i915#7920]: https://gitlab.freedesktop.org/drm/intel/issues/7920
  [i915#7977]: https://gitlab.freedesktop.org/drm/intel/issues/7977
  [i915#8442]: https://gitlab.freedesktop.org/drm/intel/issues/8442
  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7377 -> IGTPW_9373

  CI-20190529: 20190529
  CI_DRM_13360: 033ea4a472c5dc3646d343799a2971b4817acfb5 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_9373: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/index.html
  IGT_7377: d1574543ba4bb322597345530053475c07be0eb9 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git


Testlist changes
----------------

-igt@xe_sysfs_tile@physical_vram_size_bytes

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9373/index.html

[-- Attachment #2: Type: text/html, Size: 9350 bytes --]

[igt-dev] ○ CI.xeBAT: info for Speedup IGT build time (rev5)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

== Series Details ==

Series: Speedup IGT build time (rev5)
URL   : https://patchwork.freedesktop.org/series/120233/
State : info

== Summary ==

Participating hosts:
"bat-pvc-2n""bat-atsm-2n""bat-dg2-oem2n""bat-adlp-7n"Missing hosts results[0]:
Results: [IGTPW_9373](https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_9373/index.html)



[-- Attachment #2: Type: text/html, Size: 827 bytes --]

[igt-dev] ○ CI.xeBAT: info for Speedup IGT build time (rev5)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

== Series Details ==

Series: Speedup IGT build time (rev5)
URL   : https://patchwork.freedesktop.org/series/120233/
State : info

== Summary ==

Participating hosts:
bat-pvc-2
bat-atsm-2
bat-dg2-oem2
bat-adlp-7
Missing hosts results[0]:
Results: [IGTPW_9373](https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_9373/index.html)



[-- Attachment #2: Type: text/html, Size: 843 bytes --]