[igt-dev] [PATCH i-g-t 0/2] Properly fix compilation breakage when Chamelium is disabled

[igt-dev] [PATCH i-g-t 0/2] Properly fix compilation breakage when Chamelium is disabled

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

There is an issue on checking KMS documentation: there is a single JSON
config that include files for both KMS and chamelium tests.

As it is possible to disable just chamelium, validating documentation
will fail, as the commented tests won't be parsed.

Fix it the right way by disabling KMS testplan check if chamelium
is disabled.

Mauro Carvalho Chehab (2):
  testplan/meson.build: disable kms check testlist if not chamelium
  scripts/test_list.py: fix and simplify missing doc check

 docs/testplan/meson.build | 15 +++++++++------
 scripts/test_list.py      | 34 ++++------------------------------
 2 files changed, 13 insertions(+), 36 deletions(-)

-- 
2.41.0

[igt-dev] [PATCH i-g-t 1/2] testplan/meson.build: disable kms check testlist if not chamelium

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

The KMS JSON file contains some tests that can optionally be disabled.
When those are disabled, the code can't check for missing documents,
as the JSON file specifies both enabled and disabled tests.

Detect that to avoid build problems.

While here, remove a left-over dictionary (xe_test_dict).

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 docs/testplan/meson.build | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/docs/testplan/meson.build b/docs/testplan/meson.build
index 16dd578266cd..9e0c3aee06af 100644
--- a/docs/testplan/meson.build
+++ b/docs/testplan/meson.build
@@ -11,12 +11,19 @@ kms_test_config = join_paths(source_root, 'tests', 'intel', 'kms_test_config.jso
 i915_test_config = join_paths(source_root, 'tests', 'intel', 'i915_test_config.json')
 
 check_testlist = []
+kms_check_testlist = []
 if build_tests
 	doc_dependencies = testlist_files
 	# Check if documentation matches the actual tests and tests can run
 	if not meson.is_cross_build()
 		build_info += 'Will Check if documentation is in sync with testlist'
 		check_testlist = [ '--check-testlist', '--igt-build-path', build_root ]
+
+		if not chamelium.found()
+			warning('WARNING: Will not check if documentation is in sync for KMS as chamelium is disabled')
+		else
+			kms_check_testlist = check_testlist
+		endif
 	else
 		warning('WARNING: Will not check if documentation is in sync with testlist')
 	endif
@@ -24,20 +31,16 @@ else
 	doc_dependencies = []
 endif
 
-xe_test_dict = {
-		 'xe_tests': { 'input': xe_test_config, 'extra_args': check_testlist }
-	       }
-
 if build_xe
 	test_dict = {
 		'i915_tests': { 'input': i915_test_config, 'extra_args': check_testlist },
-		'kms_tests': { 'input': kms_test_config, 'extra_args': check_testlist },
+		'kms_tests': { 'input': kms_test_config, 'extra_args': kms_check_testlist },
 		'xe_tests': { 'input': xe_test_config, 'extra_args': check_testlist }
 	    }
 else
 	test_dict = {
 	      'i915_tests': { 'input': i915_test_config, 'extra_args': check_testlist },
-	      'kms_tests': { 'input': kms_test_config, 'extra_args': check_testlist }
+	      'kms_tests': { 'input': kms_test_config, 'extra_args': kms_check_testlist }
 	    }
 endif
 
-- 
2.41.0

Re: [igt-dev] [PATCH i-g-t 1/2] testplan/meson.build: disable kms check testlist if not chamelium

[Kamil Konieczny]

Hi Mauro,

thank you for a fix.

On 2023-10-10 at 15:17:19 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> The KMS JSON file contains some tests that can optionally be disabled.
> When those are disabled, the code can't check for missing documents,
> as the JSON file specifies both enabled and disabled tests.
> 
> Detect that to avoid build problems.
> 
> While here, remove a left-over dictionary (xe_test_dict).
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  docs/testplan/meson.build | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/docs/testplan/meson.build b/docs/testplan/meson.build
> index 16dd578266cd..9e0c3aee06af 100644
> --- a/docs/testplan/meson.build
> +++ b/docs/testplan/meson.build
> @@ -11,12 +11,19 @@ kms_test_config = join_paths(source_root, 'tests', 'intel', 'kms_test_config.jso
>  i915_test_config = join_paths(source_root, 'tests', 'intel', 'i915_test_config.json')
>  
>  check_testlist = []
> +kms_check_testlist = []
>  if build_tests
>  	doc_dependencies = testlist_files
>  	# Check if documentation matches the actual tests and tests can run
>  	if not meson.is_cross_build()
>  		build_info += 'Will Check if documentation is in sync with testlist'
>  		check_testlist = [ '--check-testlist', '--igt-build-path', build_root ]
> +
> +		if not chamelium.found()
> +			warning('WARNING: Will not check if documentation is in sync for KMS as chamelium is disabled')
> +		else
> +			kms_check_testlist = check_testlist
> +		endif
>  	else
>  		warning('WARNING: Will not check if documentation is in sync with testlist')
>  	endif
> @@ -24,20 +31,16 @@ else
>  	doc_dependencies = []
>  endif
>  
> -xe_test_dict = {
> -		 'xe_tests': { 'input': xe_test_config, 'extra_args': check_testlist }
> -	       }
> -
>  if build_xe
>  	test_dict = {
>  		'i915_tests': { 'input': i915_test_config, 'extra_args': check_testlist },
> -		'kms_tests': { 'input': kms_test_config, 'extra_args': check_testlist },
> +		'kms_tests': { 'input': kms_test_config, 'extra_args': kms_check_testlist },
>  		'xe_tests': { 'input': xe_test_config, 'extra_args': check_testlist }
>  	    }
>  else
>  	test_dict = {
>  	      'i915_tests': { 'input': i915_test_config, 'extra_args': check_testlist },
> -	      'kms_tests': { 'input': kms_test_config, 'extra_args': check_testlist }
> +	      'kms_tests': { 'input': kms_test_config, 'extra_args': kms_check_testlist }
>  	    }
>  endif
>  
> -- 
> 2.41.0
> 

[igt-dev] [PATCH i-g-t 2/2] scripts/test_list.py: fix and simplify missing doc check

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Instead of using regexes, we can simply use sets to check what's
missing. This make checks easier and less error-prone.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 34 ++++------------------------------
 1 file changed, 4 insertions(+), 30 deletions(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index daca290a02bd..c0462188ba8a 100644
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -1105,39 +1105,13 @@ class TestList:
 
             doc_subtests.add(subtest)
 
-        doc_subtests = list(sorted(doc_subtests))
-
         # Get a list of tests from
-        run_subtests = self.get_testlist()
+        run_subtests = set(self.get_testlist())
 
-        # Compare arrays
+        # Compare sets
 
-        run_missing = []
-        doc_uneeded = []
-
-        test_regex = r""
-        for doc_test in doc_subtests:
-            if test_regex != r"":
-                test_regex += r"|"
-            test_regex += doc_test
-
-        test_regex = re.compile(r'^(' + test_regex + r')$')
-
-        for doc_test in doc_subtests:
-            found = False
-            for run_test in run_subtests:
-                if test_regex.match(run_test):
-                    found = True
-                    break
-            if not found:
-                doc_uneeded.append(doc_test)
-
-        for run_test in run_subtests:
-            found = False
-            if test_regex.match(run_test):
-                found = True
-            if not found:
-                run_missing.append(run_test)
+        run_missing = list(sorted(run_subtests - doc_subtests))
+        doc_uneeded = list(sorted(doc_subtests - run_subtests))
 
         if doc_uneeded:
             for test_name in doc_uneeded:
-- 
2.41.0

Re: [igt-dev] [PATCH i-g-t 2/2] scripts/test_list.py: fix and simplify missing doc check

[Kamil Konieczny]

Hi Mauro,
On 2023-10-10 at 15:17:20 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Instead of using regexes, we can simply use sets to check what's
> missing. This make checks easier and less error-prone.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  scripts/test_list.py | 34 ++++------------------------------
>  1 file changed, 4 insertions(+), 30 deletions(-)
> 
> diff --git a/scripts/test_list.py b/scripts/test_list.py
> index daca290a02bd..c0462188ba8a 100644
> --- a/scripts/test_list.py
> +++ b/scripts/test_list.py
> @@ -1105,39 +1105,13 @@ class TestList:
>  
>              doc_subtests.add(subtest)
>  
> -        doc_subtests = list(sorted(doc_subtests))
> -
>          # Get a list of tests from
> -        run_subtests = self.get_testlist()
> +        run_subtests = set(self.get_testlist())
>  
> -        # Compare arrays
> +        # Compare sets
>  
> -        run_missing = []
> -        doc_uneeded = []
> -
> -        test_regex = r""
> -        for doc_test in doc_subtests:
> -            if test_regex != r"":
> -                test_regex += r"|"
> -            test_regex += doc_test
> -
> -        test_regex = re.compile(r'^(' + test_regex + r')$')
> -
> -        for doc_test in doc_subtests:
> -            found = False
> -            for run_test in run_subtests:
> -                if test_regex.match(run_test):
> -                    found = True
> -                    break
> -            if not found:
> -                doc_uneeded.append(doc_test)
> -
> -        for run_test in run_subtests:
> -            found = False
> -            if test_regex.match(run_test):
> -                found = True
> -            if not found:
> -                run_missing.append(run_test)
> +        run_missing = list(sorted(run_subtests - doc_subtests))
> +        doc_uneeded = list(sorted(doc_subtests - run_subtests))
>  
>          if doc_uneeded:
>              for test_name in doc_uneeded:
> -- 
> 2.41.0
> 

[igt-dev] ✗ Fi.CI.BAT: failure for Properly fix compilation breakage when Chamelium is disabled

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 252 bytes --]

== Series Details ==

Series: Properly fix compilation breakage when Chamelium is disabled
URL   : https://patchwork.freedesktop.org/series/124903/
State : failure

== Summary ==

Series 124903 revision 1 was fully merged or fully failed: no git log



[-- Attachment #2: Type: text/html, Size: 721 bytes --]