[igt-dev] [PATCH i-g-t 0/3] Validate Xe field descriptions at build time

[igt-dev] [PATCH i-g-t 0/3] Validate Xe field descriptions at build time

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Some fields shall be mandatory on every subtest. Add a logic to check
if those are filled, enforcing it during build time.

We should probably merge patches 1 and 2 first, as currently, 83
subtests are not properly documented on Xe driver.

Mauro Carvalho Chehab (3):
  xe_test_config.json: mark some fields are mandatory at subtest level
  scripts/test_list.py: add a check for missing features
  [RFC] scripts/test_list.py: enforce mandatory fields to be filled

 scripts/test_list.py         | 18 ++++++++++++++++--
 tests/xe/xe_test_config.json |  5 +++++
 2 files changed, 21 insertions(+), 2 deletions(-)

-- 
2.41.0

[igt-dev] [PATCH i-g-t 1/3] xe_test_config.json: mark some fields are mandatory at subtest level

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Some IGT Xe tests were added without proper documentation.
Currently, igt_doc.py doesn't know what fields are mandatory or
not, so it can't enforce it.

Add a field to mark those.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/xe/xe_test_config.json | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/xe/xe_test_config.json b/tests/xe/xe_test_config.json
index 08c39de74384..8fb65d02d840 100644
--- a/tests/xe/xe_test_config.json
+++ b/tests/xe/xe_test_config.json
@@ -5,6 +5,7 @@
     "fields": {
         "Category": {
             "_properties_": {
+                "mandatory": true,
                 "description": "Contains the major group for the tested functionality, being hardware, software or firmware"
             },
             "Mega feature": {
@@ -13,10 +14,12 @@
                 },
                 "Sub-category": {
                     "_properties_": {
+                        "mandatory": true,
                         "description": "Contains the the technical feature/functionality"
                     },
                     "Functionality": {
                         "_properties_": {
+                            "mandatory": true,
                             "description": "Groups page table tests on buckets containg more detailed functionality"
                         },
                         "Feature": {
@@ -27,6 +30,7 @@
                     },
                     "Run type": {
                         "_properties_": {
+                            "mandatory": true,
                             "description": "Defines what category of testlist it belongs",
                             "order": [
                                 "boot",
@@ -75,6 +79,7 @@
         },
         "Description" : {
             "_properties_": {
+                "mandatory": true,
                 "description": "Provides a description for the test/subtest."
             }
         }
-- 
2.41.0

[igt-dev] [PATCH i-g-t 2/3] scripts/test_list.py: add a check for missing features

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Some fields shall always be present at the documentation.

Add a check for it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index 4f580fb3de58..c23d6d735edf 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -936,13 +936,27 @@ class TestList:
         if self.filters:
             print("NOTE: test checks are affected by filters")
 
+        mandatory_fields = set()
+        for field, item in self.props.items():
+            if item["_properties_"].get("mandatory"):
+                    mandatory_fields.add(field)
+
         doc_subtests = set()
 
         args_regex = re.compile(r'\<[^\>]+\>')
 
-        for subtest in self.get_subtests()[""]:
+        missing_mandatory_fields = False
+
+        subtests = self.expand_dictionary(True)
+        for subtest, data in sorted(subtests.items()):
             subtest = "@".join(subtest.split("@")[:3])
             subtest = args_regex.sub(r'\\d+', subtest)
+
+            for field in mandatory_fields:
+                if field not in data:
+                    print(f"Warning: {subtest} {field} documentation is missing")
+                    missing_mandatory_fields = True
+
             doc_subtests.add(subtest)
 
         doc_subtests = list(sorted(doc_subtests))
-- 
2.41.0

[igt-dev] [PATCH i-g-t 3/3] [RFC] scripts/test_list.py: enforce mandatory fields to be filled

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

If they won't be filled, compilation will break.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index c23d6d735edf..d52cf23a18ad 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -1000,7 +1000,7 @@ class TestList:
         if run_missing:
             for test_name in run_missing:
                 print(f'Warning: Missing documentation for {test_name}')
-        if doc_uneeded or run_missing:
+        if doc_uneeded or run_missing or missing_mandatory_fields:
             sys.exit(1)
 
     #
-- 
2.41.0

[igt-dev] ✗ Fi.CI.BUILD: failure for Validate Xe field descriptions at build time

[Patchwork]

== Series Details ==

Series: Validate Xe field descriptions at build time
URL   : https://patchwork.freedesktop.org/series/122285/
State : failure

== Summary ==

IGT patchset build failed on latest successful build
85557af77f06ffef7e909c6008788ac34a011b65 tests/intel-ci: fix xe-fast-feedback.testlist

Tail of build.log:
Warning: igt@xe_ccs@ctrl-surf-copy-new-ctx Functionality documentation is missing
Warning: igt@xe_ccs@ctrl-surf-copy-new-ctx Sub-category documentation is missing
Warning: igt@xe_ccs@suspend-resume Category documentation is missing
Warning: igt@xe_ccs@suspend-resume Functionality documentation is missing
Warning: igt@xe_ccs@suspend-resume Sub-category documentation is missing
Warning: igt@xe_exercise_blt@fast-copy Category documentation is missing
Warning: igt@xe_exercise_blt@fast-copy Functionality documentation is missing
Warning: igt@xe_exercise_blt@fast-copy Sub-category documentation is missing
Warning: igt@xe_exercise_blt@fast-copy-emit Category documentation is missing
Warning: igt@xe_exercise_blt@fast-copy-emit Functionality documentation is missing
Warning: igt@xe_exercise_blt@fast-copy-emit Sub-category documentation is missing
Warning: igt@xe_media_fill@media-fill Category documentation is missing
Warning: igt@xe_media_fill@media-fill Functionality documentation is missing
Warning: igt@xe_media_fill@media-fill Sub-category documentation is missing
Warning: igt@xe_sysfs_scheduler@job_timeout_ms-invalid Category documentation is missing
Warning: igt@xe_sysfs_scheduler@job_timeout_ms-min-max Category documentation is missing
Warning: igt@xe_sysfs_scheduler@job_timeout_ms-nonprivileged-user Category documentation is missing
Warning: igt@xe_sysfs_scheduler@preempt_timeout_us-invalid Category documentation is missing
Warning: igt@xe_sysfs_scheduler@preempt_timeout_us-min-max Category documentation is missing
Warning: igt@xe_sysfs_scheduler@preempt_timeout_us-nonprivileged-user Category documentation is missing
Warning: igt@xe_sysfs_scheduler@timeslice_duration_us-invalid Category documentation is missing
Warning: igt@xe_sysfs_scheduler@timeslice_duration_us-min-max Category documentation is missing
Warning: igt@xe_sysfs_scheduler@timeslice_duration_us-nonprivileged-user Category documentation is missing
Warning: igt@xe_sysfs_tile@physical_vram_size_bytes Run type documentation is missing
Warning: igt@xe_sysfs_tile@physical_vram_size_bytes Category documentation is missing
Warning: igt@xe_sysfs_tile@physical_vram_size_bytes Functionality documentation is missing
Warning: igt@xe_sysfs_tile@physical_vram_size_bytes Sub-category documentation is missing
Warning: igt@xe_uevent@fake_reset_uevent_listener Run type documentation is missing
Warning: igt@xe_uevent@fake_reset_uevent_listener Category documentation is missing
Warning: igt@xe_uevent@fake_reset_uevent_listener Functionality documentation is missing
Warning: igt@xe_uevent@fake_reset_uevent_listener Sub-category documentation is missing
Warning: igt@xe_vm@mmap-style-bind-all Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-either-side-full Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-either-side-partial Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-either-side-partial-hammer Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-either-side-partial-large-page-hammer Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-either-side-partial-split-page-hammer Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-end Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-front Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-many-all Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-many-either-side-partial Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-many-either-side-partial-hammer Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-one-partial Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-userptr-all Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-userptr-either-side-full Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-userptr-either-side-partial Functionality documentation is missing
Warning: igt@xe_vm@mmap-style-bind-userptr-one-partial Functionality documentation is missing
[1618/1622] Generating kms_tests.html with a custom command.
[1619/1622] Generating i915_tests.rst with a custom command.
ninja: build stopped: subcommand failed.