[igt-dev] [PATCH i-g-t 00/11] Improve igt_doc.py

[igt-dev] [PATCH i-g-t 00/11] Improve igt_doc.py

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

This series do some improvements at the igt_doc.py tool:

- It does some tweaks at the Xe documentation hierarchy;
- It adds support to generate test lists from documentation;
- It splits the script on two parts: one with the TestList
  class definition and another one with the toll itself;
- While here, add missing documentation for two new tests;
- Check if the documentation matches the actual tests at
  build time, in order to prevent it to become outdated.

Mauro Carvalho Chehab (11):
  xe_test_config.json: do some adjustments at the output hierarchy
  scripts/igt_doc.py: move show_subtests logic to the class
  scripts/igt_doc.py: add support to generate testlists
  scripts/igt_doc.py: prepare to place TestList class on a separate file
  scripts/test_list.py: prepare to place class on a separate file
  scripts/igt_doc.py: re-introduce it by calling test_list.py
  scripts/test_list.py: better handle documentation validation issues
  scripts/test_list.py: add support for igt_simple_main
  testplan/meson.build: Validate documentation at build time
  xe/xe_vm: add documentation for igt@xe_vm@userptr-invalid
  xe/xe_noexec_ping_pong: document subtest

 docs/testplan/meson.build      |    8 +-
 meson.build                    |    1 +
 scripts/igt_doc.py             | 1038 +-----------------------------
 scripts/test_list.py           | 1084 ++++++++++++++++++++++++++++++++
 tests/xe/xe_noexec_ping_pong.c |   26 +-
 tests/xe/xe_test_config.json   |   30 +-
 tests/xe/xe_vm.c               |   10 +-
 7 files changed, 1143 insertions(+), 1054 deletions(-)
 create mode 100755 scripts/test_list.py

-- 
2.39.2

[igt-dev] [PATCH i-g-t 01/11] xe_test_config.json: do some adjustments at the output hierarchy

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Place "Mega feature" inside the same hierarchical level,
placing it on a higher level than sub-category, as this provides
a higher level of group for functionalities.

While here, better document the fields.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/xe/xe_test_config.json | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/tests/xe/xe_test_config.json b/tests/xe/xe_test_config.json
index 6c474d6053f3..666fe6cd315c 100644
--- a/tests/xe/xe_test_config.json
+++ b/tests/xe/xe_test_config.json
@@ -4,27 +4,27 @@
     "fields": {
         "Category": {
             "_properties_": {
-                "description": "Contains the major group for the tested functionality"
+                "description": "Contains the major group for the tested functionality, being hardware, software or firmware"
             },
-            "Sub-category": {
+            "Mega feature": {
                 "_properties_": {
-                    "description": "Contains the minor group of the functionality"
-                },
-                "Functionality": {
-                    "_properties_": {
-                        "description": "Groups page table tests per functionality"
-                    }
+                    "description": "Contains the mega feature for end to end use case, e. g. the 'sellable' feature."
                 },
-                "Run type": {
+                "Sub-category": {
                     "_properties_": {
-                        "description": "Defines what category of testlist it belongs"
+                        "description": "Contains the the technical feature/functionality"
+                    },
+                    "Functionality": {
+                        "_properties_": {
+                            "description": "Groups page table tests on buckets containg more detailed functionality"
+                        }
+                    },
+                    "Run type": {
+                        "_properties_": {
+                            "description": "Defines what category of testlist it belongs"
+                        }
                     }
                 }
-            },
-            "Mega feature": {
-                "_properties_": {
-                    "description": "Contains the mega feature for E2E use case"
-                }
             }
         },
         "Test category": {
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 01/11] xe_test_config.json: do some adjustments at the output hierarchy

[Kamil Konieczny]

On 2023-03-27 at 15:31:45 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Place "Mega feature" inside the same hierarchical level,
> placing it on a higher level than sub-category, as this provides
> a higher level of group for functionalities.
> 
> While here, better document the fields.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  tests/xe/xe_test_config.json | 30 +++++++++++++++---------------
>  1 file changed, 15 insertions(+), 15 deletions(-)
> 
> diff --git a/tests/xe/xe_test_config.json b/tests/xe/xe_test_config.json
> index 6c474d6053f3..666fe6cd315c 100644
> --- a/tests/xe/xe_test_config.json
> +++ b/tests/xe/xe_test_config.json
> @@ -4,27 +4,27 @@
>      "fields": {
>          "Category": {
>              "_properties_": {
> -                "description": "Contains the major group for the tested functionality"
> +                "description": "Contains the major group for the tested functionality, being hardware, software or firmware"
>              },
> -            "Sub-category": {
> +            "Mega feature": {
>                  "_properties_": {
> -                    "description": "Contains the minor group of the functionality"
> -                },
> -                "Functionality": {
> -                    "_properties_": {
> -                        "description": "Groups page table tests per functionality"
> -                    }
> +                    "description": "Contains the mega feature for end to end use case, e. g. the 'sellable' feature."
>                  },
> -                "Run type": {
> +                "Sub-category": {
>                      "_properties_": {
> -                        "description": "Defines what category of testlist it belongs"
> +                        "description": "Contains the the technical feature/functionality"
> +                    },
> +                    "Functionality": {
> +                        "_properties_": {
> +                            "description": "Groups page table tests on buckets containg more detailed functionality"
> +                        }
> +                    },
> +                    "Run type": {
> +                        "_properties_": {
> +                            "description": "Defines what category of testlist it belongs"
> +                        }
>                      }
>                  }
> -            },
> -            "Mega feature": {
> -                "_properties_": {
> -                    "description": "Contains the mega feature for E2E use case"
> -                }
>              }
>          },
>          "Test category": {
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 02/11] scripts/igt_doc.py: move show_subtests logic to the class

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Simplify the main code and avoid warnings about global vars with
pylint by moving its code to a separate function.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/igt_doc.py | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
index 67ab7984dd11..cc2768ae0847 100755
--- a/scripts/igt_doc.py
+++ b/scripts/igt_doc.py
@@ -1018,6 +1018,25 @@ class TestList:
                 sys.exit(f"{fname}:{file_ln + 1}: Error: unrecognized line. Need to add field at %s?\n\t==> %s" %
                          (self.config_fname, file_line))
 
+    def show_subtests(self, sort_field, filter_field):
+
+        """Show subtests, allowing sort and filter a field """
+
+        if sort_field:
+            test_subtests = tests.get_subtests(sort_field, filter_field)
+            for val_key in sorted(test_subtests.keys()):
+                if not test_subtests[val_key]:
+                    continue
+                if val_key == "":
+                    print("not defined:")
+                else:
+                    print(f"{val_key}:")
+                    for sub in test_subtests[val_key]:
+                        print (f"  {sub}")
+        else:
+            for sub in tests.get_subtests(sort_field, filter_field)[""]:
+                print (sub)
+
 #
 # Main
 #
@@ -1056,20 +1075,7 @@ tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files)
 RUN = 0
 if parse_args.show_subtests:
     RUN = 1
-    if parse_args.sort_field:
-        test_subtests = tests.get_subtests(parse_args.sort_field, parse_args.filter_field)
-        for val_key in sorted(test_subtests.keys()):
-            if not test_subtests[val_key]:
-                continue
-            if val_key == "":
-                print("not defined:")
-            else:
-                print(f"{val_key}:")
-            for sub in test_subtests[val_key]:
-                print (f"  {sub}")
-    else:
-        for sub in tests.get_subtests(parse_args.sort_field, parse_args.filter_field)[""]:
-            print (sub)
+    tests.show_subtests(parse_args.sort_field, parse_args.filter_field)
 
 if parse_args.check_testlist:
     RUN = 1
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 02/11] scripts/igt_doc.py: move show_subtests logic to the class

[Kamil Konieczny]

On 2023-03-27 at 15:31:46 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Simplify the main code and avoid warnings about global vars with
> pylint by moving its code to a separate function.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  scripts/igt_doc.py | 34 ++++++++++++++++++++--------------
>  1 file changed, 20 insertions(+), 14 deletions(-)
> 
> diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
> index 67ab7984dd11..cc2768ae0847 100755
> --- a/scripts/igt_doc.py
> +++ b/scripts/igt_doc.py
> @@ -1018,6 +1018,25 @@ class TestList:
>                  sys.exit(f"{fname}:{file_ln + 1}: Error: unrecognized line. Need to add field at %s?\n\t==> %s" %
>                           (self.config_fname, file_line))
>  
> +    def show_subtests(self, sort_field, filter_field):
> +
> +        """Show subtests, allowing sort and filter a field """
> +
> +        if sort_field:
> +            test_subtests = tests.get_subtests(sort_field, filter_field)
> +            for val_key in sorted(test_subtests.keys()):
> +                if not test_subtests[val_key]:
> +                    continue
> +                if val_key == "":
> +                    print("not defined:")
> +                else:
> +                    print(f"{val_key}:")
> +                    for sub in test_subtests[val_key]:
> +                        print (f"  {sub}")
> +        else:
> +            for sub in tests.get_subtests(sort_field, filter_field)[""]:
> +                print (sub)
> +
>  #
>  # Main
>  #
> @@ -1056,20 +1075,7 @@ tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files)
>  RUN = 0
>  if parse_args.show_subtests:
>      RUN = 1
> -    if parse_args.sort_field:
> -        test_subtests = tests.get_subtests(parse_args.sort_field, parse_args.filter_field)
> -        for val_key in sorted(test_subtests.keys()):
> -            if not test_subtests[val_key]:
> -                continue
> -            if val_key == "":
> -                print("not defined:")
> -            else:
> -                print(f"{val_key}:")
> -            for sub in test_subtests[val_key]:
> -                print (f"  {sub}")
> -    else:
> -        for sub in tests.get_subtests(parse_args.sort_field, parse_args.filter_field)[""]:
> -            print (sub)
> +    tests.show_subtests(parse_args.sort_field, parse_args.filter_field)
>  
>  if parse_args.check_testlist:
>      RUN = 1
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 03/11] scripts/igt_doc.py: add support to generate testlists

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

As the testlists are part of the documentation, add support
to generate them from the documentation on an specified
directory.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/igt_doc.py | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
index cc2768ae0847..efbcee5e74e2 100755
--- a/scripts/igt_doc.py
+++ b/scripts/igt_doc.py
@@ -1037,6 +1037,35 @@ class TestList:
             for sub in tests.get_subtests(sort_field, filter_field)[""]:
                 print (sub)
 
+    def gen_testlist(self, directory, sort_field, filter_field):
+
+        """Generate testlists from the test documentation"""
+
+        test_prefix = os.path.commonprefix(self.get_subtests()[""])
+        test_prefix = re.sub(r'^igt@', '', test_prefix)
+
+        test_subtests = self.get_subtests(sort_field, filter_field)
+
+        for test in test_subtests.keys():  # pylint: disable=C0201,C0206
+            if not test_subtests[test]:
+                continue
+
+            testlist = test.lower()
+            if testlist == "":
+                fname = "other"
+            elif testlist == "bat":
+                fname = "fast-feedback"
+            else:
+                fname = testlist
+
+            fname = directory + "/" + test_prefix + fname + ".testlist"
+            fname = re.sub(r"[\s_]+", "-", fname)
+
+            with open(fname, 'w', encoding='utf8') as handler:
+                for sub in test_subtests[test]:
+                    handler.write (f"{sub}\n")
+                print(f"{fname} created.")
+
 #
 # Main
 #
@@ -1065,6 +1094,8 @@ parser.add_argument("--include-plan", action="store_true",
 parser.add_argument("--igt-build-path",
                     help="Path where the IGT runner is sitting. Used by --check-testlist.",
                     default=IGT_BUILD_PATH)
+parser.add_argument("--gen-testlist",
+                    help="Generate documentation at the GEN_TESTLIST directory, using SORT_FIELD to split the tests. Requires --sort-field.")
 parser.add_argument('--files', nargs='+',
                     help="File name(s) to be processed")
 
@@ -1081,6 +1112,12 @@ if parse_args.check_testlist:
     RUN = 1
     tests.check_tests()
 
+if parse_args.gen_testlist:
+    RUN = 1
+    if not parse_args.sort_field:
+        sys.exit("Need a field to split the testlists")
+    tests.gen_testlist(parse_args.gen_testlist, parse_args.sort_field, parse_args.filter_field)
+
 if parse_args.to_json:
     RUN = 1
     tests.print_json(parse_args.to_json)
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 03/11] scripts/igt_doc.py: add support to generate testlists

[Kamil Konieczny]

On 2023-03-27 at 15:31:47 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> As the testlists are part of the documentation, add support
> to generate them from the documentation on an specified
> directory.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  scripts/igt_doc.py | 37 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 37 insertions(+)
> 
> diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
> index cc2768ae0847..efbcee5e74e2 100755
> --- a/scripts/igt_doc.py
> +++ b/scripts/igt_doc.py
> @@ -1037,6 +1037,35 @@ class TestList:
>              for sub in tests.get_subtests(sort_field, filter_field)[""]:
>                  print (sub)
>  
> +    def gen_testlist(self, directory, sort_field, filter_field):
> +
> +        """Generate testlists from the test documentation"""
> +
> +        test_prefix = os.path.commonprefix(self.get_subtests()[""])
> +        test_prefix = re.sub(r'^igt@', '', test_prefix)
> +
> +        test_subtests = self.get_subtests(sort_field, filter_field)
> +
> +        for test in test_subtests.keys():  # pylint: disable=C0201,C0206
> +            if not test_subtests[test]:
> +                continue
> +
> +            testlist = test.lower()
> +            if testlist == "":
> +                fname = "other"
> +            elif testlist == "bat":
> +                fname = "fast-feedback"
> +            else:
> +                fname = testlist
> +
> +            fname = directory + "/" + test_prefix + fname + ".testlist"
> +            fname = re.sub(r"[\s_]+", "-", fname)
> +
> +            with open(fname, 'w', encoding='utf8') as handler:
> +                for sub in test_subtests[test]:
> +                    handler.write (f"{sub}\n")
> +                print(f"{fname} created.")
> +
>  #
>  # Main
>  #
> @@ -1065,6 +1094,8 @@ parser.add_argument("--include-plan", action="store_true",
>  parser.add_argument("--igt-build-path",
>                      help="Path where the IGT runner is sitting. Used by --check-testlist.",
>                      default=IGT_BUILD_PATH)
> +parser.add_argument("--gen-testlist",
> +                    help="Generate documentation at the GEN_TESTLIST directory, using SORT_FIELD to split the tests. Requires --sort-field.")
>  parser.add_argument('--files', nargs='+',
>                      help="File name(s) to be processed")
>  
> @@ -1081,6 +1112,12 @@ if parse_args.check_testlist:
>      RUN = 1
>      tests.check_tests()
>  
> +if parse_args.gen_testlist:
> +    RUN = 1
> +    if not parse_args.sort_field:
> +        sys.exit("Need a field to split the testlists")
> +    tests.gen_testlist(parse_args.gen_testlist, parse_args.sort_field, parse_args.filter_field)
> +
>  if parse_args.to_json:
>      RUN = 1
>      tests.print_json(parse_args.to_json)
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 04/11] scripts/igt_doc.py: prepare to place TestList class on a separate file

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Do some changes at the code to avoid depending on global vars and
to declare plural ancillary function as static.

Acked-by: Jari Tahvanainen <jari.tahvanainen@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/igt_doc.py | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
index efbcee5e74e2..8eed875b0681 100755
--- a/scripts/igt_doc.py
+++ b/scripts/igt_doc.py
@@ -51,7 +51,7 @@ def _sort_using_array(item, array):
 # As suggested at https://stackoverflow.com/questions/18902608/generating-the-plural-form-of-a-noun/19018986
 #
 
-def plural(field):
+def _plural(field):
 
     """
     Poor man's conversion to plural.
@@ -232,7 +232,8 @@ class TestList:
     Description: test ioctls
     """
 
-    def __init__(self, config_fname, include_plan, file_list):
+    def __init__(self, config_fname, include_plan = False, file_list = False,
+                 igt_build_path = None, igt_runner = None):
         self.doc = {}
         self.test_number = 0
         self.config = None
@@ -240,6 +241,8 @@ class TestList:
         self.plan_filenames = []
         self.props = {}
         self.config_fname = config_fname
+        self.igt_build_path = igt_build_path
+        self.igt_runner = igt_runner
         self.level_count = 0
         self.field_list = {}
         self.title = None
@@ -264,7 +267,7 @@ class TestList:
 
                 field_lc = field.lower()
                 self.field_list[field_lc] = field
-                field_plural = plural(field_lc)
+                field_plural = _plural(field_lc)
                 if field_lc != field_plural:
                     self.field_list[field_plural] = field
 
@@ -757,6 +760,9 @@ class TestList:
 
         """Compare documented subtests with the IGT test list"""
 
+        if not self.igt_build_path or not self.igt_runner:
+            sys.exit("Need the IGT build path and igt_runner executable file name")
+
         doc_subtests = sorted(self.get_subtests()[""])
 
         for i in range(0, len(doc_subtests)): # pylint: disable=C0200
@@ -766,9 +772,9 @@ class TestList:
 
         # Get a list of tests from
         try:
-            result = subprocess.run([ f"{IGT_BUILD_PATH}/{IGT_RUNNER}",
+            result = subprocess.run([ f"{self.igt_build_path}/{self.igt_runner}",
                                     "-L", "-t",  test_prefix,
-                                    f"{IGT_BUILD_PATH}/tests"], check = True,
+                                    f"{self.igt_build_path}/tests"], check = True,
                                     stdout=subprocess.PIPE, universal_newlines=True)
         except subprocess.CalledProcessError as sub_err:
             print(sub_err.stderr)
@@ -1023,7 +1029,7 @@ class TestList:
         """Show subtests, allowing sort and filter a field """
 
         if sort_field:
-            test_subtests = tests.get_subtests(sort_field, filter_field)
+            test_subtests = self.get_subtests(sort_field, filter_field)
             for val_key in sorted(test_subtests.keys()):
                 if not test_subtests[val_key]:
                     continue
@@ -1034,7 +1040,7 @@ class TestList:
                     for sub in test_subtests[val_key]:
                         print (f"  {sub}")
         else:
-            for sub in tests.get_subtests(sort_field, filter_field)[""]:
+            for sub in self.get_subtests(sort_field, filter_field)[""]:
                 print (sub)
 
     def gen_testlist(self, directory, sort_field, filter_field):
@@ -1101,7 +1107,8 @@ parser.add_argument('--files', nargs='+',
 
 parse_args = parser.parse_args()
 
-tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files)
+tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files,
+                 IGT_BUILD_PATH, IGT_RUNNER)
 
 RUN = 0
 if parse_args.show_subtests:
-- 
2.39.2

[igt-dev] [PATCH i-g-t 05/11] scripts/test_list.py: prepare to place class on a separate file

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Rename scripts/igt_doc.py to scripts/test_list.py, preparing to
split the main function from the class definition.

That allows re-using the class by other programs as well. It
also makes the module better organized, and ensures that all
variables inside the class are not global.

While this could be folded with the next patch, by splitting it
on two, it makes a lot easier to review the code changes, as most
of the contents of the igt_doc.py are actually the TestList class.

The next patch will re-introduce scripts/igt_doc.py, dropping
the global vars and the main code from test_list.py.

Acked-by: Jari Tahvanainen <jari.tahvanainen@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/{igt_doc.py => test_list.py} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename scripts/{igt_doc.py => test_list.py} (100%)

diff --git a/scripts/igt_doc.py b/scripts/test_list.py
similarity index 100%
rename from scripts/igt_doc.py
rename to scripts/test_list.py
-- 
2.39.2

[igt-dev] [PATCH i-g-t 06/11] scripts/igt_doc.py: re-introduce it by calling test_list.py

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Now that the class on a separate file, move the argument parser
to igt_doc.py.

Acked-by: Jari Tahvanainen <jari.tahvanainen@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/igt_doc.py   | 80 ++++++++++++++++++++++++++++++++++++++++++++
 scripts/test_list.py | 67 -------------------------------------
 2 files changed, 80 insertions(+), 67 deletions(-)
 create mode 100755 scripts/igt_doc.py

diff --git a/scripts/igt_doc.py b/scripts/igt_doc.py
new file mode 100755
index 000000000000..547cb81bce02
--- /dev/null
+++ b/scripts/igt_doc.py
@@ -0,0 +1,80 @@
+#!/usr/bin/env python3
+# pylint: disable=C0301
+# SPDX-License-Identifier: (GPL-2.0 OR MIT)
+
+## Copyright (C) 2023    Intel Corporation                 ##
+## Author: Mauro Carvalho Chehab <mchehab@kernel.org>      ##
+##                                                         ##
+## Allow keeping inlined test documentation and validate   ##
+## if the documentation is kept updated.                   ##
+
+"""Maintain test plan and test implementation documentation on IGT."""
+
+import argparse
+
+from test_list import TestList
+
+IGT_BUILD_PATH = 'build'
+IGT_RUNNER = 'runner/igt_runner'
+
+parser = argparse.ArgumentParser(description = "Print formatted kernel documentation to stdout.",
+                                 formatter_class = argparse.ArgumentDefaultsHelpFormatter,
+                                 epilog = 'If no action specified, assume --rest.')
+parser.add_argument("--config", required = True,
+                    help="JSON file describing the test plan template")
+parser.add_argument("--rest",
+                    help="Output documentation from the source files in REST file.")
+parser.add_argument("--per-test", action="store_true",
+                    help="Modifies ReST output to print subtests per test.")
+parser.add_argument("--to-json",
+                    help="Output test documentation in JSON format as TO_JSON file")
+parser.add_argument("--show-subtests", action="store_true",
+                    help="Shows the name of the documented subtests in alphabetical order.")
+parser.add_argument("--sort-field",
+                    help="modify --show-subtests to sort output based on SORT_FIELD value")
+parser.add_argument("--filter-field",
+                    help="modify --show-subtests to filter output based a regex given by FILTER_FIELD=~'regex'")
+parser.add_argument("--check-testlist", action="store_true",
+                    help="Compare documentation against IGT runner testlist.")
+parser.add_argument("--include-plan", action="store_true",
+                    help="Include test plans, if any.")
+parser.add_argument("--igt-build-path",
+                    help="Path where the IGT runner is sitting. Used by --check-testlist.",
+                    default=IGT_BUILD_PATH)
+parser.add_argument("--gen-testlist",
+                    help="Generate documentation at the GEN_TESTLIST directory, using SORT_FIELD to split the tests. Requires --sort-field.")
+parser.add_argument("--igt-runner",
+                    help="Path where the IGT runner is sitting. Used by --check-testlist.",
+                    default=IGT_RUNNER)
+parser.add_argument('--files', nargs='+',
+                    help="File name(s) to be processed")
+
+parse_args = parser.parse_args()
+
+tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files,
+                 parse_args.igt_build_path, parse_args.igt_runner)
+
+RUN = 0
+if parse_args.show_subtests:
+    RUN = 1
+    tests.show_subtests(parse_args.sort_field, parse_args.filter_field)
+
+if parse_args.check_testlist:
+    RUN = 1
+    tests.check_tests()
+
+if parse_args.gen_testlist:
+    RUN = 1
+    if not parse_args.sort_field:
+        sys.exit("Need a field to split the testlists")
+    tests.gen_testlist(parse_args.gen_testlist, parse_args.sort_field, parse_args.filter_field)
+
+if parse_args.to_json:
+    RUN = 1
+    tests.print_json(parse_args.to_json)
+
+if not RUN or parse_args.rest:
+    if parse_args.per_test:
+        tests.print_rest_flat(parse_args.rest)
+    else:
+        tests.print_nested_rest(parse_args.rest)
diff --git a/scripts/test_list.py b/scripts/test_list.py
index 8eed875b0681..73c1794caec5 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -10,7 +10,6 @@
 
 """Maintain test plan and test implementation documentation on IGT."""
 
-import argparse
 import glob
 import json
 import os
@@ -22,9 +21,6 @@ MIN_PYTHON = (3, 6)
 if sys.version_info < MIN_PYTHON:
     sys.exit("Python %s.%s or later is required.\n" % MIN_PYTHON) # pylint: disable=C0209
 
-IGT_BUILD_PATH = 'build'
-IGT_RUNNER = 'runner/igt_runner'
-
 #
 # ancillary functions to sort dictionary hierarchy
 #
@@ -1071,66 +1067,3 @@ class TestList:
                 for sub in test_subtests[test]:
                     handler.write (f"{sub}\n")
                 print(f"{fname} created.")
-
-#
-# Main
-#
-
-parser = argparse.ArgumentParser(description = "Print formatted kernel documentation to stdout.",
-                                 formatter_class = argparse.ArgumentDefaultsHelpFormatter,
-                                 epilog = 'If no action specified, assume --rest.')
-parser.add_argument("--config", required = True,
-                    help="JSON file describing the test plan template")
-parser.add_argument("--rest",
-                    help="Output documentation from the source files in REST file.")
-parser.add_argument("--per-test", action="store_true",
-                    help="Modifies ReST output to print subtests per test.")
-parser.add_argument("--to-json",
-                    help="Output test documentation in JSON format as TO_JSON file")
-parser.add_argument("--show-subtests", action="store_true",
-                    help="Shows the name of the documented subtests in alphabetical order.")
-parser.add_argument("--sort-field",
-                    help="modify --show-subtests to sort output based on SORT_FIELD value")
-parser.add_argument("--filter-field",
-                    help="modify --show-subtests to filter output based a regex given by FILTER_FIELD=~'regex'")
-parser.add_argument("--check-testlist", action="store_true",
-                    help="Compare documentation against IGT runner testlist.")
-parser.add_argument("--include-plan", action="store_true",
-                    help="Include test plans, if any.")
-parser.add_argument("--igt-build-path",
-                    help="Path where the IGT runner is sitting. Used by --check-testlist.",
-                    default=IGT_BUILD_PATH)
-parser.add_argument("--gen-testlist",
-                    help="Generate documentation at the GEN_TESTLIST directory, using SORT_FIELD to split the tests. Requires --sort-field.")
-parser.add_argument('--files', nargs='+',
-                    help="File name(s) to be processed")
-
-parse_args = parser.parse_args()
-
-tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files,
-                 IGT_BUILD_PATH, IGT_RUNNER)
-
-RUN = 0
-if parse_args.show_subtests:
-    RUN = 1
-    tests.show_subtests(parse_args.sort_field, parse_args.filter_field)
-
-if parse_args.check_testlist:
-    RUN = 1
-    tests.check_tests()
-
-if parse_args.gen_testlist:
-    RUN = 1
-    if not parse_args.sort_field:
-        sys.exit("Need a field to split the testlists")
-    tests.gen_testlist(parse_args.gen_testlist, parse_args.sort_field, parse_args.filter_field)
-
-if parse_args.to_json:
-    RUN = 1
-    tests.print_json(parse_args.to_json)
-
-if not RUN or parse_args.rest:
-    if parse_args.per_test:
-        tests.print_rest_flat(parse_args.rest)
-    else:
-        tests.print_nested_rest(parse_args.rest)
-- 
2.39.2

[igt-dev] [PATCH i-g-t 07/11] scripts/test_list.py: better handle documentation validation issues

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Produce warnings and exit with an error when test plan is out of sync.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index 73c1794caec5..7d35deda3329 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -803,16 +803,14 @@ class TestList:
                 run_missing.append(run_test)
 
         if doc_uneeded:
-            print("Unused documentation")
             for test_name in doc_uneeded:
-                print(test_name)
+                print(f"Warning: Documented {test_name} doesn't exist on source files")
 
         if run_missing:
-            if doc_uneeded:
-                print()
-            print("Missing documentation")
             for test_name in run_missing:
-                print(test_name)
+                print(f'Warning: Missing documentation for {test_name}')
+        if doc_uneeded or run_missing:
+            sys.exit(1)
 
     #
     # File handling methods
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 07/11] scripts/test_list.py: better handle documentation validation issues

[Kamil Konieczny]

On 2023-03-27 at 15:31:51 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Produce warnings and exit with an error when test plan is out of sync.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  scripts/test_list.py | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/scripts/test_list.py b/scripts/test_list.py
> index 73c1794caec5..7d35deda3329 100755
> --- a/scripts/test_list.py
> +++ b/scripts/test_list.py
> @@ -803,16 +803,14 @@ class TestList:
>                  run_missing.append(run_test)
>  
>          if doc_uneeded:
> -            print("Unused documentation")
>              for test_name in doc_uneeded:
> -                print(test_name)
> +                print(f"Warning: Documented {test_name} doesn't exist on source files")
>  
>          if run_missing:
> -            if doc_uneeded:
> -                print()
> -            print("Missing documentation")
>              for test_name in run_missing:
> -                print(test_name)
> +                print(f'Warning: Missing documentation for {test_name}')
> +        if doc_uneeded or run_missing:
> +            sys.exit(1)
>  
>      #
>      # File handling methods
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 08/11] scripts/test_list.py: add support for igt_simple_main

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

when igt_simple_main is used, there will be a single unamed
subtest. Add support for it.

On such case, the documentation will be like:

	/**
	 * TEST: Test foo
	 * Category: Software building block
	 * Sub-category: foo
	 * Test category: functionality test
	 * Run type: BAT
	 *
	 * SUBTEST:
	 * Description: foo description
	 */

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 scripts/test_list.py | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/scripts/test_list.py b/scripts/test_list.py
index 7d35deda3329..075f44d5e604 100755
--- a/scripts/test_list.py
+++ b/scripts/test_list.py
@@ -176,6 +176,22 @@ class TestList:
 
     The SUBTEST contains the fields that are specific to each subtest.
 
+    Note: when igt_simple_main is used, there will be a single nameless
+    subtest. On such case, "SUBTEST:" is still needed, but without a
+    test name on it, e. g., it would be documented as:
+
+        /**
+         * TEST: some test that uses igt_simple_main
+         * Category: Software build block
+         * Sub-category: documentation
+         * Functionality: test documentation
+         * Issue: none
+         * Description: Complete description of this test
+         *
+         * SUBTEST:
+         * Description: do foo things
+         */
+
     Some IGT tests may have strings or integer wildcards, like:
         test-%s-%ld-size
 
@@ -376,8 +392,9 @@ class TestList:
         subtest_array = []
 
         for subtest in self.doc[test]["subtest"].keys():
-            summary = test_name + '@' + self.doc[test]["subtest"][subtest]["Summary"]
-
+            summary = test_name
+            if self.doc[test]["subtest"][subtest]["Summary"] != '':
+                 summary += '@' + self.doc[test]["subtest"][subtest]["Summary"]
             if not summary:
                 continue
 
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 08/11] scripts/test_list.py: add support for igt_simple_main

[Kamil Konieczny]

On 2023-03-27 at 15:31:52 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> when igt_simple_main is used, there will be a single unamed
> subtest. Add support for it.
> 
> On such case, the documentation will be like:
> 
> 	/**
> 	 * TEST: Test foo
> 	 * Category: Software building block
> 	 * Sub-category: foo
> 	 * Test category: functionality test
> 	 * Run type: BAT
> 	 *
> 	 * SUBTEST:
> 	 * Description: foo description
> 	 */
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  scripts/test_list.py | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/test_list.py b/scripts/test_list.py
> index 7d35deda3329..075f44d5e604 100755
> --- a/scripts/test_list.py
> +++ b/scripts/test_list.py
> @@ -176,6 +176,22 @@ class TestList:
>  
>      The SUBTEST contains the fields that are specific to each subtest.
>  
> +    Note: when igt_simple_main is used, there will be a single nameless
> +    subtest. On such case, "SUBTEST:" is still needed, but without a
> +    test name on it, e. g., it would be documented as:
> +
> +        /**
> +         * TEST: some test that uses igt_simple_main
> +         * Category: Software build block
> +         * Sub-category: documentation
> +         * Functionality: test documentation
> +         * Issue: none
> +         * Description: Complete description of this test
> +         *
> +         * SUBTEST:
> +         * Description: do foo things
> +         */
> +
>      Some IGT tests may have strings or integer wildcards, like:
>          test-%s-%ld-size
>  
> @@ -376,8 +392,9 @@ class TestList:
>          subtest_array = []
>  
>          for subtest in self.doc[test]["subtest"].keys():
> -            summary = test_name + '@' + self.doc[test]["subtest"][subtest]["Summary"]
> -
> +            summary = test_name
> +            if self.doc[test]["subtest"][subtest]["Summary"] != '':
> +                 summary += '@' + self.doc[test]["subtest"][subtest]["Summary"]
>              if not summary:
>                  continue
>  
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 09/11] testplan/meson.build: Validate documentation at build time

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

If the tests are build too, check if the documentation match
the name of the tests.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 docs/testplan/meson.build | 8 ++++++--
 meson.build               | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/docs/testplan/meson.build b/docs/testplan/meson.build
index fee568868167..d1ad6ea0461b 100644
--- a/docs/testplan/meson.build
+++ b/docs/testplan/meson.build
@@ -8,12 +8,16 @@ stylesheet = join_paths(meson.current_source_dir(), 'testplan.css')
 
 xe_test_config = join_paths(source_root, 'tests', 'xe', 'xe_test_config.json')
 
-if not build_tests
+check_testlist = ''
+if build_tests
+	# Check if documentation matches the actual tests
+	check_testlist = [ '--check-testlist', '--igt-build-path', build_root ]
+else
 	test_executables = []
 endif
 
 test_dict = { 'xe_tests':
-		{ 'input': xe_test_config, 'extra_args': [] }
+		{ 'input': xe_test_config, 'extra_args': check_testlist }
 	    }
 
 foreach testplan, fields: test_dict
diff --git a/meson.build b/meson.build
index 7360634fe199..bef590f465dc 100644
--- a/meson.build
+++ b/meson.build
@@ -15,6 +15,7 @@ endif
 
 # meson.source_root is marked as a future-deprecated feature
 source_root = meson.current_source_dir()
+build_root = meson.current_build_dir()
 
 cc = meson.get_compiler('c')
 
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 09/11] testplan/meson.build: Validate documentation at build time

[Kamil Konieczny]

On 2023-03-27 at 15:31:53 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> If the tests are build too, check if the documentation match
> the name of the tests.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  docs/testplan/meson.build | 8 ++++++--
>  meson.build               | 1 +
>  2 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/testplan/meson.build b/docs/testplan/meson.build
> index fee568868167..d1ad6ea0461b 100644
> --- a/docs/testplan/meson.build
> +++ b/docs/testplan/meson.build
> @@ -8,12 +8,16 @@ stylesheet = join_paths(meson.current_source_dir(), 'testplan.css')
>  
>  xe_test_config = join_paths(source_root, 'tests', 'xe', 'xe_test_config.json')
>  
> -if not build_tests
> +check_testlist = ''
> +if build_tests
> +	# Check if documentation matches the actual tests
> +	check_testlist = [ '--check-testlist', '--igt-build-path', build_root ]
> +else
>  	test_executables = []
>  endif
>  
>  test_dict = { 'xe_tests':
> -		{ 'input': xe_test_config, 'extra_args': [] }
> +		{ 'input': xe_test_config, 'extra_args': check_testlist }
>  	    }
>  
>  foreach testplan, fields: test_dict
> diff --git a/meson.build b/meson.build
> index 7360634fe199..bef590f465dc 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -15,6 +15,7 @@ endif
>  
>  # meson.source_root is marked as a future-deprecated feature
>  source_root = meson.current_source_dir()
> +build_root = meson.current_build_dir()
>  
>  cc = meson.get_compiler('c')
>  
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 10/11] xe/xe_vm: add documentation for igt@xe_vm@userptr-invalid

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Solves this build-time warning:

	Warning: Missing documentation for igt@xe_vm@userptr-invalid

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/xe/xe_vm.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
index 15356c704d2c..9ba00f0885f5 100644
--- a/tests/xe/xe_vm.c
+++ b/tests/xe/xe_vm.c
@@ -294,9 +294,13 @@ static void unbind_all(int fd, int n_vmas)
 
 #define	MAP_ADDRESS	0x00007fadeadbe000
 
-/*
- * This tests verifies that mapping an invalid userptr returns -EFAULT,
- * and that it is correctly handled.
+/**
+ * SUBTEST: userptr-invalid
+ * Description:
+ *	Verifies that mapping an invalid userptr returns -EFAULT,
+ *	and that it is correctly handled.
+ * Run type: FULL
+ * TODO: change ``'Run type' == FULL`` to a better category
  */
 static void userptr_invalid(int fd)
 {
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 10/11] xe/xe_vm: add documentation for igt@xe_vm@userptr-invalid

[Kamil Konieczny]

On 2023-03-27 at 15:31:54 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Solves this build-time warning:
> 
> 	Warning: Missing documentation for igt@xe_vm@userptr-invalid
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  tests/xe/xe_vm.c | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
> index 15356c704d2c..9ba00f0885f5 100644
> --- a/tests/xe/xe_vm.c
> +++ b/tests/xe/xe_vm.c
> @@ -294,9 +294,13 @@ static void unbind_all(int fd, int n_vmas)
>  
>  #define	MAP_ADDRESS	0x00007fadeadbe000
>  
> -/*
> - * This tests verifies that mapping an invalid userptr returns -EFAULT,
> - * and that it is correctly handled.
> +/**
> + * SUBTEST: userptr-invalid
> + * Description:
> + *	Verifies that mapping an invalid userptr returns -EFAULT,
> + *	and that it is correctly handled.
> + * Run type: FULL
> + * TODO: change ``'Run type' == FULL`` to a better category
>   */
>  static void userptr_invalid(int fd)
>  {
> -- 
> 2.39.2
> 

[igt-dev] [PATCH i-g-t 11/11] xe/xe_noexec_ping_pong: document subtest

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Use the SUBTEST tag to add the subtest's description to the
IGT test plan documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
 tests/xe/xe_noexec_ping_pong.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/tests/xe/xe_noexec_ping_pong.c b/tests/xe/xe_noexec_ping_pong.c
index dc14f8cf3f2d..367671883a28 100644
--- a/tests/xe/xe_noexec_ping_pong.c
+++ b/tests/xe/xe_noexec_ping_pong.c
@@ -20,20 +20,22 @@
  * Category: Software building block
  * Sub-category: compute
  * Test category: functionality test
- */
-
-/*
- * This test creates compute vms, binds a couple of bos and an engine each,
- * thus redying it for execution. However, VRAM memory is over-
- * committed and while there is still nothing to execute, an eviction
- * will trigger the VM's rebind worker to rebind the evicted bo, which
- * will in turn trigger another eviction and so on.
  *
- * Since we don't have eviction stats yet we need to watch "top" for
- * the rebind kworkers using a lot of CPU while the test idles.
+ * SUBTEST:
+ * Description:
+ *	This test creates compute vms, binds a couple of bos and an engine each,
+ *	thus redying it for execution. However, VRAM memory is over-
+ *	committed and while there is still nothing to execute, an eviction
+ *	will trigger the VM's rebind worker to rebind the evicted bo, which
+ *	will in turn trigger another eviction and so on.
+ *
+ *	Since we don't have eviction stats yet we need to watch "top" for
+ *	the rebind kworkers using a lot of CPU while the test idles.
  *
- * The correct driver behaviour should be not to rebind anything unless
- * there is worked queued on one of the VM's compute engines.
+ *	The correct driver behaviour should be not to rebind anything unless
+ *	there is worked queued on one of the VM's compute engines.
+ * Run type: FULL
+ * TODO: change ``'Run type' == FULL`` to a better category
  */
 static void test_ping_pong(int fd, struct drm_xe_engine_class_instance *eci)
 {
-- 
2.39.2

Re: [igt-dev] [PATCH i-g-t 11/11] xe/xe_noexec_ping_pong: document subtest

[Kamil Konieczny]

On 2023-03-27 at 15:31:55 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> Use the SUBTEST tag to add the subtest's description to the
> IGT test plan documentation.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> ---
>  tests/xe/xe_noexec_ping_pong.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/tests/xe/xe_noexec_ping_pong.c b/tests/xe/xe_noexec_ping_pong.c
> index dc14f8cf3f2d..367671883a28 100644
> --- a/tests/xe/xe_noexec_ping_pong.c
> +++ b/tests/xe/xe_noexec_ping_pong.c
> @@ -20,20 +20,22 @@
>   * Category: Software building block
>   * Sub-category: compute
>   * Test category: functionality test
> - */
> -
> -/*
> - * This test creates compute vms, binds a couple of bos and an engine each,
> - * thus redying it for execution. However, VRAM memory is over-
> - * committed and while there is still nothing to execute, an eviction
> - * will trigger the VM's rebind worker to rebind the evicted bo, which
> - * will in turn trigger another eviction and so on.
>   *
> - * Since we don't have eviction stats yet we need to watch "top" for
> - * the rebind kworkers using a lot of CPU while the test idles.
> + * SUBTEST:
> + * Description:
> + *	This test creates compute vms, binds a couple of bos and an engine each,
> + *	thus redying it for execution. However, VRAM memory is over-
> + *	committed and while there is still nothing to execute, an eviction
> + *	will trigger the VM's rebind worker to rebind the evicted bo, which
> + *	will in turn trigger another eviction and so on.
> + *
> + *	Since we don't have eviction stats yet we need to watch "top" for
> + *	the rebind kworkers using a lot of CPU while the test idles.
>   *
> - * The correct driver behaviour should be not to rebind anything unless
> - * there is worked queued on one of the VM's compute engines.
> + *	The correct driver behaviour should be not to rebind anything unless
> + *	there is worked queued on one of the VM's compute engines.
> + * Run type: FULL
> + * TODO: change ``'Run type' == FULL`` to a better category
>   */
>  static void test_ping_pong(int fd, struct drm_xe_engine_class_instance *eci)
>  {
> -- 
> 2.39.2
> 

[igt-dev] ✗ GitLab.Pipeline: warning for Improve igt_doc.py

[Patchwork]

== Series Details ==

Series: Improve igt_doc.py
URL   : https://patchwork.freedesktop.org/series/115667/
State : warning

== Summary ==

Pipeline status: FAILED.

see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/840355 for the overview.

build:tests-debian-minimal has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/38824636):
  Program rst2man-3 found: NO
  Program rst2man found: NO
  Program rst2man.sh found: YES (/builds/gfx-ci/igt-ci-tags/man/rst2man.sh)
  Program igt_doc.py found: YES (/builds/gfx-ci/igt-ci-tags/scripts/igt_doc.py)
  Program gen_rst_index found: YES (/builds/gfx-ci/igt-ci-tags/scripts/gen_rst_index)
  Dependency gtk-doc found: NO (tried pkgconfig and cmake)
  Program sphinx-build found: NO
  Program rst2html-3 found: NO
  Program rst2html found: NO
  Program rst2pdf found: NO
  
  docs/testplan/meson.build:24:1: ERROR:  Invalid use of addition: can only concatenate list (not "str") to list
  
  A full log can be found at /builds/gfx-ci/igt-ci-tags/build/meson-logs/meson-log.txt
  section_end:1679932066:step_script
  section_start:1679932066:cleanup_file_variables
  Cleaning up project directory and file based variables
  section_end:1679932068:cleanup_file_variables
  ERROR: Job failed: exit code 1

== Logs ==

For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/840355

[igt-dev] ✓ Fi.CI.BAT: success for Improve igt_doc.py

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 2753 bytes --]

== Series Details ==

Series: Improve igt_doc.py
URL   : https://patchwork.freedesktop.org/series/115667/
State : success

== Summary ==

CI Bug Log - changes from IGT_7221 -> IGTPW_8686
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html

Participating hosts (36 -> 36)
------------------------------

  No changes in participating hosts

Known issues
------------

  Here are the changes found in IGTPW_8686 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@i915_selftest@live@reset:
    - bat-rpls-2:         [PASS][1] -> [ABORT][2] ([i915#4983] / [i915#7913])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/bat-rpls-2/igt@i915_selftest@live@reset.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/bat-rpls-2/igt@i915_selftest@live@reset.html

  * igt@i915_selftest@live@slpc:
    - bat-rpls-1:         [PASS][3] -> [DMESG-FAIL][4] ([i915#6367] / [i915#7996])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/bat-rpls-1/igt@i915_selftest@live@slpc.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/bat-rpls-1/igt@i915_selftest@live@slpc.html

  * igt@kms_pipe_crc_basic@nonblocking-crc-frame-sequence:
    - bat-dg2-11:         NOTRUN -> [SKIP][5] ([i915#5354])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/bat-dg2-11/igt@kms_pipe_crc_basic@nonblocking-crc-frame-sequence.html

  
#### Possible fixes ####

  * igt@i915_pm_rps@basic-api:
    - bat-dg2-11:         [FAIL][6] ([i915#8308]) -> [PASS][7]
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/bat-dg2-11/igt@i915_pm_rps@basic-api.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/bat-dg2-11/igt@i915_pm_rps@basic-api.html

  
  [i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
  [i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
  [i915#7996]: https://gitlab.freedesktop.org/drm/intel/issues/7996
  [i915#8308]: https://gitlab.freedesktop.org/drm/intel/issues/8308


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7221 -> IGTPW_8686

  CI-20190529: 20190529
  CI_DRM_12920: 78054149ebf825810cd893726be90865a6faf25e @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8686: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html
  IGT_7221: 4b77c6d85024d22ca521d510f8eee574128fe04f @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html

[-- Attachment #2: Type: text/html, Size: 3425 bytes --]

[igt-dev] ✓ Fi.CI.IGT: success for Improve igt_doc.py

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 11638 bytes --]

== Series Details ==

Series: Improve igt_doc.py
URL   : https://patchwork.freedesktop.org/series/115667/
State : success

== Summary ==

CI Bug Log - changes from IGT_7221_full -> IGTPW_8686_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html

Participating hosts (7 -> 7)
------------------------------

  No changes in participating hosts

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_8686_full:

### IGT changes ###

#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@api_intel_allocator@reserve-simple:
    - {shard-dg1}:        NOTRUN -> [DMESG-WARN][1]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-dg1-15/igt@api_intel_allocator@reserve-simple.html

  
Known issues
------------

  Here are the changes found in IGTPW_8686_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_exec_fair@basic-deadline:
    - shard-glk:          [PASS][2] -> [FAIL][3] ([i915#2846])
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk1/igt@gem_exec_fair@basic-deadline.html
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk1/igt@gem_exec_fair@basic-deadline.html

  * igt@gem_render_copy@y-tiled-to-vebox-x-tiled:
    - shard-snb:          NOTRUN -> [SKIP][4] ([fdo#109271]) +18 similar issues
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-snb5/igt@gem_render_copy@y-tiled-to-vebox-x-tiled.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
    - shard-glk:          [PASS][5] -> [FAIL][6] ([i915#2346])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk1/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk8/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html

  * igt@kms_dither@fb-8bpc-vs-panel-6bpc@pipe-a-hdmi-a-1:
    - shard-glk:          NOTRUN -> [SKIP][7] ([fdo#109271])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk3/igt@kms_dither@fb-8bpc-vs-panel-6bpc@pipe-a-hdmi-a-1.html

  * igt@kms_flip@dpms-vs-vblank-race-interruptible@c-hdmi-a1:
    - shard-glk:          [PASS][8] -> [FAIL][9] ([i915#407])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk1/igt@kms_flip@dpms-vs-vblank-race-interruptible@c-hdmi-a1.html
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk9/igt@kms_flip@dpms-vs-vblank-race-interruptible@c-hdmi-a1.html

  * igt@kms_flip@flip-vs-expired-vblank-interruptible@b-hdmi-a1:
    - shard-glk:          [PASS][10] -> [FAIL][11] ([i915#79])
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk5/igt@kms_flip@flip-vs-expired-vblank-interruptible@b-hdmi-a1.html
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk7/igt@kms_flip@flip-vs-expired-vblank-interruptible@b-hdmi-a1.html

  * igt@kms_vblank@pipe-b-ts-continuation-suspend:
    - shard-snb:          [PASS][12] -> [SKIP][13] ([fdo#109271])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-snb2/igt@kms_vblank@pipe-b-ts-continuation-suspend.html
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-snb2/igt@kms_vblank@pipe-b-ts-continuation-suspend.html

  
#### Possible fixes ####

  * igt@i915_selftest@perf@engine_cs:
    - shard-snb:          [ABORT][14] ([i915#4528]) -> [PASS][15]
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-snb5/igt@i915_selftest@perf@engine_cs.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-snb5/igt@i915_selftest@perf@engine_cs.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
    - shard-glk:          [FAIL][16] ([i915#2346]) -> [PASS][17]
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

  * igt@kms_flip@2x-plain-flip-fb-recreate@ab-hdmi-a1-hdmi-a2:
    - shard-glk:          [FAIL][18] ([i915#2122]) -> [PASS][19]
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7221/shard-glk8/igt@kms_flip@2x-plain-flip-fb-recreate@ab-hdmi-a1-hdmi-a2.html
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/shard-glk9/igt@kms_flip@2x-plain-flip-fb-recreate@ab-hdmi-a1-hdmi-a2.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
  [fdo#109300]: https://bugs.freedesktop.org/show_bug.cgi?id=109300
  [fdo#109315]: https://bugs.freedesktop.org/show_bug.cgi?id=109315
  [fdo#109506]: https://bugs.freedesktop.org/show_bug.cgi?id=109506
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [fdo#112054]: https://bugs.freedesktop.org/show_bug.cgi?id=112054
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
  [i915#1937]: https://gitlab.freedesktop.org/drm/intel/issues/1937
  [i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2433]: https://gitlab.freedesktop.org/drm/intel/issues/2433
  [i915#2434]: https://gitlab.freedesktop.org/drm/intel/issues/2434
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3361]: https://gitlab.freedesktop.org/drm/intel/issues/3361
  [i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
  [i915#3469]: https://gitlab.freedesktop.org/drm/intel/issues/3469
  [i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#3742]: https://gitlab.freedesktop.org/drm/intel/issues/3742
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3938]: https://gitlab.freedesktop.org/drm/intel/issues/3938
  [i915#407]: https://gitlab.freedesktop.org/drm/intel/issues/407
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213
  [i915#426]: https://gitlab.freedesktop.org/drm/intel/issues/426
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
  [i915#4528]: https://gitlab.freedesktop.org/drm/intel/issues/4528
  [i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
  [i915#4565]: https://gitlab.freedesktop.org/drm/intel/issues/4565
  [i915#4579]: https://gitlab.freedesktop.org/drm/intel/issues/4579
  [i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
  [i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
  [i915#4833]: https://gitlab.freedesktop.org/drm/intel/issues/4833
  [i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
  [i915#4854]: https://gitlab.freedesktop.org/drm/intel/issues/4854
  [i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
  [i915#4873]: https://gitlab.freedesktop.org/drm/intel/issues/4873
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5288]: https://gitlab.freedesktop.org/drm/intel/issues/5288
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#5325]: https://gitlab.freedesktop.org/drm/intel/issues/5325
  [i915#5431]: https://gitlab.freedesktop.org/drm/intel/issues/5431
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5563]: https://gitlab.freedesktop.org/drm/intel/issues/5563
  [i915#5723]: https://gitlab.freedesktop.org/drm/intel/issues/5723
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6301]: https://gitlab.freedesktop.org/drm/intel/issues/6301
  [i915#6433]: https://gitlab.freedesktop.org/drm/intel/issues/6433
  [i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6944]: https://gitlab.freedesktop.org/drm/intel/issues/6944
  [i915#6946]: https://gitlab.freedesktop.org/drm/intel/issues/6946
  [i915#7116]: https://gitlab.freedesktop.org/drm/intel/issues/7116
  [i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
  [i915#7561]: https://gitlab.freedesktop.org/drm/intel/issues/7561
  [i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
  [i915#7701]: https://gitlab.freedesktop.org/drm/intel/issues/7701
  [i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79
  [i915#7975]: https://gitlab.freedesktop.org/drm/intel/issues/7975
  [i915#8155]: https://gitlab.freedesktop.org/drm/intel/issues/8155
  [i915#8292]: https://gitlab.freedesktop.org/drm/intel/issues/8292
  [i915#8304]: https://gitlab.freedesktop.org/drm/intel/issues/8304


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7221 -> IGTPW_8686

  CI-20190529: 20190529
  CI_DRM_12920: 78054149ebf825810cd893726be90865a6faf25e @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8686: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html
  IGT_7221: 4b77c6d85024d22ca521d510f8eee574128fe04f @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8686/index.html

[-- Attachment #2: Type: text/html, Size: 6878 bytes --]

Re: [igt-dev] [PATCH i-g-t 00/11] Improve igt_doc.py

[Kamil Konieczny]

Hi Mauro,

On 2023-03-27 at 15:31:44 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> This series do some improvements at the igt_doc.py tool:
> 
> - It does some tweaks at the Xe documentation hierarchy;
> - It adds support to generate test lists from documentation;
> - It splits the script on two parts: one with the TestList
>   class definition and another one with the toll itself;
> - While here, add missing documentation for two new tests;
> - Check if the documentation matches the actual tests at
>   build time, in order to prevent it to become outdated.
> 
> Mauro Carvalho Chehab (11):
>   xe_test_config.json: do some adjustments at the output hierarchy
>   scripts/igt_doc.py: move show_subtests logic to the class
>   scripts/igt_doc.py: add support to generate testlists
>   scripts/igt_doc.py: prepare to place TestList class on a separate file
>   scripts/test_list.py: prepare to place class on a separate file
>   scripts/igt_doc.py: re-introduce it by calling test_list.py
>   scripts/test_list.py: better handle documentation validation issues
>   scripts/test_list.py: add support for igt_simple_main
>   testplan/meson.build: Validate documentation at build time

Shouldn't you merge this with your other patch:
runner/meson.build: fix minimal build

as now GitLab shows warning during build ?

Regards,
Kamil

>   xe/xe_vm: add documentation for igt@xe_vm@userptr-invalid
>   xe/xe_noexec_ping_pong: document subtest
> 
>  docs/testplan/meson.build      |    8 +-
>  meson.build                    |    1 +
>  scripts/igt_doc.py             | 1038 +-----------------------------
>  scripts/test_list.py           | 1084 ++++++++++++++++++++++++++++++++
>  tests/xe/xe_noexec_ping_pong.c |   26 +-
>  tests/xe/xe_test_config.json   |   30 +-
>  tests/xe/xe_vm.c               |   10 +-
>  7 files changed, 1143 insertions(+), 1054 deletions(-)
>  create mode 100755 scripts/test_list.py
> 
> -- 
> 2.39.2
> 

Re: [igt-dev] [PATCH i-g-t 00/11] Improve igt_doc.py

[Mauro Carvalho Chehab]

On Wed, 29 Mar 2023 19:59:25 +0200
Kamil Konieczny <kamil.konieczny@linux.intel.com> wrote:

> Hi Mauro,
> 
> On 2023-03-27 at 15:31:44 +0200, Mauro Carvalho Chehab wrote:
> > From: Mauro Carvalho Chehab <mchehab@kernel.org>
> > 
> > This series do some improvements at the igt_doc.py tool:
> > 
> > - It does some tweaks at the Xe documentation hierarchy;
> > - It adds support to generate test lists from documentation;
> > - It splits the script on two parts: one with the TestList
> >   class definition and another one with the toll itself;
> > - While here, add missing documentation for two new tests;
> > - Check if the documentation matches the actual tests at
> >   build time, in order to prevent it to become outdated.
> > 
> > Mauro Carvalho Chehab (11):
> >   xe_test_config.json: do some adjustments at the output hierarchy
> >   scripts/igt_doc.py: move show_subtests logic to the class
> >   scripts/igt_doc.py: add support to generate testlists
> >   scripts/igt_doc.py: prepare to place TestList class on a separate file
> >   scripts/test_list.py: prepare to place class on a separate file
> >   scripts/igt_doc.py: re-introduce it by calling test_list.py
> >   scripts/test_list.py: better handle documentation validation issues
> >   scripts/test_list.py: add support for igt_simple_main
> >   testplan/meson.build: Validate documentation at build time  

Kamil,

Thanks for the review!

> Shouldn't you merge this with your other patch:
> runner/meson.build: fix minimal build
> 
> as now GitLab shows warning during build ?

Yeah, it makes sense to apply both this series and the one
addressing the warning together. I'll submit a new version of
this series with your R-B + the warning fix, in order to let
CI test them altogether.

Regards,
Mauro