[igt-dev] [PATCH i-g-t] .gitlab-ci: Fix listing undocumented subtests

[igt-dev] [PATCH i-g-t] .gitlab-ci: Fix listing undocumented subtests

[Arkadiusz Hiler]

Looking at this code:

  description = ""
  current_subtest = None

  for line in proc.stdout.decode().splitlines():
  if line.startswith("SUB "):
      output += [Subtest(current_subtest, description)]

So if there is no documentation on the top level we will get subtest ==
None and description == "".

Let's check for those properly so we won't falsely flag the whole binary.

Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
---
 .gitlab-ci/list_undocumented_tests.py | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci/list_undocumented_tests.py b/.gitlab-ci/list_undocumented_tests.py
index 0ce3f330..d6bb0a5c 100755
--- a/.gitlab-ci/list_undocumented_tests.py
+++ b/.gitlab-ci/list_undocumented_tests.py
@@ -47,13 +47,11 @@ def main():
     for test in tests:
         subtests = get_subtests(testdir, test)
 
-        if subtests and subtests[0].name is None:
-            # top level description missing, list binary
-            print(test)
-
         for name, description in subtests:
-            if name is None: # top level description, skipping
-                continue
+            if name is None: # top level description
+                if not description: # is empty
+                    print(test) # mention the test binary
+                continue # and skip because it's not a subtest
 
             if "NO DOCUMENTATION!" in description:
                 print("{}@{}".format(test, name))
-- 
2.21.0

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for .gitlab-ci: Fix listing undocumented subtests

[Patchwork]

== Series Details ==

Series: .gitlab-ci: Fix listing undocumented subtests
URL   : https://patchwork.freedesktop.org/series/66419/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6852 -> IGTPW_3431
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/66419/revisions/1/mbox/

Known issues
------------

  Here are the changes found in IGTPW_3431 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_exec_reloc@basic-write-read-noreloc:
    - fi-icl-u3:          [PASS][1] -> [DMESG-WARN][2] ([fdo#107724])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-icl-u3/igt@gem_exec_reloc@basic-write-read-noreloc.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-icl-u3/igt@gem_exec_reloc@basic-write-read-noreloc.html

  * igt@kms_chamelium@dp-crc-fast:
    - fi-cml-u2:          [PASS][3] -> [FAIL][4] ([fdo#110627])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-cml-u2/igt@kms_chamelium@dp-crc-fast.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-cml-u2/igt@kms_chamelium@dp-crc-fast.html

  * igt@kms_chamelium@dp-edid-read:
    - fi-kbl-7500u:       [PASS][5] -> [WARN][6] ([fdo#109483])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-kbl-7500u/igt@kms_chamelium@dp-edid-read.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-kbl-7500u/igt@kms_chamelium@dp-edid-read.html

  
#### Possible fixes ####

  * igt@gem_ctx_create@basic-files:
    - fi-icl-u2:          [INCOMPLETE][7] ([fdo#107713] / [fdo#109100]) -> [PASS][8]
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-icl-u2/igt@gem_ctx_create@basic-files.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-icl-u2/igt@gem_ctx_create@basic-files.html

  * igt@gem_exec_fence@nb-await-default:
    - fi-icl-u3:          [DMESG-WARN][9] ([fdo#107724]) -> [PASS][10]
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-icl-u3/igt@gem_exec_fence@nb-await-default.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-icl-u3/igt@gem_exec_fence@nb-await-default.html

  * igt@kms_frontbuffer_tracking@basic:
    - fi-icl-u3:          [FAIL][11] ([fdo#103167]) -> [PASS][12]
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/fi-icl-u3/igt@kms_frontbuffer_tracking@basic.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/fi-icl-u3/igt@kms_frontbuffer_tracking@basic.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
  [fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
  [fdo#107724]: https://bugs.freedesktop.org/show_bug.cgi?id=107724
  [fdo#109100]: https://bugs.freedesktop.org/show_bug.cgi?id=109100
  [fdo#109483]: https://bugs.freedesktop.org/show_bug.cgi?id=109483
  [fdo#110627]: https://bugs.freedesktop.org/show_bug.cgi?id=110627
  [fdo#111108]: https://bugs.freedesktop.org/show_bug.cgi?id=111108


Participating hosts (52 -> 47)
------------------------------

  Additional (2): fi-skl-6770hq fi-skl-6700k2 
  Missing    (7): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-icl-y fi-byt-clapper fi-bdw-samus 


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_5173 -> IGTPW_3431

  CI-20190529: 20190529
  CI_DRM_6852: d45d78ff950be956657e1236785714509a7d43be @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_3431: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/
  IGT_5173: 3fb0f227d8856008f89a797879e27094745ce97e @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.IGT: success for .gitlab-ci: Fix listing undocumented subtests

[Patchwork]

== Series Details ==

Series: .gitlab-ci: Fix listing undocumented subtests
URL   : https://patchwork.freedesktop.org/series/66419/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6852_full -> IGTPW_3431_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/66419/revisions/1/mbox/

Known issues
------------

  Here are the changes found in IGTPW_3431_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_switch@legacy-bsd2-heavy:
    - shard-iclb:         [PASS][1] -> [SKIP][2] ([fdo#109276]) +12 similar issues
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb4/igt@gem_ctx_switch@legacy-bsd2-heavy.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb3/igt@gem_ctx_switch@legacy-bsd2-heavy.html

  * igt@gem_eio@unwedge-stress:
    - shard-snb:          [PASS][3] -> [FAIL][4] ([fdo#109661])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-snb7/igt@gem_eio@unwedge-stress.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-snb5/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_schedule@preempt-queue-contexts-bsd:
    - shard-iclb:         [PASS][5] -> [SKIP][6] ([fdo#111325])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb8/igt@gem_exec_schedule@preempt-queue-contexts-bsd.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb2/igt@gem_exec_schedule@preempt-queue-contexts-bsd.html

  * igt@gem_workarounds@suspend-resume:
    - shard-apl:          [PASS][7] -> [DMESG-WARN][8] ([fdo#108566]) +4 similar issues
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-apl2/igt@gem_workarounds@suspend-resume.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-apl7/igt@gem_workarounds@suspend-resume.html

  * igt@kms_flip@2x-flip-vs-expired-vblank-interruptible:
    - shard-glk:          [PASS][9] -> [FAIL][10] ([fdo#105363]) +1 similar issue
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-glk5/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-glk2/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible.html

  * igt@kms_flip@flip-vs-panning-vs-hang-interruptible:
    - shard-iclb:         [PASS][11] -> [INCOMPLETE][12] ([fdo#107713])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb7/igt@kms_flip@flip-vs-panning-vs-hang-interruptible.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb7/igt@kms_flip@flip-vs-panning-vs-hang-interruptible.html

  * igt@kms_flip@flip-vs-wf_vblank-interruptible:
    - shard-hsw:          [PASS][13] -> [INCOMPLETE][14] ([fdo#103540])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-hsw2/igt@kms_flip@flip-vs-wf_vblank-interruptible.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-hsw1/igt@kms_flip@flip-vs-wf_vblank-interruptible.html

  * igt@kms_flip@plain-flip-fb-recreate-interruptible:
    - shard-apl:          [PASS][15] -> [INCOMPLETE][16] ([fdo#103927])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-apl1/igt@kms_flip@plain-flip-fb-recreate-interruptible.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-apl5/igt@kms_flip@plain-flip-fb-recreate-interruptible.html

  * igt@kms_frontbuffer_tracking@fbc-stridechange:
    - shard-iclb:         [PASS][17] -> [FAIL][18] ([fdo#103167]) +5 similar issues
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb2/igt@kms_frontbuffer_tracking@fbc-stridechange.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb6/igt@kms_frontbuffer_tracking@fbc-stridechange.html

  * igt@kms_psr@psr2_sprite_blt:
    - shard-iclb:         [PASS][19] -> [SKIP][20] ([fdo#109441]) +1 similar issue
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb2/igt@kms_psr@psr2_sprite_blt.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb8/igt@kms_psr@psr2_sprite_blt.html

  
#### Possible fixes ####

  * igt@gem_ctx_shared@exec-single-timeline-bsd:
    - shard-iclb:         [SKIP][21] ([fdo#110841]) -> [PASS][22]
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb2/igt@gem_ctx_shared@exec-single-timeline-bsd.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb8/igt@gem_ctx_shared@exec-single-timeline-bsd.html

  * igt@gem_exec_schedule@preempt-bsd1:
    - shard-iclb:         [SKIP][23] ([fdo#109276]) -> [PASS][24] +15 similar issues
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb3/igt@gem_exec_schedule@preempt-bsd1.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb2/igt@gem_exec_schedule@preempt-bsd1.html

  * igt@gem_exec_schedule@preempt-other-bsd:
    - shard-iclb:         [SKIP][25] ([fdo#111325]) -> [PASS][26] +2 similar issues
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb4/igt@gem_exec_schedule@preempt-other-bsd.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb8/igt@gem_exec_schedule@preempt-other-bsd.html

  * igt@gem_tiled_swapping@non-threaded:
    - shard-apl:          [DMESG-WARN][27] ([fdo#108686]) -> [PASS][28]
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-apl6/igt@gem_tiled_swapping@non-threaded.html
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-apl2/igt@gem_tiled_swapping@non-threaded.html

  * igt@i915_suspend@fence-restore-untiled:
    - shard-apl:          [DMESG-WARN][29] ([fdo#108566]) -> [PASS][30] +5 similar issues
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-apl3/igt@i915_suspend@fence-restore-untiled.html
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-apl2/igt@i915_suspend@fence-restore-untiled.html

  * igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic:
    - shard-glk:          [FAIL][31] ([fdo#104873]) -> [PASS][32]
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-glk1/igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic.html
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-glk7/igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic.html

  * igt@kms_flip@flip-vs-suspend:
    - shard-hsw:          [INCOMPLETE][33] ([fdo#103540]) -> [PASS][34]
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-hsw5/igt@kms_flip@flip-vs-suspend.html
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-hsw2/igt@kms_flip@flip-vs-suspend.html

  * igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-pwrite:
    - shard-iclb:         [FAIL][35] ([fdo#103167]) -> [PASS][36] +2 similar issues
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb5/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-pwrite.html
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb4/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-pwrite.html

  * igt@kms_frontbuffer_tracking@fbc-stridechange:
    - shard-glk:          [FAIL][37] ([fdo#103167]) -> [PASS][38]
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-glk4/igt@kms_frontbuffer_tracking@fbc-stridechange.html
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-glk2/igt@kms_frontbuffer_tracking@fbc-stridechange.html
    - shard-apl:          [FAIL][39] ([fdo#103167]) -> [PASS][40]
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-apl2/igt@kms_frontbuffer_tracking@fbc-stridechange.html
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-apl6/igt@kms_frontbuffer_tracking@fbc-stridechange.html
    - shard-kbl:          [FAIL][41] ([fdo#103167]) -> [PASS][42]
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-kbl1/igt@kms_frontbuffer_tracking@fbc-stridechange.html
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-kbl6/igt@kms_frontbuffer_tracking@fbc-stridechange.html

  * igt@kms_plane_lowres@pipe-a-tiling-y:
    - shard-iclb:         [FAIL][43] ([fdo#103166]) -> [PASS][44]
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb5/igt@kms_plane_lowres@pipe-a-tiling-y.html
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb5/igt@kms_plane_lowres@pipe-a-tiling-y.html

  * igt@kms_psr2_su@frontbuffer:
    - shard-iclb:         [SKIP][45] ([fdo#109642] / [fdo#111068]) -> [PASS][46]
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb3/igt@kms_psr2_su@frontbuffer.html
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb2/igt@kms_psr2_su@frontbuffer.html

  * igt@kms_psr@psr2_sprite_render:
    - shard-iclb:         [SKIP][47] ([fdo#109441]) -> [PASS][48]
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6852/shard-iclb3/igt@kms_psr@psr2_sprite_render.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/shard-iclb2/igt@kms_psr@psr2_sprite_render.html

  
  [fdo#103166]: https://bugs.freedesktop.org/show_bug.cgi?id=103166
  [fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
  [fdo#103540]: https://bugs.freedesktop.org/show_bug.cgi?id=103540
  [fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
  [fdo#104873]: https://bugs.freedesktop.org/show_bug.cgi?id=104873
  [fdo#105363]: https://bugs.freedesktop.org/show_bug.cgi?id=105363
  [fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
  [fdo#108566]: https://bugs.freedesktop.org/show_bug.cgi?id=108566
  [fdo#108686]: https://bugs.freedesktop.org/show_bug.cgi?id=108686
  [fdo#109276]: https://bugs.freedesktop.org/show_bug.cgi?id=109276
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
  [fdo#109661]: https://bugs.freedesktop.org/show_bug.cgi?id=109661
  [fdo#110841]: https://bugs.freedesktop.org/show_bug.cgi?id=110841
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111325]: https://bugs.freedesktop.org/show_bug.cgi?id=111325


Participating hosts (10 -> 6)
------------------------------

  Missing    (4): pig-skl-6260u shard-skl pig-hsw-4770r pig-glk-j5005 


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_5173 -> IGTPW_3431
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_6852: d45d78ff950be956657e1236785714509a7d43be @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_3431: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/
  IGT_5173: 3fb0f227d8856008f89a797879e27094745ce97e @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3431/
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev