[igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[Arkadiusz Hiler]

Provide the reader with the taint names and a short explanation, as well
as point in the direction of the dmesg for more details.

Cc: Petri Latvala <petri.latvala@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 runner/executor.c | 42 ++++++++++++++++++++++++++++++++----------
 1 file changed, 32 insertions(+), 10 deletions(-)

diff --git a/runner/executor.c b/runner/executor.c
index 2038c3fd..954cd9a7 100644
--- a/runner/executor.c
+++ b/runner/executor.c
@@ -139,24 +139,46 @@ static char *handle_lockdep(void)
 
 static char *handle_taint(void)
 {
-	const unsigned long bad_taints =
-		0x20  | /* TAINT_PAGE */
-		0x80  | /* TAINT_DIE */
-		0x200; /* TAINT_OOPS */
-	unsigned long taints = 0;
+	/* see Linux's include/linux/kernel.h */
+	static const struct {
+		unsigned long bit;
+		const char *explanation;
+	} taints[] = {
+	  {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."},
+	  {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."},
+	  {(1 << 9), "TAINT_WARN: WARN_ON has happened."},
+	  {0, 0}};
+
+	unsigned long system_taints = 0;
 	char *reason = NULL;
 	FILE *f;
 
+	unsigned long bad_taints = 0;
+	for (typeof(*taints) *taint = taints; taint->bit; taint++)
+		bad_taints |= taint->bit;
+
 	f = fopen("/proc/sys/kernel/tainted", "r");
 	if (f) {
-		fscanf(f, "%lu", &taints);
+		fscanf(f, "%lu", &system_taints);
 		fclose(f);
 	}
 
-	if (taints & bad_taints)
-		asprintf(&reason,
-			 "Kernel tainted (%#lx -- %lx)",
-			 taints, taints & bad_taints);
+	if (system_taints & bad_taints) {
+		asprintf(&reason, "Kernel badly tainted (%#lx) "
+			 "(check dmesg for details if possible):\n",
+			 system_taints);
+
+		for (typeof(*taints) *taint = taints; taint->bit; taint++) {
+			if (taint->bit & system_taints) {
+				char *old_reason = reason;
+				asprintf(&reason, "%s\t(%#lx) %s\n",
+						old_reason,
+						taint->bit,
+						taint->explanation);
+				free(old_reason);
+			}
+		}
+	}
 
 	return reason;
 }
-- 
2.19.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[Chris Wilson]

Quoting Arkadiusz Hiler (2018-11-26 08:52:22)
> Provide the reader with the taint names and a short explanation, as well
> as point in the direction of the dmesg for more details.
> 
> Cc: Petri Latvala <petri.latvala@intel.com>
> Cc: Martin Peres <martin.peres@linux.intel.com>
> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> ---
>  runner/executor.c | 42 ++++++++++++++++++++++++++++++++----------
>  1 file changed, 32 insertions(+), 10 deletions(-)
> 
> diff --git a/runner/executor.c b/runner/executor.c
> index 2038c3fd..954cd9a7 100644
> --- a/runner/executor.c
> +++ b/runner/executor.c
> @@ -139,24 +139,46 @@ static char *handle_lockdep(void)
>  
>  static char *handle_taint(void)
>  {
> -       const unsigned long bad_taints =
> -               0x20  | /* TAINT_PAGE */
> -               0x80  | /* TAINT_DIE */
> -               0x200; /* TAINT_OOPS */
> -       unsigned long taints = 0;
> +       /* see Linux's include/linux/kernel.h */
> +       static const struct {
> +               unsigned long bit;
> +               const char *explanation;
> +       } taints[] = {
> +         {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."},
> +         {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."},
> +         {(1 << 9), "TAINT_WARN: WARN_ON has happened."},

Pure noise, these don't add any useful information, so why? The only
information is the explanation in dmesg. Checking tainted is just
shorthand for us to determine a significant event.

If you want to add the kmsg output filtered by WARN+, that would be more
useful than hinting to the user they need to do it themselves. But that
should be redundant as that is already part of the runner -- so is not
the problem the actual integration, that this appears as a separate test
and not flagging the igt itself, with exceptions to handle tainted
before a test?
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[Arkadiusz Hiler]

On Mon, Nov 26, 2018 at 09:12:46AM +0000, Chris Wilson wrote:
> Quoting Arkadiusz Hiler (2018-11-26 08:52:22)
> > Provide the reader with the taint names and a short explanation, as well
> > as point in the direction of the dmesg for more details.
> > 
> > Cc: Petri Latvala <petri.latvala@intel.com>
> > Cc: Martin Peres <martin.peres@linux.intel.com>
> > Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> > ---
> >  runner/executor.c | 42 ++++++++++++++++++++++++++++++++----------
> >  1 file changed, 32 insertions(+), 10 deletions(-)
> > 
> > diff --git a/runner/executor.c b/runner/executor.c
> > index 2038c3fd..954cd9a7 100644
> > --- a/runner/executor.c
> > +++ b/runner/executor.c
> > @@ -139,24 +139,46 @@ static char *handle_lockdep(void)
> >  
> >  static char *handle_taint(void)
> >  {
> > -       const unsigned long bad_taints =
> > -               0x20  | /* TAINT_PAGE */
> > -               0x80  | /* TAINT_DIE */
> > -               0x200; /* TAINT_OOPS */
> > -       unsigned long taints = 0;
> > +       /* see Linux's include/linux/kernel.h */
> > +       static const struct {
> > +               unsigned long bit;
> > +               const char *explanation;
> > +       } taints[] = {
> > +         {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."},
> > +         {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."},
> > +         {(1 << 9), "TAINT_WARN: WARN_ON has happened."},
> 
> Pure noise, these don't add any useful information, so why? The only
> information is the explanation in dmesg. Checking tainted is just
> shorthand for us to determine a significant event.

People not knowing what taint means (or that it is a "bit field") were
getting confused by those aborts. I just got tired of explaining that we
don't check for "unsafe module option taint" which is usually the first
result they get while greping dmesg for "taint".

> If you want to add the kmsg output filtered by WARN+, that would be more
> useful than hinting to the user they need to do it themselves. But that
> should be redundant as that is already part of the runner -- so is not
> the problem the actual integration, that this appears as a separate test
> and not flagging the igt itself, with exceptions to handle tainted
> before a test?
> -Chris

Providing more logs, as a part of the aborted results, indeed sounds
good, but it is harder to implement, as we would have to keep track of
dmesg between taint checks too. Petri, what do you think?

I believe that this is, for now, a good middle ground. We give people
more context, explain what we hit so they know that they have to look
for a warn, oops or a slab spill in the logs.

-- 
Cheers,
Arek
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[Petri Latvala]

On Mon, Nov 26, 2018 at 12:36:44PM +0200, Arkadiusz Hiler wrote:
> On Mon, Nov 26, 2018 at 09:12:46AM +0000, Chris Wilson wrote:
> > Quoting Arkadiusz Hiler (2018-11-26 08:52:22)
> > > Provide the reader with the taint names and a short explanation, as well
> > > as point in the direction of the dmesg for more details.
> > > 
> > > Cc: Petri Latvala <petri.latvala@intel.com>
> > > Cc: Martin Peres <martin.peres@linux.intel.com>
> > > Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> > > ---
> > >  runner/executor.c | 42 ++++++++++++++++++++++++++++++++----------
> > >  1 file changed, 32 insertions(+), 10 deletions(-)
> > > 
> > > diff --git a/runner/executor.c b/runner/executor.c
> > > index 2038c3fd..954cd9a7 100644
> > > --- a/runner/executor.c
> > > +++ b/runner/executor.c
> > > @@ -139,24 +139,46 @@ static char *handle_lockdep(void)
> > >  
> > >  static char *handle_taint(void)
> > >  {
> > > -       const unsigned long bad_taints =
> > > -               0x20  | /* TAINT_PAGE */
> > > -               0x80  | /* TAINT_DIE */
> > > -               0x200; /* TAINT_OOPS */
> > > -       unsigned long taints = 0;
> > > +       /* see Linux's include/linux/kernel.h */
> > > +       static const struct {
> > > +               unsigned long bit;
> > > +               const char *explanation;
> > > +       } taints[] = {
> > > +         {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."},
> > > +         {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."},
> > > +         {(1 << 9), "TAINT_WARN: WARN_ON has happened."},
> > 
> > Pure noise, these don't add any useful information, so why? The only
> > information is the explanation in dmesg. Checking tainted is just
> > shorthand for us to determine a significant event.
> 
> People not knowing what taint means (or that it is a "bit field") were
> getting confused by those aborts. I just got tired of explaining that we
> don't check for "unsafe module option taint" which is usually the first
> result they get while greping dmesg for "taint".


Yeah this has been causing confusion in all kinds of circles.

In general it's always better to explain numbers in outputs meant for
humans. Surely if people ask about stuff, adding explanations to it is
useful information by definition?


> 
> > If you want to add the kmsg output filtered by WARN+, that would be more
> > useful than hinting to the user they need to do it themselves. But that
> > should be redundant as that is already part of the runner -- so is not
> > the problem the actual integration, that this appears as a separate test
> > and not flagging the igt itself, with exceptions to handle tainted
> > before a test?
> > -Chris
> 
> Providing more logs, as a part of the aborted results, indeed sounds
> good, but it is harder to implement, as we would have to keep track of
> dmesg between taint checks too. Petri, what do you think?


It's possible [citation needed] to inject the previous test
execution's dmesg into the aborted test result, but only in the form
of igt_runner CI is using today. Resuming, especially when using
multiple-mode, will make it harder. Actually, even today CI is not
fully able to visualize shard results when multiple shards abort.

What might be easier is injecting the aborting blame into the
previously executed test but even that is not a one-liner.

Either way, aborting and its reporting is not done and abandoned yet. WIP.



-- 
Petri Latvala
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Make taint aborts more verbose

[Petri Latvala]

On Mon, Nov 26, 2018 at 02:22:00PM +0200, Petri Latvala wrote:
> On Mon, Nov 26, 2018 at 12:36:44PM +0200, Arkadiusz Hiler wrote:
> > On Mon, Nov 26, 2018 at 09:12:46AM +0000, Chris Wilson wrote:
> > > Quoting Arkadiusz Hiler (2018-11-26 08:52:22)
> > > > Provide the reader with the taint names and a short explanation, as well
> > > > as point in the direction of the dmesg for more details.
> > > > 
> > > > Cc: Petri Latvala <petri.latvala@intel.com>
> > > > Cc: Martin Peres <martin.peres@linux.intel.com>
> > > > Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> > > > ---
> > > >  runner/executor.c | 42 ++++++++++++++++++++++++++++++++----------
> > > >  1 file changed, 32 insertions(+), 10 deletions(-)
> > > > 
> > > > diff --git a/runner/executor.c b/runner/executor.c
> > > > index 2038c3fd..954cd9a7 100644
> > > > --- a/runner/executor.c
> > > > +++ b/runner/executor.c
> > > > @@ -139,24 +139,46 @@ static char *handle_lockdep(void)
> > > >  
> > > >  static char *handle_taint(void)
> > > >  {
> > > > -       const unsigned long bad_taints =
> > > > -               0x20  | /* TAINT_PAGE */
> > > > -               0x80  | /* TAINT_DIE */
> > > > -               0x200; /* TAINT_OOPS */
> > > > -       unsigned long taints = 0;
> > > > +       /* see Linux's include/linux/kernel.h */
> > > > +       static const struct {
> > > > +               unsigned long bit;
> > > > +               const char *explanation;
> > > > +       } taints[] = {
> > > > +         {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."},
> > > > +         {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."},
> > > > +         {(1 << 9), "TAINT_WARN: WARN_ON has happened."},
> > > 
> > > Pure noise, these don't add any useful information, so why? The only
> > > information is the explanation in dmesg. Checking tainted is just
> > > shorthand for us to determine a significant event.
> > 
> > People not knowing what taint means (or that it is a "bit field") were
> > getting confused by those aborts. I just got tired of explaining that we
> > don't check for "unsafe module option taint" which is usually the first
> > result they get while greping dmesg for "taint".
> 
> 
> Yeah this has been causing confusion in all kinds of circles.
> 
> In general it's always better to explain numbers in outputs meant for
> humans. Surely if people ask about stuff, adding explanations to it is
> useful information by definition?


We've had too many questions about the aborts right about now.

Reviewed-by: Petri Latvala <petri.latvala@intel.com>
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for runner: Make taint aborts more verbose

[Patchwork]

== Series Details ==

Series: runner: Make taint aborts more verbose
URL   : https://patchwork.freedesktop.org/series/53002/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4726 -> IGTPW_2090 =

== Summary - SUCCESS ==

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/53002/revisions/1/mbox/

== Known issues ==

  Here are the changes found in IGTPW_2090 that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@gem_ctx_create@basic-files:
      fi-bsw-n3050:       PASS -> FAIL (fdo#108656)
      fi-icl-u2:          PASS -> DMESG-WARN (fdo#107724)

    igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence:
      fi-byt-clapper:     PASS -> FAIL (fdo#103191, fdo#107362)

    
    ==== Possible fixes ====

    igt@gem_ctx_create@basic-files:
      {fi-icl-u3}:        DMESG-WARN (fdo#107724) -> PASS

    igt@gem_exec_suspend@basic-s3:
      fi-icl-u2:          DMESG-WARN (fdo#107724) -> PASS

    igt@kms_frontbuffer_tracking@basic:
      fi-hsw-peppy:       DMESG-WARN (fdo#102614) -> PASS

    igt@kms_pipe_crc_basic@read-crc-pipe-a:
      fi-byt-clapper:     FAIL (fdo#107362) -> PASS

    igt@pm_rpm@module-reload:
      fi-skl-iommu:       INCOMPLETE (fdo#107807) -> PASS

    
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  fdo#102614 https://bugs.freedesktop.org/show_bug.cgi?id=102614
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#107362 https://bugs.freedesktop.org/show_bug.cgi?id=107362
  fdo#107724 https://bugs.freedesktop.org/show_bug.cgi?id=107724
  fdo#107807 https://bugs.freedesktop.org/show_bug.cgi?id=107807
  fdo#108656 https://bugs.freedesktop.org/show_bug.cgi?id=108656


== Participating hosts (48 -> 45) ==

  Additional (4): fi-ivb-3520m fi-kbl-r fi-skl-6260u fi-pnv-d510 
  Missing    (7): fi-kbl-soraka fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-gdg-551 


== Build changes ==

    * IGT: IGT_4726 -> IGTPW_2090

  CI_DRM_5195: b6df470d38bf4580c00e3f5008d795ec2a901066 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2090: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2090/
  IGT_4726: f48bebb15d3d2c1e6382e1f11b0aeac06fae6082 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2090/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.IGT: success for runner: Make taint aborts more verbose

[Patchwork]

== Series Details ==

Series: runner: Make taint aborts more verbose
URL   : https://patchwork.freedesktop.org/series/53002/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4726_full -> IGTPW_2090_full =

== Summary - WARNING ==

  Minor unknown changes coming with IGTPW_2090_full need to be verified
  manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_2090_full, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/53002/revisions/1/mbox/

== Possible new issues ==

  Here are the unknown changes that may have been introduced in IGTPW_2090_full:

  === IGT changes ===

    ==== Warnings ====

    igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-pwrite:
      shard-snb:          PASS -> SKIP +1

    
== Known issues ==

  Here are the changes found in IGTPW_2090_full that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@gem_ctx_isolation@rcs0-s3:
      shard-kbl:          PASS -> DMESG-WARN (fdo#103313)

    igt@gem_exec_big:
      shard-snb:          PASS -> INCOMPLETE (fdo#105411)

    igt@kms_available_modes_crc@available_mode_test_crc:
      shard-apl:          PASS -> FAIL (fdo#106641)

    igt@kms_busy@extended-pageflip-hang-newfb-render-b:
      shard-apl:          NOTRUN -> DMESG-WARN (fdo#107956) +1

    igt@kms_busy@extended-pageflip-modeset-hang-oldfb-render-c:
      shard-kbl:          NOTRUN -> DMESG-WARN (fdo#107956)

    igt@kms_ccs@pipe-a-crc-sprite-planes-basic:
      shard-apl:          NOTRUN -> FAIL (fdo#106510, fdo#105458)
      shard-kbl:          PASS -> FAIL (fdo#107725, fdo#108145)

    igt@kms_color@pipe-b-degamma:
      shard-apl:          PASS -> FAIL (fdo#104782)

    igt@kms_cursor_crc@cursor-256x256-dpms:
      shard-apl:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-256x256-random:
      shard-glk:          PASS -> FAIL (fdo#103232) +4

    igt@kms_cursor_crc@cursor-64x21-onscreen:
      shard-kbl:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-64x64-suspend:
      shard-apl:          PASS -> FAIL (fdo#103232, fdo#103191)
      shard-kbl:          PASS -> FAIL (fdo#103232, fdo#103191)

    igt@kms_flip@2x-flip-vs-expired-vblank-interruptible:
      shard-glk:          PASS -> FAIL (fdo#105363)

    igt@kms_flip@flip-vs-expired-vblank-interruptible:
      shard-glk:          PASS -> FAIL (fdo#105363, fdo#102887)

    igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-pwrite:
      shard-glk:          PASS -> FAIL (fdo#103167) +2

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-mmap-cpu:
      shard-apl:          NOTRUN -> FAIL (fdo#103167)
      shard-kbl:          PASS -> FAIL (fdo#103167)

    igt@kms_plane@plane-position-covered-pipe-a-planes:
      shard-apl:          PASS -> FAIL (fdo#103166) +2
      shard-kbl:          PASS -> FAIL (fdo#103166) +1

    igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max:
      shard-glk:          PASS -> FAIL (fdo#108145) +1
      shard-kbl:          PASS -> FAIL (fdo#108145)
      shard-apl:          PASS -> FAIL (fdo#108145)

    igt@kms_setmode@basic:
      shard-hsw:          PASS -> FAIL (fdo#99912)

    igt@kms_universal_plane@universal-plane-pipe-c-functional:
      shard-glk:          PASS -> FAIL (fdo#103166) +5

    
    ==== Possible fixes ====

    igt@kms_busy@extended-modeset-hang-newfb-with-reset-render-b:
      shard-snb:          DMESG-WARN (fdo#107956) -> PASS

    igt@kms_color@pipe-a-ctm-0-5:
      shard-apl:          INCOMPLETE (fdo#103927) -> PASS

    igt@kms_cursor_crc@cursor-128x128-sliding:
      shard-hsw:          DMESG-FAIL (fdo#102614, fdo#103232) -> PASS

    igt@kms_cursor_crc@cursor-256x256-suspend:
      shard-apl:          FAIL (fdo#103232, fdo#103191) -> PASS

    igt@kms_cursor_crc@cursor-64x21-random:
      shard-apl:          FAIL (fdo#103232) -> PASS +1

    igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-xtiled:
      shard-glk:          FAIL (fdo#107791) -> PASS

    igt@kms_flip@dpms-vs-vblank-race-interruptible:
      shard-hsw:          DMESG-FAIL (fdo#102614, fdo#103060) -> PASS

    igt@kms_flip@flip-vs-expired-vblank:
      shard-kbl:          FAIL (fdo#105363, fdo#102887) -> PASS
      shard-glk:          FAIL (fdo#105363) -> PASS

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-blt:
      shard-kbl:          FAIL (fdo#103167) -> PASS

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-pwrite:
      shard-apl:          FAIL (fdo#103167) -> PASS +1
      shard-glk:          FAIL (fdo#103167) -> PASS +3

    igt@kms_plane@plane-position-hole-dpms-pipe-c-planes:
      shard-hsw:          FAIL (fdo#107814) -> PASS

    igt@kms_plane_multiple@atomic-pipe-a-tiling-y:
      shard-glk:          FAIL (fdo#103166) -> PASS +1
      shard-apl:          FAIL (fdo#103166) -> PASS

    
  fdo#102614 https://bugs.freedesktop.org/show_bug.cgi?id=102614
  fdo#102887 https://bugs.freedesktop.org/show_bug.cgi?id=102887
  fdo#103060 https://bugs.freedesktop.org/show_bug.cgi?id=103060
  fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
  fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#103232 https://bugs.freedesktop.org/show_bug.cgi?id=103232
  fdo#103313 https://bugs.freedesktop.org/show_bug.cgi?id=103313
  fdo#103927 https://bugs.freedesktop.org/show_bug.cgi?id=103927
  fdo#104782 https://bugs.freedesktop.org/show_bug.cgi?id=104782
  fdo#105363 https://bugs.freedesktop.org/show_bug.cgi?id=105363
  fdo#105411 https://bugs.freedesktop.org/show_bug.cgi?id=105411
  fdo#105458 https://bugs.freedesktop.org/show_bug.cgi?id=105458
  fdo#106510 https://bugs.freedesktop.org/show_bug.cgi?id=106510
  fdo#106641 https://bugs.freedesktop.org/show_bug.cgi?id=106641
  fdo#107725 https://bugs.freedesktop.org/show_bug.cgi?id=107725
  fdo#107791 https://bugs.freedesktop.org/show_bug.cgi?id=107791
  fdo#107814 https://bugs.freedesktop.org/show_bug.cgi?id=107814
  fdo#107956 https://bugs.freedesktop.org/show_bug.cgi?id=107956
  fdo#108145 https://bugs.freedesktop.org/show_bug.cgi?id=108145
  fdo#99912 https://bugs.freedesktop.org/show_bug.cgi?id=99912


== Participating hosts (7 -> 5) ==

  Missing    (2): shard-skl shard-iclb 


== Build changes ==

    * IGT: IGT_4726 -> IGTPW_2090

  CI_DRM_5195: b6df470d38bf4580c00e3f5008d795ec2a901066 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2090: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2090/
  IGT_4726: f48bebb15d3d2c1e6382e1f11b0aeac06fae6082 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2090/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev