[igt-dev] [PATCH i-g-t] runner: Be patient for processes to die

[igt-dev] [PATCH i-g-t] runner: Be patient for processes to die

[Chris Wilson]

Some machines are very slow and some processes hog a lot of resources
and so take much longer than a mere 2s to be terminated. Be patient.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108073
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
 runner/executor.c | 44 ++++++++++++++++++++++++++++++--------------
 1 file changed, 30 insertions(+), 14 deletions(-)

diff --git a/runner/executor.c b/runner/executor.c
index 2038c3fda..927696044 100644
--- a/runner/executor.c
+++ b/runner/executor.c
@@ -137,27 +137,36 @@ static char *handle_lockdep(void)
 	return NULL;
 }
 
-static char *handle_taint(void)
+
+static unsigned long tainted(unsigned long *taints)
 {
 	const unsigned long bad_taints =
 		0x20  | /* TAINT_PAGE */
 		0x80  | /* TAINT_DIE */
 		0x200; /* TAINT_OOPS */
-	unsigned long taints = 0;
-	char *reason = NULL;
 	FILE *f;
 
+	*taints = 0;
+
 	f = fopen("/proc/sys/kernel/tainted", "r");
 	if (f) {
-		fscanf(f, "%lu", &taints);
+		fscanf(f, "%lu", taints);
 		fclose(f);
 	}
 
-	if (taints & bad_taints)
-		asprintf(&reason,
-			 "Kernel tainted (%#lx -- %lx)",
-			 taints, taints & bad_taints);
+	return *taints & bad_taints;
+}
+
+static char *handle_taint(void)
+{
+	unsigned long taints, bad_taints;
+	char *reason;
+
+	bad_taints = tainted(&taints);
+	if (!bad_taints)
+		return NULL;
 
+	asprintf(&reason, "Kernel tainted (%#lx -- %lx)", taints, bad_taints);
 	return reason;
 }
 
@@ -437,6 +446,7 @@ static int monitor_output(pid_t child,
 	int wd_extra = 10;
 	int killed = 0; /* 0 if not killed, signal number otherwise */
 	struct timespec time_beg, time_end;
+	unsigned long taints = 0;
 	bool aborting = false;
 
 	igt_gettime(&time_beg);
@@ -493,10 +503,8 @@ static int monitor_output(pid_t child,
 		}
 
 		if (n == 0) {
-			intervals_left--;
-			if (intervals_left) {
+			if (--intervals_left)
 				continue;
-			}
 
 			ping_watchdogs();
 
@@ -504,6 +512,7 @@ static int monitor_output(pid_t child,
 			case 0:
 				if (settings->log_level >= LOG_LEVEL_NORMAL) {
 					printf("Timeout. Killing the current test with SIGTERM.\n");
+					fflush(stdout);
 				}
 
 				killed = SIGTERM;
@@ -514,13 +523,14 @@ static int monitor_output(pid_t child,
 				 * Now continue the loop and let the
 				 * dying child be handled normally.
 				 */
-				timeout = 2; /* Timeout for waiting selected by fair dice roll. */
-				watchdogs_set_timeout(20);
+				timeout = 20;
+				watchdogs_set_timeout(120);
 				intervals_left = timeout_intervals = 1;
 				break;
 			case SIGTERM:
 				if (settings->log_level >= LOG_LEVEL_NORMAL) {
 					printf("Timeout. Killing the current test with SIGKILL.\n");
+					fflush(stdout);
 				}
 
 				killed = SIGKILL;
@@ -530,9 +540,15 @@ static int monitor_output(pid_t child,
 				intervals_left = timeout_intervals = 1;
 				break;
 			case SIGKILL:
+				if (kill(0, child) == 0 && !tainted(&taints)) {
+					intervals_left =  1;
+					break;
+				}
+
 				/* Nothing that can be done, really. Let's tell the caller we want to abort. */
 				if (settings->log_level >= LOG_LEVEL_NORMAL) {
-					fprintf(stderr, "Child refuses to die. Aborting.\n");
+					fprintf(stderr, "Child refuses to die, tainted %x. Aborting.\n",
+						taints);
 				}
 				close_watchdogs(settings);
 				free(outbuf);
-- 
2.20.0.rc1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for runner: Be patient for processes to die

[Patchwork]

== Series Details ==

Series: runner: Be patient for processes to die
URL   : https://patchwork.freedesktop.org/series/53033/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4729 -> IGTPW_2096 =

== Summary - SUCCESS ==

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/53033/revisions/1/mbox/

== Known issues ==

  Here are the changes found in IGTPW_2096 that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@gem_ctx_switch@basic-default:
      fi-icl-u2:          PASS -> DMESG-WARN (fdo#107724)

    igt@gem_exec_suspend@basic-s3:
      fi-blb-e6850:       PASS -> INCOMPLETE (fdo#107718)

    igt@kms_pipe_crc_basic@read-crc-pipe-b:
      fi-byt-clapper:     PASS -> FAIL (fdo#107362)

    
    ==== Possible fixes ====

    igt@gem_ctx_create@basic-files:
      fi-icl-u2:          DMESG-WARN (fdo#107724) -> PASS

    igt@i915_selftest@live_hangcheck:
      {fi-icl-u3}:        INCOMPLETE (fdo#108315) -> PASS

    igt@kms_chamelium@hdmi-hpd-fast:
      {fi-kbl-7500u}:     FAIL -> PASS

    igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence:
      fi-byt-clapper:     FAIL (fdo#103191, fdo#107362) -> PASS +1

    
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#107362 https://bugs.freedesktop.org/show_bug.cgi?id=107362
  fdo#107718 https://bugs.freedesktop.org/show_bug.cgi?id=107718
  fdo#107724 https://bugs.freedesktop.org/show_bug.cgi?id=107724
  fdo#108315 https://bugs.freedesktop.org/show_bug.cgi?id=108315


== Participating hosts (51 -> 45) ==

  Missing    (6): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-glk-j4005 


== Build changes ==

    * IGT: IGT_4729 -> IGTPW_2096

  CI_DRM_5205: 48f2a38e885181b63502de679e4975551955ea74 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2096: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2096/
  IGT_4729: 2388bbd062c17b5912039101efd4603e8d876c88 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2096/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.IGT: success for runner: Be patient for processes to die

[Patchwork]

== Series Details ==

Series: runner: Be patient for processes to die
URL   : https://patchwork.freedesktop.org/series/53033/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4729_full -> IGTPW_2096_full =

== Summary - SUCCESS ==

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/53033/revisions/1/mbox/

== Known issues ==

  Here are the changes found in IGTPW_2096_full that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@kms_available_modes_crc@available_mode_test_crc:
      shard-apl:          PASS -> FAIL (fdo#106641)

    igt@kms_busy@extended-modeset-hang-newfb-with-reset-render-a:
      shard-glk:          NOTRUN -> DMESG-WARN (fdo#107956)

    igt@kms_cursor_crc@cursor-128x128-sliding:
      shard-kbl:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-64x64-onscreen:
      shard-apl:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-64x64-sliding:
      shard-glk:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-64x64-suspend:
      shard-apl:          PASS -> FAIL (fdo#103191, fdo#103232)
      shard-kbl:          PASS -> FAIL (fdo#103191, fdo#103232)

    igt@kms_flip@plain-flip-ts-check:
      shard-kbl:          PASS -> DMESG-WARN (fdo#105602, fdo#103558) +44

    igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-draw-mmap-cpu:
      shard-glk:          PASS -> FAIL (fdo#103167) +2

    igt@kms_plane_alpha_blend@pipe-c-alpha-opaque-fb:
      shard-kbl:          PASS -> FAIL (fdo#108145)
      shard-apl:          PASS -> FAIL (fdo#108145)

    igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max:
      shard-glk:          PASS -> FAIL (fdo#108145) +2

    igt@kms_plane_multiple@atomic-pipe-b-tiling-y:
      shard-apl:          PASS -> FAIL (fdo#103166) +3

    igt@kms_setmode@basic:
      shard-kbl:          PASS -> FAIL (fdo#99912)

    igt@kms_universal_plane@universal-plane-pipe-b-functional:
      shard-glk:          PASS -> FAIL (fdo#103166) +3

    igt@kms_vblank@pipe-c-ts-continuation-suspend:
      shard-kbl:          PASS -> INCOMPLETE (fdo#103665)

    
    ==== Possible fixes ====

    igt@kms_ccs@pipe-a-crc-sprite-planes-basic:
      shard-glk:          FAIL (fdo#108145) -> PASS +1

    igt@kms_color@pipe-b-ctm-max:
      shard-apl:          DMESG-WARN (fdo#105602, fdo#103558) -> PASS +18

    igt@kms_color@pipe-c-ctm-max:
      shard-apl:          FAIL (fdo#108147) -> PASS

    igt@kms_cursor_crc@cursor-128x128-random:
      shard-kbl:          FAIL (fdo#103232) -> PASS

    igt@kms_cursor_crc@cursor-128x128-suspend:
      shard-apl:          FAIL (fdo#103191, fdo#103232) -> PASS +1

    igt@kms_cursor_crc@cursor-128x42-onscreen:
      shard-glk:          FAIL (fdo#103232) -> PASS +5

    igt@kms_cursor_crc@cursor-256x256-suspend:
      shard-kbl:          FAIL (fdo#103191, fdo#103232) -> PASS +1

    igt@kms_cursor_crc@cursor-64x21-random:
      shard-apl:          FAIL (fdo#103232) -> PASS +6

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-pwrite:
      shard-apl:          FAIL (fdo#103167) -> PASS +4
      shard-kbl:          FAIL (fdo#103167) -> PASS

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-move:
      shard-glk:          FAIL (fdo#103167) -> PASS +8

    igt@kms_plane@plane-position-covered-pipe-b-planes:
      shard-glk:          FAIL (fdo#103166) -> PASS +3

    igt@kms_plane_multiple@atomic-pipe-a-tiling-y:
      shard-apl:          FAIL (fdo#103166) -> PASS

    igt@kms_setmode@basic:
      shard-apl:          FAIL (fdo#99912) -> PASS

    igt@kms_vblank@pipe-c-ts-continuation-dpms-suspend:
      shard-kbl:          INCOMPLETE (fdo#103665) -> PASS

    igt@prime_vgem@coherency-gtt:
      shard-snb:          INCOMPLETE (fdo#105411) -> PASS

    
    ==== Warnings ====

    igt@kms_cursor_crc@cursor-64x21-sliding:
      shard-kbl:          FAIL (fdo#103232) -> DMESG-WARN (fdo#105602, fdo#103558)

    igt@kms_plane_alpha_blend@pipe-c-alpha-basic:
      shard-kbl:          FAIL (fdo#108145) -> DMESG-FAIL (fdo#105602, fdo#103558, fdo#108145)

    igt@kms_plane_alpha_blend@pipe-c-alpha-transparant-fb:
      shard-apl:          DMESG-FAIL (fdo#105602, fdo#103558, fdo#108145) -> FAIL (fdo#108145)

    
  fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
  fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#103232 https://bugs.freedesktop.org/show_bug.cgi?id=103232
  fdo#103558 https://bugs.freedesktop.org/show_bug.cgi?id=103558
  fdo#103665 https://bugs.freedesktop.org/show_bug.cgi?id=103665
  fdo#105411 https://bugs.freedesktop.org/show_bug.cgi?id=105411
  fdo#105602 https://bugs.freedesktop.org/show_bug.cgi?id=105602
  fdo#106641 https://bugs.freedesktop.org/show_bug.cgi?id=106641
  fdo#107956 https://bugs.freedesktop.org/show_bug.cgi?id=107956
  fdo#108145 https://bugs.freedesktop.org/show_bug.cgi?id=108145
  fdo#108147 https://bugs.freedesktop.org/show_bug.cgi?id=108147
  fdo#99912 https://bugs.freedesktop.org/show_bug.cgi?id=99912


== Participating hosts (6 -> 5) ==

  Missing    (1): shard-skl 


== Build changes ==

    * IGT: IGT_4729 -> IGTPW_2096

  CI_DRM_5205: 48f2a38e885181b63502de679e4975551955ea74 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2096: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2096/
  IGT_4729: 2388bbd062c17b5912039101efd4603e8d876c88 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2096/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Be patient for processes to die

[Petri Latvala]

On Mon, Nov 26, 2018 at 09:37:44PM +0000, Chris Wilson wrote:
> Some machines are very slow and some processes hog a lot of resources
> and so take much longer than a mere 2s to be terminated. Be patient.
> 
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108073
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
>  runner/executor.c | 44 ++++++++++++++++++++++++++++++--------------
>  1 file changed, 30 insertions(+), 14 deletions(-)
> 
> diff --git a/runner/executor.c b/runner/executor.c
> index 2038c3fda..927696044 100644
> --- a/runner/executor.c
> +++ b/runner/executor.c
> @@ -137,27 +137,36 @@ static char *handle_lockdep(void)
>  	return NULL;
>  }
>  
> -static char *handle_taint(void)
> +
> +static unsigned long tainted(unsigned long *taints)


An extra newline slipped in here.


Reviewed-by: Petri Latvala <petri.latvala@intel.com>



>  {
>  	const unsigned long bad_taints =
>  		0x20  | /* TAINT_PAGE */
>  		0x80  | /* TAINT_DIE */
>  		0x200; /* TAINT_OOPS */
> -	unsigned long taints = 0;
> -	char *reason = NULL;
>  	FILE *f;
>  
> +	*taints = 0;
> +
>  	f = fopen("/proc/sys/kernel/tainted", "r");
>  	if (f) {
> -		fscanf(f, "%lu", &taints);
> +		fscanf(f, "%lu", taints);
>  		fclose(f);
>  	}
>  
> -	if (taints & bad_taints)
> -		asprintf(&reason,
> -			 "Kernel tainted (%#lx -- %lx)",
> -			 taints, taints & bad_taints);
> +	return *taints & bad_taints;
> +}
> +
> +static char *handle_taint(void)
> +{
> +	unsigned long taints, bad_taints;
> +	char *reason;
> +
> +	bad_taints = tainted(&taints);
> +	if (!bad_taints)
> +		return NULL;
>  
> +	asprintf(&reason, "Kernel tainted (%#lx -- %lx)", taints, bad_taints);
>  	return reason;
>  }
>  
> @@ -437,6 +446,7 @@ static int monitor_output(pid_t child,
>  	int wd_extra = 10;
>  	int killed = 0; /* 0 if not killed, signal number otherwise */
>  	struct timespec time_beg, time_end;
> +	unsigned long taints = 0;
>  	bool aborting = false;
>  
>  	igt_gettime(&time_beg);
> @@ -493,10 +503,8 @@ static int monitor_output(pid_t child,
>  		}
>  
>  		if (n == 0) {
> -			intervals_left--;
> -			if (intervals_left) {
> +			if (--intervals_left)
>  				continue;
> -			}
>  
>  			ping_watchdogs();
>  
> @@ -504,6 +512,7 @@ static int monitor_output(pid_t child,
>  			case 0:
>  				if (settings->log_level >= LOG_LEVEL_NORMAL) {
>  					printf("Timeout. Killing the current test with SIGTERM.\n");
> +					fflush(stdout);
>  				}
>  
>  				killed = SIGTERM;
> @@ -514,13 +523,14 @@ static int monitor_output(pid_t child,
>  				 * Now continue the loop and let the
>  				 * dying child be handled normally.
>  				 */
> -				timeout = 2; /* Timeout for waiting selected by fair dice roll. */
> -				watchdogs_set_timeout(20);
> +				timeout = 20;
> +				watchdogs_set_timeout(120);
>  				intervals_left = timeout_intervals = 1;
>  				break;
>  			case SIGTERM:
>  				if (settings->log_level >= LOG_LEVEL_NORMAL) {
>  					printf("Timeout. Killing the current test with SIGKILL.\n");
> +					fflush(stdout);
>  				}
>  
>  				killed = SIGKILL;
> @@ -530,9 +540,15 @@ static int monitor_output(pid_t child,
>  				intervals_left = timeout_intervals = 1;
>  				break;
>  			case SIGKILL:
> +				if (kill(0, child) == 0 && !tainted(&taints)) {
> +					intervals_left =  1;
> +					break;
> +				}
> +
>  				/* Nothing that can be done, really. Let's tell the caller we want to abort. */
>  				if (settings->log_level >= LOG_LEVEL_NORMAL) {
> -					fprintf(stderr, "Child refuses to die. Aborting.\n");
> +					fprintf(stderr, "Child refuses to die, tainted %x. Aborting.\n",
> +						taints);
>  				}
>  				close_watchdogs(settings);
>  				free(outbuf);
> -- 
> 2.20.0.rc1
> 
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t] runner: Be patient for processes to die

[Chris Wilson]

Quoting Petri Latvala (2018-11-27 09:32:32)
> On Mon, Nov 26, 2018 at 09:37:44PM +0000, Chris Wilson wrote:
> > Some machines are very slow and some processes hog a lot of resources
> > and so take much longer than a mere 2s to be terminated. Be patient.
> > 
> > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108073
> > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > ---
> >  runner/executor.c | 44 ++++++++++++++++++++++++++++++--------------
> >  1 file changed, 30 insertions(+), 14 deletions(-)
> > 
> > diff --git a/runner/executor.c b/runner/executor.c
> > index 2038c3fda..927696044 100644
> > --- a/runner/executor.c
> > +++ b/runner/executor.c
> > @@ -137,27 +137,36 @@ static char *handle_lockdep(void)
> >       return NULL;
> >  }
> >  
> > -static char *handle_taint(void)
> > +
> > +static unsigned long tainted(unsigned long *taints)
> 
> 
> An extra newline slipped in here.
> 
> 
> Reviewed-by: Petri Latvala <petri.latvala@intel.com>

Fingers crossed since CI skipped skl-shards...
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev