[OSSTEST PATCH 1/4] Email reports: Sort out recipients and Reply-to

[OSSTEST PATCH 1/4] Email reports: Sort out recipients and Reply-to

[Ian Jackson]

No longer send reports, or copies, to named individuals.  Instead,
send all output to 1. appropriate development mailing lists 2. the new
osstest-admin@xenproject administrator alias, and 3. Bcc the new
osstest-output list.

Bisection progress emails go only to osstest-output.  In this case we
provide a Reply-To.

After this patch goes live, people interested in bisection progress
emails will find them in osstest-output.  (There are a lot of these.)
Such people can also subscribe to osstest-output and get copies of
only the main test reports and then use recipient-side filtering to
see the ones they find interesting.

(Configurations for `adhoc' and `play' runs remain unchanged and still
have a tendency to refer to my personal address @citrix.)

Deployment notes:

Keir and Stefano will no longer receive automatic CCs of certain
reports, and should subscribe to the osstest-output list, with
appropriate filtering, if they care.

The new osstest-output list has (I think) been configured to accept
the BCCs.

My personal chiark mail-to-news gateway has already been subscribed to
osstest-output.

One substantial change is that the osstest admin now gets copies of
each final bisection failure.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: keir@xen.org
CC: stefano.stabellini@eu.citrix.com
CC: Lars Kurth <lars.kurth@citrix.com>
---
 daily-cron-email-osstest                          |    3 ++-
 daily-cron-email-real                             |    5 +++--
 daily-cron-email-real--rumpuserxen                |    5 +++--
 daily-cron-email-real-bisectcomplete              |    7 +++----
 daily-cron-email-real-bisectcomplete--rumpuserxen |    5 +++--
 daily-cron-email-real-bisectdone                  |    3 ++-
 daily-cron-email-real-bisectrun                   |    3 ++-
 7 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/daily-cron-email-osstest b/daily-cron-email-osstest
index 8a9fac4..50dc9cd 100644
--- a/daily-cron-email-osstest
+++ b/daily-cron-email-osstest
@@ -1 +1,2 @@
-To: ian.jackson@eu.citrix.com,ian.campbell@eu.citrix.com
+To: osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real b/daily-cron-email-real
index e43d780..139069c 100644
--- a/daily-cron-email-real
+++ b/daily-cron-email-real
@@ -1,2 +1,3 @@
-To: xen-devel@lists.xensource.com
-Cc: ian.jackson@eu.citrix.com
+To: xen-devel@lists.xensource.com,
+    osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real--rumpuserxen b/daily-cron-email-real--rumpuserxen
index a9166a0..0e55978 100644
--- a/daily-cron-email-real--rumpuserxen
+++ b/daily-cron-email-real--rumpuserxen
@@ -1,3 +1,4 @@
 To: xen-devel@lists.xensource.com,
-    rumpkernel-builds@freelists.org
-Cc: ian.jackson@eu.citrix.com
+    rumpkernel-builds@freelists.org,
+    osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real-bisectcomplete b/daily-cron-email-real-bisectcomplete
index 18ae6ad..139069c 100644
--- a/daily-cron-email-real-bisectcomplete
+++ b/daily-cron-email-real-bisectcomplete
@@ -1,4 +1,3 @@
-To: xen-devel@lists.xensource.com
-Cc: keir@xen.org,
-    stefano.stabellini@eu.citrix.com,
-    ian.jackson@eu.citrix.com
+To: xen-devel@lists.xensource.com,
+    osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real-bisectcomplete--rumpuserxen b/daily-cron-email-real-bisectcomplete--rumpuserxen
index a9166a0..0e55978 100644
--- a/daily-cron-email-real-bisectcomplete--rumpuserxen
+++ b/daily-cron-email-real-bisectcomplete--rumpuserxen
@@ -1,3 +1,4 @@
 To: xen-devel@lists.xensource.com,
-    rumpkernel-builds@freelists.org
-Cc: ian.jackson@eu.citrix.com
+    rumpkernel-builds@freelists.org,
+    osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real-bisectdone b/daily-cron-email-real-bisectdone
index eef18cd..50dc9cd 100644
--- a/daily-cron-email-real-bisectdone
+++ b/daily-cron-email-real-bisectdone
@@ -1 +1,2 @@
-To: list-a__chiark.users.ijackson.xen.cron.osstest@chiark.greenend.org.uk
+To: osstest-admin@xenproject.org
+Bcc: osstest-output@lists.xenproject.org
diff --git a/daily-cron-email-real-bisectrun b/daily-cron-email-real-bisectrun
index eef18cd..efb5e87 100644
--- a/daily-cron-email-real-bisectrun
+++ b/daily-cron-email-real-bisectrun
@@ -1 +1,2 @@
-To: list-a__chiark.users.ijackson.xen.cron.osstest@chiark.greenend.org.uk
+To: osstest-output@lists.xenproject.org
+Reply-to: osstest-admin@xenproject.org
-- 
1.7.10.4

[OSSTEST PATCH 2/4] Reporting: Make HTML reports for bisection flights too

[Ian Jackson]

Provide an appropriate --html-dir option to sg-report-flight.
(Bisection flights still do not run sg-report-host-history and
sg-report-job-history, which are fairly expensive.)

Make references to jobs from sg-report-job-history and
sg-report-host-history go to the /info.html page, not the webserver
directory listing.  (sg-report-flight's references already do.)

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 cri-bisect             |    5 ++++-
 sg-report-host-history |    2 +-
 sg-report-job-history  |    3 ++-
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/cri-bisect b/cri-bisect
index 03b2e29..2ce9945 100644
--- a/cri-bisect
+++ b/cri-bisect
@@ -110,7 +110,10 @@ END
         cat >&3 $reportfile
         if $flight_executed; then
                 echo >&3 ----------------------------------------
-                ./sg-report-flight >&3 $flight
+		if [ "x$OSSTEST_HTMLPUB_DIR" != x ]; then
+			sgr_args+="$OSSTEST_HTMLPUB_DIR/$flight/"
+		fi
+                ./sg-report-flight $sgr_args >&3 $flight
                 ./cr-publish-flight-logs --push-harness $flight
         fi
         exec 3>&-
diff --git a/sg-report-host-history b/sg-report-host-history
index f33fafa..8e0e2aa 100755
--- a/sg-report-host-history
+++ b/sg-report-host-history
@@ -238,7 +238,7 @@ END
 	my $url= "$c{ReportHtmlPubBaseUrl}/$jr->{flight}";
 	print H "<td><a href=\"$url\">$jr->{flight}</a></td>\n";
 	$url= "$c{ReportHtmlPubBaseUrl}/$jr->{flight}/".
-	    encode_entities($jr->{job})."/";
+	    encode_entities($jr->{job})."/info.html";
 	print H "<td>$ir->{branch}</td>";
 	print H "<td>$ir->{intended}</td>";
 	print H "<td>";
diff --git a/sg-report-job-history b/sg-report-job-history
index 031bd4a..e20c376 100755
--- a/sg-report-job-history
+++ b/sg-report-job-history
@@ -244,7 +244,8 @@ END
             $url= "$c{ReportHtmlPubBaseUrl}/$flt";
             print H "<td><a href=\"$url\">$flt</a></td>\n";
             print H "<td>".encode_entities($r->{Flight}{branch})."</td>\n";
-            $url= "$c{ReportHtmlPubBaseUrl}/$flt/".encode_entities($j)."/";
+            $url= "$c{ReportHtmlPubBaseUrl}/$flt/".encode_entities($j).
+		"/info.html";
             print H "<td $r->{ColourAttr}><a href=\"$url\">".
                 $r->{Content}."</a></td>\n";
             my $lastrev;
-- 
1.7.10.4

Re: [OSSTEST PATCH 2/4] Reporting: Make HTML reports for bisection flights too

[Ian Campbell]

On Wed, 2015-07-08 at 16:21 +0100, Ian Jackson wrote:
> Provide an appropriate --html-dir option to sg-report-flight.
> (Bisection flights still do not run sg-report-host-history and
> sg-report-job-history, which are fairly expensive.)
> 
> Make references to jobs from sg-report-job-history and
> sg-report-host-history go to the /info.html page, not the webserver
> directory listing.  (sg-report-flight's references already do.)
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

[OSSTEST PATCH 3/4] mg-branch-setup: Get the direction of the tree-bisect link right

[Ian Jackson]

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-branch-setup |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mg-branch-setup b/mg-branch-setup
index d465882..f8bcb2e 100755
--- a/mg-branch-setup
+++ b/mg-branch-setup
@@ -107,8 +107,8 @@ set -e
 make_tree branches
 if $bisect; then
 	make_tree bisects
-	x ln -sf $HOME/branches/for-$branch.git \
-		$HOME/bisects/for-$branch.git/tree-bisect
+	x ln -sf $HOME/bisects/for-$branch.git \
+		$HOME/branches/for-$branch.git/tree-bisect
 fi
 
 echo "Branch $branch set up ok."
-- 
1.7.10.4

Re: [OSSTEST PATCH 3/4] mg-branch-setup: Get the direction of the tree-bisect link right

[Ian Campbell]

On Wed, 2015-07-08 at 16:21 +0100, Ian Jackson wrote:
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

(I think you cleared up the existing mess already)

> ---
>  mg-branch-setup |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mg-branch-setup b/mg-branch-setup
> index d465882..f8bcb2e 100755
> --- a/mg-branch-setup
> +++ b/mg-branch-setup
> @@ -107,8 +107,8 @@ set -e
>  make_tree branches
>  if $bisect; then
>  	make_tree bisects
> -	x ln -sf $HOME/branches/for-$branch.git \
> -		$HOME/bisects/for-$branch.git/tree-bisect
> +	x ln -sf $HOME/bisects/for-$branch.git \
> +		$HOME/branches/for-$branch.git/tree-bisect
>  fi
>  
>  echo "Branch $branch set up ok."

[OSSTEST PATCH 4/4] JobDB/Executive: Improve an internal `die' error

[Ian Jackson]

When this assertion fails, dump the troublesome harness revision and
requested host flags, too.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/JobDB/Executive.pm |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/Osstest/JobDB/Executive.pm b/Osstest/JobDB/Executive.pm
index ba51bdf..1ec947e 100644
--- a/Osstest/JobDB/Executive.pm
+++ b/Osstest/JobDB/Executive.pm
@@ -127,14 +127,17 @@ sub host_check_allocated ($$) { #method
     $ho->{SharedMaybeOthers}=
         $ho->{Shared} &&
         $ho->{Shared}{State} eq 'ready';
+    my $harness = get_harness_rev();
+    my @flags = get_hostflags($ho->{Ident});
     $ho->{SharedReady}=
 	$ho->{SharedMaybeOthers} &&
-        !! (grep { $_." ".get_harness_rev() eq "share-".$ho->{Shared}{Type} }
-	    get_hostflags($ho->{Ident}));
+        !! (grep { $_." ".$harness eq "share-".$ho->{Shared}{Type} }
+	    @flags);
     $ho->{SharedOthers}=
         $ho->{Shared} ? $ho->{Shared}{Others} : 0;
     
-    die Dumper($ho)." ?" if $ho->{SharedOthers} && !$ho->{SharedReady};
+    die Dumper($ho, $harness, \@flags)." ?"
+	if $ho->{SharedOthers} && !$ho->{SharedReady};
 }
 
 sub jobdb_postfork ($) { #method
-- 
1.7.10.4

Re: [OSSTEST PATCH 4/4] JobDB/Executive: Improve an internal `die' error

[Ian Campbell]

On Wed, 2015-07-08 at 16:21 +0100, Ian Jackson wrote:
> When this assertion fails, dump the troublesome harness revision and
> requested host flags, too.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 1/4] Email reports: Sort out recipients and Reply-to

[Ian Campbell]

On Wed, 2015-07-08 at 16:21 +0100, Ian Jackson wrote:
> No longer send reports, or copies, to named individuals.  Instead,
> send all output to 1. appropriate development mailing lists 2. the new
> osstest-admin@xenproject administrator alias, and 3. Bcc the new
> osstest-output list.
> 
> Bisection progress emails go only to osstest-output.  In this case we
> provide a Reply-To.
> 
> After this patch goes live, people interested in bisection progress
> emails will find them in osstest-output.  (There are a lot of these.)
> Such people can also subscribe to osstest-output and get copies of
> only the main test reports and then use recipient-side filtering to
> see the ones they find interesting.
> 
> (Configurations for `adhoc' and `play' runs remain unchanged and still
> have a tendency to refer to my personal address @citrix.)
> 
> Deployment notes:
> 
> Keir and Stefano will no longer receive automatic CCs of certain
> reports, and should subscribe to the osstest-output list, with
> appropriate filtering, if they care.
> 
> The new osstest-output list has (I think) been configured to accept
> the BCCs.
> 
> My personal chiark mail-to-news gateway has already been subscribed to
> osstest-output.
> 
> One substantial change is that the osstest admin now gets copies of
> each final bisection failure.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> CC: keir@xen.org
> CC: stefano.stabellini@eu.citrix.com
> CC: Lars Kurth <lars.kurth@citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 1/4] Email reports: Sort out recipients and Reply-to

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 1/4] Email reports: Sort out recipients and Reply-to"):
> On Wed, 2015-07-08 at 16:21 +0100, Ian Jackson wrote:
> > Deployment notes:
> > 
> > Keir and Stefano will no longer receive automatic CCs of certain
> > reports, and should subscribe to the osstest-output list, with
> > appropriate filtering, if they care.

This patch has now been pushed (along with the others) to the osstest
colo's pretest branch.  It will be automatically deployed if and when
it has passed the self-tests.

Ian.