[OSSTEST PATCH 0/4] A few more bisector fixes

[OSSTEST PATCH 0/4] A few more bisector fixes

[Ian Jackson]

This contains one bugfix to a bug reported to me by Ian Campbell, and
three useful cleanups prompted by annoyances I tripped over on the
way.

[OSSTEST PATCH 1/4] adhoc-revtuple-generator: Turn off a set -x

[Ian Jackson]

This set -x uglified the output.  Provide a way (OSSTEST_AHRTG_SETX)
to turn it on again.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 adhoc-revtuple-generator |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/adhoc-revtuple-generator b/adhoc-revtuple-generator
index 2b04c11..5a44d63 100755
--- a/adhoc-revtuple-generator
+++ b/adhoc-revtuple-generator
@@ -99,8 +99,8 @@ sub git_generator ($) {
     my $treename= $tree->{Treename};
     my $eff_url = git_massage_url($tree->{Url});
     my $upcmd= <<END;
-        set -xe; exec >&2
-        if ! sh -xe <<ENDIFCOND
+        set -e\$OSSTEST_AHRTG_SETX; exec >&2
+        if ! sh -e\$OSSTEST_AHRTG_SETX <<ENDIFCOND
             # the "set -e"-disarmament done by "if" leaks into ( ) !
 	    cd $c{Repos}/$treename
 	    git remote set-url origin $eff_url
-- 
1.7.10.4

[OSSTEST PATCH 2/4] adhoc-revtuple-generator: Honour OSSTEST_AHRTG_SETX elsewhere

[Ian Jackson]

Find all the places where adhoc-revtuple-generator runs subprograms
and have it add set -x (either by adding $OSSTEST_AHRTG_SETX to an
existing set -e, or using $setx which is either : or `set -x').

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 adhoc-revtuple-generator |   17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/adhoc-revtuple-generator b/adhoc-revtuple-generator
index 5a44d63..c920e9c 100755
--- a/adhoc-revtuple-generator
+++ b/adhoc-revtuple-generator
@@ -58,6 +58,8 @@ while (@ARGV && $ARGV[0] =~ m/^-/) {
     }
 }
 
+my $setx = ($ENV{OSSTEST_AHRTG_SETX} // '') =~ m/x/ ? 'set -x' : ':';
+
 readglobalconfig();
 
 grabrepolock_reexec(@org_argv);
@@ -113,7 +115,7 @@ ENDIFCOND
 END
     print DEBUG "GIT-GEN UPCMD\n$upcmd\n";
     shellcmd($upcmd) if $doupdate;
-    my $cmd= "cd $c{Repos}/$treename &&".
+    my $cmd= "$setx; cd $c{Repos}/$treename &&".
         " git log --pretty=raw --date-order $tree->{Latest}";
     print DEBUG "GIT-GEN CMD $cmd\n";
     my $fh= new IO::File;
@@ -175,7 +177,7 @@ sub hg_revshower ($) {
     my $treedir= "$c{Repos}/$tree->{Treename}";
     chdir $treedir or die $treedir;
     my $revspec = "$tree->{Latest}:$tree->{Earliest}";
-    my $cmd = "hg log -vr'$revspec' |";
+    my $cmd = "$setx; hg log -vr'$revspec' |";
     print DEBUG "HG-REVSHOW CMD $cmd\n";
     open P, $cmd or die $!;
     while (<P>) {
@@ -195,8 +197,9 @@ sub hg_generator ($) {
     local $_;
     print DEBUG "HG-GEN ".pmap($tree).".\n";
     my $treename= $tree->{Treename};
-    shellcmd("set -e; exec >&2; cd $c{Repos}/$treename && hg pull -u")
-        if $doupdate;
+    shellcmd(
+ "set -e\$OSSTEST_AHRTG_SETX; exec >&2; cd $c{Repos}/$treename && hg pull -u"
+	) if $doupdate;
     my $cmd= "cd $c{Repos}/$treename && hg log -r $tree->{Latest}:1".
         " --template ".("'{node|short}\n".
                          "{node|short} {date}\n".
@@ -204,7 +207,7 @@ sub hg_generator ($) {
                          "{parents}\n".
                          "'");
     my $fh= new IO::File;
-    open $fh, "$cmd |" or die "$cmd $!";
+    open $fh, "$setx; $cmd |" or die "$cmd $!";
     $_= <$fh>;
     m/^\w+$/ or die "$_ $! ?";
     return sub {
@@ -301,7 +304,7 @@ sub xu_withtag_generator ($) {
 	if ($targetqemu eq 'unknown') {
 	    my $nodeonly= $xenrev->{Commit};
 	    $nodeonly =~ s/^\d+\://;
-	    open CMK, "cd $c{Repos}/$xentree &&
+	    open CMK, "$setx; cd $c{Repos}/$xentree &&
                        hg cat -r $nodeonly Config.mk |"
 			 or die $!;
 	    while (<CMK>) {
@@ -312,7 +315,7 @@ sub xu_withtag_generator ($) {
 		die if $targetqemu =~ m/[^-+._0-9a-zA-Z]/;
 		die unless $targetqemu =~ m/^[^-]/;
 		$!=0; $?=0;
-		$targetqemu= `cd $c{Repos}/$qemutree &&
+		$targetqemu= `$setx; cd $c{Repos}/$qemutree &&
                               git rev-parse '$targetqemu^0'`;
 		die "$! $?" if (!defined $targetqemu) or $?;
 		chomp $targetqemu;
-- 
1.7.10.4

[OSSTEST PATCH 3/4] cs-bisection-step: Print our command line at the start

[Ian Jackson]

The usual approach for debugging the cs-bisection-step is to repro the
problem (with --max-flight), which is most easily done by copying the
command line provided during a run which did the wrong thing.

Print the command line at startup, so that it appears in the report.
This will save us grobbling through the logs and cron mail.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 cs-bisection-step |   13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/cs-bisection-step b/cs-bisection-step
index 42bf20b..37efbea 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -114,6 +114,18 @@ our (@treeinfos);
 our $restrictflight_cond= restrictflight_cond();
 our $blessingscond= '('.join(" OR  ", map { "blessing='$_'" } @blessings).')';
 
+sub reportcmdline () {
+    print STDERR "Running cs-bisection-step";
+    foreach (my @noalias = @org_argv) {
+	if (!length || m{[^-+,./_=:A-Z0-9a-z]}) {
+	    s{'}{'\\''}g;
+	    $_ = "'$_'";
+	}
+	print STDERR " ", $_;
+    }
+    print STDERR "\n";
+}
+
 sub psummary ($) {
     my ($s) = @_;
     print STDERR $s or die $!;
@@ -1326,6 +1338,7 @@ sub compute_exitstatus () {
 
 csreadconfig();
 grabrepolock_reexec(@org_argv);
+reportcmdline();
 findbasics();
 digraph_whole();
 digraph_relevant();
-- 
1.7.10.4

[OSSTEST PATCH 4/4] cs-bisection-step: Properly handle external job refs in template

[Ian Jackson]

cs-bisection-step has had, for a long time, code which is supposed to
handle the situation where the template flight contains build job
references to other flights.

However:

 - The regexp to spot these other-flight job reference runvars would
   never match because it said \s where \S was probably intended (and
   . would be better);

 - If it were to match, the flight and job arguments to the recursive
   preparejob invocation were teh wrong way round.  preparejob takes
   the job name first.

Fix these two bugs.  Now it does seem to work properly.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 cs-bisection-step |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/cs-bisection-step b/cs-bisection-step
index 37efbea..b676044 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1190,8 +1190,8 @@ END
     foreach my $subjob (@$subjobs) {
         my $target;
         my $jobspec= $subjob->{val};
-        if ($jobspec =~ m/^(\d+)\.(\s+)$/) {
-            $target= preparejob($1, $2, 1);
+        if ($jobspec =~ m/^(\d+)\.(.+)$/) {
+            $target= preparejob($2, $1, 1);
         } else {
             $target= preparejob($jobspec, $copyflight, 1);
         }
-- 
1.7.10.4

Re: [OSSTEST PATCH 4/4] cs-bisection-step: Properly handle external job refs in template

[Ian Campbell]

On Fri, 2015-09-04 at 11:56 +0100, Ian Jackson wrote:
> cs-bisection-step has had, for a long time, code which is supposed to
> handle the situation where the template flight contains build job
> references to other flights.
> 
> However:
> 
>  - The regexp to spot these other-flight job reference runvars would
>    never match because it said \s where \S was probably intended (and
>    . would be better);
> 
>  - If it were to match, the flight and job arguments to the recursive
>    preparejob invocation were teh wrong way round.  preparejob takes

"teh"

Otherwise all 4 of these patches LGTM:
Acked-by: Ian Campbell <ian.campbell@citrix.com>

>    the job name first.
> 
> Fix these two bugs.  Now it does seem to work properly.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> ---
>  cs-bisection-step |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/cs-bisection-step b/cs-bisection-step
> index 37efbea..b676044 100755
> --- a/cs-bisection-step
> +++ b/cs-bisection-step
> @@ -1190,8 +1190,8 @@ END
>      foreach my $subjob (@$subjobs) {
>          my $target;
>          my $jobspec= $subjob->{val};
> -        if ($jobspec =~ m/^(\d+)\.(\s+)$/) {
> -            $target= preparejob($1, $2, 1);
> +        if ($jobspec =~ m/^(\d+)\.(.+)$/) {
> +            $target= preparejob($2, $1, 1);
>          } else {
>              $target= preparejob($jobspec, $copyflight, 1);
>          }