[PATCH 00/10] Repeat rump kernel xenstorels test

[PATCH 00/10] Repeat rump kernel xenstorels test

[Ian Jackson]

This 10-patch series(!) arranges to repeat the xenstorels test 50
times.  In my ad-hoc tests this takes about 3 minutes so is clearly
fine from a resource POV.

This is in aid of debugging the problem seen in 33690 and 33416 (which
we think is the same).  However, in my ad-hoc tests it hasn't failed
even once (with the bits from flight 33866 which is after we think the
bug appeared).  I think it would be valuable to introduce this new
repeat test in any case.

The series is in four parts:

 - 01..03  Minor improvements in TestSupport.pm

 - 04..07 Changes to testid generation to give better control of the
	   testid for tests done with the ts-repeat-test script from
	   patch 08.

 - 08..10  Actually do the repetition

04..07 need a thorough full-flight test which I have not yet done.
But this sql rune suggests that nothing is horribly broken at least
for the rump test job:

  select * from (select testid,flight,stepno,step,status,started,finished from steps where flight=33866 and job='test-amd64-i386-rumpuserxen-i386') a full outer join (select * from steps where flight=34273 and job='test-amd64-i386-rumpuserxen-i386') b using (testid) order by b.stepno, a.stepno;

[OSSTEST PATCH 01/10] TestSupport: Provide target_editfile_cancel

[Ian Jackson]

This allows a caller of target_editfile to cancel the edit from within
their supplied code block.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm |   20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 6a02757..5a4c41f 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,6 +55,7 @@ BEGIN {
                       target_putfilecontents_stash
 		      target_putfilecontents_root_stash
                       target_put_guest_image target_editfile
+                      target_editfile_cancel
                       target_editfile_root target_file_exists
                       target_run_apt
                       target_install_packages target_install_packages_norec
@@ -483,6 +484,14 @@ sub target_file_exists ($$) {
     die "$rfile $out ?";
 }
 
+our $target_editfile_cancel_exception =
+    bless { }, 'Osstest::TestSupport::TargetEditfileCancelException';
+
+sub target_editfile_cancel ($) {
+    logm("cancelling edit: @_");
+    die $target_editfile_cancel_exception;
+}
+
 sub teditfileex {
     my $user= shift @_;
     my $code= pop @_;
@@ -512,12 +521,19 @@ sub teditfileex {
     open '::EI', "$lfile" or die "$lfile: $!";
     open '::EO', "> $lfile.new" or die "$lfile.new: $!";
 
-    &$code;
+    my $install = 1;
+
+    eval { &$code };
+    if ($@) {
+	ref $@ && $@ == $target_editfile_cancel_exception or die $@;
+	$install = 0;
+    }
 
     '::EI'->error and die $!;
     close '::EI' or die $!;
     close '::EO' or die $!;
-    tputfileex($user, $ho, 60, "$lfile.new", $rdest);
+    tputfileex($user, $ho, 60, "$lfile.new", $rdest)
+	if $install;
 }
 
 sub target_editfile_root ($$$;$$) { teditfileex('root',@_); }
-- 
1.7.10.4

Re: [OSSTEST PATCH 01/10] TestSupport: Provide target_editfile_cancel

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> This allows a caller of target_editfile to cancel the edit from within
> their supplied code block.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

This contains more Perl exception handling voodoo than I'm really
qualified for, but based on a bit of searching and if your tests work
then:
Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 01/10] TestSupport: Provide target_editfile_cancel

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 01/10] TestSupport: Provide target_editfile_cancel"):
> On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> > This allows a caller of target_editfile to cancel the edit from within
> > their supplied code block.
> > 
> > Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> 
> This contains more Perl exception handling voodoo than I'm really
> qualified for, but based on a bit of searching and if your tests work
> then:
> Acked-by: Ian Campbell <ian.campbell@citrix.com>

Yes, my tests do work :-).

Thanks,
Ian.

[OSSTEST PATCH 02/10] TestSupport: Introduce next_unique_name

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm |   11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 5a4c41f..55541b1 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -46,7 +46,7 @@ BEGIN {
 
                       store_runvar get_runvar get_runvar_maybe
                       get_runvar_default need_runvars flight_otherjob
-                      unique_incrementing_runvar 
+                      unique_incrementing_runvar next_unique_name
 
                       target_cmd_root target_cmd target_cmd_build
                       target_cmd_output_root target_cmd_output
@@ -484,6 +484,11 @@ sub target_file_exists ($$) {
     die "$rfile $out ?";
 }
 
+sub next_unique_name ($) {
+    my ($fnref) = @_;
+    $$fnref .= '+';
+}
+
 our $target_editfile_cancel_exception =
     bless { }, 'Osstest::TestSupport::TargetEditfileCancelException';
 
@@ -509,7 +514,7 @@ sub teditfileex {
             $! == &ENOENT or die "$lfile $!";
             last;
         }
-        $lleaf .= '+';
+        next_unique_name \$lleaf;
     }
     if ($rdest eq $rfile) {
         logm("editing $rfile as $lfile".'{,.new}');
@@ -929,7 +934,7 @@ sub open_unique_stashfile ($) {
         $dh= new IO::File "$stash/$df", O_WRONLY|O_EXCL|O_CREAT;
         last if $dh;
         die "$df $!" unless $!==&EEXIST;
-        $$leafref .= '+';
+        next_unique_name $leafref;
     }
     return $dh;
 }
-- 
1.7.10.4

Re: [OSSTEST PATCH 02/10] TestSupport: Introduce next_unique_name

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> No functional change.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

[OSSTEST PATCH 03/10] TestSupport: Make next_unique_name count in decimal, not unary

[Ian Jackson]

Ie, we add `+<counter>' rather than an ever-longer series of `+'s.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 55541b1..8aed285 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -486,7 +486,8 @@ sub target_file_exists ($$) {
 
 sub next_unique_name ($) {
     my ($fnref) = @_;
-    $$fnref .= '+';
+    my $num = $$fnref =~ s/\+([1-9]\d*)$// ? $1 : 0;
+    $$fnref .= '+'.($num+1);
 }
 
 our $target_editfile_cancel_exception =
-- 
1.7.10.4

Re: [OSSTEST PATCH 03/10] TestSupport: Make next_unique_name count in decimal, not unary

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> Ie, we add `+<counter>' rather than an ever-longer series of `+'s.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

[OSSTEST PATCH 04/10] tcl/osstestlib.tcl: Provide lshift

[Ian Jackson]

No callers yet.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 tcl/osstestlib.tcl |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tcl/osstestlib.tcl b/tcl/osstestlib.tcl
index 1531c87..a0413c4 100644
--- a/tcl/osstestlib.tcl
+++ b/tcl/osstestlib.tcl
@@ -67,3 +67,10 @@ proc lremove {listvar item} {
     if {$ix<0} return
     set list [lreplace $list $ix $ix]
 }
+
+proc lshift {listvar} {
+    upvar 1 $listvar list
+    set head [lindex $list 0]
+    set list [lrange $list 1 end]
+    return $head
+}
-- 
1.7.10.4

Re: [OSSTEST PATCH 04/10] tcl/osstestlib.tcl: Provide lshift

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> No callers yet.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> ---
>  tcl/osstestlib.tcl |    7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/tcl/osstestlib.tcl b/tcl/osstestlib.tcl
> index 1531c87..a0413c4 100644
> --- a/tcl/osstestlib.tcl
> +++ b/tcl/osstestlib.tcl
> @@ -67,3 +67,10 @@ proc lremove {listvar item} {
>      if {$ix<0} return
>      set list [lreplace $list $ix $ix]
>  }
> +
> +proc lshift {listvar} {
> +    upvar 1 $listvar list
> +    set head [lindex $list 0]
> +    set list [lrange $list 1 end]

You calculate but then throw away list here?

Perhaps I need to read up on tcl scoping rules ;-)

Anyway, with or without this last quoted line, whichever you intend:
Acked-by: Ian Campbell <ian.campbell@citirx.com>

Re: [OSSTEST PATCH 04/10] tcl/osstestlib.tcl: Provide lshift

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 04/10] tcl/osstestlib.tcl: Provide lshift"):
> On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> > +proc lshift {listvar} {
> > +    upvar 1 $listvar list
> > +    set head [lindex $list 0]
> > +    set list [lrange $list 1 end]
> 
> You calculate but then throw away list here?

No.

> Perhaps I need to read up on tcl scoping rules ;-)

Yes :-).

"upvar" is a magic scoping tool.  "upvar $listvar list" means "take
the variable which in my callers scope has the name contained in the
variable which in my variable `listvar', and bring it into my own
scope as a variable called `list'".  The caller is expected to pass
the name of its variable as the actual parameter.

The effect is that someone who host
   set l {1 2 3}
   set f [lshift l]
finds that afterwards l contains `2 3' and f contains `1'.

> Anyway, with or without this last quoted line, whichever you intend:
> Acked-by: Ian Campbell <ian.campbell@citirx.com>

Thanks,
Ian.

[OSSTEST PATCH 05/10] sg-run-job: testid generation: Move spawn-step-begin

[Ian Jackson]

Run spawn-step-begin until after the loop over $args, and after the
computation of the basic deftestid.

No functional change: nothing in that loop looks at stepno.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-run-job |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 2cf810a..68d6c65 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -115,8 +115,6 @@ proc spawn-ts {iffail testid ts args} {
 
     if {![string compare . $iffail]} { set iffail fail }
 
-    jobdb::spawn-step-begin $flight $jobinfo(job) $ts stepno
-
     set real_args {}
     set adding 1
     set host_testid_suffix {}
@@ -132,6 +130,8 @@ proc spawn-ts {iffail testid ts args} {
     regsub {^ts-} $ts {} deftestid
     append deftestid /@
 
+    jobdb::spawn-step-begin $flight $jobinfo(job) $ts stepno
+
     if {[string match =* $testid]} {
         set testid "$deftestid[string range $testid 1 end]"
     } elseif {![string compare $testid *]} {
-- 
1.7.10.4

Re: [OSSTEST PATCH 05/10] sg-run-job: testid generation: Move spawn-step-begin

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> Run spawn-step-begin until after the loop over $args, and after the
> computation of the basic deftestid.
> 
> No functional change: nothing in that loop looks at stepno.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

TBH any Ack I gave here (and to the following TCL patches) would be
pretty cursory. I agree with what you seem to be trying to do even if
I'm not really in a position to confirm it's what you are actually
achieving. I think you can be trusted, and you are running a test
flight, and osstest has a push gate of its own. IOW I think you should
go ahead on patches 5..8 without an explicit ack...

Ian.

Re: [OSSTEST PATCH 05/10] sg-run-job: testid generation: Move spawn-step-begin

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 05/10] sg-run-job: testid generation: Move spawn-step-begin"):
> TBH any Ack I gave here (and to the following TCL patches) would be
> pretty cursory. I agree with what you seem to be trying to do even if
> I'm not really in a position to confirm it's what you are actually
> achieving. I think you can be trusted, and you are running a test
> flight, and osstest has a push gate of its own. IOW I think you should
> go ahead on patches 5..8 without an explicit ack...

OK, thanks.

I need to compare the results of my test run to check none of the
testids have changed.  (The reason I've broken it up into such small
pieces is to try to make it less likely I would make that kind of
mistake.)

Ian.

[OSSTEST PATCH 06/10] sg-run-job: testid generation: Introduce $testid_args

[Ian Jackson]

Rather than accumulating into host_testid_suffix directly, accumulate
into a list testid_args first.

No functional change: all we do is defer the construction of
host_testid_suffix, which is not used until later in this function.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-run-job |    7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 68d6c65..5c192ab 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -117,19 +117,22 @@ proc spawn-ts {iffail testid ts args} {
 
     set real_args {}
     set adding 1
-    set host_testid_suffix {}
+    set testid_args {}
     foreach arg $args {
         if {![string compare + $arg]} {
             set adding [expr {!$adding}]
             continue
         }
         lappend real_args $arg
-        if {$adding} { append host_testid_suffix "/$arg" }
+        if {$adding} { lappend testid_args $arg }
     }
 
     regsub {^ts-} $ts {} deftestid
     append deftestid /@
 
+    set host_testid_suffix {}
+    foreach arg $testid_args { append host_testid_suffix "/$arg" }
+
     jobdb::spawn-step-begin $flight $jobinfo(job) $ts stepno
 
     if {[string match =* $testid]} {
-- 
1.7.10.4

[OSSTEST PATCH 07/10] sg-run-job: testid generation: Process ts more like rest of args

[Ian Jackson]

Remove ts as a separate parameter to spawn-ts.  The test script now
becomes the first entry in args.

We process it through the arg loop as before.  Currently there are no
calls where the first arg is `+' so the test script name ends up in
both real_args and testid_args.

We split it out of real_args into the ts variable with lshift.
We split it out of testid_args into the deftestid with lshift.

So afterwards in spawn-ts, all the variables (including real_args, ts,
deftestid and testid_args and hence host_testid_suffix) have the
values they would have had before.

Therefore there is no functional change for any existing calls.

However, because the first argument is not treated specially for the
`+' procesing loop, it is now possible to specify `+' as the first
entry in args to spawn-ts (ie where ts used to be) to arrange that the
deftestid (and hence, probably, the testid) is computed using later
arguments.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-run-job |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 5c192ab..070534d 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -103,12 +103,12 @@ proc run-ts {args} {
     if {![reap-ts $reap]} { error "test script failed" }
 }
 
-proc spawn-ts {iffail testid ts args} {
+proc spawn-ts {iffail testid args} {
     global flight c jobinfo reap_details env
 
     if {[file exists abort]} {
         jobdb::logputs stdout \
-            "aborting - not executing $flight.$jobinfo(job) $ts $args"
+            "aborting - not executing $flight.$jobinfo(job) $args"
         job-set-status $flight $jobinfo(job) aborted
         return {}
     }
@@ -127,7 +127,9 @@ proc spawn-ts {iffail testid ts args} {
         if {$adding} { lappend testid_args $arg }
     }
 
-    regsub {^ts-} $ts {} deftestid
+    set ts [lshift real_args]
+
+    regsub {^ts-} [lshift testid_args] {} deftestid
     append deftestid /@
 
     set host_testid_suffix {}
-- 
1.7.10.4

[OSSTEST PATCH 08/10] sg-run-job, etc.: Infrastructure for test script repetition

[Ian Jackson]

Provide:
 * ts-repeat-test, a script to run multiple other test scripts in a loop
 * repeat-ts, a proc in sg-run-job which invokes it

No callers yet.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-run-job     |    4 ++++
 ts-repeat-test |   66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)
 create mode 100755 ts-repeat-test

diff --git a/sg-run-job b/sg-run-job
index 070534d..0a49c93 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -103,6 +103,10 @@ proc run-ts {args} {
     if {![reap-ts $reap]} { error "test script failed" }
 }
 
+proc repeat-ts {reps testid args} {
+     eval [list run-ts . $testid + ts-repeat-test $reps +] $args
+}
+
 proc spawn-ts {iffail testid args} {
     global flight c jobinfo reap_details env
 
diff --git a/ts-repeat-test b/ts-repeat-test
new file mode 100755
index 0000000..e3b1426
--- /dev/null
+++ b/ts-repeat-test
@@ -0,0 +1,66 @@
+#!/usr/bin/perl -w
+#
+# usage:
+#   ./ts-repeat-test COUNT ARGSPECS...
+#   ./ts-repeat-test COUNT [-] ts-SCRIPT [ARGS...] [; ...]
+#
+# ts-SCRIPT  will be prefixed with ./ before execution
+#             (provided it actually starts with `ts-')
+# ;          separates multiple scripts to be run
+# -          before script name means to ignore errors
+# \          at the start of any ARGSPEC is removed (after the checks above)
+
+use strict;
+use Osstest::TestSupport;
+
+use Data::Dumper;
+
+tsreadconfig();
+
+my $reps = shift @ARGV;
+die unless @ARGV && $reps =~ m/^\d+$/;
+
+my @cmdis = ();
+my $cmdi = { };
+# $cmds[]{L} = qw(./ts-foo-bar arg arg...);
+# $cmds[]{IgnoreError} = undef or 1
+
+push @ARGV, ';';
+
+foreach (@ARGV) {
+    if ($_ eq ';') {
+	if (%$cmdi) {
+	    push @cmdis, $cmdi;
+	    $cmdi = { };
+	}
+    } else {
+	if (!$cmdi->{L}) {
+	    if ($_ eq '-') {
+		$cmdi->{IgnoreError} = 1;
+		next;
+	    }
+	    s#^(?=ts-)#./#;
+	}
+	s#^\\##;
+	push @{ $cmdi->{L} }, $_;
+    }
+}
+
+my $dumper = new Data::Dumper [\@cmdis], [qw(*cmdis)];
+$dumper->Indent(0);
+print $dumper->Dump,"\n";
+
+foreach my $rep (1..$reps) {
+    logm("========== rep $rep ==========");
+    foreach my $cmdi (@cmdis) {
+	my $l = $cmdi->{L};
+	logm("---------- rep $rep @$l ----------");
+	my $r = system @$l;
+	if ($r) {
+	    my $m = "$l->[0]: ".($r==-1 ? "$!" : "status $?")."\n";
+	    if ($cmdi->{IgnoreError}) { warn $m; } else { die $m; }
+	}
+    }
+}
+
+logm("========== did $reps ==========");
-- 
1.7.10.4

[OSSTEST PATCH 09/10] rump kernel tests: Cancel unneeded edits of guest config

[Ian Jackson]

If the guest config is already set up to preserve, cancel the edit.
We are going to repeat this test, and this avoids creating many
identical copies of the same file in the log output.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-rumpuserxen-demo-xenstorels |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/ts-rumpuserxen-demo-xenstorels b/ts-rumpuserxen-demo-xenstorels
index ed46843..a54fbf4 100755
--- a/ts-rumpuserxen-demo-xenstorels
+++ b/ts-rumpuserxen-demo-xenstorels
@@ -32,7 +32,11 @@ our $gn = $gho->{Guest};
 sub arrangepreserve () {
     target_editfile_root($ho,$r{"$gho->{Guest}_cfgpath"}, sub {
 	while (<EI>) {
-	    next if m/^\s*on_poweroff\s*=/;
+	    if (m/^\s*on_poweroff\s*=/) {
+		target_editfile_cancel("already configured to preserve")
+		    if m/\bpreserve\b/;
+		next;
+	    }
 	    print EO or die $!;
 	}
 	print EO "\n","on_poweroff='preserve'\n" or die $!;
-- 
1.7.10.4

Re: [OSSTEST PATCH 09/10] rump kernel tests: Cancel unneeded edits of guest config

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> If the guest config is already set up to preserve, cancel the edit.
> We are going to repeat this test, and this avoids creating many
> identical copies of the same file in the log output.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

[OSSTEST PATCH 10/10] rump kernel tests: Repeat the xenstorels test 50 times

[Ian Jackson]

Add a new step which uses repeat-ts to run
ts-rumpuserxen-demo-xenstorels many times.

We have to run ts-guest-destroy-hard first each time, to destroy the
guest which might exist at each previous step.  To help with ad-hoc
debugging runs, we specify `-' on the destroy, so that if the destroy
fails (probably because the guest doesn't exist) we don't mind.

Strategically placed `+'s in the repeat-ts command line arrange that
the testid ends up being
   rumpuserxen-demo-xenstorels/xenstorels.repeat

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-run-job |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 0a49c93..ebbf53f 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -327,6 +327,9 @@ proc run-job/test-rumpuserxen {} {
     set g xenstorels
     run-ts . =   ts-rumpuserxen-demo-setup      + host + $g
     run-ts . =   ts-rumpuserxen-demo-xenstorels + host + $g
+    repeat-ts 50 =.repeat \
+             + - ts-guest-destroy-hard            host   $g \; + \
+                 ts-rumpuserxen-demo-xenstorels + host + $g
     run-ts . =   ts-guest-destroy-hard          + host + $g
 }
 
-- 
1.7.10.4

Re: [OSSTEST PATCH 10/10] rump kernel tests: Repeat the xenstorels test 50 times

[Ian Campbell]

On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> Add a new step which uses repeat-ts to run
> ts-rumpuserxen-demo-xenstorels many times.
> 
> We have to run ts-guest-destroy-hard first each time, to destroy the
> guest which might exist at each previous step.  To help with ad-hoc
> debugging runs, we specify `-' on the destroy, so that if the destroy
> fails (probably because the guest doesn't exist) we don't mind.
> 
> Strategically placed `+'s in the repeat-ts command line arrange that
> the testid ends up being
>    rumpuserxen-demo-xenstorels/xenstorels.repeat
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbelL@citrix.com>

> ---
>  sg-run-job |    3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/sg-run-job b/sg-run-job
> index 0a49c93..ebbf53f 100755
> --- a/sg-run-job
> +++ b/sg-run-job
> @@ -327,6 +327,9 @@ proc run-job/test-rumpuserxen {} {
>      set g xenstorels
>      run-ts . =   ts-rumpuserxen-demo-setup      + host + $g
>      run-ts . =   ts-rumpuserxen-demo-xenstorels + host + $g
> +    repeat-ts 50 =.repeat \
> +             + - ts-guest-destroy-hard            host   $g \; + \
> +                 ts-rumpuserxen-demo-xenstorels + host + $g
>      run-ts . =   ts-guest-destroy-hard          + host + $g
>  }
>  

[OSSTEST PATCH v2 10/10] rump kernel tests: Repeat the xenstorels test 50 times

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 10/10] rump kernel tests: Repeat the xenstorels test 50 times"):
> On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> > Add a new step which uses repeat-ts to run
> > ts-rumpuserxen-demo-xenstorels many times.
> 
> Acked-by: Ian Campbell <ian.campbelL@citrix.com>

Thanks.

After reviewing the output of my full-flight adhoc test, I decided
that the order of steps ought to be different, so here is a v2 of this
patch.

(The rest of the series, including the testid fiddling, worked
correctly.)

Thanks,
Ian.

>From f9b70e699a4175ff871f165cbf70ff07847c3abd Mon Sep 17 00:00:00 2001
From: Ian Jackson <ian.jackson@eu.citrix.com>
Date: Fri, 6 Feb 2015 17:09:40 +0000
Subject: [OSSTEST PATCH] rump kernel tests: Repeat the xenstorels test 50
 times

Add a new step which uses repeat-ts to run
ts-rumpuserxen-demo-xenstorels many times.

We have to run ts-guest-destroy-hard after each time, to destroy the
guest which the demo script leaves lying about.

Strategically placed `+'s in the repeat-ts command line arrange that
the testid ends up being
   rumpuserxen-demo-xenstorels/xenstorels.repeat

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Run the test after, rather than before, the explicit
     ts-guest-destroy-hard.  That will avoid blocking the single
     destroy test if the repeat fails.

    No longer specify to tolerate failures of the post-run-demo
     destroy, as if the test passes so must the destroy.  Now by-hand
     testing may need a different ts-repeat-test rune, but in practice
     by-hand testing will probably involve a shell loop or something
     anyway.
---
 sg-run-job |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 0a49c93..94d091b 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -328,6 +328,9 @@ proc run-job/test-rumpuserxen {} {
     run-ts . =   ts-rumpuserxen-demo-setup      + host + $g
     run-ts . =   ts-rumpuserxen-demo-xenstorels + host + $g
     run-ts . =   ts-guest-destroy-hard          + host + $g
+    repeat-ts 50 =.repeat \
+                 ts-rumpuserxen-demo-xenstorels + host + $g     \; \
+             +   ts-guest-destroy-hard            host   $g   +
 }
 
 #---------- builds ----------
-- 
1.7.10.4

Re: [OSSTEST PATCH v2 10/10] rump kernel tests: Repeat the xenstorels test 50 times

[Ian Campbell]

On Wed, 2015-02-11 at 14:37 +0000, Ian Jackson wrote:
> Ian Campbell writes ("Re: [OSSTEST PATCH 10/10] rump kernel tests: Repeat the xenstorels test 50 times"):
> > On Fri, 2015-02-06 at 19:17 +0000, Ian Jackson wrote:
> > > Add a new step which uses repeat-ts to run
> > > ts-rumpuserxen-demo-xenstorels many times.
> > 
> > Acked-by: Ian Campbell <ian.campbelL@citrix.com>
> 
> Thanks.
> 
> After reviewing the output of my full-flight adhoc test, I decided
> that the order of steps ought to be different, so here is a v2 of this
> patch.

Still looks fine. Ack.

Re: [PATCH 00/10] Repeat rump kernel xenstorels test

[Ian Jackson]

Ian Jackson writes ("[PATCH 00/10] Repeat rump kernel xenstorels test"):
> 04..07 need a thorough full-flight test which I have not yet done.
> But this sql rune suggests that nothing is horribly broken at least
> for the rump test job:
> 
>   select * from (select testid,flight,stepno,step,status,started,finished from steps where flight=33866 and job='test-amd64-i386-rumpuserxen-i386') a full outer join (select * from steps where flight=34273 and job='test-amd64-i386-rumpuserxen-i386') b using (testid) order by b.stepno, a.stepno;

For my own records: said full-flight test is now running as 34137.

Ian.