[OSSTEST PATCH [RFC] 0/3] Series short description

[OSSTEST PATCH [RFC] 0/3] Series short description

[Dario Faggioli]

OSSTest: allow for handling multiple guests at the same time

via the ts-debian-install, ts-debian-fixup and
ts-guest-{start,stop,destroy} test scripts.

The idea is to enable something like that:

 $ OSSTEST_JOB=test-amd64-amd64-xl
 $ export OSSTEST_JOB
 $ ./ts-debian-install host=tg03 debian1 debian2 debian3
 $ ./ts-debian-fixup host=tg03 debian1 debian2 debian3
 $ ./ts-guest-start host=tg03 debian1 debian2 debian3
 $ ./ts-guest-stop host=tg03 debian1 debian2
 $ ./ts-guest-destroy host=tg03 debian3

In fact, when wanting to use OSSTest for some fairly complex test
or benchamrk that involves more than one or two VMs, I think
it could be convinient to install, start, stop, etc., them like
this, rather than having to call each script for all the domains.

It should also save some time, when it comes, for example, to
starting domains, as the operation is first requsted for all of
them, and only after that we start checking wether they're up
(the idea being that when we start checking, after having started
the last one, the first one have probably finished booting).

It's going to be mostly useful in standalone mode, but
I guess it could allow to write more compact recipes too.

I tested this both as reported above and by issueing a full
`./sg-run-job test-amd64-amd64-xl'.

I can go ahead and do the same for other ts-* scripts but:
1) I don't think is that much relevant/useful to do something
   like this in, for instance, ts-guest-migrate or
   ts-guest-saverestore
2) given my limited perl skills, I though I better ask for
   some early feedback. :-)

Any comments are welcome.

Regards,
Dario

---

Dario Faggioli (3):
      ts-debian-install: support for installing multiple guests
      ts-debian-fixup: support multiple guests
      ts-guest-start, -stop, -destroy: support multiple guests


 ts-debian-fixup   |   30 ++++++++++++++++++---------
 ts-debian-install |   59 +++++++++++++++++++++++++++++++----------------------
 ts-guest-destroy  |   28 +++++++++++++++++++------
 ts-guest-start    |   35 ++++++++++++++++++++++---------
 ts-guest-stop     |   30 ++++++++++++++++++++-------
 5 files changed, 123 insertions(+), 59 deletions(-)

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[OSSTEST PATCH [RFC] 1/3] ts-debian-install: support for installing multiple guests

[Dario Faggioli]

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2

This assumes that, if an host is to be explicitly specified,
that happens via the first argument and in the form 'host=somehost',
as it typically is the case for standalone mode.

If no first argument in 'host=' form is found, we assume no explicit
host designation at all, as it typicall is the case for non-standalone
mode, and we consider all the parameters a list of guest names.
We do this also if the first parametr is just 'host', as this seem to
be another way to ask to just use the host from the runvars (typical
in non-standalone, but functional in both modes).

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 ts-debian-install |   59 +++++++++++++++++++++++++++++++----------------------
 1 file changed, 35 insertions(+), 24 deletions(-)

diff --git a/ts-debian-install b/ts-debian-install
index 519edc9..a9a89dd 100755
--- a/ts-debian-install
+++ b/ts-debian-install
@@ -22,9 +22,14 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($whhost,$gn) = @ARGV;
-$whhost ||= 'host';
-$gn ||= 'debian';
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+
+our (@guests) = @ARGV;
+$guests[0] = 'debian' unless defined(@guests);
 
 our $ho= selecthost($whhost);
 
@@ -32,27 +37,30 @@ our $ram_mb=    512;
 our $swap_mb=  1000;
 our $disk_mb= 10000;
 
-our $guesthost= "$gn.guest.osstest";
-our $gho;
+our $guesthost;
+our %gho;
+
+sub prep ($) {
+    my ($gn) = @_;
 
-sub prep () {
     target_install_packages_norec($ho, qw(lvm2 xen-tools));
 
-    $gho= prepareguest($ho, $gn, $guesthost, 22,
-                       $swap_mb + $disk_mb + 2,
-                       40);
-    target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+    $gho{$gn}= prepareguest($ho, $gn, $guesthost, 22,
+                            $swap_mb + $disk_mb + 2,
+                            40);
+    target_cmd_root($ho, "umount $gho{$gn}->{Lvdev} ||:");
 }
 
-sub ginstall () {
-    my $arch= $r{"$gho->{Guest}_arch"};
+sub ginstall ($) {
+    my ($gn) = @_;
+    my $arch= $r{"$gho{$gn}->{Guest}_arch"};
     my $archarg= defined($arch) ? "--arch $arch" : '';
-    my $gsuite= guest_var($gho,'suite',$c{GuestDebianSuite});
+    my $gsuite= guest_var($gho{$gn},'suite',$c{GuestDebianSuite});
 
-    my $kernpath = guest_var($gho,'kernel_path',$r{xen_kernel_path});
-    my $initrd = guest_var($gho,'initrd_path',$r{xen_initrd_path});
+    my $kernpath = guest_var($gho{$gn},'kernel_path',$r{xen_kernel_path});
+    my $initrd = guest_var($gho{$gn},'initrd_path',$r{xen_initrd_path});
     if (!$kernpath) {
-	my $kernver= guest_var($gho,'kernel_ver',$r{xen_kernel_ver});
+	my $kernver= guest_var($gho{$gn},'kernel_ver',$r{xen_kernel_ver});
 	$kernver ||= target_cmd_output($ho, 'uname -r');
 	$kernpath = "/boot/vmlinuz-$kernver";
 	$initrd ||= "/boot/initrd.img-$kernver";
@@ -65,21 +73,24 @@ sub ginstall () {
     
     target_cmd_root($ho, <<END, 2000);
         xen-create-image \\
-            --dhcp --mac $gho->{Ether} \\
+            --dhcp --mac $gho{$gn}->{Ether} \\
             --memory ${ram_mb}Mb --swap ${swap_mb}Mb \\
             --dist $gsuite \\
             --mirror http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} \\
-            --hostname $gho->{Name} \\
-            --lvm $gho->{Vg} --force \\
+            --hostname $gho{$gn}->{Name} \\
+            --lvm $gho{$gn}->{Vg} --force \\
             --kernel $kernpath \\
             --genpass 0 --password xenroot \\
             $initrd_opt \\
             $archarg
 END
-    my $cfg_xend= "/etc/xen/$gho->{Name}.cfg";
-    store_runvar("$gho->{Guest}_cfgpath", $cfg_xend);
-    store_runvar("$gho->{Guest}_swap_lv", "$gho->{Name}-swap");
+    my $cfg_xend= "/etc/xen/$gho{$gn}->{Name}.cfg";
+    store_runvar("$gho{$gn}->{Guest}_cfgpath", $cfg_xend);
+    store_runvar("$gho{$gn}->{Guest}_swap_lv", "$gho{$gn}->{Name}-swap");
 }
 
-prep();
-ginstall();
+foreach my $g (@guests) {
+    $guesthost = "$g.guest.osstest";
+    prep($g);
+    ginstall($g);
+}

[OSSTEST PATCH [RFC] 2/3] ts-debian-fixup: support multiple guests

[Dario Faggioli]

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2
$ ./ts-debian-fixup host=tg03 debian1 debian2

This again assumes that either the host is explicitly specified
via a 'host=somehost' first argument, or the argument are all
guest names.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 ts-debian-fixup |   30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index f001418..e281ad7 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -22,7 +22,14 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,$gho);
 
 our ($cfgfile,$cfgstash,$cfg);
 
@@ -163,12 +170,15 @@ sub writecfg () {
     target_putfile_root($ho,10, $cfgstash, $cfgfile);
 }
 
-savecfg();
-ether();
-target_kernkind_check($gho);
-access();
-console();
-filesystems();
-otherfixupcfg();
-writecfg();
-unmount();
+foreach my $g (@guests) {
+    ($ho,$gho) = ts_get_host_guest($whhost, $g);
+    savecfg();
+    ether();
+    target_kernkind_check($gho);
+    access();
+    console();
+    filesystems();
+    otherfixupcfg();
+    writecfg();
+    unmount();
+}

[OSSTEST PATCH [RFC] 3/3] ts-guest-start, -stop, -destroy: support multiple guests

[Dario Faggioli]

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2 debian3
$ ./ts-debian-fixup host=tg03 debian1 debian2 debian3

$ ./ts-guest-start host=tg03 debian1 debian2 debian3
$ ./ts-guest-stop host=tg03 debian1 debian2
$ ./ts-guest-destroy host=tg03 debian3

This again assumes that either the host is explicitly specified
via a 'host=somehost' first argument, or the arguments are all
guest names.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 ts-guest-destroy |   28 +++++++++++++++++++++-------
 ts-guest-start   |   35 +++++++++++++++++++++++++----------
 ts-guest-stop    |   30 ++++++++++++++++++++++--------
 3 files changed, 68 insertions(+), 25 deletions(-)

diff --git a/ts-guest-destroy b/ts-guest-destroy
index 738650a..794f38d 100755
--- a/ts-guest-destroy
+++ b/ts-guest-destroy
@@ -22,13 +22,27 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub destroy () {
-    guest_destroy($ho, $gho);
-    guest_checkrunning($ho, $gho) and die $gho->{Name};
+sub destroy ($) {
+    my ($gn) = @_;
+    guest_destroy($ho, $gho{$gn});
+    guest_checkrunning($ho, $gho{$gn}) and die $gho{$gn}->{Name};
 }
 
-guest_await_dhcp_tcp($gho, 5);
-destroy();
-target_ping_check_down($gho);
+# Let's first destroy all domains and then check they're gone
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    guest_await_dhcp_tcp($gho{$g}, 5);
+    destroy($g);
+}
+foreach my $g (@guests) {
+    target_ping_check_down($gho{$g});
+}
diff --git a/ts-guest-start b/ts-guest-start
index 057afe6..410f5fd 100755
--- a/ts-guest-start
+++ b/ts-guest-start
@@ -22,20 +22,35 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub start () {
-    guest_umount_lv($ho, $gho);
+sub start ($) {
+    my ($gn) = @_;
+    guest_umount_lv($ho, $gho{$gn});
     my $cmd= toolstack()->{Command}." create ".
-        $r{ $gho->{Guest}.'_'. toolstack()->{CfgPathVar} };
+        $r{ $gho{$gn}->{Guest}.'_'. toolstack()->{CfgPathVar} };
     target_cmd_root($ho, $cmd, 30);
 }
 
-sub checkstart () {
-    guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+sub checkstart ($) {
+    my ($gn) = @_;
+    guest_checkrunning($ho, $gho{$gn}) or die "$gho{$gn}->{Name} not running";
 }
 
-start();
-checkstart();
-guest_await($gho, target_var($gho,'boot_timeout'));
-guest_check_up($gho);
+# Let's first ask all domains to start and then check they're all actually up
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    start($g);
+    checkstart($g);
+}
+foreach my $g (@guests) {
+    guest_await($gho{$g}, target_var($gho{$g},'boot_timeout'));
+    guest_check_up($gho{$g});
+}
diff --git a/ts-guest-stop b/ts-guest-stop
index cc7db4c..a4fddf4 100755
--- a/ts-guest-stop
+++ b/ts-guest-stop
@@ -22,17 +22,31 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub stop () {
-    guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+sub stop ($) {
+    my ($gn) = @_;
+    guest_checkrunning($ho, $gho{$gn}) or die "$gho{$gn}->{Name} not running";
     target_cmd_root($ho,
 		    toolstack()->{Command}
 		    ." shutdown -w "
-		    .$gho->{Name}, 200);
-    guest_checkrunning($ho, $gho) and die $gho->{Name};
+		    .$gho{$gn}->{Name}, 200);
+    guest_checkrunning($ho, $gho{$gn}) and die $gho{$gn}->{Name};
 }
 
-guest_await_dhcp_tcp($gho, 5);
-stop();
-target_ping_check_down($gho);
+# Let's first ask all domains to stop and then check they're down
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    guest_await_dhcp_tcp($gho{$g}, 5);
+    stop($g);
+}
+foreach my $g (@guests) {
+    target_ping_check_down($gho{$g});
+}

Re: [OSSTEST PATCH [RFC] 0/3] Series short description

[Ian Jackson]

Dario Faggioli writes ("[OSSTEST PATCH [RFC] 0/3] Series short description"):
> OSSTest: allow for handling multiple guests at the same time
> 
> via the ts-debian-install, ts-debian-fixup and
> ts-guest-{start,stop,destroy} test scripts.
> 
> The idea is to enable something like that:
> 
>  $ OSSTEST_JOB=test-amd64-amd64-xl
>  $ export OSSTEST_JOB
>  $ ./ts-debian-install host=tg03 debian1 debian2 debian3

I don't much like this approach, I'm afraid.

* It's more serialised that necessary
* The iteration is distributed over bunches of scripts
* You can't set up heterogeonous systems
* It's nearly-duplicating functionality from sg-run-job
* The logfiles from the different guest operations are mixed up
* The multiple guests are conflated into a single cell in the
  test results matrix

sg-run-job already knows how to parallelise operations on multiple
hosts.  It wouldn't be too hard to make it capable of doing these kind
of stepwise operations on multiple guests.

> I can go ahead and do the same for other ts-* scripts but:

Thanks for asking!

> 2) given my limited perl skills, I though I better ask for
>    some early feedback. :-)

How's your Tcl ? :-)

Ian.

Re: [OSSTEST PATCH [RFC] 0/3] Series short description

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2097 bytes --]

On gio, 2013-12-05 at 15:25 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("[OSSTEST PATCH [RFC] 0/3] Series short description"):
> > OSSTest: allow for handling multiple guests at the same time
> > 
> > via the ts-debian-install, ts-debian-fixup and
> > ts-guest-{start,stop,destroy} test scripts.
> > 
> > The idea is to enable something like that:
> > 
> >  $ OSSTEST_JOB=test-amd64-amd64-xl
> >  $ export OSSTEST_JOB
> >  $ ./ts-debian-install host=tg03 debian1 debian2 debian3
> 
> I don't much like this approach, I'm afraid.
> 
Fair enough, I'm happy I sent this out pretty early then! :-)

Just one thing.

> * It's more serialised that necessary
> * The iteration is distributed over bunches of scripts
> * You can't set up heterogeonous systems
What do you mean with this?

> * It's nearly-duplicating functionality from sg-run-job
> * The logfiles from the different guest operations are mixed up
> * The multiple guests are conflated into a single cell in the
>   test results matrix
> 
> sg-run-job already knows how to parallelise operations on multiple
> hosts.  It wouldn't be too hard to make it capable of doing these kind
> of stepwise operations on multiple guests.
> 
Ok. So, given  that ...

> > 2) given my limited perl skills, I though I better ask for
> >    some early feedback. :-)
> 
> How's your Tcl ? :-)
> 
... I knew a language called like that existed, but that is mostly it,
and while I try to understand that fancy syntax, any pointer to what I
should look at to understand how it "already knows how to parallelise
operations on multiple hosts" and hence how I'd be able to "make it
capable of doing these kind of stepwise operations on multiple guests"?

I found a 'proc per-host-ts'. Is that a good starting point?

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTEST PATCH [RFC] 0/3] Series short description

[Ian Jackson]

Dario Faggioli writes ("Re: [Xen-devel] [OSSTEST PATCH [RFC] 0/3] Series short description"):
> On gio, 2013-12-05 at 15:25 +0000, Ian Jackson wrote:
> > * You can't set up heterogeonous systems
>
> What do you mean with this?

I mean, you can't easily set up a mixed setup with a number of
different guest OS's, because each script only makes a particular kind
of guest.  If you wanted to do that you'd have to do it in sg-run-job -
and if you were going to write that then it could take care of the
whole problem.

> > > 2) given my limited perl skills, I though I better ask for
> > >    some early feedback. :-)
> > 
> > How's your Tcl ? :-)
> > 
> ... I knew a language called like that existed, but that is mostly it,
> and while I try to understand that fancy syntax, any pointer to what I
> should look at to understand how it "already knows how to parallelise
> operations on multiple hosts" and hence how I'd be able to "make it
> capable of doing these kind of stepwise operations on multiple guests"?

Tcl itself is just a programming language.  It happens to have very
good support for writing event-continuation-passing-style programs,
with little code.  (I also really like it.)

In sg-run-job the core of this is done with the spawn-ts and reap-ts
procedures.  Mostly in sg-run-job, these are combined into run-ts,
which spawns and then immediately reaps.

But there is an existing proc per-host-ts which interprets its
arguments as a specification of a set of hosts, and spawns the script
on each host and then reaps them again, so that the same script runs
in parallel on all the hosts.

This is used in the main run-job procedure for host installation, and
actually takes parallel effect in the paired migration tests.  But it
could be refactored to be slightly more general and then we could have
a similar per-guest-ts procedure.

The tcl.tk website has comprehensive language documentation of course,
but I would be happy to help/review/etc.  Do you like learning new
languages ... ?

Ian.

Re: [OSSTEST PATCH [RFC] 0/3] Series short description

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1973 bytes --]

On gio, 2013-12-05 at 17:34 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("Re: [Xen-devel] [OSSTEST PATCH [RFC] 0/3] Series short description"):
> > ... I knew a language called like that existed, but that is mostly it,
> > and while I try to understand that fancy syntax, any pointer to what I
> > should look at to understand how it "already knows how to parallelise
> > operations on multiple hosts" and hence how I'd be able to "make it
> > capable of doing these kind of stepwise operations on multiple guests"?
> 
> Tcl itself is just a programming language.  It happens to have very
> good support for writing event-continuation-passing-style programs,
> with little code.
>
Yes, that I knew. I've seen it being used for event based simulation,
but I never got to that.

>   (I also really like it.)
> 
That I suspected. :-P

> In sg-run-job the core of this is done with the spawn-ts and reap-ts
> procedures.  Mostly in sg-run-job, these are combined into run-ts,
> which spawns and then immediately reaps.
> 
Ok.

> But there is an existing proc per-host-ts which interprets its
> arguments as a specification of a set of hosts, and spawns the script
> on each host and then reaps them again, so that the same script runs
> in parallel on all the hosts.
> 
Right, as I said, I felt like that could be the spot. I'll look into it
and try to understand what goes on there (and the how to extend that as
per my needs).

> The tcl.tk website has comprehensive language documentation of course,
> but I would be happy to help/review/etc.  Do you like learning new
> languages ... ?
> 
I certainly do! :-)

Thanks for the explanation and the pointers.
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel