[PATCH OSSTEST v2 0/7] Fixes for distros-debian-* flights on armhf

[PATCH OSSTEST v2 0/7] Fixes for distros-debian-* flights on armhf

[Ian Campbell]

The armhf tests in distros-debian-* have always failed since they did not
shutdown at the end of installation. I've now investigated that and the fix
is patch #2 in this series.

The remainder of the series consists of various fixes, extensions to
existing quirks and workarounds, mainly to the use of the pv-menu-list
package on ARM when actually starting the guest (which was previously
blocked by the install failure).

In addition to this a xen patch ("libxl: use correct command line for arm
guests.") is also needed. That should be applied for 4.6 pretty soon
(probably right after I've sent this mail...)

This will fix the armhf tests in distros-debian-jessie flights (and    
later) but won't fix the armhf -raw/qcow/vhd tests in other flights    
since they also suffer from lack of Xen support in Wheezy's kernel.    
Once we switch osstest to using Jessie by default then together with    
this series those things ought to spring into life.

Ian.

[PATCH OSSTEST v2 1/7] ts-logs-capture: Collect /var/log/xen/bootloader.*.log

[Ian Campbell]

This is the pygrub debug log.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-logs-capture | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-logs-capture b/ts-logs-capture
index 0081372..b99b1db 100755
--- a/ts-logs-capture
+++ b/ts-logs-capture
@@ -143,6 +143,7 @@ sub fetch_logs_host () {
                   /var/log/xen/xend-debug.log*
                   /var/log/xen/xen-hotplug.log*
                   /var/log/xen/domain-builder-ng.log*
+                  /var/log/xen/bootloader.*.log
                   /var/log/xen/qemu-dm*
                   /var/log/xen/xl*.log
                   /var/log/xen/osstest*
-- 
2.1.4

[PATCH OSSTEST v2 2/7] ts-debian-di-install: Use exit/poweroff in preference to exit/always_halt

[Ian Campbell]

always_halt results in d-i calling "halt", which does not necessarily
poweroff the host (it seems to for x86/PV Xen guests, but does not for
ARM). Using exit/poweroff calls "poweroff" which is equivalent to
"halt -p", doing so results in ARM guests powering off as desired.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-debian-di-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 34b8e1e..612e0fa 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -210,7 +210,7 @@ END
     }
 
     my @cmdline = ();
-    push @cmdline, "debian-installer/exit/always_halt=true";
+    push @cmdline, "debian-installer/exit/poweroff=true";
     push @cmdline, "domain=$c{TestHostDomain}";
     push @cmdline, "console=hvc0";
     push @cmdline, di_installcmdline_core($gho, $ps_url);
-- 
2.1.4

[PATCH OSSTEST v2 3/7] ts-debian-di-install: Install pv-grub-menu in ARM guests, always.

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: s/pv-menu-lst/pv-grub-menu/
    Add note that pvgrub == grub1
    Ungarble a comment
---
 ts-debian-di-install | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 612e0fa..eef4681 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -190,9 +190,20 @@ END
 
 	$method_cfg = setup_netboot($tmpdir, $arch, $suite);
 
+	# We need the pv-grub-menu package:
+	# - On x86 when running pvgrub, since pvgrub only speaks grub1
+	#   menu.lst syntax and grub2 is now the only grub in
+	#   Debian.
+	# - On ARM, which uses pygrub, because grub2 is not installed
+	#   on ARM by default (except for, maybe, UEFI guests in the
+	#   future, but not today).
+	#
+	# NB: "pvgrub" is grub1/grub-legacy, not PV grub2.
+	my $pvmenulst = ($bl eq "pvgrub" || $arch =~ /arm/);
+
 	$ps_url = preseed_create_guest($gho, $arch, '',
 				       Suite=>$suite,
-				       PvMenuLst=>($bl eq "pvgrub"));
+				       PvMenuLst=>$pvmenulst);
 
 	$extra_disk = "";
     }
-- 
2.1.4

Re: [PATCH OSSTEST v2 3/7] ts-debian-di-install: Install pv-grub-menu in ARM guests, always.

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST v2 3/7] ts-debian-di-install: Install pv-grub-menu in ARM guests, always."):
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

[PATCH OSSTEST v2 4/7] ts-debian-di-install: Use the suite in the default hostname

[Ian Campbell]

By appending ".$suite" if the suite is in the runvars.

This is more useful in standalone mode than having everything be
"debian".

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Keep using gn but append the suite if given. Not sure what the
    best syntax for that ($x ? ".$x " : "") is.
---
 ts-debian-di-install | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index eef4681..9905691 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -66,7 +66,9 @@ our $ho= selecthost($whhost);
 our $ram_mb=    512;
 our $disk_mb= 10000;
 
-our $guesthost= "$gn.guest.osstest";
+our $guesthost= $gn.
+    ($r{"${gn}_suite"} ? ".".$r{"${gn}_suite"} : "").
+    ".guest.osstest";
 our $gho;
 
 sub prep () {
-- 
2.1.4

Re: [PATCH OSSTEST v2 4/7] ts-debian-di-install: Use the suite in the default hostname

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST v2 4/7] ts-debian-di-install: Use the suite in the default hostname"):
> By appending ".$suite" if the suite is in the runvars.
> 
> This is more useful in standalone mode than having everything be
> "debian".

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

> v2: Keep using gn but append the suite if given. Not sure what the
>     best syntax for that ($x ? ".$x " : "") is.

This will do.

Ian.

[PATCH OSSTEST v2 5/7] Debian: ARM: only apply no bootloader workaround if xopts{PvMenuLst}

[Ian Campbell]

This workaround is only necessary because of how pv-menu-list works,
so we should only apply both or neither of them.

This results in a long line and I'm about to add a second workaround
to this block, so switch to a regular if block instead of postfixing
on the one command. Move the comment inside that block in preparation
for other workarounds as well.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Add a \n after each &&
---
 Osstest/Debian.pm | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index f0bcf06..1c07108 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -915,15 +915,20 @@ d-i     grub-installer/bootdev          string /dev/xvda
 
 END
 
-    # Debian doesn't currently know what bootloader to install in a
-    # Xen guest on ARM. We install pv-grub-menu above which actually
-    # does what we need, but the installer doesn't treat that as a
-    # "bootloader".
     logm("\$arch is $arch, \$suite is $suite");
-    $preseed_file.= (<<END) if $arch =~ /^arm/ && $suite =~ /wheezy|jessie|sid/;
+    if ($xopts{PvMenuLst} &&
+	$arch =~ /^arm/ &&
+	$suite =~ /wheezy|jessie|sid/ ) {
+
+	# Debian doesn't currently know what bootloader to install in
+	# a Xen guest on ARM. We install pv-grub-menu above which
+	# actually does what we need, but the installer doesn't treat
+	# that as a "bootloader".
+	$preseed_file.= (<<END);
 d-i     nobootloader/confirmation_common boolean true
 
 END
+    }
 
     $preseed_file .= preseed_hook_cmds();
 
-- 
2.1.4

Re: [PATCH OSSTEST v2 5/7] Debian: ARM: only apply no bootloader workaround if xopts{PvMenuLst}

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST v2 5/7] Debian: ARM: only apply no bootloader workaround if xopts{PvMenuLst}"):
> This workaround is only necessary because of how pv-menu-list works,
                                                   ^^^^^^^^^^^^
pv-grub-menu.

Ian.

[PATCH OSSTEST v2 6/7] Debian: ARM has no bootloader (for Xen) even in Stretch.

[Ian Campbell]

Realistically this isn't going to change until we have either u-boot
or UEFI in an arm32 guest.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 1c07108..7fc39d5 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -918,7 +918,7 @@ END
     logm("\$arch is $arch, \$suite is $suite");
     if ($xopts{PvMenuLst} &&
 	$arch =~ /^arm/ &&
-	$suite =~ /wheezy|jessie|sid/ ) {
+	$suite =~ /wheezy|jessie|stretch|sid/ ) {
 
 	# Debian doesn't currently know what bootloader to install in
 	# a Xen guest on ARM. We install pv-grub-menu above which
-- 
2.1.4

[PATCH OSSTEST v2 7/7] Debian: Create /boot/boot -> . symlink on ARM when PvMenuLst enabled

[Ian Campbell]

This is under the same conditional as the nobootloader confirmation
one, since they effectively both stem from the lack of a boot loader
and the consequential use of the pv-grub-menu package.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Add the second bit of the commit message.
---
 Osstest/Debian.pm | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 7fc39d5..9f8f60b 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -926,7 +926,15 @@ END
 	# that as a "bootloader".
 	$preseed_file.= (<<END);
 d-i     nobootloader/confirmation_common boolean true
+END
 
+        # Debian Bug #771949 means that update-menu-list always
+        # generates a full absolute path to the kernel + initrd, while
+        # by default the partition layout on ARM has a separate /boot.
+        preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+ln -s . /target/boot/boot
 END
     }
 
-- 
2.1.4

Re: [PATCH OSSTEST v2 7/7] Debian: Create /boot/boot -> . symlink on ARM when PvMenuLst enabled

[Ian Campbell]

On Thu, 2015-08-13 at 17:52 +0100, Ian Campbell wrote:
> This is under the same conditional as the nobootloader confirmation
> one, since they effectively both stem from the lack of a boot loader
> and the consequential use of the pv-grub-menu package.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

I resent this before you acked v1 (after I explained the condition
where this was added).

Since all I did this time was add that explanation to the commit
message I will retain that Ack and have therefore pushed this fully
acked series to pretest.

Thanks,
Ian