[OSSTEST PATCH 00/35] ARM64 softiron support

[OSSTEST PATCH 00/35] ARM64 softiron support

[Ian Jackson]

I currently have this working well enough to run d-i and install the
host.  I'm running a commissioning flight on both softiron boxes
(laxton0, laxton1).

If this works well enough I will push this seris to osstest pretest
and, assuming that it passes, put laxton* into production.

Note that I have not yet booted the Xen hypervisor.  If the builds
work, this will be tested by my commissioning flight.  But I think it
is probably better to put this new test code, and new hardware, into
service, even if the actual hypervisor tests fail for some reason.  We
will then debug this in, and with the benefit of, the normal
infrastructure.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 01/35] Serial console: honour LinuxSerialConsole in bootloader setup

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5eecf92..52527bf 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -67,8 +67,9 @@ sub debian_boot_setup ($$$$$;$) {
 
     my $kopt;
     my $console= target_var($ho,'console');
+    $console ||= get_host_native_linux_console($ho);
     if (defined $console && length $console) {
-        $kopt= "console=$console";
+        $kopt= $console eq 'NONE' ? '' : "console=$console";
     } else {
         $kopt= "xencons=ttyS console=ttyS0,$c{Baud}n8";
     }
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 02/35] ts-host-install: Expect "pool" in ntp.conf, not only "server"

[Ian Jackson]

This is helpful for playing around with stretch, which seems to use
this option by default.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-host-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index ecf5f0b..99ba130 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -134,7 +134,7 @@ END
 	target_editfile_root($ho, '/etc/ntp.conf', sub {
 	    my $done= 0;
 	    while (<EI>) {
-		if (m/^server\s/) {
+		if (m/^server\b|^pool\b\s/) {
 		    if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie/) {
 			$_= $done ? "" : "server $ntpserver\n";
 		    } else {
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 03/35] ts-host-install: NTP not honoured bug remains

[Ian Jackson]

Debian #778564 remains open.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-host-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index 99ba130..770c51b 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -135,7 +135,7 @@ END
 	    my $done= 0;
 	    while (<EI>) {
 		if (m/^server\b|^pool\b\s/) {
-		    if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie/) {
+		    if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie|stretch/) {
 			$_= $done ? "" : "server $ntpserver\n";
 		    } else {
 			m/^server \Q$ntpserver\E\s/ or
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 04/35] mg-debian-installer-update: Break out default setting of src=

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-debian-installer-update | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index bb1ae62..00cc735 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -31,17 +31,17 @@ packages="$3"
 site=http://ftp.debian.org/debian/
 sbase=$site/dists/$suite
 
+src=$sbase/main/installer-$arch/current/images/netboot/
+
 case ${suite}_${arch} in
     lenny_armhf|squeeze_armhf|lenny_arm64|squeeze_arm64|wheezy_arm64)
         # No such thing.
         exit 0
         ;;
     *_i386|*_amd64)
-        src=$sbase/main/installer-$arch/current/images/netboot/
         files="debian-installer/$arch/initrd.gz:initrd.gz debian-installer/$arch/linux:linux xen/vmlinuz:vmlinuz-xen xen/initrd.gz:initrd.gz-xen"
         ;;
     *_arm64)
-        src=$sbase/main/installer-$arch/current/images/netboot/
         files="debian-installer/$arch/initrd.gz:initrd.gz \
                debian-installer/$arch/linux:linux"
         dtbs=$sbase/main/installer-$arch/current/images/device-tree/
@@ -53,7 +53,6 @@ case ${suite}_${arch} in
         files='initrd.gz' # no kernel -- needs a newer kernel
         ;;
     *_armhf)
-        src=$sbase/main/installer-$arch/current/images/netboot
         files='vmlinuz:linux initrd.gz'
         dtbs=$sbase/main/installer-$arch/current/images/device-tree/
         ;;
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 05/35] mg-debian-installer-update: Print working directory to stderr

[Ian Jackson]

No other change

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-debian-installer-update | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 00cc735..d0aabb5 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -70,6 +70,7 @@ esac
 cd $dstroot
 mkdir -p $dst
 cd $dst
+echo >&2 "working in $dstroot/$dst"
 
 for f in $files; do
         echo >&2 "fetching $f"
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 06/35] mg-debian-installer-update: Log our version into our output

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-debian-installer-update | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index d0aabb5..444f80b 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -62,6 +62,7 @@ tftpdiversion=`getconfig_TftpDiVersion_suite $suite`
 dstroot=`getconfig TftpPath`/`getconfig TftpDiBase`/
 date=`date +%Y-%m-%d`
 dst=$arch/$date-$suite
+harness_rev=$(perl -e 'use Osstest; print get_harness_rev();')
 
 case "`getconfig HttpProxy`" in
 ?*)	export http_proxy=`getconfig HttpProxy` ;;
@@ -72,6 +73,9 @@ mkdir -p $dst
 cd $dst
 echo >&2 "working in $dstroot/$dst"
 
+now=$(date -R)
+echo >>osstest-touched.log "$now: $harness_rev"
+
 for f in $files; do
         echo >&2 "fetching $f"
         s=${f/:*} ; d=${f/*:}
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 07/35] mg-debian-installer-update: backports: Provide pps and ptp

[Ian Jackson]

Our ARM64 softiron boxes need these to find their network and disks.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-debian-installer-update | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 444f80b..12586f8 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -154,6 +154,8 @@ if [ x$bpok != x ]; then
                          -path \*/kernel/drivers/i2c/busses/\* -o \
                          -path \*/kernel/drivers/md/\* -o \
                          -path \*/kernel/drivers/net/\* -o \
+                         -path \*/kernel/drivers/pps/\* -o \
+                         -path \*/kernel/drivers/ptp/\* -o \
                          -path \*/kernel/drivers/regulator/\* -o \
                          -path \*/kernel/drivers/scsi/\* -o \
                          -path \*/kernel/drivers/usb/dwc3/\* -o \
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 08/35] mg-debian-installer-update: backports: Add modules for EFI

[Ian Jackson]

UEFI uses vfat for the boot partiton.
This also means we need some charset modules.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-debian-installer-update | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 12586f8..079d3e8 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -146,6 +146,8 @@ if [ x$bpok != x ]; then
                          -path \*/kernel/fs/mbcache.ko -o \
                          -path \*/kernel/fs/ext\* -o \
                          -path \*/kernel/fs/jbd\* -o \
+                         -path \*/kernel/fs/fat\* -o \
+                         -path \*/kernel/fs/nls/\* -o \
                          -path \*/kernel/drivers/clk/\* -o \
                          -path \*/kernel/drivers/mmc/\* -o \
                          -path \*/kernel/drivers/phy/\* -o \
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 09/35] mg-debian-installer-update: backports: Provide crypto

[Ian Jackson]

Filesystem drivers now refer to this.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-debian-installer-update | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 079d3e8..82a1804 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -146,6 +146,7 @@ if [ x$bpok != x ]; then
                          -path \*/kernel/fs/mbcache.ko -o \
                          -path \*/kernel/fs/ext\* -o \
                          -path \*/kernel/fs/jbd\* -o \
+                         -path \*/kernel/fs/crypto\* -o \
                          -path \*/kernel/fs/fat\* -o \
                          -path \*/kernel/fs/nls/\* -o \
                          -path \*/kernel/drivers/clk/\* -o \
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 10/35] backports handling: Refactor kernel deb handling

[Ian Jackson]

Introduce private function $kdeb->().  No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 45 +++++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 16 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 52527bf..5a083c2 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1114,32 +1114,45 @@ END
 	$kp = "need-kernel-deb-wheezy-backports" if $kp eq "need-kernel-deb-armmp";
 	$kp =~ s/need-kernel-deb-$ho->{Suite}-// or next;
 
-	my $kern = "$d_i/$kp.deb";
-	my $kurl = create_webfile($ho, "kernel", sub {
-	    copy($kern, $_[0])
-		or die "Copy kernel $kern failed: $!";
-        });
-
-	my $ird = "$d_i/initramfs-tools.deb";
-	my $iurl = create_webfile($ho, "initramfs-tools", sub {
-	    copy($ird, $_[0])
-		or die "Copy initramfs-tools $ird failed: $!";
-        });
-
-	preseed_hook_command($ho, 'late_command', $sfx, <<END);
+        my $cmds = <<END;
 #!/bin/sh
 set -ex
 
 r=/target
 
-$preseed_wget -O \$r/tmp/kern.deb $kurl
-$preseed_wget -O \$r/tmp/initramfs-tools.deb $iurl
+END
+
+        my @kdebs;
+        my $kdeb = sub {
+            my ($what, $file) = @_;
+
+            logm("for late_command using $what $file");
+            my $furl = create_webfile($ho, $what, sub {
+	        copy($file, $_[0]) or die "Copy $what $file failed: $!";
+            });
+            my $tmpfn = "/tmp/$what.deb";
+
+            $cmds .= <<END;
+$preseed_wget -O \$r$tmpfn $furl
+END
+            push @kdebs, $tmpfn;
+        };
+
+        my $kern = "$d_i/$kp.deb";
+        $kdeb->("kernel", $kern);
+
+        my $ird = "$d_i/initramfs-tools.deb";
+        $kdeb->("initramfs-tools", $ird);
+
+        $cmds .= <<END;
 
 # This will fail due to dependencies...
-in-target dpkg -i /tmp/kern.deb /tmp/initramfs-tools.deb || true
+in-target dpkg -i @kdebs || true
 # ... Now fix everything up...
 in-target apt-get install -f -y
 END
+
+	preseed_hook_command($ho, 'late_command', $sfx, $cmds);
     }
 
     if ( $ho->{Flags}{'need-uboot-bootscr'} ) {
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 11/35] backports handling: Make backports initramfs-tools optional

[Ian Jackson]

If mg-debian-installer-update decided not to provide an initramfs.deb,
just skip that.

No functional change with earlier d-i fetches produced by existing
versions of mg-debian-installer-update.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5a083c2..dacefb3 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1142,7 +1142,11 @@ END
         $kdeb->("kernel", $kern);
 
         my $ird = "$d_i/initramfs-tools.deb";
-        $kdeb->("initramfs-tools", $ird);
+        if (stat $ird) {
+            $kdeb->("initramfs-tools", $ird);
+        } else {
+            die "$ird $!" unless $!==ENOENT;
+        }
 
         $cmds .= <<END;
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 12/35] backports handling: Cope with dtbs in subdirs

[Ian Jackson]

Some of the arm64 kernels have backports in subdirectories.  So use a
find rune instead of a plain mv.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-debian-installer-update | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 82a1804..c4abc43 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -169,7 +169,8 @@ if [ x$bpok != x ]; then
     if [ -d x/usr/lib/linux-image-*-${bpok}/ ] ; then
         rm -rf backports-dtbs/
         mkdir backports-dtbs/
-        mv x/usr/lib/linux-image-*-${bpok}/*.dtb backports-dtbs/
+        find x/usr/lib/linux-image-*-${bpok} -name '*.dtb' -print0 | \
+             xargs -0r sh -ec 'mv -- "$@" backports-dtbs/' x
         tar --mtime=./x/usr/lib -cf backports-dtbs.tar backports-dtbs
         gzip -9nf backports-dtbs.tar
     fi
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 13/35] d-i: Cope if .ssh already exists (in /target/root/ and also /home/...)

[Ian Jackson]

Use mkdir -p in both cases.  Otherwise these late_commands may fail.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index dacefb3..987c7c6 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -735,13 +735,13 @@ r=/target/root
 cd \$r
 
 umask 022
-mkdir .ssh
+mkdir -p .ssh
 $preseed_wget -O .ssh/authorized_keys '$authkeys_url'
 $preseed_wget -O .ssh/known_hosts     '$knownhosts_url'
 
 u=osstest
 h=/home/\$u
-mkdir /target\$h/.ssh
+mkdir -p /target\$h/.ssh
 cp .ssh/authorized_keys /target\$h/.ssh
 chroot /target chown -R \$u.\$u \$h/.ssh
 END
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 14/35] d-i grub: Extend UEFI workaround to stretch

[Ian Jackson]

Debian bug #789798 remains open.  I have pinged the blocking bug in
grub2, #792547.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 987c7c6..5cd6433 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1276,7 +1276,7 @@ END
     preseed_microcode($ho,$sfx);
 
     if (get_host_property($ho, "firmware",'') eq "uefi") {
-	die unless $ho->{Suite} =~ m/jessie/;
+	die unless $ho->{Suite} =~ m/jessie|stretch/;
 	# Prevent grub-install from making a new Debian boot entry, so
 	# we always reboot from PXE. Debian bug #789798 proposes a
 	# properly preseedable solution to this.
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 15/35] pxe templates: Do defaulting outside host_pxefile

[Ian Jackson]

host_pxefile now returns undef if the specified $templatekey is not
configured, rather than defaulting to the plain PxeTemplates.

This is going to useful because we want host_pxefile not to hardcode
the default: with netgrub, the settings are all different.

If $templatekey is 'PxeTemplates' then this only has any effect if the
PxeTempltes is not defined - ie, a broken case.

There is one place where $templatekey is not 'PxeTemplates', in
mg-hosts.  The defaulting to the value of PxeTemplates now occurs there.

So, overall, no functional change in non-broken cases.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 3 ++-
 mg-hosts               | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 888f0ac..4c6a677 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2370,7 +2370,8 @@ sub host_pxefile ($;$) {
     my ($ho, $templatekey) = @_;
     my %v = %r;
     $templatekey //= 'PxeTemplates';
-    my $templates = $ho->{Tftp}{$templatekey} || $ho->{Tftp}{PxeTemplates};
+    my $templates = $ho->{Tftp}{$templatekey};
+    return undef unless defined $templates;
     if (defined $ho->{Ether}) {
 	my $eth = $v{'ether'} = $ho->{Ether};
 	$eth =~ y/A-Z/a-z/;
diff --git a/mg-hosts b/mg-hosts
index f785454..ed51952 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -120,6 +120,7 @@ sub cmd_mkpxedir () {
         my $ho= selecthost("host=$hn");
 	my $pxefile = host_pxefile($ho);
 	my $pxerealfile = host_pxefile($ho, 'PxeTemplatesReal');
+	$pxerealfile //= $pxefile;
 	my $dirname = dirname $pxefile;
 	my $cmd = <<END;
             set -e
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 16/35] netboot: Rework interface to host_netboot_file (was host_pxefile)

[Ian Jackson]

* Now it takes the tail of the template key (currently, normally
  `Templates') rather than the whole of it (`PxeTemplates'); the
  `Pxe' part is implied.

* Now it normally returns the whole path, including PxeDir, which the
  caller is no longer expected to prepend.  In any case, it supplies
  the dir value.

* Rename it.

* All call sites adjusted.

Overall, no functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 20 +++++++++++++-------
 mg-hosts               |  6 +++---
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 4c6a677..54ff552 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -115,7 +115,7 @@ BEGIN {
 
                       await_webspace_fetch_byleaf create_webfile
                       file_link_contents get_timeout
-                      setup_pxeboot_di setup_pxeboot_local host_pxefile
+                      setup_pxeboot_di setup_pxeboot_local host_netboot_file
 
                       ether_prefix
 
@@ -2366,12 +2366,18 @@ sub file_link_contents ($$$) {
     logm("wrote $fn". (defined $stash ? " (stashed as $stash)" : ""));
 }
 
-sub host_pxefile ($;$) {
-    my ($ho, $templatekey) = @_;
+sub host_netboot_file ($;$) {
+    my ($ho, $templatekeytail) = @_;
+    # returns the full netboot filename path
+    # in array context, returns (dir, pathtail)
+    #  where dir does not depend on $templatekeytail
     my %v = %r;
-    $templatekey //= 'PxeTemplates';
+    my $templatekeybase = 'Pxe';
+    $templatekeytail //= 'Templates';
+    my $templatekey = $templatekeybase.$templatekeytail;
     my $templates = $ho->{Tftp}{$templatekey};
     return undef unless defined $templates;
+    my $dir = $ho->{Tftp}{$templatekeybase.'Dir'};
     if (defined $ho->{Ether}) {
 	my $eth = $v{'ether'} = $ho->{Ether};
 	$eth =~ y/A-Z/a-z/;
@@ -2396,7 +2402,7 @@ sub host_pxefile ($;$) {
 		    next;
 		 }ge;
 	# and return the first pattern we managed to completely substitute
-        return $pat;
+        return wantarray ? ($dir,$pat) : $dir.$pat;
     }
     die "no pxe template ($templatekey) matched $templates ".
         (join ",", sort keys %v)." ?";
@@ -2404,8 +2410,8 @@ sub host_pxefile ($;$) {
 
 sub setup_pxelinux_bootcfg ($$) {
     my ($ho, $bootfile) = @_;
-    my $f= host_pxefile($ho);
-    file_link_contents("$ho->{Tftp}{Path}$ho->{Tftp}{PxeDir}$f", $bootfile,
+    my $f= host_netboot_file($ho);
+    file_link_contents("$ho->{Tftp}{Path}$f", $bootfile,
 	hostnamepath($ho)."-pxelinux.cfg");
 }
 
diff --git a/mg-hosts b/mg-hosts
index ed51952..ed26df7 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -118,13 +118,13 @@ sub cmd_mkpxedir () {
     my $sudo = $ENV{'OSSTEST_SUDO'} // 'sudo';
     foreach my $hn (@ARGV) {
         my $ho= selecthost("host=$hn");
-	my $pxefile = host_pxefile($ho);
-	my $pxerealfile = host_pxefile($ho, 'PxeTemplatesReal');
+	my ($dir, $pxefile) = host_netboot_file($ho);
+	my ($rdir, $pxerealfile) = host_netboot_file($ho, 'TemplatesReal');
 	$pxerealfile //= $pxefile;
 	my $dirname = dirname $pxefile;
 	my $cmd = <<END;
             set -e
-	    cd $ho->{Tftp}{Path}$ho->{Tftp}{PxeDir}
+           cd $dir
 END
         $cmd .= <<END if $dirname ne '.';
 	    $sudo mkdir -p $dirname
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 17/35] netboot: Rename pxeboot functions

[Ian Jackson]

 git-ls-files | xargs perl -i~ -pe 's/\bsetup_pxeboot_/setup_netboot_/g'
 git-ls-files | xargs perl -i~ -pe 's/\bsetup_pxelinux_/setup_netboot_/g'

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 28 ++++++++++++++--------------
 ts-host-install        |  8 ++++----
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 54ff552..9dbf1a7 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -115,7 +115,7 @@ BEGIN {
 
                       await_webspace_fetch_byleaf create_webfile
                       file_link_contents get_timeout
-                      setup_pxeboot_di setup_pxeboot_local host_netboot_file
+                      setup_netboot_di setup_netboot_local host_netboot_file
 
                       ether_prefix
 
@@ -2408,7 +2408,7 @@ sub host_netboot_file ($;$) {
         (join ",", sort keys %v)." ?";
 }
 
-sub setup_pxelinux_bootcfg ($$) {
+sub setup_netboot_bootcfg ($$) {
     my ($ho, $bootfile) = @_;
     my $f= host_netboot_file($ho);
     file_link_contents("$ho->{Tftp}{Path}$f", $bootfile,
@@ -2416,10 +2416,10 @@ sub setup_pxelinux_bootcfg ($$) {
 }
 
 # Systems using BIOS are configured to use pxelinux
-sub setup_pxeboot_di_bios ($$$$$;%) {
+sub setup_netboot_di_bios ($$$$$;%) {
     my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
     my $dtbs = $xopts{dtbs} ? "fdtdir $xopts{dtbs}" : "";
-    setup_pxelinux_bootcfg($ho, <<END);
+    setup_netboot_bootcfg($ho, <<END);
     serial 0 $c{Baud}
 timeout 5
 label overwrite
@@ -2433,9 +2433,9 @@ default overwrite
 END
 }
 
-sub setup_pxeboot_local_bios ($) {
+sub setup_netboot_local_bios ($) {
     my ($ho) = @_;
-    setup_pxelinux_bootcfg($ho, <<END);
+    setup_netboot_bootcfg($ho, <<END);
 serial 0 $c{Baud}
 timeout 5
 label local
@@ -2445,8 +2445,8 @@ END
 }
 
 # uboot emulates pxelinux, so reuse BIOS stuff
-sub setup_pxeboot_di_uboot ($$$$$;%) { return &setup_pxeboot_di_bios; }
-sub setup_pxeboot_local_uboot ($) { return &setup_pxeboot_local_bios; }
+sub setup_netboot_di_uboot ($$$$$;%) { return &setup_netboot_di_bios; }
+sub setup_netboot_local_uboot ($) { return &setup_netboot_local_bios; }
 
 sub setup_grub_efi_bootcfg ($$) {
     my ($ho, $bootfile) = @_;
@@ -2464,7 +2464,7 @@ sub setup_grub_efi_bootcfg ($$) {
 }
 
 # UEFI systems PXE boot using grub.efi
-sub setup_pxeboot_di_uefi ($$$$$;%) {
+sub setup_netboot_di_uefi ($$$$$;%) {
     my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
     setup_grub_efi_bootcfg($ho, <<END);
 set default=0
@@ -2476,7 +2476,7 @@ menuentry 'overwrite' {
 END
 }
 
-sub setup_pxeboot_local_uefi ($) {
+sub setup_netboot_local_uefi ($) {
     my ($ho) = @_;
     my %efi_archs = qw(amd64 X64
                        arm32 ARM
@@ -2499,20 +2499,20 @@ menuentry 'local' {
 END
 }
 
-sub setup_pxeboot_local ($) {
+sub setup_netboot_local ($) {
     my ($ho) = @_;
     my $firmware = get_host_property($ho, "firmware", "bios");
     $firmware =~ s/-/_/g;
     no strict qw(refs);
-    return &{"setup_pxeboot_local_${firmware}"}($ho);
+    return &{"setup_netboot_local_${firmware}"}($ho);
 }
 
-sub setup_pxeboot_di ($$$$$;%) {
+sub setup_netboot_di ($$$$$;%) {
     my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
     my $firmware = get_host_property($ho, "firmware", "bios");
     $firmware =~ s/-/_/g;
     no strict qw(refs);
-    return &{"setup_pxeboot_di_${firmware}"}($ho,$kern,$initrd,
+    return &{"setup_netboot_di_${firmware}"}($ho,$kern,$initrd,
 					     join(' ',@{$dicmd}),
 					     join(' ',@{$hocmd}),
 					     %xopts);
diff --git a/ts-host-install b/ts-host-install
index 770c51b..e151c5b 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -66,7 +66,7 @@ d-i netcfg/get_hostname string $ho->{Name}
 d-i netcfg/get_ipaddress string $ho->{Ip}
 END
 
-    setup_pxeboot_firstboot($ps_url);
+    setup_netboot_firstboot($ps_url);
     power_cycle_sleep($ho);
 
     my $logtailer= Osstest::Logtailer->new($c{WebspaceLog});
@@ -82,7 +82,7 @@ END
 	exit 0;
     }
 
-    setup_pxeboot_local($ho);
+    setup_netboot_local($ho);
 
     await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14,$ho);
 
@@ -154,7 +154,7 @@ END
     logm('OK: install completed');
 }   
 
-sub setup_pxeboot_firstboot($) {
+sub setup_netboot_firstboot($) {
     my ($ps_url) = @_;
     
     my $d_i= $ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.$ho->{DiVersion}.'-'.$ho->{Suite};
@@ -275,7 +275,7 @@ END
         get_host_property($ho, "linux-boot-append $ho->{Suite} $r{arch}", '');
 
     $xopts{ipappend} = $ipappend;
-    setup_pxeboot_di($ho, $kernel, "/$initrd", \@dicmdline, \@hocmdline,
+    setup_netboot_di($ho, $kernel, "/$initrd", \@dicmdline, \@hocmdline,
 		     %xopts);
 }
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 18/35] netboot: Change pxe to netboot in comments, docs and messages

[Ian Jackson]

UEFI grub netbooting is not PXE.  Indeed PXE is a BIOS thing only.

No functional change other than to messages.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 INSTALL.production     | 10 +++++-----
 Osstest/Debian.pm      |  2 +-
 Osstest/TestSupport.pm |  6 +++---
 README                 |  4 ++--
 mg-hosts               |  2 +-
 5 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/INSTALL.production b/INSTALL.production
index 2114ccc..61ad095 100644
--- a/INSTALL.production
+++ b/INSTALL.production
@@ -59,8 +59,8 @@ OwnerDaemonHost and QueueDaemonHost configuration options. If they
 happen to run on the same host then setting ControlDaemonHost will
 configure both in one go.
 
-DHCP AND PXE
-------------
+DHCP AND TFTP (eg PXE)
+----------------------
 
 osstest does not require a dedicated DHCP server but one must be
 present on the network and osstest requires access to its leases
@@ -69,9 +69,9 @@ to which will dump the lease file. This is configured via the
 DhcpWatchMethod host prop (default configured via
 HostProp_DhcpWatchMethod)
 
-Likewise osstest does not require a dedicated PXE server but one must
-exist on the network and be accessible via a local path on the osstest
-control VM (e.g. over NFS)
+Likewise osstest does not require a dedicated TFTP (netboot) server
+but one must exist on the network and be accessible via a local path
+on the osstest control VM (e.g. over NFS)
 
 CONFIGURATION
 =============
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5cd6433..4c73318 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1278,7 +1278,7 @@ END
     if (get_host_property($ho, "firmware",'') eq "uefi") {
 	die unless $ho->{Suite} =~ m/jessie|stretch/;
 	# Prevent grub-install from making a new Debian boot entry, so
-	# we always reboot from PXE. Debian bug #789798 proposes a
+	# we always reboot from the network. Debian bug #789798 proposes a
 	# properly preseedable solution to this.
 	preseed_hook_installscript($ho, $sfx,
 		'/usr/lib/base-installer.d/',
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 9dbf1a7..fde796a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2332,7 +2332,7 @@ sub create_webfile ($$$) {
     return $wf_url;
 }
 
-#---------- pxe handling ----------
+#---------- netboot handling ----------
 
 sub file_link_contents ($$$) {
     my ($fn, $contents, $stash) = @_;
@@ -2404,7 +2404,7 @@ sub host_netboot_file ($;$) {
 	# and return the first pattern we managed to completely substitute
         return wantarray ? ($dir,$pat) : $dir.$pat;
     }
-    die "no pxe template ($templatekey) matched $templates ".
+    die "no netboot template ($templatekey) matched $templates ".
         (join ",", sort keys %v)." ?";
 }
 
@@ -2463,7 +2463,7 @@ sub setup_grub_efi_bootcfg ($$) {
 		       $bootfile,  hostnamepath($ho)."-pxegrub.cfg");
 }
 
-# UEFI systems PXE boot using grub.efi
+# UEFI systems netboot using grub.efi
 sub setup_netboot_di_uefi ($$$$$;%) {
     my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
     setup_grub_efi_bootcfg($ho, <<END);
diff --git a/README b/README
index e3519a4..f77922c 100644
--- a/README
+++ b/README
@@ -296,7 +296,7 @@ To run osstest in standalone mode:
    so you need to insert those missing runvars into standalone.db with
    sqlite3.
 
- - Don't forget to set the machine to pxeboot in the BIOS.
+ - Don't forget to set the machine to netboot in the BIOS.
 
  - Currently you need a serial console at 115200 8n1; see TODO.
 
@@ -508,7 +508,7 @@ Tftp*
     TftpDiBase        The path under `Path' to the root of the debian
                       installer images.
     TftpGrubBase      The path under `Path' to the root of the grub
-                      EFI PXE images.
+                      EFI netboot images.
 
 Tftp<setting>_<scope>
 
diff --git a/mg-hosts b/mg-hosts
index ed26df7..60b0742 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -20,7 +20,7 @@
 # Usages:
 #
 #  ./mg-hosts mkpxedir HOST...
-#               Create directories for pxeboot as expected by the rest
+#               Create directories for netboot as expected by the rest
 #               of osstest.  Will use "sudo". The HOST(s) must be
 #               allocated (via mg-allocate HOST).
 #
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 19/35] netboot: Change pxe to netboot in variable names

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm |  6 +++---
 mg-hosts               | 14 +++++++-------
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index fde796a..bb72fe5 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2453,10 +2453,10 @@ sub setup_grub_efi_bootcfg ($$) {
     my $f = "grub.cfg-$ho->{Ether}";
     my $grub= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{GrubBase}.'/'.
 	$c{TftpGrubVersion}."/pxegrub-$r{arch}.efi";
-    my $pxe=$ho->{Tftp}{Path}.'/'.hostnamepath($ho).'/pxe.img';
+    my $img=$ho->{Tftp}{Path}.'/'.hostnamepath($ho).'/pxe.img';
 
-    logm("Copy $grub => $pxe");
-    copy($grub, $pxe) or die "Copy $grub to $pxe failed: $!";
+    logm("Copy $grub => $img");
+    copy($grub, $img) or die "Copy $grub to $img failed: $!";
 
     logm("grub_efi bootcfg into $f");
     file_link_contents("$ho->{Tftp}{Path}$ho->{Tftp}{TmpDir}$f",
diff --git a/mg-hosts b/mg-hosts
index 60b0742..20f40b0 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -118,10 +118,10 @@ sub cmd_mkpxedir () {
     my $sudo = $ENV{'OSSTEST_SUDO'} // 'sudo';
     foreach my $hn (@ARGV) {
         my $ho= selecthost("host=$hn");
-	my ($dir, $pxefile) = host_netboot_file($ho);
-	my ($rdir, $pxerealfile) = host_netboot_file($ho, 'TemplatesReal');
-	$pxerealfile //= $pxefile;
-	my $dirname = dirname $pxefile;
+	my ($dir, $file) = host_netboot_file($ho);
+	my ($rdir, $realfile) = host_netboot_file($ho, 'TemplatesReal');
+	$realfile //= $file;
+	my $dirname = dirname $file;
 	my $cmd = <<END;
             set -e
            cd $dir
@@ -133,10 +133,10 @@ END
             $sudo chown root.$ho->{Tftp}{PxeGroup} $dirname
             $sudo chmod 2775 $dirname
 END
-        if ($pxefile ne $pxerealfile) {
-	    my $subdirs = "../" x $pxefile =~ m#/#;
+        if ($file ne $realfile) {
+	    my $subdirs = "../" x $file =~ m#/#;
             $cmd .= <<END;
-	    $sudo ln -sf $subdirs$pxefile $pxerealfile
+	    $sudo ln -sf $subdirs$file $realfile
 END
         }
         print $cmd;
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 20/35] netboot: Rename mg-netgrub-loader-update

[Ian Jackson]

This used to be mg-pxe-loader-update, but it updates grub and not
anything to do with PXE.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 README                   |  2 +-
 mg-netgrub-loader-update | 84 ++++++++++++++++++++++++++++++++++++++++++++++++
 mg-pxe-loader-update     | 84 ------------------------------------------------
 production-config        |  2 +-
 4 files changed, 86 insertions(+), 86 deletions(-)
 create mode 100755 mg-netgrub-loader-update
 delete mode 100755 mg-pxe-loader-update

diff --git a/README b/README
index f77922c..01644b0 100644
--- a/README
+++ b/README
@@ -257,7 +257,7 @@ To run osstest in standalone mode:
 
  - Optional:
      ipmitool -- for hosts which use IPMI for power control
-     grub-common -- for mg-pxe-loader-update
+     grub-common -- for mg-netgrub-loader-update
 
  - Write a config file
     ~/.xen-osstest/config
diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
new file mode 100755
index 0000000..ca8abcb
--- /dev/null
+++ b/mg-netgrub-loader-update
@@ -0,0 +1,84 @@
+#!/bin/bash
+# usage
+#   ./mg-netgrub-loader-update jessie
+#
+# Requires grub-mkimage (Debian package: grub-common)
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e -o posix
+
+. ./cri-getconfig
+. ./mgi-common
+
+suite=$1
+
+site=http://ftp.debian.org/debian/
+sbase=$site/dists/$suite
+
+archs="amd64 i386 arm64"
+
+dstroot=`getconfig TftpPath`/`getconfig TftpGrubBase`/
+date=`date +%Y-%m-%d`
+dst=$date
+
+grubpfx=`getconfig TftpGrubBase`/$date/grub
+grubcfg=`getconfig TftpTmpDir`/grub.cfg-'$net_default_mac'
+
+mkdir -p $dstroot
+cd $dstroot
+mkdir -p $dst
+cd $dst
+rm -rf grub
+mkdir grub
+
+cat >grub/grub.cfg <<EOF
+set stage1=yes
+configfile $grubcfg
+EOF
+for arch in $archs ; do
+    case $arch in
+	amd64) grubdeb=grub-efi-amd64-bin; platform=x86_64-efi;;
+	i386)  grubdeb=grub-efi-ia32-bin;  platform=i386-efi;;
+	arm64) grubdeb=grub-efi-arm64-bin; platform=arm64-efi;;
+	*) echo "No grub on $arch" >&2; exit 0;;
+    esac
+
+    pfile=$sbase/main/binary-$arch/Packages.gz
+
+    fetch $pfile >Packages.gz
+
+    echo >&2 "collecting $grubdeb"
+    fetch_debian_package $site Packages.gz -PX $grubdeb >$grubdeb.deb
+
+    dpkg-deb -x $grubdeb.deb x
+
+    mv x/usr/lib/grub/* grub/
+
+    rm -rf x
+
+    rm Packages.gz
+
+    grub-mkimage -O "$platform" \
+		 -d ./grub/$platform \
+		 -o pxegrub-$arch.efi -p "$grubpfx" \
+		 search configfile normal efinet tftp net
+done
+
+echo $date
+echo >&2 "downloaded $dstroot/$date"
diff --git a/mg-pxe-loader-update b/mg-pxe-loader-update
deleted file mode 100755
index e0e347d..0000000
--- a/mg-pxe-loader-update
+++ /dev/null
@@ -1,84 +0,0 @@
-#!/bin/bash
-# usage
-#   ./mg-pxe-loader-update jessie
-#
-# Requires grub-mkimage (Debian package: grub-common)
-
-# This is part of "osstest", an automated testing framework for Xen.
-# Copyright (C) 2015 Citrix Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-
-set -e -o posix
-
-. ./cri-getconfig
-. ./mgi-common
-
-suite=$1
-
-site=http://ftp.debian.org/debian/
-sbase=$site/dists/$suite
-
-archs="amd64 i386 arm64"
-
-dstroot=`getconfig TftpPath`/`getconfig TftpGrubBase`/
-date=`date +%Y-%m-%d`
-dst=$date
-
-grubpfx=`getconfig TftpGrubBase`/$date/grub
-grubcfg=`getconfig TftpTmpDir`/grub.cfg-'$net_default_mac'
-
-mkdir -p $dstroot
-cd $dstroot
-mkdir -p $dst
-cd $dst
-rm -rf grub
-mkdir grub
-
-cat >grub/grub.cfg <<EOF
-set stage1=yes
-configfile $grubcfg
-EOF
-for arch in $archs ; do
-    case $arch in
-	amd64) grubdeb=grub-efi-amd64-bin; platform=x86_64-efi;;
-	i386)  grubdeb=grub-efi-ia32-bin;  platform=i386-efi;;
-	arm64) grubdeb=grub-efi-arm64-bin; platform=arm64-efi;;
-	*) echo "No grub on $arch" >&2; exit 0;;
-    esac
-
-    pfile=$sbase/main/binary-$arch/Packages.gz
-
-    fetch $pfile >Packages.gz
-
-    echo >&2 "collecting $grubdeb"
-    fetch_debian_package $site Packages.gz -PX $grubdeb >$grubdeb.deb
-
-    dpkg-deb -x $grubdeb.deb x
-
-    mv x/usr/lib/grub/* grub/
-
-    rm -rf x
-
-    rm Packages.gz
-
-    grub-mkimage -O "$platform" \
-		 -d ./grub/$platform \
-		 -o pxegrub-$arch.efi -p "$grubpfx" \
-		 search configfile normal efinet tftp net
-done
-
-echo $date
-echo >&2 "downloaded $dstroot/$date"
diff --git a/production-config b/production-config
index 2e91f21..9a299ac 100644
--- a/production-config
+++ b/production-config
@@ -99,7 +99,7 @@ DebianImageVersion_jessie 8.2.0
 MicrocodeUpdateAmd64 microcode.x86.2015-06-12.cpio
 MicrocodeUpdateI386 microcode.x86.2015-06-12.cpio
 
-# Update with ./mg-pxe-loader-update
+# Update with ./mg-netgrub-loader-update
 TftpGrubVersion XXXX-XX-XX
 
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 21/35] netboot: Rename ./mg-hosts mknetbootdir subcommand

[Ian Jackson]

Was ./mg-hosts mkpxedir, but it is more general.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 README     | 4 ++--
 README.dev | 2 +-
 mg-hosts   | 4 ++--
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/README b/README
index 01644b0..91d0328 100644
--- a/README
+++ b/README
@@ -538,8 +538,8 @@ TftpPxeTemplates
     are known is used, with TftpPath and TftpPxeDir prepended.
 
 TftpPxeTemplatesReal
-    Template filename which mg-hosts mkpxedir should make be a symlink to
-    the TftpPxeTemplates.  Not used otherwise.
+    Template filename which mg-hosts mknetbootdir should make be a
+    symlink to the TftpPxeTemplates.  Not used otherwise.
 
 Timezone
     Olson TZ name, used by host and guest installers
diff --git a/README.dev b/README.dev
index a5b23b0..09b9531 100644
--- a/README.dev
+++ b/README.dev
@@ -45,7 +45,7 @@ $ mg-hosts setprops marilith-n4 -- dhcp-watch-method 'leases dhcp3 dns-cam1.uk.x
 Handles multiple hosts:
 $ mg-hosts setprops HOSTA HOSTB -- "PROP" "VALUE"
 
-$ mg-hosts mkpxedir HOSTA HOSTB
+$ mg-hosts mknetbootdir HOSTA HOSTB
 
 Create a flight:
 $ FLIGHT=`OSSTEST_CONFIG=production-config ./cs-flight-create commission xen-unstable`
diff --git a/mg-hosts b/mg-hosts
index 20f40b0..07283c8 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -19,7 +19,7 @@
 
 # Usages:
 #
-#  ./mg-hosts mkpxedir HOST...
+#  ./mg-hosts mknetbootdir HOST...
 #               Create directories for netboot as expected by the rest
 #               of osstest.  Will use "sudo". The HOST(s) must be
 #               allocated (via mg-allocate HOST).
@@ -111,7 +111,7 @@ sub checkhost ($) {
 
 sub l ($) { return split /,/, $_[0]; }
 
-sub cmd_mkpxedir () {
+sub cmd_mknetbootdir () {
     my $dryrun = 0;
     if (@ARGV && $ARGV[0] eq '-n') { shift @ARGV; $dryrun= 1; }
     die unless @ARGV>=1;
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 22/35] netboot: Break out subst_netboot_template

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index bb72fe5..990ddbc 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -116,6 +116,7 @@ BEGIN {
                       await_webspace_fetch_byleaf create_webfile
                       file_link_contents get_timeout
                       setup_netboot_di setup_netboot_local host_netboot_file
+		      subst_netboot_template
 
                       ether_prefix
 
@@ -2366,6 +2367,22 @@ sub file_link_contents ($$$) {
     logm("wrote $fn". (defined $stash ? " (stashed as $stash)" : ""));
 }
 
+sub subst_netboot_template ($$$) {
+    my ($templates, $v, $what) = (@_);
+    foreach my $pat (split /\s+/, $templates) {
+	# we skip patterns that contain any references to undefined %var%s
+	$pat =~ s{\%(\w*)\%}{
+		    $1 eq '' ? '%' :
+		    defined($v->{$1}) ? $v->{$1} :
+		    next;
+		 }ge;
+	# and return the first pattern we managed to completely substitute
+        return $pat;
+    }
+    die "no netboot template ($what) matched $templates ".
+        (join ",", sort keys %$v)." ?";
+}
+
 sub host_netboot_file ($;$) {
     my ($ho, $templatekeytail) = @_;
     # returns the full netboot filename path
@@ -2394,18 +2411,8 @@ sub host_netboot_file ($;$) {
 	$v{'ipaddr'} = $ip;
 	$v{'ipaddrhex'} = sprintf "%02X%02X%02X%02X", split /\./, $ip;
     }
-    foreach my $pat (split /\s+/, $templates) {
-	# we skip patterns that contain any references to undefined %var%s
-	$pat =~ s{\%(\w*)\%}{
-		    $1 eq '' ? '%' :
-		    defined($v{$1}) ? $v{$1} :
-		    next;
-		 }ge;
-	# and return the first pattern we managed to completely substitute
-        return wantarray ? ($dir,$pat) : $dir.$pat;
-    }
-    die "no netboot template ($templatekey) matched $templates ".
-        (join ",", sort keys %v)." ?";
+    my $got = subst_netboot_template($templates, \%v, $templatekey);
+    return wantarray ? ($dir,$got) : $dir.$got;
 }
 
 sub setup_netboot_bootcfg ($$) {
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 23/35] netboot: Introduce TftpNetGrub*

[Ian Jackson]

* When expanding netboot filenames, look for NetGrub config keys
  rather than Pxe filenames.  Currently this only affects mg-hosts
  because the files are created by setup_grub_efi_bootcfg, and the
  files are of course referred to only by the actual netboot grub
  image (which may be made by mg-netgrub-loader-update: neither of
  which honour these variables.

* Update the documentation, the default configs, and the
  production-config-*.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest.pm                  |  2 ++
 Osstest/TestSupport.pm      |  6 ++++--
 README                      | 23 +++++++++++++++++------
 production-config           |  3 +++
 production-config-cambridge |  3 +++
 5 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/Osstest.pm b/Osstest.pm
index 26fd9d9..4ebd922 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -221,6 +221,8 @@ sub readglobalconfig () {
     $c{TftpPath} ||= "/tftpboot/";
     $c{TftpPxeDir} ||= "pxelinux.cfg/";
     $c{TftpPxeTemplates} ||= '%ipaddrhex% 01-%etherhyph%';
+    $c{TftpNetGrubDir} ||= "Netgub.cfg/";
+    $c{TftpNetGrubTemplates} ||= '%ether%';
     $c{TftpPlayDir} ||= "$c{Username}/osstest/";
     $c{TftpTmpDir} ||= "$c{TftpPlayDir}tmp/";
 
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 990ddbc..6d0f967 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -994,7 +994,8 @@ sub selecthost ($) {
     $ho->{Tftp} = { };
     $ho->{Tftp}{$_} = $c{"Tftp${_}_${tftpscope}"} || $c{"Tftp${_}"}
         foreach qw(Path TmpDir PxeDir PxeGroup PxeTemplates PxeTemplatesReal
-                   DiBase GrubBase);
+                   DiBase GrubBase
+                   NetGrubDir NetGrubTemplates NetGrubTemplatesReal);
 
     #----- finalise -----
 
@@ -2389,7 +2390,8 @@ sub host_netboot_file ($;$) {
     # in array context, returns (dir, pathtail)
     #  where dir does not depend on $templatekeytail
     my %v = %r;
-    my $templatekeybase = 'Pxe';
+    my $firmware = get_host_property($ho, "firmware", "bios");
+    my $templatekeybase = $firmware eq 'uefi' ? 'NetGrub' : 'Pxe';
     $templatekeytail //= 'Templates';
     my $templatekey = $templatekeybase.$templatekeytail;
     my $templates = $ho->{Tftp}{$templatekey};
diff --git a/README b/README
index 91d0328..9dcb65e 100644
--- a/README
+++ b/README
@@ -501,14 +501,21 @@ Tftp*
 
     TftpPxeDir        The path under `Path' to the PXE configuration directory
                       (e.g. pxelinux.cfg/).   Include the trailing /.
-    TftpPxeGroup      The Unix group which should own files under `PxeDir'.
-    TftpPxeTemplates  See TftpPxeTemplates
-    TftpPxeTemplatesReal
+    TftpNetGrubDir    The path under `Path' to the grub EFI netboot directory
+                      Include the trailing /.
+
+    TftpPxeGroup      The Unix group which should own files under `PxeDir'
+                      and NetGrubDir.
+
+    TftpPxeTemplates          } See below
+    TftpPxeTemplatesReal      }
+    TftpNetGrubTemplates      }
+    TftpNetGrubTemplatesReal  }
 
     TftpDiBase        The path under `Path' to the root of the debian
                       installer images.
     TftpGrubBase      The path under `Path' to the root of the grub
-                      EFI netboot images.
+                      EFI netboot (`netgrub') images.
 
 Tftp<setting>_<scope>
 
@@ -524,6 +531,7 @@ Tftp<setting>_<scope>
     Tftp<setting>_default is not defined.
 
 TftpPxeTemplates
+TftpNetGrubTemplates
     List (space-separated) of template filenames for writing
     The templates contain variable substitutions %var%
     The variables are the runvars plus
@@ -535,11 +543,14 @@ TftpPxeTemplates
     Templates containing references to unknown %var%s - particularly,
     the host mac address when not known, or the guest's dynamic
     ip address - are skipped.  The first template all of whose ingredients
-    are known is used, with TftpPath and TftpPxeDir prepended.
+    are known is used, with TftpPath and TftpPxeDir or TftpNetGrubDir
+    prepended.
 
 TftpPxeTemplatesReal
+TftpNetGrubTemplatesReal
     Template filename which mg-hosts mknetbootdir should make be a
-    symlink to the TftpPxeTemplates.  Not used otherwise.
+    symlink to the TftpPxeTemplates or TftpNetGrubTemplates.  Not used
+    otherwise.
 
 Timezone
     Olson TZ name, used by host and guest installers
diff --git a/production-config b/production-config
index 9a299ac..1243d7c 100644
--- a/production-config
+++ b/production-config
@@ -84,6 +84,9 @@ TftpTmpDir osstest/tmp/
 TftpPxeDir /
 TftpPxeTemplates %name%/pxelinux.cfg
 TftpPxeTemplatesReal pxelinux.cfg/%ipaddrhex%
+TftpNetGrubDir /
+TftpNetGrubTemplates %name%/netgrub.cfg
+TftpNetGrubTemplatesReal Netgrub.cfg/%ether%
 
 TftpPxeGroup osstest
 # Update with ./mg-debian-installer-update(-all)
diff --git a/production-config-cambridge b/production-config-cambridge
index 9968caa..5aefe8f 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -67,6 +67,9 @@ TftpTmpDir osstest/tmp/
 TftpPxeDir /
 TftpPxeTemplates %name%/pxelinux.cfg
 TftpPxeTemplatesReal pxelinux.cfg/%ipaddrhex%
+TftpNetGrubDir /
+TftpNetGrubTemplates %name%/netgrub.cfg
+TftpNetGrubTemplatesReal Netgrub.cfg/%ether%
 
 TftpPxeGroup osstest
 TftpDiVersion_wheezy 2016-06-08
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 24/35] netboot: Change stash file name to not be pxelinux-specific

[Ian Jackson]

This will be wrong when this is used for netgrub too.

Only functional change is to log outputs.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 6d0f967..9be40a2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2421,7 +2421,7 @@ sub setup_netboot_bootcfg ($$) {
     my ($ho, $bootfile) = @_;
     my $f= host_netboot_file($ho);
     file_link_contents("$ho->{Tftp}{Path}$f", $bootfile,
-	hostnamepath($ho)."-pxelinux.cfg");
+	hostnamepath($ho)."-netboot.cfg");
 }
 
 # Systems using BIOS are configured to use pxelinux
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 25/35] mg-netgrub-loader-update: Introduce $grubbase

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-netgrub-loader-update | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
index ca8abcb..9eb20d5 100755
--- a/mg-netgrub-loader-update
+++ b/mg-netgrub-loader-update
@@ -33,12 +33,12 @@ sbase=$site/dists/$suite
 
 archs="amd64 i386 arm64"
 
-dstroot=`getconfig TftpPath`/`getconfig TftpGrubBase`/
+grubbase=`getconfig TftpGrubBase`/
+dstroot=`getconfig TftpPath`/$grubbase
 date=`date +%Y-%m-%d`
 dst=$date
 
-grubpfx=`getconfig TftpGrubBase`/$date/grub
-grubcfg=`getconfig TftpTmpDir`/grub.cfg-'$net_default_mac'
+grubpfx=$grubbase/$date/grub
 
 mkdir -p $dstroot
 cd $dstroot
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 26/35] mg-netgrub-loader-update: Introduce $img

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-netgrub-loader-update | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
index 9eb20d5..ec6dfd3 100755
--- a/mg-netgrub-loader-update
+++ b/mg-netgrub-loader-update
@@ -74,9 +74,11 @@ for arch in $archs ; do
 
     rm Packages.gz
 
+    img=pxegrub-$arch.efi
+
     grub-mkimage -O "$platform" \
 		 -d ./grub/$platform \
-		 -o pxegrub-$arch.efi -p "$grubpfx" \
+		 -o $img -p "$grubpfx" \
 		 search configfile normal efinet tftp net
 done
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 27/35] netboot: UEFI grub: Do it more like pxe

[Ian Jackson]

* Abolish setup_grub_efi_bootcfg and use setup_netboot_bootcfg
  instead.  The latter function now expects to be given a config file
  for the host's bootloader.  For a UEFI host, that's a grub config
  file, which it will write to the configured NetGrub location for the
  specific host.

* Consequently, we no longer copy an actual grub image around every
  time we do a bootloader setup.  Instead, we expect that the tftp
  server will be configured to point relevant hosts at a suitable
  fixed image, which will go to our config file.

* Also, consequently, write the NetGrubTemplate path when we want to
  setup the bootloader, rather than the ad-hoc grub-%ether%.

* Update mg-netgrub-loader-update jessie so that it writes the
  NetGrubTemplateReal from the config into the grub.cfg so that the
  grub image will read that file and then read the file created by
  setup_netboot_bootcfg.

* Document this in its head comment.

Currently this new mg-netgrub-loader-update is untested because jessie
i386 grub (which is what we would run on the osstest controllers) does
not understand the arm64-efi target.  But the new Netgrub.cfg location
is in use.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm   | 19 ++-----------------
 mg-netgrub-loader-update | 17 +++++++++++++++++
 2 files changed, 19 insertions(+), 17 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 9be40a2..ed70579 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2457,25 +2457,10 @@ END
 sub setup_netboot_di_uboot ($$$$$;%) { return &setup_netboot_di_bios; }
 sub setup_netboot_local_uboot ($) { return &setup_netboot_local_bios; }
 
-sub setup_grub_efi_bootcfg ($$) {
-    my ($ho, $bootfile) = @_;
-    my $f = "grub.cfg-$ho->{Ether}";
-    my $grub= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{GrubBase}.'/'.
-	$c{TftpGrubVersion}."/pxegrub-$r{arch}.efi";
-    my $img=$ho->{Tftp}{Path}.'/'.hostnamepath($ho).'/pxe.img';
-
-    logm("Copy $grub => $img");
-    copy($grub, $img) or die "Copy $grub to $img failed: $!";
-
-    logm("grub_efi bootcfg into $f");
-    file_link_contents("$ho->{Tftp}{Path}$ho->{Tftp}{TmpDir}$f",
-		       $bootfile,  hostnamepath($ho)."-pxegrub.cfg");
-}
-
 # UEFI systems netboot using grub.efi
 sub setup_netboot_di_uefi ($$$$$;%) {
     my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
-    setup_grub_efi_bootcfg($ho, <<END);
+    setup_netboot_bootcfg($ho, <<END);
 set default=0
 set timeout=5
 menuentry 'overwrite' {
@@ -2493,7 +2478,7 @@ sub setup_netboot_local_uefi ($) {
                        i386  IA32);
     die "EFI arch" unless $efi_archs{ $r{arch} };
     my $efi = $efi_archs{ $r{arch} };
-    setup_grub_efi_bootcfg($ho, <<END);
+    setup_netboot_bootcfg($ho, <<END);
 set default=0
 set timeout=5
 menuentry 'local' {
diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
index ec6dfd3..21157a2 100755
--- a/mg-netgrub-loader-update
+++ b/mg-netgrub-loader-update
@@ -3,6 +3,12 @@
 #   ./mg-netgrub-loader-update jessie
 #
 # Requires grub-mkimage (Debian package: grub-common)
+#
+# Creates $c{TftpPath}/$c{TftpGrubBase}/YYYY-MM-DD/grub/*.efi
+# which expect to load .../YYYY-MM-DD/grub/grub.cfg
+# which in turn loads .../Netgrub.cfg/EE:EE:EE:EE:EE:EE\b
+# (where EE:EE:EE:EE:EE:EE is the MAC address)
+# (or whatever else is specified in TftpNetGrubTemplates).
 
 # This is part of "osstest", an automated testing framework for Xen.
 # Copyright (C) 2015 Citrix Inc.
@@ -40,6 +46,17 @@ dst=$date
 
 grubpfx=$grubbase/$date/grub
 
+grubcfg=$(perl -e '
+	use Osstest;
+	use Osstest::TestSupport;
+	readglobalconfig;
+        my $k = qw(TftpNetGrubTemplatesReal);
+	my $file = subst_netboot_template $c{$k},
+	    { ether => qw(${net_default_mac}) },
+            $k;
+	print $file, "\n" or die $!;
+')
+
 mkdir -p $dstroot
 cd $dstroot
 mkdir -p $dst
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 28/35] netboot: mg-netgrub-loader-update: Print sample dhcp config fragment

[Ian Jackson]

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mg-netgrub-loader-update | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
index 21157a2..73deb19 100755
--- a/mg-netgrub-loader-update
+++ b/mg-netgrub-loader-update
@@ -93,11 +93,17 @@ for arch in $archs ; do
 
     img=pxegrub-$arch.efi
 
+    output+="  filename \"$img\";
+"
+
     grub-mkimage -O "$platform" \
 		 -d ./grub/$platform \
 		 -o $img -p "$grubpfx" \
 		 search configfile normal efinet tftp net
 done
 
-echo $date
-echo >&2 "downloaded $dstroot/$date"
+echo >&2 "downloaded $grubbase/$dst/
+
+echo "host ... {
+  ...
+$output}"
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 29/35] netboot: Rename TftpNetbootGroup (from TftpPxeGroup)

[Ian Jackson]

This applies to NetGrub too.  Honour the old setting for
compatibility.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm      | 6 ++++--
 README                      | 2 +-
 mg-hosts                    | 2 +-
 production-config           | 2 +-
 production-config-cambridge | 2 +-
 5 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index ed70579..f55729b 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -993,9 +993,11 @@ sub selecthost ($) {
     logm("TftpScope is $tftpscope");
     $ho->{Tftp} = { };
     $ho->{Tftp}{$_} = $c{"Tftp${_}_${tftpscope}"} || $c{"Tftp${_}"}
-        foreach qw(Path TmpDir PxeDir PxeGroup PxeTemplates PxeTemplatesReal
+        foreach qw(Path TmpDir PxeDir NetbootGroup PxeTemplates PxeTemplatesReal
                    DiBase GrubBase
-                   NetGrubDir NetGrubTemplates NetGrubTemplatesReal);
+                   NetGrubDir NetGrubTemplates NetGrubTemplatesReal
+                   PxeGroup);
+    $ho->{TftpNetbootGroup} //= $ho->{TftpPxeGroup}; # compatibility
 
     #----- finalise -----
 
diff --git a/README b/README
index 9dcb65e..0e93f43 100644
--- a/README
+++ b/README
@@ -504,7 +504,7 @@ Tftp*
     TftpNetGrubDir    The path under `Path' to the grub EFI netboot directory
                       Include the trailing /.
 
-    TftpPxeGroup      The Unix group which should own files under `PxeDir'
+    TftpNetbootGroup  The Unix group which should own files under `PxeDir'
                       and NetGrubDir.
 
     TftpPxeTemplates          } See below
diff --git a/mg-hosts b/mg-hosts
index 07283c8..831a6e1 100755
--- a/mg-hosts
+++ b/mg-hosts
@@ -130,7 +130,7 @@ END
 	    $sudo mkdir -p $dirname
 END
         $cmd .= <<END;
-            $sudo chown root.$ho->{Tftp}{PxeGroup} $dirname
+            $sudo chown root.$ho->{Tftp}{NetbootGroup} $dirname
             $sudo chmod 2775 $dirname
 END
         if ($file ne $realfile) {
diff --git a/production-config b/production-config
index 1243d7c..cb35817 100644
--- a/production-config
+++ b/production-config
@@ -88,7 +88,7 @@ TftpNetGrubDir /
 TftpNetGrubTemplates %name%/netgrub.cfg
 TftpNetGrubTemplatesReal Netgrub.cfg/%ether%
 
-TftpPxeGroup osstest
+TftpNetbootGroup osstest
 # Update with ./mg-debian-installer-update(-all)
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2017-01-16
diff --git a/production-config-cambridge b/production-config-cambridge
index 5aefe8f..1389698 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -71,7 +71,7 @@ TftpNetGrubDir /
 TftpNetGrubTemplates %name%/netgrub.cfg
 TftpNetGrubTemplatesReal Netgrub.cfg/%ether%
 
-TftpPxeGroup osstest
+TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2017-01-16
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 30/35] netboot: mg-netgrub-loader-update: Add some more docs

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-netgrub-loader-update | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/mg-netgrub-loader-update b/mg-netgrub-loader-update
index 73deb19..22ef67c 100755
--- a/mg-netgrub-loader-update
+++ b/mg-netgrub-loader-update
@@ -10,6 +10,18 @@
 # (where EE:EE:EE:EE:EE:EE is the MAC address)
 # (or whatever else is specified in TftpNetGrubTemplates).
 
+# Running this script is optional.  Any approach will do which
+# arranges for the dhcp server to provide `filename' referring to a
+# copy of grub which will eventually load the config file specified by
+# TftpNetGrubPattern.  An example would be an appropriate
+# debian-installer netboot grub image, which will load 
+#   $c{TftpPath}/debian-installer/arm64/grub/grub.cfg
+# which could for example be made to contain
+#   configfile Netgrub.cfg/${net_default_mac}
+
+# This script does not currently support non-default Tftp scopes.
+# (See README.)
+
 # This is part of "osstest", an automated testing framework for Xen.
 # Copyright (C) 2015 Citrix Inc.
 #
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 31/35] d-i: preseed_create_guest: Break out preseed_backports_packages

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 39 +++++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 4c73318..f496354 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -927,6 +927,31 @@ END
     return $preseed;
 }
 
+sub preseed_backports_packages ($$$$@) {
+    my ($ho, $sfx, $xopts, $suite, @pkgs) = @_;
+
+    if (! $xopts->{BackportsSourcesAlreadyAdded}++) {
+	preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+cat <<EOF >>/target/etc/apt/sources.list
+
+# $suite backports
+deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+EOF
+in-target apt-get update
+END
+    }
+
+    preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+in-target apt-get install -y -t $suite-backports @pkgs
+END
+}
+
 sub preseed_create_guest ($$$;@) {
     my ($ho, $arch, $sfx, %xopts) = @_;
 
@@ -948,18 +973,8 @@ sub preseed_create_guest ($$$;@) {
             # the Grub device.map isn't present at pkgsel/include time
             # but it is by late_command time. This was fixed by
             # version 1.3 which is in Jessie onwards.
-            preseed_hook_command($ho, 'late_command', $sfx, <<END);
-#!/bin/sh
-set -ex
-
-cat <<EOF >>/target/etc/apt/sources.list
-
-# $suite backports
-deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
-EOF
-in-target apt-get update
-in-target apt-get install -y -t wheezy-backports pv-grub-menu
-END
+	    preseed_backports_packages($ho, $sfx, \%xopts, $suite,
+				       qw(pv-grub-menu));
         } else {
             $extra_packages = "pv-grub-menu";
         }
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 32/35] d-i: backports: When using backports kernel for a host, install linux-base

[Ian Jackson]

Newer kernels may need a newer linux-base.  This call has the happy
side effect of adding backports to its sources.list, which is also
generally needed.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index f496354..0137fc4 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1129,6 +1129,11 @@ END
 	$kp = "need-kernel-deb-wheezy-backports" if $kp eq "need-kernel-deb-armmp";
 	$kp =~ s/need-kernel-deb-$ho->{Suite}-// or next;
 
+	if ($kp eq 'backports') {
+	    preseed_backports_packages($ho, $sfx, \%xopts, $suite,
+				       qw(linux-base));
+	}
+
         my $cmds = <<END;
 #!/bin/sh
 set -ex
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 33/35] d-i: When using backports kernel, suppress kernel modules warning

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 0137fc4..06918ef 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1124,12 +1124,15 @@ in-target tar -C /boot -xaf /tmp/dtbs.tar.gz
 END
     }
 
+    my $backports_kernel;
+
     foreach my $kp (keys %{ $ho->{Flags} }) {
 	# Backwards compatibility
 	$kp = "need-kernel-deb-wheezy-backports" if $kp eq "need-kernel-deb-armmp";
 	$kp =~ s/need-kernel-deb-$ho->{Suite}-// or next;
 
 	if ($kp eq 'backports') {
+            $backports_kernel = 1;
 	    preseed_backports_packages($ho, $sfx, \%xopts, $suite,
 				       qw(linux-base));
 	}
@@ -1322,7 +1325,7 @@ END
 
     $preseed_file .= preseed_hook_cmds();
 
-    if ($ho->{Flags}{'no-di-kernel'}) {
+    if ($backports_kernel || $ho->{Flags}{'no-di-kernel'}) {
 	$preseed_file .= <<END;
 d-i anna/no_kernel_modules boolean true
 d-i base-installer/kernel/skip-install boolean true
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 34/35] d-i: Add comment about interrupting the install

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/Debian.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 06918ef..acc4618 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -890,6 +890,8 @@ d-i grub-installer/only_debian boolean true
 d-i grub-installer/bootdev string default
 
 d-i finish-install/keep-consoles boolean true
+
+# comment these out to get prompted at the end of the install:
 d-i finish-install/reboot_in_progress note
 d-i cdrom-detect/eject boolean false
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 35/35] ARM64: Create jobs, and provide the installer (with backports kernel)

[Ian Jackson]

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 make-flight                |  9 ++++++---
 mfi-common                 | 27 +++++++++++++++++++++++----
 mg-debian-installer-update |  1 +
 3 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/make-flight b/make-flight
index a374884..f513b80 100755
--- a/make-flight
+++ b/make-flight
@@ -72,6 +72,8 @@ job_create_build_filter_callback () {
         build-amd64-libvirt)	;;
         build-armhf)		;;
         build-armhf-pvops)	;;
+        build-arm64)		;;
+        build-arm64-pvops)	;;
         *)			return 1 ;;
       esac
     ;;
@@ -107,6 +109,7 @@ job_create_test_filter_callback () {
       case "$job" in
         test-amd64-amd64-libvirt)                  return 0 ;;
         test-armhf-armhf-xl)                       return 0 ;;
+        test-arm64-arm64-xl-xsm)                   return 0 ;;
         test-amd64-amd64-xl-qemuu-debianhvm-i386)  return 0 ;;
         *)                                         return 1 ;;
       esac
@@ -184,7 +187,7 @@ job_create_test_filter_callback () {
 arch_branch_filter_callback () {
   local arch=$1
   case "$arch" in
-  armhf)
+  arm*)
         case "$branch" in
         linux-3.0) return 1;;
         linux-3.4) return 1;;
@@ -572,7 +575,7 @@ do_pv_debian_tests () {
   # Within each fmt we rotate through the list of arches
   # The starting list rotates once per ts, so that we try to
   # exercise each fmt on each arch family.
-  local fmtarches_outer="i386 armhf amd64 armhf"
+  local fmtarches_outer="i386 armhf amd64 arm64"
   local endfmt="do_pv_debian_tests-missing-ts-fmt-for-dom0arch="
 
   for ts in xl libvirt ; do
@@ -628,7 +631,7 @@ test_matrix_do_one () {
   do_credit2_tests
 
   # No further arm tests at the moment
-  if [ $dom0arch = armhf ]; then
+  if [ $dom0arch = armhf -o $dom0arch = arm64 ]; then
       return
   fi
 
diff --git a/mfi-common b/mfi-common
index 76b93d1..ec31e2e 100644
--- a/mfi-common
+++ b/mfi-common
@@ -109,7 +109,7 @@ create_build_jobs () {
      BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
   fi
 
-  for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do
+  for arch in ${BUILD_ARCHES- i386 amd64 armhf arm64 }; do
 
     if [ "x$arch" = xdisable ]; then continue; fi
 
@@ -131,6 +131,16 @@ create_build_jobs () {
         kconfig_override_y=CONFIG_EXT4_FS
       "
       ;;
+    arm64)
+      case "$xenbranch" in
+      xen-3.*-testing) continue;;
+      xen-4.[0-6]-testing) continue;;
+      esac
+      pvops_kernel="
+        tree_linux=$TREE_LINUX_ARM
+        revision_linux=${REVISION_LINUX_ARM:-${DEFAULT_REVISION_LINUX_ARM}}
+      "
+      ;;
     *)
       pvops_kernel="
         tree_linux=$TREE_LINUX
@@ -316,7 +326,7 @@ create_build_jobs () {
     fi
 
     case "$arch" in
-    armhf) continue;; # don't do any other kernel builds
+    arm*) continue;; # don't do any other kernel builds
     esac
 
     if [ "x$REVISION_LINUX_OLD" != xdisable ]; then
@@ -410,7 +420,7 @@ test_matrix_iterate () {
       ;;
   esac
 
-  for xenarch in ${TEST_ARCHES- i386 amd64 armhf } ; do
+  for xenarch in ${TEST_ARCHES- i386 amd64 armhf arm64 } ; do
 
     if [ "x$xenarch" = xdisable ]; then continue; fi
 
@@ -427,6 +437,14 @@ test_matrix_iterate () {
           *) ;;
           esac
           ;;
+    arm64)
+          # ARM64 from 4.6 onwards only
+          case "$xenbranch" in
+          xen-3.*-testing) continue;;
+          xen-4.[0-6]-testing) continue;;
+          *) ;;
+          esac
+          ;;
     i386)
           # 32-bit Xen is dropped from 4.3 onwards
           case "$xenbranch" in
@@ -460,13 +478,14 @@ test_matrix_iterate () {
       *)          echo >&2 "kernkind ?  $kern"; exit 1 ;;
       esac
 
-      for dom0arch in i386 amd64 armhf; do
+      for dom0arch in i386 amd64 armhf arm64; do
 
         case ${xenarch}_${dom0arch} in
             amd64_amd64) ;;
             amd64_i386) ;;
             i386_i386) ;;
             armhf_armhf) ;;
+            arm64_arm64) ;;
             *) continue ;;
         esac
 
diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index c4abc43..37cec58 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -112,6 +112,7 @@ done
 # backports.
 case ${suite}_${arch} in
     wheezy_armhf) bpok=armmp; need_initramfs=y;;
+    jessie_arm64) bpok=arm64                  ;;
 esac
 if [ x$bpok != x ]; then
     bp="$sbase-backports"
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel