Pygrub backports

Pygrub backports

[Ian Jackson]

I think the following commits from master should be considered for
backport:

0c12e5b7427b4dfd2dfabf21f6b0e6e24bc8e864
tools/pygrub: Fix extlinux when /boot is a separate partition from /

d1b93ea2615bd789ee28901f1f1c05ffb319cb61
tools/pygrub: Make pygrub understand default entry in string format

4ee393f9d6528640c29a0554fdc6cb3e795fb6e8
pygrub: fix non-interactive parsing of grub1 config files

3b279811707dab4bab95c2e952e94ebf4d6badd9
pygrub: Fix regression from c/s d1b93ea, attempt 2


Existing Xen 4.4.1 as found in Ubuntu cannot parse the grub.cfg files
that Ubuntu itself generates, which was:

Reported-by: Owen Dunn <osd1000@cam.ac.uk>

Owen kindly tested pygrub from xen.git#master (merged with the
Debian/Ubuntu patchset, provided by me) and reports that it worked in
his setup.


Opinions ?

Ian.

Re: Pygrub backports

[Ian Campbell]

On Fri, 2015-02-27 at 12:29 +0000, Ian Jackson wrote:
> I think the following commits from master should be considered for
> backport:
> 
> 0c12e5b7427b4dfd2dfabf21f6b0e6e24bc8e864
> tools/pygrub: Fix extlinux when /boot is a separate partition from /
> 
> d1b93ea2615bd789ee28901f1f1c05ffb319cb61
> tools/pygrub: Make pygrub understand default entry in string format
> 
> 4ee393f9d6528640c29a0554fdc6cb3e795fb6e8
> pygrub: fix non-interactive parsing of grub1 config files
> 
> 3b279811707dab4bab95c2e952e94ebf4d6badd9
> pygrub: Fix regression from c/s d1b93ea, attempt 2
> 
> 
> Existing Xen 4.4.1 as found in Ubuntu cannot parse the grub.cfg files
> that Ubuntu itself generates, which was:
> 
> Reported-by: Owen Dunn <osd1000@cam.ac.uk>
> 
> Owen kindly tested pygrub from xen.git#master (merged with the
> Debian/Ubuntu patchset, provided by me) and reports that it worked in
> his setup.
> 
> 
> Opinions ?

Sounds good. If we could also get an example of the problematic grub.cfg
to be checked into xen.git/tools/pygrub/examples that would be handy
too.

Ian.

Re: Pygrub backports

[Ian Jackson]

Ian Campbell writes ("Re: Pygrub backports"):
> Sounds good. If we could also get an example of the problematic grub.cfg
> to be checked into xen.git/tools/pygrub/examples that would be handy
> too.

I have asked the reporter for a (suitably-laundered) copy and some
info about how it was generated.

Ian.

[PATCH] tools/pygrub: Add example from Ubuntu 14.04 LTS

[Ian Jackson]

(This grub.cfg does not work with Xen 4.4.1's pygrub, but does work
with the extra 4 patches which are in Xen 4.5.0.)

Contributed-by: Owen Dunn <osd1000@cam.ac.uk>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Owen Dunn <osd1000@cam.ac.uk>
---
 tools/pygrub/examples/ubuntu-14.04-lts.grub2 |  234 ++++++++++++++++++++++++++
 1 file changed, 234 insertions(+)
 create mode 100644 tools/pygrub/examples/ubuntu-14.04-lts.grub2

diff --git a/tools/pygrub/examples/ubuntu-14.04-lts.grub2 b/tools/pygrub/examples/ubuntu-14.04-lts.grub2
new file mode 100644
index 0000000..a14c8df
--- /dev/null
+++ b/tools/pygrub/examples/ubuntu-14.04-lts.grub2
@@ -0,0 +1,234 @@
+#
+# DO NOT EDIT THIS FILE
+#
+# It is automatically generated by grub-mkconfig using templates
+# from /etc/grub.d and settings from /etc/default/grub
+#
+
+### BEGIN /etc/grub.d/00_header ###
+if [ -s $prefix/grubenv ]; then
+  set have_grubenv=true
+  load_env
+fi
+if [ "${next_entry}" ] ; then
+   set default="${next_entry}"
+   set next_entry=
+   save_env next_entry
+   set boot_once=true
+else
+   set default="0"
+fi
+
+if [ x"${feature_menuentry_id}" = xy ]; then
+  menuentry_id_option="--id"
+else
+  menuentry_id_option=""
+fi
+
+export menuentry_id_option
+
+if [ "${prev_saved_entry}" ]; then
+  set saved_entry="${prev_saved_entry}"
+  save_env saved_entry
+  set prev_saved_entry=
+  save_env prev_saved_entry
+  set boot_once=true
+fi
+
+function savedefault {
+  if [ -z "${boot_once}" ]; then
+    saved_entry="${chosen}"
+    save_env saved_entry
+  fi
+}
+function recordfail {
+  set recordfail=1
+  if [ -n "${have_grubenv}" ]; then if [ -z "${boot_once}" ]; then save_env recordfail; fi; fi
+}
+function load_video {
+  if [ x$feature_all_video_module = xy ]; then
+    insmod all_video
+  else
+    insmod efi_gop
+    insmod efi_uga
+    insmod ieee1275_fb
+    insmod vbe
+    insmod vga
+    insmod video_bochs
+    insmod video_cirrus
+  fi
+}
+
+if [ x$feature_default_font_path = xy ] ; then
+   font=unicode
+else
+insmod part_msdos
+insmod lvm
+insmod ext2
+set root='lvmid/VFRfK8-JAgW-a2Rt-svO9-f06E-Frur-fzowWw/2zIHcW-s2DX-h7hm-V32p-6nz5-bH1A-fNVEWg'
+if [ x$feature_platform_search_hint = xy ]; then
+  search --no-floppy --fs-uuid --set=root --hint='lvmid/VFRfK8-JAgW-a2Rt-svO9-f06E-Frur-fzowWw/2zIHcW-s2DX-h7hm-V32p-6nz5-bH1A-fNVEWg'  c7a4b4ca-71da-4d03-b374-a6b76ebcfc90
+else
+  search --no-floppy --fs-uuid --set=root c7a4b4ca-71da-4d03-b374-a6b76ebcfc90
+fi
+    font="/usr/share/grub/unicode.pf2"
+fi
+
+if loadfont $font ; then
+  set gfxmode=auto
+  load_video
+  insmod gfxterm
+fi
+terminal_output gfxterm
+if [ "${recordfail}" = 1 ] ; then
+  set timeout=-1
+else
+  if [ x$feature_timeout_style = xy ] ; then
+    set timeout_style=menu
+    set timeout=5
+  # Fallback normal timeout code in case the timeout_style feature is
+  # unavailable.
+  else
+    set timeout=5
+  fi
+fi
+### END /etc/grub.d/00_header ###
+
+### BEGIN /etc/grub.d/05_debian_theme ###
+set menu_color_normal=white/black
+set menu_color_highlight=black/light-gray
+### END /etc/grub.d/05_debian_theme ###
+
+### BEGIN /etc/grub.d/10_linux ###
+function gfxmode {
+	set gfxpayload="${1}"
+	if [ "${1}" = "keep" ]; then
+		set vt_handoff=vt.handoff=7
+	else
+		set vt_handoff=
+	fi
+}
+if [ "${recordfail}" != 1 ]; then
+  if [ -e ${prefix}/gfxblacklist.txt ]; then
+    if hwmatch ${prefix}/gfxblacklist.txt 3; then
+      if [ ${match} = 0 ]; then
+        set linux_gfx_mode=keep
+      else
+        set linux_gfx_mode=text
+      fi
+    else
+      set linux_gfx_mode=text
+    fi
+  else
+    set linux_gfx_mode=keep
+  fi
+else
+  set linux_gfx_mode=text
+fi
+export linux_gfx_mode
+menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-simple-c7a4b4ca-71da-4d03-b374-a6b76ebcfc90' {
+	load_video
+	gfxmode $linux_gfx_mode
+	insmod gzio
+	insmod part_msdos
+	insmod ext2
+	if [ x$feature_platform_search_hint = xy ]; then
+	  search --no-floppy --fs-uuid --set=root  86ba9198-4319-4809-908d-6dbe6938b19a
+	else
+	  search --no-floppy --fs-uuid --set=root 86ba9198-4319-4809-908d-6dbe6938b19a
+	fi
+	linux	/vmlinuz-3.13.0-44-generic root=/dev/mapper/BoxenSys00-root ro biosdevname=0 quiet
+	initrd	/initrd.img-3.13.0-44-generic
+}
+submenu 'Advanced options for Ubuntu' --unrestricted $menuentry_id_option 'gnulinux-advanced-c7a4b4ca-71da-4d03-b374-a6b76ebcfc90' {
+	menuentry 'Ubuntu, with Linux 3.13.0-44-generic' --class ubuntu --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.13.0-44-generic-advanced-c7a4b4ca-71da-4d03-b374-a6b76ebcfc90' {
+		load_video
+		gfxmode $linux_gfx_mode
+		insmod gzio
+		insmod part_msdos
+		insmod ext2
+		if [ x$feature_platform_search_hint = xy ]; then
+		  search --no-floppy --fs-uuid --set=root  86ba9198-4319-4809-908d-6dbe6938b19a
+		else
+		  search --no-floppy --fs-uuid --set=root 86ba9198-4319-4809-908d-6dbe6938b19a
+		fi
+		echo	'Loading Linux 3.13.0-44-generic ...'
+		linux	/vmlinuz-3.13.0-44-generic root=/dev/mapper/BoxenSys00-root ro biosdevname=0 quiet
+		echo	'Loading initial ramdisk ...'
+		initrd	/initrd.img-3.13.0-44-generic
+	}
+	menuentry 'Ubuntu, with Linux 3.13.0-44-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.13.0-44-generic-recovery-c7a4b4ca-71da-4d03-b374-a6b76ebcfc90' {
+		load_video
+		insmod gzio
+		insmod part_msdos
+		insmod ext2
+		if [ x$feature_platform_search_hint = xy ]; then
+		  search --no-floppy --fs-uuid --set=root  86ba9198-4319-4809-908d-6dbe6938b19a
+		else
+		  search --no-floppy --fs-uuid --set=root 86ba9198-4319-4809-908d-6dbe6938b19a
+		fi
+		echo	'Loading Linux 3.13.0-44-generic ...'
+		linux	/vmlinuz-3.13.0-44-generic root=/dev/mapper/BoxenSys00-root ro recovery nomodeset biosdevname=0
+		echo	'Loading initial ramdisk ...'
+		initrd	/initrd.img-3.13.0-44-generic
+	}
+}
+
+### END /etc/grub.d/10_linux ###
+
+### BEGIN /etc/grub.d/20_linux_xen ###
+
+### END /etc/grub.d/20_linux_xen ###
+
+### BEGIN /etc/grub.d/20_memtest86+ ###
+menuentry 'Memory test (memtest86+)' {
+	insmod part_msdos
+	insmod ext2
+	if [ x$feature_platform_search_hint = xy ]; then
+	  search --no-floppy --fs-uuid --set=root  86ba9198-4319-4809-908d-6dbe6938b19a
+	else
+	  search --no-floppy --fs-uuid --set=root 86ba9198-4319-4809-908d-6dbe6938b19a
+	fi
+	knetbsd	/memtest86+.elf
+}
+menuentry 'Memory test (memtest86+, serial console 115200)' {
+	insmod part_msdos
+	insmod ext2
+	if [ x$feature_platform_search_hint = xy ]; then
+	  search --no-floppy --fs-uuid --set=root  86ba9198-4319-4809-908d-6dbe6938b19a
+	else
+	  search --no-floppy --fs-uuid --set=root 86ba9198-4319-4809-908d-6dbe6938b19a
+	fi
+	linux16	/memtest86+.bin console=ttyS0,115200n8
+}
+### END /etc/grub.d/20_memtest86+ ###
+
+### BEGIN /etc/grub.d/30_os-prober ###
+### END /etc/grub.d/30_os-prober ###
+
+### BEGIN /etc/grub.d/30_uefi-firmware ###
+### END /etc/grub.d/30_uefi-firmware ###
+
+### BEGIN /etc/grub.d/40_custom ###
+# This file provides an easy way to add custom menu entries.  Simply type the
+# menu entries you want to add after this comment.  Be careful not to change
+# the 'exec tail' line above.
+set superusers="root"
+password_pbkdf2 root grub.pbkdf2.sha512.
+### END /etc/grub.d/40_custom ###
+
+### BEGIN /etc/grub.d/40_custom.bakPW ###
+# This file provides an easy way to add custom menu entries.  Simply type the
+# menu entries you want to add after this comment.  Be careful not to change
+# the 'exec tail' line above.
+set superusers="root"
+password_pbkdf2 root dummy
+### END /etc/grub.d/40_custom.bakPW ###
+
+### BEGIN /etc/grub.d/41_custom ###
+if [ -f  ${config_directory}/custom.cfg ]; then
+  source ${config_directory}/custom.cfg
+elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
+  source $prefix/custom.cfg;
+fi
+### END /etc/grub.d/41_custom ###
-- 
1.7.10.4

Re: [PATCH] tools/pygrub: Add example from Ubuntu 14.04 LTS

[Ian Campbell]

On Fri, 2015-02-27 at 15:22 +0000, Ian Jackson wrote:
> (This grub.cfg does not work with Xen 4.4.1's pygrub, but does work
> with the extra 4 patches which are in Xen 4.5.0.)
> 
> Contributed-by: Owen Dunn <osd1000@cam.ac.uk>
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> CC: Owen Dunn <osd1000@cam.ac.uk>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [PATCH] tools/pygrub: Add example from Ubuntu 14.04 LTS

[Ian Campbell]

On Fri, 2015-02-27 at 15:26 +0000, Ian Campbell wrote:
> On Fri, 2015-02-27 at 15:22 +0000, Ian Jackson wrote:
> > (This grub.cfg does not work with Xen 4.4.1's pygrub, but does work
> > with the extra 4 patches which are in Xen 4.5.0.)
> > 
> > Contributed-by: Owen Dunn <osd1000@cam.ac.uk>
> > Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> > CC: Owen Dunn <osd1000@cam.ac.uk>
> 
> Acked-by: Ian Campbell <ian.campbell@citrix.com>

applied, thanks.

Re: Pygrub backports

[Jan Beulich]

>>> On 27.02.15 at 13:29, <Ian.Jackson@eu.citrix.com> wrote:
> I think the following commits from master should be considered for
> backport:

Looks reasonable. Question is - do you still want this for 4.4.2 or
only afterwards? If for it, then can these please go in before RC2
(which really is only pending a push on the branches)?

Jan

> 0c12e5b7427b4dfd2dfabf21f6b0e6e24bc8e864
> tools/pygrub: Fix extlinux when /boot is a separate partition from /
> 
> d1b93ea2615bd789ee28901f1f1c05ffb319cb61
> tools/pygrub: Make pygrub understand default entry in string format
> 
> 4ee393f9d6528640c29a0554fdc6cb3e795fb6e8
> pygrub: fix non-interactive parsing of grub1 config files
> 
> 3b279811707dab4bab95c2e952e94ebf4d6badd9
> pygrub: Fix regression from c/s d1b93ea, attempt 2
> 
> 
> Existing Xen 4.4.1 as found in Ubuntu cannot parse the grub.cfg files
> that Ubuntu itself generates, which was:
> 
> Reported-by: Owen Dunn <osd1000@cam.ac.uk>
> 
> Owen kindly tested pygrub from xen.git#master (merged with the
> Debian/Ubuntu patchset, provided by me) and reports that it worked in
> his setup.
> 
> 
> Opinions ?
> 
> Ian.

Re: Pygrub backports

[Ian Jackson]

Jan Beulich writes ("Re: Pygrub backports"):
> >>> On 27.02.15 at 13:29, <Ian.Jackson@eu.citrix.com> wrote:
> > I think the following commits from master should be considered for
> > backport:
> 
> Looks reasonable. Question is - do you still want this for 4.4.2 or
> only afterwards? If for it, then can these please go in before RC2
> (which really is only pending a push on the branches)?

Well, TBH I was kind of surprised that we hadn't queued these as
backports anyway.  Backporting pygrub improvements is important for
compatibility with newer guests.

So if you don't mind too much, can we have them in 4.4.2 ?  In which
case I would push them right away.

Ian.

Re: Pygrub backports

[Jan Beulich]

>>> On 27.02.15 at 15:41, <Ian.Jackson@eu.citrix.com> wrote:
> Jan Beulich writes ("Re: Pygrub backports"):
>> >>> On 27.02.15 at 13:29, <Ian.Jackson@eu.citrix.com> wrote:
>> > I think the following commits from master should be considered for
>> > backport:
>> 
>> Looks reasonable. Question is - do you still want this for 4.4.2 or
>> only afterwards? If for it, then can these please go in before RC2
>> (which really is only pending a push on the branches)?
> 
> Well, TBH I was kind of surprised that we hadn't queued these as
> backports anyway.  Backporting pygrub improvements is important for
> compatibility with newer guests.
> 
> So if you don't mind too much, can we have them in 4.4.2 ?  In which
> case I would push them right away.

Sure, go ahead.

Jan

Re: Pygrub backports

[Ian Jackson]

Jan Beulich writes ("Re: Pygrub backports"):
> On 27.02.15 at 15:41, <Ian.Jackson@eu.citrix.com> wrote:
> > So if you don't mind too much, can we have them in 4.4.2 ?  In which
> > case I would push them right away.
> 
> Sure, go ahead.

Thanks.  They were already in 4.5.  I have backported them all to 4.3
and 4.4.  There was one trivial conflict.

Ian.