Re: [PATCH v2 00/23] x86: multiboot2 protocol support

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[PGNet Dev]

Re:

	> The final goal is xen.efi binary file which could be loaded by EFI
	> loader, multiboot (v1) protocol (only on legacy BIOS platforms) and
	> multiboot2 protocol. This way we will have:
	>   - smaller Xen code base,
	>   - one code base for xen.gz and xen.efi,
	>   - one build method for xen.gz and xen.efi;
	>     xen.efi will be extracted from xen file
	>     using objcopy; PE header will be contained
	>     in ELF file and will precede Xen code,
	>   - xen.efi build will not so strongly depend
	>     on a given GCC and binutils version.


As of v220, systemd provides systemd-boot (formerly gummiboot)

     https://wiki.archlinux.org/index.php/Boot_loaders#systemd-boot
     https://wiki.archlinux.org/index.php/Systemd-boot#EFI_boot

and is apparently capable of booting EFISTUB kernels.

Will the aforementioned goals enable direct boot using systemd-boot on 
EFI platforms ?  Or is it possible even now?

As my current test distro is Opensuse, with systemd v < 220, I've not 
yet tested.

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[Daniel Kiper]

On Fri, Oct 30, 2015 at 08:44:53AM -0700, PGNet Dev wrote:
> Re:
>
> 	> The final goal is xen.efi binary file which could be loaded by EFI
> 	> loader, multiboot (v1) protocol (only on legacy BIOS platforms) and
> 	> multiboot2 protocol. This way we will have:
> 	>   - smaller Xen code base,
> 	>   - one code base for xen.gz and xen.efi,
> 	>   - one build method for xen.gz and xen.efi;
> 	>     xen.efi will be extracted from xen file
> 	>     using objcopy; PE header will be contained
> 	>     in ELF file and will precede Xen code,
> 	>   - xen.efi build will not so strongly depend
> 	>     on a given GCC and binutils version.
>
>
> As of v220, systemd provides systemd-boot (formerly gummiboot)
>
>     https://wiki.archlinux.org/index.php/Boot_loaders#systemd-boot
>     https://wiki.archlinux.org/index.php/Systemd-boot#EFI_boot
>
> and is apparently capable of booting EFISTUB kernels.
>
> Will the aforementioned goals enable direct boot using systemd-boot
> on EFI platforms ?  Or is it possible even now?

If it supports multiboot2 with my extensions then it should work.
However, I do not think it is true because all is in development
state and nothing is set in stone. I am going to release next version
of my patches in November. I hope then we will be able to establish
main things and maybe it will be later sucked by systemd-boot. I think
that you should ask systemd-boot guys what they think about that.

Daniel

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[PGNet Dev]

On 10/30/2015 10:29 AM, Daniel Kiper wrote:
>> Will the aforementioned goals enable direct boot using systemd-boot
>> on EFI platforms ?  Or is it possible even now?
>
> If it supports multiboot2 with my extensions then it should work.
> However, I do not think it is true because all is in development
> state and nothing is set in stone. I am going to release next version
> of my patches in November. I hope then we will be able to establish
> main things and maybe it will be later sucked by systemd-boot. I think
> that you should ask systemd-boot guys what they think about that.

fwiw, from #systemd IRC

	Q: does/will systemd-boot work to boot Xen Dom0?
	A: systemd-boot will boot anything that's an EFI program
	Q: that's to mean, any EFI executable on the EFI partition?
	A: yes. apparently that includes Xen itself, according to
	   https://wiki.archlinux.org/index.php/Xen#With_UEFI_support
	   "cp /usr/lib/efi/xen-4.4.0.efi /boot"
	   that's not the dom0 however, that's just the hypervisor
	   so the question is, how the hypervisor loads dom0
	   ...
	   if you copy xen-x.y.z.efi as /boot/EFI/Boot/BOOTX64.EFI it'll be 
booted by UEFI by default
	   you don't even need gummiboot/sd-boot
	   or you could `efibootmgr` it into your native UEFI boot menu
	   I never used xen, but it seems to read xen.cfg and load dom0 itself, 
which is just fine


which, IIUC, is "probably should ... sort of".

I suspect wait and see is called for.

I'm hoping the Xen-on-UEFI work you're doing helps to smooth out the 
boot process.

Thanks.

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[Daniel Kiper]

On Fri, Oct 30, 2015 at 11:06:53AM -0700, PGNet Dev wrote:
> On 10/30/2015 10:29 AM, Daniel Kiper wrote:
> >>Will the aforementioned goals enable direct boot using systemd-boot
> >>on EFI platforms ?  Or is it possible even now?
> >
> >If it supports multiboot2 with my extensions then it should work.
> >However, I do not think it is true because all is in development
> >state and nothing is set in stone. I am going to release next version
> >of my patches in November. I hope then we will be able to establish
> >main things and maybe it will be later sucked by systemd-boot. I think
> >that you should ask systemd-boot guys what they think about that.
>
> fwiw, from #systemd IRC
>
> 	Q: does/will systemd-boot work to boot Xen Dom0?
> 	A: systemd-boot will boot anything that's an EFI program
> 	Q: that's to mean, any EFI executable on the EFI partition?
> 	A: yes. apparently that includes Xen itself, according to
> 	   https://wiki.archlinux.org/index.php/Xen#With_UEFI_support
> 	   "cp /usr/lib/efi/xen-4.4.0.efi /boot"
> 	   that's not the dom0 however, that's just the hypervisor
> 	   so the question is, how the hypervisor loads dom0
> 	   ...
> 	   if you copy xen-x.y.z.efi as /boot/EFI/Boot/BOOTX64.EFI it'll be
> booted by UEFI by default
> 	   you don't even need gummiboot/sd-boot
> 	   or you could `efibootmgr` it into your native UEFI boot menu
> 	   I never used xen, but it seems to read xen.cfg and load dom0
> itself, which is just fine

This is correct. I missed that you are talking about PE executable. However,
as it was said you must create xen.cfg in advance and put it in the same
directory with xen.efi. Please check xen/docs/misc/efi.markdown for more details.

> which, IIUC, is "probably should ... sort of".
>
> I suspect wait and see is called for.
>
> I'm hoping the Xen-on-UEFI work you're doing helps to smooth out the
> boot process.

I am too.

Daniel

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[PGNet Dev]

On 10/30/2015 11:23 AM, Daniel Kiper wrote:
> I missed that you are talking about PE executable. However,
> as it was said you must create xen.cfg in advance and put it in the same
> directory with xen.efi. Please check xen/docs/misc/efi.markdown for more details.

TBH I'm not clear on whether the executable is -- or more importantly, 
still will be -- needed.

Atm, I'm manually creating the xen.cfg/xen.ini and chainloading via 
GRUB2.  Hence the mess, and asking about simpler approaches.

IIUC, systemd-boot is a rename (fork?) of gummiboot.  WAS going to try 
it awhile ago, as a simpler alternative to Grub2, but saw only that the 
project had been emptied out.  Didn't realize that it'd been subsumed.

Interestingly, from https://en.wikipedia.org/wiki/Gummiboot_(software)

	"Red Hat's Fedora Project does not use gummiboot for booting UEFI 
systems; instead, it will use efilinux to chainload GRUB."

Trying to find out the WHY in the different boot-Xen-on-UEFI approaches, 
let alone simply what just works, is a challenge.

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[Daniel Kiper]

On Fri, Oct 30, 2015 at 11:36:03AM -0700, PGNet Dev wrote:
> On 10/30/2015 11:23 AM, Daniel Kiper wrote:
> >I missed that you are talking about PE executable. However,
> >as it was said you must create xen.cfg in advance and put it in the same
> >directory with xen.efi. Please check xen/docs/misc/efi.markdown for more details.
>
> TBH I'm not clear on whether the executable is -- or more
> importantly, still will be -- needed.

I am not sure what do you mean here.

> Atm, I'm manually creating the xen.cfg/xen.ini and chainloading via
> GRUB2.  Hence the mess, and asking about simpler approaches.

multiboot2 (on which I am working on) is solution for your problem.

Daniel

[PATCH v2 00/23] x86: multiboot2 protocol support

[Daniel Kiper]

Hi,

I am sending, long awaited, second version of multiboot2 protocol
support for legacy BIOS and EFI platforms. It fixes all major issues
discovered until now. There are still some minor problems which should
be fixed in one way or another. I will address them in next releases.

This series, in general, is not targeted to Xen 4.6. However, there are
some fixes at the beginning of it which are worth considering, I think.

The final goal is xen.efi binary file which could be loaded by EFI
loader, multiboot (v1) protocol (only on legacy BIOS platforms) and
multiboot2 protocol. This way we will have:
  - smaller Xen code base,
  - one code base for xen.gz and xen.efi,
  - one build method for xen.gz and xen.efi;
    xen.efi will be extracted from xen file
    using objcopy; PE header will be contained
    in ELF file and will precede Xen code,
  - xen.efi build will not so strongly depend
    on a given GCC and binutils version.

ARM guys should check at least patches #9 - #18 and #20. In general
earlier mentioned patches touches common EFI code but they are not
change functionality significantly.

GRUB2 patch series will follow this patch series.

GRUB2 guys should check patches #20 and #23 but I am sending to you
all Xen related patches just in case.

If you are not interested in this patch series at all please drop me
a line and I will remove you from distribution list.

Daniel

 .gitignore                        |    5 +-
 xen/arch/x86/Makefile             |   21 ++--
 xen/arch/x86/Rules.mk             |    4 +
 xen/arch/x86/boot/Makefile        |   10 +-
 xen/arch/x86/boot/build32.mk      |    4 +-
 xen/arch/x86/boot/cmdline.S       |  367 ---------------------------------------------------------------
 xen/arch/x86/boot/cmdline.c       |  396 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/x86/boot/edd.S           |    3 -
 xen/arch/x86/boot/head.S          |  474 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
 xen/arch/x86/boot/reloc.c         |  242 +++++++++++++++++++++++++++++++++++-------
 xen/arch/x86/boot/trampoline.S    |   25 ++++-
 xen/arch/x86/boot/video.S         |    6 --
 xen/arch/x86/boot/wakeup.S        |    6 +-
 xen/arch/x86/boot/x86_64.S        |   34 +++---
 xen/arch/x86/dmi_scan.c           |    4 +-
 xen/arch/x86/domain_page.c        |    2 +-
 xen/arch/x86/efi/Makefile         |   16 +--
 xen/arch/x86/efi/efi-boot.h       |   68 ++++++++++--
 xen/arch/x86/efi/stub.c           |   16 ++-
 xen/arch/x86/mm.c                 |    3 +-
 xen/arch/x86/mpparse.c            |    4 +-
 xen/arch/x86/setup.c              |   50 ++++-----
 xen/arch/x86/shutdown.c           |    2 +-
 xen/arch/x86/time.c               |    2 +-
 xen/arch/x86/x86_64/asm-offsets.c |   10 ++
 xen/arch/x86/x86_64/mm.c          |    2 +-
 xen/arch/x86/xen.lds.S            |    6 +-
 xen/common/efi/boot.c             |  461 ++++++++++++++++++++++++++++++++++++++++++++++---------------------------------
 xen/common/efi/runtime.c          |   23 ++--
 xen/drivers/acpi/osl.c            |    2 +-
 xen/include/asm-x86/config.h      |    3 +
 xen/include/asm-x86/page.h        |    2 +-
 xen/include/xen/efi.h             |   17 ++-
 xen/include/xen/multiboot2.h      |  182 ++++++++++++++++++++++++++++++++
 34 files changed, 1709 insertions(+), 763 deletions(-)

Daniel Kiper (23):
      x86/boot: remove unneeded instruction
      x86/boot: copy only text section from *.lnk file to *.bin file
      x86: zero BSS using stosl instead of stosb
      x86/boot: call reloc() using cdecl calling convention
      x86/boot/reloc: create generic alloc and copy functions
      x86/boot: use %ecx instead of %eax
      x86/boot/reloc: Rename some variables and rearrange code a bit
      x86: add multiboot2 protocol support
      efi: create efi_enabled()
      efi: build xen.gz with EFI code
      efi: split out efi_init()
      efi: split out efi_console_set_mode()
      efi: split out efi_get_gop()
      efi: split out efi_find_gop_mode()
      efi: split out efi_tables()
      efi: split out efi_variables()
      efi: split out efi_set_gop_mode()
      efi: split out efi_exit_boot()
      x86/efi: create new early memory allocator
      x86: add multiboot2 protocol support for EFI platforms
      x86/boot: implement early command line parser in C
      x86: make Xen early boot code relocatable
      x86: add multiboot2 protocol support for relocatable images

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[Jan Beulich]

>>> On 20.07.15 at 16:28, <daniel.kiper@oracle.com> wrote:
> This series, in general, is not targeted to Xen 4.6. However, there are
> some fixes at the beginning of it which are worth considering, I think.

I looked at the first few, and didn't spot any fixes. If you meant
just cleanup or other cosmetic adjustments, then I don't view them
as candidates for going in now. If I overlooked some aspect, then
please be more precise.

Jan

Re: [PATCH v2 00/23] x86: multiboot2 protocol support

[Jan Beulich]

>>> On 20.07.15 at 16:28, <daniel.kiper@oracle.com> wrote:
> This series, in general, is not targeted to Xen 4.6. However, there are
> some fixes at the beginning of it which are worth considering, I think.

I looked at the first few, and didn't spot any fixes. If you meant
just cleanup or other cosmetic adjustments, then I don't view them
as candidates for going in now. If I overlooked some aspect, then
please be more precise.

Jan

[PATCH v2 00/23] x86: multiboot2 protocol support

[Daniel Kiper]

Hi,

I am sending, long awaited, second version of multiboot2 protocol
support for legacy BIOS and EFI platforms. It fixes all major issues
discovered until now. There are still some minor problems which should
be fixed in one way or another. I will address them in next releases.

This series, in general, is not targeted to Xen 4.6. However, there are
some fixes at the beginning of it which are worth considering, I think.

The final goal is xen.efi binary file which could be loaded by EFI
loader, multiboot (v1) protocol (only on legacy BIOS platforms) and
multiboot2 protocol. This way we will have:
  - smaller Xen code base,
  - one code base for xen.gz and xen.efi,
  - one build method for xen.gz and xen.efi;
    xen.efi will be extracted from xen file
    using objcopy; PE header will be contained
    in ELF file and will precede Xen code,
  - xen.efi build will not so strongly depend
    on a given GCC and binutils version.

ARM guys should check at least patches #9 - #18 and #20. In general
earlier mentioned patches touches common EFI code but they are not
change functionality significantly.

GRUB2 patch series will follow this patch series.

GRUB2 guys should check patches #20 and #23 but I am sending to you
all Xen related patches just in case.

If you are not interested in this patch series at all please drop me
a line and I will remove you from distribution list.

Daniel

 .gitignore                        |    5 +-
 xen/arch/x86/Makefile             |   21 ++--
 xen/arch/x86/Rules.mk             |    4 +
 xen/arch/x86/boot/Makefile        |   10 +-
 xen/arch/x86/boot/build32.mk      |    4 +-
 xen/arch/x86/boot/cmdline.S       |  367 ---------------------------------------------------------------
 xen/arch/x86/boot/cmdline.c       |  396 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/x86/boot/edd.S           |    3 -
 xen/arch/x86/boot/head.S          |  474 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
 xen/arch/x86/boot/reloc.c         |  242 +++++++++++++++++++++++++++++++++++-------
 xen/arch/x86/boot/trampoline.S    |   25 ++++-
 xen/arch/x86/boot/video.S         |    6 --
 xen/arch/x86/boot/wakeup.S        |    6 +-
 xen/arch/x86/boot/x86_64.S        |   34 +++---
 xen/arch/x86/dmi_scan.c           |    4 +-
 xen/arch/x86/domain_page.c        |    2 +-
 xen/arch/x86/efi/Makefile         |   16 +--
 xen/arch/x86/efi/efi-boot.h       |   68 ++++++++++--
 xen/arch/x86/efi/stub.c           |   16 ++-
 xen/arch/x86/mm.c                 |    3 +-
 xen/arch/x86/mpparse.c            |    4 +-
 xen/arch/x86/setup.c              |   50 ++++-----
 xen/arch/x86/shutdown.c           |    2 +-
 xen/arch/x86/time.c               |    2 +-
 xen/arch/x86/x86_64/asm-offsets.c |   10 ++
 xen/arch/x86/x86_64/mm.c          |    2 +-
 xen/arch/x86/xen.lds.S            |    6 +-
 xen/common/efi/boot.c             |  461 ++++++++++++++++++++++++++++++++++++++++++++++---------------------------------
 xen/common/efi/runtime.c          |   23 ++--
 xen/drivers/acpi/osl.c            |    2 +-
 xen/include/asm-x86/config.h      |    3 +
 xen/include/asm-x86/page.h        |    2 +-
 xen/include/xen/efi.h             |   17 ++-
 xen/include/xen/multiboot2.h      |  182 ++++++++++++++++++++++++++++++++
 34 files changed, 1709 insertions(+), 763 deletions(-)

Daniel Kiper (23):
      x86/boot: remove unneeded instruction
      x86/boot: copy only text section from *.lnk file to *.bin file
      x86: zero BSS using stosl instead of stosb
      x86/boot: call reloc() using cdecl calling convention
      x86/boot/reloc: create generic alloc and copy functions
      x86/boot: use %ecx instead of %eax
      x86/boot/reloc: Rename some variables and rearrange code a bit
      x86: add multiboot2 protocol support
      efi: create efi_enabled()
      efi: build xen.gz with EFI code
      efi: split out efi_init()
      efi: split out efi_console_set_mode()
      efi: split out efi_get_gop()
      efi: split out efi_find_gop_mode()
      efi: split out efi_tables()
      efi: split out efi_variables()
      efi: split out efi_set_gop_mode()
      efi: split out efi_exit_boot()
      x86/efi: create new early memory allocator
      x86: add multiboot2 protocol support for EFI platforms
      x86/boot: implement early command line parser in C
      x86: make Xen early boot code relocatable
      x86: add multiboot2 protocol support for relocatable images