[Qemu-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Qemu-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Gerd Hoffmann]

  Hi,

ipxe just got a EFI_PXE_BASE_CODE_PROTOCOL implementation.  I've updated
ipxe and commented the EFI_DOWNGRADE_UX define in the qemu config.  If
you want to test this you can fetch the qemu branch with the updated
roms here:

https://www.kraxel.org/cgit/qemu/log/?h=rebase/roms-next

Ran a quick test.  The good news are that shim.efi can now fetch files,
it happily chainloads grub.efi.  The bad news are that grub.efi has
trouble loading files.  It doesn't present the boot menu but greets me
with a prompt, which usually happens if grub fails to load the config
file.  So things have improved, but there still is something broken ...

cheers,
  Gerd

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Michael Brown]

On 07/09/15 11:14, Gerd Hoffmann wrote:
> ipxe just got a EFI_PXE_BASE_CODE_PROTOCOL implementation.  I've updated
> ipxe and commented the EFI_DOWNGRADE_UX define in the qemu config.  If
> you want to test this you can fetch the qemu branch with the updated
> roms here:
>
> https://www.kraxel.org/cgit/qemu/log/?h=rebase/roms-next
>
> Ran a quick test.  The good news are that shim.efi can now fetch files,
> it happily chainloads grub.efi.  The bad news are that grub.efi has
> trouble loading files.  It doesn't present the boot menu but greets me
> with a prompt, which usually happens if grub fails to load the config
> file.  So things have improved, but there still is something broken ...

Thank you for picking this up!

My local grub.efi (built from source) does nothing with 
EFI_PXE_BASE_CODE_PROTOCOL except retrieve the cached DHCP packets.  I 
confirmed that it was doing this by typing "net_ls_addr" within the GRUB 
shell, before instructing GRUB to attempt DHCP.  With iPXE's 
EFI_PXE_BASE_CODE_PROTOCOL enabled, grub.efi did show the correct IP 
address even before attempting DHCP; without iPXE's 
EFI_PXE_BASE_CODE_PROTOCOL, it did not.

I also verified that grub.efi was able to use the exposed 
EFI_SIMPLE_NETWORK_PROTOCOL NIC by typing "net_bootp" within the GRUB 
shell.  This was able to successfully obtain a new DHCP lease.

I find GRUB2 difficult to work with since it seems not to support manual 
creation of configuration files.  Do you have a known-working grub.efi 
configuration (DHCP options, configuration files, etc) that you could 
send me so I can reproduce the problem?

Thanks,

Michael

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Gerd Hoffmann]

  Hi,

> I find GRUB2 difficult to work with since it seems not to support manual 
> creation of configuration files.  Do you have a known-working grub.efi 
> configuration (DHCP options, configuration files, etc) that you could 
> send me so I can reproduce the problem?

You can get away with a pretty simple config file even though the ones
generated by grub2-mkconfig are horrible.  I have this one:

======================== cut here ==========================

set timeout=5

menuentry 'RHEL-7.1-20150219.1 Workstation' --class os {
        echo 'loading rhel/RHEL-7.1-20150219.1-Workstation/vmlinuz ...'
        linuxefi rhel/RHEL-7.1-20150219.1-Workstation/vmlinuz
inst.repo=http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os/
        echo 'loading
rhel/RHEL-7.1-20150219.1-Workstation/initrd.img ...'
        initrdefi rhel/RHEL-7.1-20150219.1-Workstation/initrd.img
        echo 'booting ...'
}

menuentry 'RHEL-7.1-20150219.1 Workstation (trying http)' --class os {
        echo 'loading
http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os//images/pxeboot/vmlinuz ...'
        linuxefi
http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os//images/pxeboot/vmlinuz inst.repo=http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os/
        echo 'loading
http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os//images/pxeboot/initrd.img ...'
        initrdefi
http://spunk.home.kraxel.org/mirror/rhel/7/RHEL-7.1-20150219.1/compose/Workstation/x86_64/os//images/pxeboot/initrd.img
        echo 'booting ...'
}

======================== cut here ==========================

This is generated by a script which does also copy the kernels to the
tftproot, there are more simliar entries for more RHEL versions but this
should be enough to get the idea ...

First entry actually works.  Second does not, but I'm hoping some day it
does.  Probably depends on how grub actually tries to fetches the files,
i.e. whenever the full http url is passed on to ipxe somehow so it can
do its magic.

The file is simply placed as "grub.cfg" in tftproot, next to grubx64.efi

cheers,
  Gerd

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Michael Brown]

On 07/09/15 15:58, Gerd Hoffmann wrote:
> This is generated by a script which does also copy the kernels to the
> tftproot, there are more simliar entries for more RHEL versions but this
> should be enough to get the idea ...
>
> First entry actually works.  Second does not, but I'm hoping some day it
> does.  Probably depends on how grub actually tries to fetches the files,
> i.e. whenever the full http url is passed on to ipxe somehow so it can
> do its magic.
>
> The file is simply placed as "grub.cfg" in tftproot, next to grubx64.efi

Thanks.  From my experiments, it seems that GRUB is using the hardcoded 
path /efi/boot/grub.cfg to locate the configuration file, regardless of 
where grub.efi was loaded from.

I can't immediately see where in the GRUB source this path gets derived.

Michael

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Gerd Hoffmann]

  Hi,

> > The file is simply placed as "grub.cfg" in tftproot, next to grubx64.efi
> 
> Thanks.  From my experiments, it seems that GRUB is using the hardcoded 
> path /efi/boot/grub.cfg to locate the configuration file, regardless of 
> where grub.efi was loaded from.

Hmm, probably depends on how you are building it.  There are also tools
like grub2-mknetdir and grub2-mkstandalone.  Pretty confusing :(

I'm using the binary shipped with RHEL.  The centos one should show the
same behavior.

https://ftp.fau.de/centos/7.1.1503/os/x86_64/EFI/BOOT/

There is a pretty readable grub.cfg file too btw.

Given there are no grub2 modules anywhere I suspect that binary was
created with grub2-mkstandalone.

HTH,
  Gerd

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Michael Brown]

On 10/09/15 16:20, Gerd Hoffmann wrote:
>> Thanks.  From my experiments, it seems that GRUB is using the hardcoded
>> path /efi/boot/grub.cfg to locate the configuration file, regardless of
>> where grub.efi was loaded from.
>
> Hmm, probably depends on how you are building it.  There are also tools
> like grub2-mknetdir and grub2-mkstandalone.  Pretty confusing :(
>
> I'm using the binary shipped with RHEL.  The centos one should show the
> same behavior.
>
> https://ftp.fau.de/centos/7.1.1503/os/x86_64/EFI/BOOT/

That grubx64.efi binary makes no attempt to even look for an 
EFI_PXE_BASE_CODE_PROTOCOL.  Here's the complete log of calls that it makes:

OpenProtocol ( 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, 
LoadedImage, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f1aac00 ) -> 0x3e90349b
LocateProtocol ( ConsoleControl, 0x0 ) = NOT_FOUND ( 0x0 ) -> 0x3e90344b
LocateProtocol ( ConsoleControl, 0x0 ) = NOT_FOUND ( 0x0 ) -> 0x3e90344b
OpenProtocol ( ConIn, SimpleTextInputEx, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3ed3ffc0 ) -> 0x3e90349b
LocateHandle ( ByProtocol, BlockIo, 0x0, 64 ) = 0 ( 32, { 
PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0), 
PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0), 
PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1), 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1) } ) -> 0x3e903478
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0), DevicePath, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f398f98 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0), BlockIo, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f3980a8 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0), DevicePath, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f372998 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0), BlockIo, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f35be68 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1), DevicePath, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f3726d8 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1), BlockIo, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f35c128 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1), 
DevicePath, PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, 
<NULL>, G ) = 0 ( 0x3ea75b48 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1), BlockIo, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3ea72e50 ) -> 0x3e90349b
LocateProtocol ( UgaDraw, 0x0 ) = NOT_FOUND ( 0x0 ) -> 0x3e90344b
LocateHandle ( ByProtocol, GraphicsOutput, 0x0, 64 ) = 0 ( 16, { 
PciRoot(0x0)/Pci(0x2,0x0)/AcpiAdr(0x80010100), ConOut } ) -> 0x3e903478
OpenProtocol ( PciRoot(0x0)/Pci(0x2,0x0)/AcpiAdr(0x80010100), 
GraphicsOutput, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f3750b8 ) -> 0x3e90349b
LocateHandle ( ByProtocol, SimpleNetwork, 0x0, 64 ) = 0 ( 8, { 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1) } ) -> 0x3e903478
OpenProtocol ( PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1), 
SimpleNetwork, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3ea755d0 ) -> 0x3e90349b
OpenProtocol ( 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, 
LoadedImage, 
PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
= 0 ( 0x3f1aac00 ) -> 0x3e90349b
OpenProtocol ( PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1), 
DevicePath, PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, 
<NULL>, G ) = 0 ( 0x3ea75b48 ) -> 0x3e90349b

It's looking for and successfully opening our 
EFI_SIMPLE_NETWORK_PROTOCOL and it is able to obtain an IP address using 
"net_bootp".  It doesn't look anywhere for EFI_PXE_BASE_CODE_PROTOCOL.

Do you have a set of instructions (starting from "git clone 
git://git.savannah.gnu.org/grub.git") which will allow me to rebuild 
that grubx64.efi binary from the source?

Thanks,

Michael

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Gerd Hoffmann]

> > https://ftp.fau.de/centos/7.1.1503/os/x86_64/EFI/BOOT/
> 
> That grubx64.efi binary makes no attempt to even look for an 
> EFI_PXE_BASE_CODE_PROTOCOL.  Here's the complete log of calls that it makes:

With grubx86.efi loaded via tftp I assume?

> OpenProtocol ( 
> PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, 
> LoadedImage, 
> PciRoot(0x0)/Pci(0x3,0x0)/MAC(525400123456,0x1)/grubx64.efi, <NULL>, G ) 
> = 0 ( 0x3f1aac00 ) -> 0x3e90349b

... yes, seems so.

The very same grub binary boots from network and disk, so I guess here
it tries to figure how it was loaded to then go fetch the grub.cfg file
from the same location.

On a successfull boot (with EFI_DOWNGRADE_UX) I see this in the ovmf
log:

[ ... lot of stuff snipped ... ]
Booting EFI Network
InstallProtocolInterface: 245DCA21-FB7B-11D3-8F01-00A0C969723B 6FD60E0
InstallProtocolInterface: 41D94CD2-35B6-455A-8258-D4E51334AADD 7073B20
InstallProtocolInterface: 3AD9DF29-4501-478D-B1F8-7F7FE70E50F3 7074BF8
InstallProtocolInterface: F4B427BB-BA21-4F16-BC4E-43E416AB619C 70798B0
InstallProtocolInterface: 245DCA21-FB7B-11D3-8F01-00A0C969723B 6FD60E0
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 70ADB40
Loading driver at 0x00006605000 EntryPoint=0x00006605400 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7094898
PixelBlueGreenRedReserved8BitPerColor

The guids translate to:

245DCA21-FB7B-11D3-8F01-00A0C969723B gEfiPxeBaseCodeCallbackProtocolGuid
41D94CD2-35B6-455A-8258-D4E51334AADD gEfiIp4ProtocolGuid
3AD9DF29-4501-478D-B1F8-7F7FE70E50F3 gEfiUdp4ProtocolGuid
F4B427BB-BA21-4F16-BC4E-43E416AB619C gEfiArpProtocolGuid
245DCA21-FB7B-11D3-8F01-00A0C969723B gEfiPxeBaseCodeCallbackProtocolGuid
5B1B31A1-9562-11D2-8E3F-00A0C969723B gEfiLoadedImageProtocolGuid
BC62157E-3E33-4FEC-9920-2D3B36D750DF
gEfiLoadedImageDevicePathProtocolGuid

Hmm, EfiPxeBaseCode*Callback*Protocol?

> Do you have a set of instructions (starting from "git clone 
> git://git.savannah.gnu.org/grub.git") which will allow me to rebuild 
> that grubx64.efi binary from the source?

The rpm spec file for grub2 is here:

https://git.centos.org/blob/rpms!
grub2/c695181b7bb165516116ba7c7d94805c799ae977/SPECS!grub2.spec

grubx64.efi is created with grub-mkimage:

<quote>
  ./grub-mkimage -O %{grubefiarch} -o %{grubefiname}.orig \
    -p /EFI/%{efidir} -d grub-core ${GRUB_MODULES}
</quote>

cheers,
  Gerd

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Gerd Hoffmann]

  Hi,

> The guids translate to:
> 
> 245DCA21-FB7B-11D3-8F01-00A0C969723B gEfiPxeBaseCodeCallbackProtocolGuid
> 41D94CD2-35B6-455A-8258-D4E51334AADD gEfiIp4ProtocolGuid
> 3AD9DF29-4501-478D-B1F8-7F7FE70E50F3 gEfiUdp4ProtocolGuid
> F4B427BB-BA21-4F16-BC4E-43E416AB619C gEfiArpProtocolGuid
> 245DCA21-FB7B-11D3-8F01-00A0C969723B gEfiPxeBaseCodeCallbackProtocolGuid
> 5B1B31A1-9562-11D2-8E3F-00A0C969723B gEfiLoadedImageProtocolGuid
> BC62157E-3E33-4FEC-9920-2D3B36D750DF
> gEfiLoadedImageDevicePathProtocolGuid
> 
> Hmm, EfiPxeBaseCode*Callback*Protocol?

Ping.  Any progress here?

Meanwhile I've noticed efi grub2 fails to load grub.cfg on microsoft
hyper-v too, so possibly this is something in grub not ipxe ...

cheers,
  Gerd

Re: [Qemu-devel] [ipxe-devel] EFI_PXE_BASE_CODE_PROTOCOL

[Michael Brown]

On 20/10/15 09:16, Gerd Hoffmann wrote:
>> Hmm, EfiPxeBaseCode*Callback*Protocol?
>
> Ping.  Any progress here?
>
> Meanwhile I've noticed efi grub2 fails to load grub.cfg on microsoft
> hyper-v too, so possibly this is something in grub not ipxe ...

Haven't had time to look at it yet, sorry.  It _is_ still on my 
things-to-be-fixed list.

Michael