Mac OS X entries don't work (fail or crash), part 1

Mac OS X entries don't work (fail or crash), part 1

[Chris Murphy]

Fedora 18, grub2-efi-2.00-13.fc18

When I choose any Mac OS X entry I get:

error: can't find command 'xnu_uuid' 
error: can't find command 'xnu_kernel64' 
error: can't find command 'xnu_kextdir'

Press any key to continue…

I already know how I can fix this, I'm inquiring on what exactly is wrong and how it needs to get fixed in Fedora so it works.

1. The Fedora installed grubx64.efi doesn't have the necessary xnu modules baked into it. Should it? Or is it better for it to depend on finding the modules and loading them on demand?

2. The grubx64.efi prefix is looking on the ESP in /EFI/fedora/x86_64-efi for the modules. Is this the correct location for modules on (U)EFI systems?

3. The Fedora grub2-efi package doesn't install any modules to either location (not to the ESP, not to /boot/grub2). Should it?

4. The Fedora installer, doesn't call grub2-install, to cause the modules to be installed anywhere. Should it?

Clearly either the grubx64.efi needs the xnu mod files baked in, OR the modules need to be installed by the grub2 package in a location where grubx64.efi can find them; OR the installer needs to call grub2-install to get them installed to a location where grubx64.efi can find them. I'm just not clear where they are supposed to go on a (U)EFI system.

The grub2-install help description for --boot-directory= says they should go in /boot/grub2, apparently even on (U)EFI systems, not on the ESP. If I use this to direct it to /boot/efi, they still end up in the wrong place because a grub2 folder is created.

When I run grub2-install by itself, the modules are installed to /boot/grub2/x86_64-efi, but I also get a message:

/boot/efi doesn't look like an EFI partition.

So grub2-install presumably wanted to do something on the ESP? What? I see nothing written.  Is the failure because Fedora is using an HFS+ volume as the EFI System partition, grub2-install then falls back to installing modules into /boot/grub2?

Results from grub2-install --debug:


[root@f18s ~]# grub2-install --debug
+ setup_verbose=--verbose
+ efi_quiet=-q
+ '[' -z '' ']'
+ bootdir=/boot
+ '[' -n '' ']'
++ echo /boot/grub2
++ sed 's,//*,/,g'
+ grubdir=/boot/grub2
+ device_map=/boot/grub2/device.map
+ '[' x86_64-efi = i386-pc ']'
+ '[' x86_64-efi = sparc64-ieee1275 ']'
+ set /usr/bin/grub2-mkimage dummy
+ test -f /usr/bin/grub2-mkimage
+ :
+ '[' xefi = xefi ']'
+ test -n ''
+ test -d /boot/efi
++ /usr/sbin/grub2-probe --target=device --device-map= /boot/efi
+ install_device=/dev/sda4
++ /usr/sbin/grub2-probe --target=device --device-map= /boot
+ test x/dev/sda4 '!=' x/dev/sda6
+ efidir=/boot/efi
+ test -n /boot/efi
++ /usr/sbin/grub2-probe --target=fs --device-map=/boot/grub2/device.map /boot/efi
+ efi_fs=hfsplus
+ test xhfsplus = xfat
+ gettext_printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
+ gettext_printf_format='%s doesn'\''t look like an EFI partition.\n'
+ shift
++ gettext '%s doesn'\''t look like an EFI partition.\n'
+ printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
/boot/efi doesn't look like an EFI partition.
+ efidir=
+ test -n ''
+ efidir=/boot/grub2
+ efi_distributor=
+ efi_file=grub.efi
+ mkdir -p /boot/grub2
+ mkdir -p /boot/grub2/x86_64-efi
+ test no = yes
+ test -f /boot/grub2/device.map
+ device_map=

I have previously fixed the missing modules part of this, but then get a kernel panic when choosing the OS X options. I'll discuss that part after I resolve where everything is supposed to go so I can file an appropriate Fedora bug, and the work around for where things should go in the future.


Chris Murphy

Re: Mac OS X entries don't work (fail or crash), part 1

[Andrey Borzenkov]

On Fri, Jan 25, 2013 at 10:00 AM, Chris Murphy <lists@colorremedies.com> wrote:
>
> Fedora 18, grub2-efi-2.00-13.fc18
>
> When I choose any Mac OS X entry I get:
>
> error: can't find command 'xnu_uuid'
> error: can't find command 'xnu_kernel64'
> error: can't find command 'xnu_kextdir'
>
> Press any key to continue…
>
> I already know how I can fix this, I'm inquiring on what exactly is wrong and how it needs to get fixed in Fedora so it works.
>

How it should be implemented in Fedora is probably better discussed
with Fedora developers. They obviously had some reasons to implement
it this way ...

> 1. The Fedora installed grubx64.efi doesn't have the necessary xnu modules baked into it. Should it? Or is it better for it to depend on finding the modules and loading them on demand?
>

grubx64.efi is what is known as core.img elsewhere. Normally core.img
includes just enough to be able to access ${prefix} directory.

Is grubx64.efi shipped with and installed by RPM or built dynamically?

> 2. The grubx64.efi prefix is looking on the ESP in /EFI/fedora/x86_64-efi for the modules. Is this the correct location for modules on (U)EFI systems?
>

At the end it is up to distribution to decide. grub-install right now
does not support it in this exact form.

> 3. The Fedora grub2-efi package doesn't install any modules to either location (not to the ESP, not to /boot/grub2). Should it?
>

Do you have /boot/grub2-efi? openSUSE 12.2 shipped two packages with
different default ${prefix}.

> 4. The Fedora installer, doesn't call grub2-install, to cause the modules to be installed anywhere. Should it?
>

Ask Fedora :)

> Clearly either the grubx64.efi needs the xnu mod files baked in, OR the modules need to be installed by the grub2 package in a location where grubx64.efi can find them; OR the installer needs to call grub2-install to get them installed to a location where grubx64.efi can find them. I'm just not clear where they are supposed to go on a (U)EFI system.
>

Well ... without secure boot it does not really matter. Having them in
ESP has marginal advantage of having everything in one place, but

- you need framework to manage multiple installations of the same OS.
They obviously cannot all go into the same place
- you cannot easily use software RAID to protect ${prefix}
- you cannot use encrypted partitions

With secure boot using ESP as ${prefix} allows you to ship signed
grubx64.efi and (with current support added to grub) signed modules
and do not modify grubx64.efi to encode ${prefix}. Of course, you
still need to somehow sign grub.cfg which is generated on end system.
Which needs secure way to tell grub to use end user key ... which
probably means putting it somewhere into firmware boot-ony variable.

Without support for signature verification in grub your only option is
to disable module loading in which case it does not matter what prefix
is set to :)

> The grub2-install help description for --boot-directory= says they should go in /boot/grub2, apparently even on (U)EFI systems, not on the ESP. If I use this to direct it to /boot/efi, they still end up in the wrong place because a grub2 folder is created.
>

I think using /EFI/fedora/grub(2)/x86_64-efi in this case would be
better and compatible with current grub-install. After all, there
could be multiple bootladers installed in ESP as well.

> When I run grub2-install by itself, the modules are installed to /boot/grub2/x86_64-efi, but I also get a message:
>
> /boot/efi doesn't look like an EFI partition.
>
> So grub2-install presumably wanted to do something on the ESP? What?

Surprise - install grubx64.efi :)

> I see nothing written.  Is the failure because Fedora is using an HFS+ volume as the EFI System partition, grub2-install then falls back to installing modules into /boot/grub2?
>

Currently grub is always installing modules in /boot/grub (or
${bootdir}/${grubdirname} to be sure). Only core.img a.k.a.
grubx64.efi goes into ESP.


> + test -n /boot/efi
> ++ /usr/sbin/grub2-probe --target=fs --device-map=/boot/grub2/device.map /boot/efi
> + efi_fs=hfsplus
> + test xhfsplus = xfat
> + gettext_printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
> + gettext_printf_format='%s doesn'\''t look like an EFI partition.\n'
> + shift
> ++ gettext '%s doesn'\''t look like an EFI partition.\n'
> + printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
> /boot/efi doesn't look like an EFI partition.
> + efidir=

ESP is by definition FAT. Firmware is free to support additional
filesystems but ESP is the only partition that is guaranteed to be
supported by firmware. I do believe that Mac can boot off HFS+ though.

Re: Mac OS X entries don't work (fail or crash), part 1

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 4854 bytes --]

On 25.01.2013 07:00, Chris Murphy wrote:

> 
> Fedora 18, grub2-efi-2.00-13.fc18
> 
> When I choose any Mac OS X entry I get:
> 
> error: can't find command 'xnu_uuid' 
> error: can't find command 'xnu_kernel64' 
> error: can't find command 'xnu_kextdir'
> 
> Press any key to continue…
> 
> I already know how I can fix this, I'm inquiring on what exactly is wrong and how it needs to get fixed in Fedora so it works.
> 
> 1. The Fedora installed grubx64.efi doesn't have the necessary xnu modules baked into it. Should it? Or is it better for it to depend on finding the modules and loading them on demand?
> 
> 2. The grubx64.efi prefix is looking on the ESP in /EFI/fedora/x86_64-efi for the modules. Is this the correct location for modules on (U)EFI systems?
> 
> 3. The Fedora grub2-efi package doesn't install any modules to either location (not to the ESP, not to /boot/grub2). Should it?
> 
> 4. The Fedora installer, doesn't call grub2-install, to cause the modules to be installed anywhere. Should it?
> 

GRUB supports only modules under $prefix/$cpu-$platform which can be on
memdisk. It seems that some people start shuffling it around using
custom stuff. It's not supported by upstream and if it fails like in
your case we don't care. Unless you use one of official ways
(grub-install, grub-mknetdir, grub-mkrescue or grub-mkstandalone) you
have an unsupported config which may fail in a number of ways but it's
not my problem. Using unofficial install way = no support. I haven't
seen any valid reason to use any other install (except when preparing
coreboot or loongson ROM image). If there are valid reasons a
corresponding install tool can be added (like it was the case of
grub-mknetdir).
Use grub-install to reinstall properly.

> Clearly either the grubx64.efi needs the xnu mod files baked in, OR the modules need to be installed by the grub2 package in a location where grubx64.efi can find them; OR the installer needs to call grub2-install to get them installed to a location where grubx64.efi can find them. I'm just not clear where they are supposed to go on a (U)EFI system.
> 
> The grub2-install help description for --boot-directory= says they should go in /boot/grub2, apparently even on (U)EFI systems, not on the ESP. If I use this to direct it to /boot/efi, they still end up in the wrong place because a grub2 folder is created.
> 
> When I run grub2-install by itself, the modules are installed to /boot/grub2/x86_64-efi, but I also get a message:
> 
> /boot/efi doesn't look like an EFI partition.
> 
> So grub2-install presumably wanted to do something on the ESP? What? I see nothing written.  Is the failure because Fedora is using an HFS+ volume as the EFI System partition, grub2-install then falls back to installing modules into /boot/grub2?
> 
> Results from grub2-install --debug:
> 
> 
> [root@f18s ~]# grub2-install --debug
> + setup_verbose=--verbose
> + efi_quiet=-q
> + '[' -z '' ']'
> + bootdir=/boot
> + '[' -n '' ']'
> ++ echo /boot/grub2
> ++ sed 's,//*,/,g'
> + grubdir=/boot/grub2
> + device_map=/boot/grub2/device.map
> + '[' x86_64-efi = i386-pc ']'
> + '[' x86_64-efi = sparc64-ieee1275 ']'
> + set /usr/bin/grub2-mkimage dummy
> + test -f /usr/bin/grub2-mkimage
> + :
> + '[' xefi = xefi ']'
> + test -n ''
> + test -d /boot/efi
> ++ /usr/sbin/grub2-probe --target=device --device-map= /boot/efi
> + install_device=/dev/sda4
> ++ /usr/sbin/grub2-probe --target=device --device-map= /boot
> + test x/dev/sda4 '!=' x/dev/sda6
> + efidir=/boot/efi
> + test -n /boot/efi
> ++ /usr/sbin/grub2-probe --target=fs --device-map=/boot/grub2/device.map /boot/efi
> + efi_fs=hfsplus
> + test xhfsplus = xfat
> + gettext_printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
> + gettext_printf_format='%s doesn'\''t look like an EFI partition.\n'
> + shift
> ++ gettext '%s doesn'\''t look like an EFI partition.\n'
> + printf '%s doesn'\''t look like an EFI partition.\n' /boot/efi
> /boot/efi doesn't look like an EFI partition.
> + efidir=
> + test -n ''
> + efidir=/boot/grub2
> + efi_distributor=
> + efi_file=grub.efi
> + mkdir -p /boot/grub2
> + mkdir -p /boot/grub2/x86_64-efi
> + test no = yes
> + test -f /boot/grub2/device.map
> + device_map=
> 
> I have previously fixed the missing modules part of this, but then get a kernel panic when choosing the OS X options. I'll discuss that part after I resolve where everything is supposed to go so I can file an appropriate Fedora bug, and the work around for where things should go in the future.
> 
> 
> Chris Murphy
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
> 



-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Mac OS X entries don't work (fail or crash), part 1

[Chris Murphy]

On Jan 25, 2013, at 1:40 AM, Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com> wrote:

> 
> GRUB supports only modules under $prefix/$cpu-$platform which can be on
> memdisk.

I do not understand $prefix. I do understand $cpu-platform. I don't understand memdisk.

If the grubx64.efi prefix points to the actual location where modules are stored, is this supported?

> It seems that some people start shuffling it around using
> custom stuff. It's not supported by upstream and if it fails like in
> your case we don't care.

I'm not asking for upstream support. I'm asking to understand design and intent, so I can properly communicate the bug to Fedora, and to users so the problem can be fixed. Not just for me.

I totally understand that a grubx64.efi that doesn't contain the modules called for in grub.cfg, and points to ESP//EFI/fedora, which doesn't contain an x86_64-efi folder containing GRUB modules, isn't going to work. And I know how to fix this for myself, but just because I can get it to work for me, doesn't mean that's the recommended best practice downstream.

So, should the modules be stored on the ESP? Or should they be installed in /boot/grub, just as with BIOS computers? Or does it not matter?


> Unless you use one of official ways
> (grub-install, grub-mknetdir, grub-mkrescue or grub-mkstandalone) you
> have an unsupported config which may fail in a number of ways but it's
> not my problem. Using unofficial install way = no support. I haven't
> seen any valid reason to use any other install (except when preparing
> coreboot or loongson ROM image). If there are valid reasons a
> corresponding install tool can be added (like it was the case of
> grub-mknetdir).
> Use grub-install to reinstall properly.

My understanding is they do this because grubx64.efi is signed, so they include one prebaked and signed grubx64.efi for everyone. But it appears to be using an improper prefix to search for additional GRUB modules, thus even if they were installed, they wouldn't be found. So that may be the Fedora bug? I've filed it here:

https://bugzilla.redhat.com/show_bug.cgi?id=903937

But even if we aren't talking about Fedora custom stuff, there is a problem for Macs and grub-install. The problem is that for Macs the grub-install result is not persistent. The grubx64.efi is not found by the Mac EFI boot manager, or the OS X Startup Disk panel. Once the user clears NVRAM, or chooses to boot OS X from the Startup Disk panel, it's no longer possible to find and thus load grubx64.efi.

For this to work persistently on Macs, the grubx64.efi needs to be installed on a JHFS+ volume which contains:

1. a dummy mach_kernel file
2. the grubx64.efi needs to be installed to /System/Library/CoreServices/boot.efi
3. a SystemVersion.plist file in the same location as the boot.efi, with at least a ProductName field, so that the boot option has a recognizable name.

So basically, if you want to support booting on Macs, then there are special requirements because they aren't exactly doing things strictly the UEFI way.

Further, grub-install doesn't like JHFS+ ESPs, and refuses to install a grubx64.efi file if the ESP is not FAT32. I just tried replacing the JHFS+ ESP with a FAT32 ESP, and I no longer get the error, and a new grubx64.efi is installed, whereas with a JHFS+ ESP this is not the case.

So across the board, this isn't working for EFI Macs out of the box.

Chris Murphy

Re: Mac OS X entries don't work (fail or crash), part 2

[Chris Murphy]

OK so now for the kernel panic problem. I've fixed the modules not found problem by eliminating the JHFS+ ESP, and using the correct FAT32 ESP. After mounting that, grub-install creates a new grubx64.efi which points to a location that contains all GRUB modules. I've also used grub-mkconfig to produce a grub.cfg.

When I choose either the OS X 32-bit or 64-bit options, I get a pause for 10 seconds with a black screen, and then a kernel panic.

The grub.cfg is here:

https://dl.dropbox.com/u/3253801/grub.cfg

The kernel panic photo is here:

https://dl.dropbox.com/u/3253801/IMG_2220.jpg

Chris Murphy