How does Grub find itself being booted from UEFI CD ElTorito image ?

How does Grub find itself being booted from UEFI CD ElTorito image ?

[adrian15sgd]

I actually have an specific question for Grub developers / maintainers / 
contributors so if you are one of them you can skip to: (2) .

1) INTRODUCTION

1.1) I am working on liveid ( https://github.com/rescatux/liveid ), this 
is sort of an earmark for live cds.
I am currently trying to push some changes onto the live-build and 
live-boot packages from Debian GNU/Linux distribution.

I was quite convinced I knew how Grub booted from a Debian's ElTorito 
image bundled into a Debian Live CD but as it turns out I was wrong.


1.2) The current way this is implemented from a Debian point of view 
(based on GRUB 2.06) can be described in two steps.
1.2.1) The first step is creating a gcdx64.efi file that has an embedded 
memdisk with a custom grub.cfg.

So... the gcdx64.efi image is created thanks to a custom Debian script 
named `build-efi-images` .

A memdisk image is being created first:
( 
https://salsa.debian.org/grub-team/grub/-/blob/c5fc69927aefefb713d43a3753a7e7062c20ffa9/debian/build-efi-images#L57-76 
)
and then it's being used on the final gcdx64.efi file:
( 
https://salsa.debian.org/grub-team/grub/-/blob/c5fc69927aefefb713d43a3753a7e7062c20ffa9/debian/build-efi-images#L217-227 
)
.

Please note that the embedded grub.cfg contents are:
```
if [ -z "\$prefix" -o ! -e "\$prefix" ]; then
     if ! search --file --set=root /.disk/info; then
         search --file --set=root /.disk/mini-info
     fi
     set prefix=(\$root)/boot/grub
fi
if [ -e \$prefix/$platform/grub.cfg ]; then
     source \$prefix/$platform/grub.cfg
elif [ -e \$prefix/grub.cfg ]; then
     source \$prefix/grub.cfg
else
     source \$cmdpath/grub.cfg
fi
```
.

So, as you can see it tries to find the kernel and initrd thanks to info 
and mini-info files.

1.2.2) The second step is that the Debian live cd is going to have an 
ElTorito image inside of it ( boot/grub/efi.img ) which it's essentially 
a FAT partition.
You can find the specific code for that creation on: 
https://salsa.debian.org/live-team/live-build/-/blob/ccf1f49bb9d185b2bb426ae1d4e84f4c4e4547ff/scripts/build/binary_grub-efi#L287-296

Basically a FAT partition image is created reusing some SecureBoot 
signed EFI images (if available, otherwise you would be using non-signed 
EFI images).
In the end what the FAT partition image contents are is somewhat similar to:

```
EFI
EFI/boot
EFI/boot/bootx64.efi
EFI/boot/grubx64.efi
boot
boot/grub
boot/grub/grub.cfg
```

The grubx64.efi file that you see above is actually a copy of the 
gcdx64.efi file created above in (1.2.1).

1.3) How does Debian image boot thanks to this ElTorito image from an 
UEFI CD boot?

UEFI firmware reads whatever defines ElTorito image and finds the 
contents of boot/grub/efi.img .
UEFI firmware only loads into memory EFI/boot/bootx64.efi on the 
ElTorito image.
This is usually shim which will boot onto grub.
But let's suppose that this is grub itself for the sake of simplicity.

When EFI/boot/grubx64.efi ( which it's actually a copy of gcdx64.efi ) 
is loaded into RAM as it has an embedded memdisk, its prefix gets set to 
the memdisk image and then the snippet mentioned above that searches for 
`/.disk/info` and `/.disk/mini-info` files is run.

Please note that whatever `boot/grub/grub.cfg` file was inside of the 
ElTorito image gets **ignored**.
Also please note that we are not reading information about the CD unless 
the CD has such files and it is found first in the Grub devices search.
In other words if you have a `/.disk/info` file in your internal hard 
disk or another device this might get messed up.

So, usually what happens is that the `/.disk/info` is found on the (cd0) 
device and then everything boots correctly.

1.4) Building a barebones ElTorito image.

So I have modified both Debian live-build's package and Debian grub 
`build-efi-images` script.

1.4.1) From the point of view of ElTorito image you only get:

```
EFI
EFI/boot
EFI/boot/bootx64.efi
```

with the modified `gcdx64.efi` contents.

1.4.2) And what about `gcdx64.efi`, well, it's no longer created with a 
embedded memdisk so it just boots and that's it.

1.4.3) My only test suggests that somehow Grub knows that its root is 
`(cd0)` and prefix is changed onto the usual `(cd0)/boot/grub` .

So that's very great because you can run the `/boot/grub/grub.cfg` file 
in the cdrom. That's what we all wanted all of the way down.

2) If I build a grub image with cdrom modules without any embedded 
memdisk and I use it inside of an ElTorito image and I boot from an UEFI 
firmware CD... Grub knows how to autodetect itself from RAM and finds (cd0).

QUESTION: Can you please point me to the Grub source code where that 
autodetection is coded?

I want to make sure this autodetection won't fail detecting another 
quite similar cd (cd0) instead of (cd1) if booted from (cd1).

ElTorito image which it's usually found in `boot/grub/efi.img` would 
have had:

```
EFI
EFI/boot
EFI/boot/bootx64.efi
```
contents.

That bootx64.efi is a grub image created thanks to this command ( Yes, 
it's actually a copy of gcdx64.efi ):
```
"obj/grub-pc/grub-mkimage"     -O "x86_64-efi"     -o 
"obj/monolithic/grub-efi-amd64/gcdx64.efi"     -d 
"obj/grub-efi-amd64/grub-core"     -p /boot/grub     --sbat 
"/tmp/tmp.OGhv28NUZq/sbat.debian.csv" all_video boot btrfs cat chain 
configfile echo efifwsetup efinet ext2 fat font f2fs gettext gfxmenu 
gfxterm gfxterm_background gzio halt help hfsplus iso9660 jfs jpeg 
keystatus loadenv loopback linux ls lsefi lsefimmap lsefisystab lssal 
memdisk minicmd normal ntfs part_apple part_msdos part_gpt 
password_pbkdf2 png probe reboot regexp search search_fs_uuid 
search_fs_file search_label serial sleep smbios squash4 test true video 
xfs zfs zfscrypt zfsinfo cpuid linuxefi play tpm
```
.

3) QUESTION: Do you know if Grub could autodetect its device if a 
regular grubx64.efi image is being loaded from 'Boot from file' UEFI option?
Or are we supposed to add an embedded memdisk with an embedded grub.cfg 
that helps the image find the rest of it? (This is for a live cd finding 
itself).

Actually I think this is the reason why the current Debian 
implementation is done in that embedded way.
If Grub manages to find itself consistently in a cd I'll send a pull 
request to Debian project about using this rediscovered method of using 
Grub for cds.

adrian15

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: How does Grub find itself being booted from UEFI CD ElTorito image ?

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1.1: Type: text/plain, Size: 7454 bytes --]

The code for auto detect of root device on EFI is
https://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/kern/efi/init.c#n130
. It will have no problem detecting right device even if there are similar
devices. The only case I know of, where it's unreliable is if partition
module for partition in question is missing but the disk is always detected
correctly unless firmware lies or we have a crazy bug.

Le dim. 15 sept. 2024, 14:07, adrian15sgd <adrian15sgd@gmail.com> a écrit :

> I actually have an specific question for Grub developers / maintainers /
> contributors so if you are one of them you can skip to: (2) .
>
> 1) INTRODUCTION
>
> 1.1) I am working on liveid ( https://github.com/rescatux/liveid ), this
> is sort of an earmark for live cds.
> I am currently trying to push some changes onto the live-build and
> live-boot packages from Debian GNU/Linux distribution.
>
> I was quite convinced I knew how Grub booted from a Debian's ElTorito
> image bundled into a Debian Live CD but as it turns out I was wrong.
>
>
> 1.2) The current way this is implemented from a Debian point of view
> (based on GRUB 2.06) can be described in two steps.
> 1.2.1) The first step is creating a gcdx64.efi file that has an embedded
> memdisk with a custom grub.cfg.
>
> So... the gcdx64.efi image is created thanks to a custom Debian script
> named `build-efi-images` .
>
> A memdisk image is being created first:
> (
>
> https://salsa.debian.org/grub-team/grub/-/blob/c5fc69927aefefb713d43a3753a7e7062c20ffa9/debian/build-efi-images#L57-76
> )
> and then it's being used on the final gcdx64.efi file:
> (
>
> https://salsa.debian.org/grub-team/grub/-/blob/c5fc69927aefefb713d43a3753a7e7062c20ffa9/debian/build-efi-images#L217-227
> )
> .
>
> Please note that the embedded grub.cfg contents are:
> ```
> if [ -z "\$prefix" -o ! -e "\$prefix" ]; then
>      if ! search --file --set=root /.disk/info; then
>          search --file --set=root /.disk/mini-info
>      fi
>      set prefix=(\$root)/boot/grub
> fi
> if [ -e \$prefix/$platform/grub.cfg ]; then
>      source \$prefix/$platform/grub.cfg
> elif [ -e \$prefix/grub.cfg ]; then
>      source \$prefix/grub.cfg
> else
>      source \$cmdpath/grub.cfg
> fi
> ```
> .
>
> So, as you can see it tries to find the kernel and initrd thanks to info
> and mini-info files.
>
> 1.2.2) The second step is that the Debian live cd is going to have an
> ElTorito image inside of it ( boot/grub/efi.img ) which it's essentially
> a FAT partition.
> You can find the specific code for that creation on:
>
> https://salsa.debian.org/live-team/live-build/-/blob/ccf1f49bb9d185b2bb426ae1d4e84f4c4e4547ff/scripts/build/binary_grub-efi#L287-296
>
> Basically a FAT partition image is created reusing some SecureBoot
> signed EFI images (if available, otherwise you would be using non-signed
> EFI images).
> In the end what the FAT partition image contents are is somewhat similar
> to:
>
> ```
> EFI
> EFI/boot
> EFI/boot/bootx64.efi
> EFI/boot/grubx64.efi
> boot
> boot/grub
> boot/grub/grub.cfg
> ```
>
> The grubx64.efi file that you see above is actually a copy of the
> gcdx64.efi file created above in (1.2.1).
>
> 1.3) How does Debian image boot thanks to this ElTorito image from an
> UEFI CD boot?
>
> UEFI firmware reads whatever defines ElTorito image and finds the
> contents of boot/grub/efi.img .
> UEFI firmware only loads into memory EFI/boot/bootx64.efi on the
> ElTorito image.
> This is usually shim which will boot onto grub.
> But let's suppose that this is grub itself for the sake of simplicity.
>
> When EFI/boot/grubx64.efi ( which it's actually a copy of gcdx64.efi )
> is loaded into RAM as it has an embedded memdisk, its prefix gets set to
> the memdisk image and then the snippet mentioned above that searches for
> `/.disk/info` and `/.disk/mini-info` files is run.
>
> Please note that whatever `boot/grub/grub.cfg` file was inside of the
> ElTorito image gets **ignored**.
> Also please note that we are not reading information about the CD unless
> the CD has such files and it is found first in the Grub devices search.
> In other words if you have a `/.disk/info` file in your internal hard
> disk or another device this might get messed up.
>
> So, usually what happens is that the `/.disk/info` is found on the (cd0)
> device and then everything boots correctly.
>
> 1.4) Building a barebones ElTorito image.
>
> So I have modified both Debian live-build's package and Debian grub
> `build-efi-images` script.
>
> 1.4.1) From the point of view of ElTorito image you only get:
>
> ```
> EFI
> EFI/boot
> EFI/boot/bootx64.efi
> ```
>
> with the modified `gcdx64.efi` contents.
>
> 1.4.2) And what about `gcdx64.efi`, well, it's no longer created with a
> embedded memdisk so it just boots and that's it.
>
> 1.4.3) My only test suggests that somehow Grub knows that its root is
> `(cd0)` and prefix is changed onto the usual `(cd0)/boot/grub` .
>
> So that's very great because you can run the `/boot/grub/grub.cfg` file
> in the cdrom. That's what we all wanted all of the way down.
>
> 2) If I build a grub image with cdrom modules without any embedded
> memdisk and I use it inside of an ElTorito image and I boot from an UEFI
> firmware CD... Grub knows how to autodetect itself from RAM and finds
> (cd0).
>
> QUESTION: Can you please point me to the Grub source code where that
> autodetection is coded?
>
> I want to make sure this autodetection won't fail detecting another
> quite similar cd (cd0) instead of (cd1) if booted from (cd1).
>
> ElTorito image which it's usually found in `boot/grub/efi.img` would
> have had:
>
> ```
> EFI
> EFI/boot
> EFI/boot/bootx64.efi
> ```
> contents.
>
> That bootx64.efi is a grub image created thanks to this command ( Yes,
> it's actually a copy of gcdx64.efi ):
> ```
> "obj/grub-pc/grub-mkimage"     -O "x86_64-efi"     -o
> "obj/monolithic/grub-efi-amd64/gcdx64.efi"     -d
> "obj/grub-efi-amd64/grub-core"     -p /boot/grub     --sbat
> "/tmp/tmp.OGhv28NUZq/sbat.debian.csv" all_video boot btrfs cat chain
> configfile echo efifwsetup efinet ext2 fat font f2fs gettext gfxmenu
> gfxterm gfxterm_background gzio halt help hfsplus iso9660 jfs jpeg
> keystatus loadenv loopback linux ls lsefi lsefimmap lsefisystab lssal
> memdisk minicmd normal ntfs part_apple part_msdos part_gpt
> password_pbkdf2 png probe reboot regexp search search_fs_uuid
> search_fs_file search_label serial sleep smbios squash4 test true video
> xfs zfs zfscrypt zfsinfo cpuid linuxefi play tpm
> ```
> .
>
> 3) QUESTION: Do you know if Grub could autodetect its device if a
> regular grubx64.efi image is being loaded from 'Boot from file' UEFI
> option?
> Or are we supposed to add an embedded memdisk with an embedded grub.cfg
> that helps the image find the rest of it? (This is for a live cd finding
> itself).
>
> Actually I think this is the reason why the current Debian
> implementation is done in that embedded way.
> If Grub manages to find itself consistently in a cd I'll send a pull
> request to Debian project about using this rediscovered method of using
> Grub for cds.
>
> adrian15
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #1.2: Type: text/html, Size: 9247 bytes --]

[-- Attachment #2: Type: text/plain, Size: 141 bytes --]

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel