[PATCH 1/7] bootimg_pcbios: support grub hybrid boot

[PATCH 1/7] bootimg_pcbios: support grub hybrid boot

[Vincent Davis Jr]

If caller wants to primarily leverage
grub as the primary boot loader current
oe-core doesn't support booting grub when
the boot firmware is legacy BIOS based and
the partition table format is GPT based.

Issue GPT header reside where core.img
should be located (at byte 512).

To navigate around issue core.img was
moved to a seperate partition.

If disk is a GPT disk caller must specify
the file system type as none and set the
partition type as BIOS boot. No filesystem
will be created on partition. This also
allows wic plugin to know where to dd
core.img.

Unfortunately No deep dive into grub-install
was perform to know the exact bytes changed
in hybrid boot case. To see the change in
bytes generated boot.img + core.img was compared
to boot.img + core.img after grub-install was
executed using the xxd command.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 .../lib/wic/plugins/source/bootimg_pcbios.py  | 55 ++++++++++++++++++-
 1 file changed, 53 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg_pcbios.py b/scripts/lib/wic/plugins/source/bootimg_pcbios.py
index caabda6318..a7fb2eca4e 100644
--- a/scripts/lib/wic/plugins/source/bootimg_pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg_pcbios.py
@@ -101,7 +101,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
                      disk_name, full_path, disk.min_size)
 
         if cls.loader == 'grub':
-            cls._do_install_grub(creator, kernel_dir,
+            cls._do_install_grub(disk, creator, kernel_dir,
                             native_sysroot, full_path)
         elif cls.loader == 'syslinux':
             cls._do_install_syslinux(creator, bootimg_dir,
@@ -340,6 +340,12 @@ class BootimgPcbiosPlugin(SourcePlugin):
 
     @classmethod
     def _do_configure_grub(cls, part, creator, cr_workdir):
+        # If partition type is either EFI System or
+        # BIOS boot no need to generate/copy grub config.
+        if part.part_type == '21686148-6449-6E6F-744E-656564454649' or \
+           part.part_type == 'C12A7328-F81F-11D2-BA4B-00A0C93EC93B':
+            return 0
+
         hdddir = "%s/hdd" % cr_workdir
         bootloader = creator.ks.bootloader
 
@@ -438,6 +444,12 @@ class BootimgPcbiosPlugin(SourcePlugin):
          grub_mods_path, core_img, builtin_modules)
         exec_native_cmd(grub_mkimage, native_sysroot)
 
+        # If partition type is either EFI System or
+        # BIOS boot no need to copy in grub modules.
+        if part.part_type == '21686148-6449-6E6F-744E-656564454649' or \
+           part.part_type == 'C12A7328-F81F-11D2-BA4B-00A0C93EC93B':
+            return 0
+
         # Copy grub modules
         install_dir = '%s/%s/%s' % (hdddir, grub_prefix_path, grub_format)
         os.makedirs(install_dir, exist_ok=True)
@@ -454,7 +466,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
                             native_sysroot, False)
 
     @classmethod
-    def _do_install_grub(cls, creator, kernel_dir,
+    def _do_install_grub(cls, disk, creator, kernel_dir,
                          native_sysroot, full_path):
         core_img = '%s/grub-bios-core.img' % (kernel_dir)
 
@@ -478,6 +490,45 @@ class BootimgPcbiosPlugin(SourcePlugin):
             # Install core.img or grub stage 1.5
             dd_cmd = "dd if=%s of=%s conv=notrunc bs=1 seek=512" % (core_img, full_path)
             exec_cmd(dd_cmd, native_sysroot)
+        elif creator.ptable_format == 'gpt':
+            # Issue GPT headers reside where core.img should be (at byte 512).
+            # To navigate around issue core.img was moved to a seperate partition.
+            #
+            # If disk is a GPT disk caller must specify the file system
+            # type as none. As no filesystem may be created. Caller must
+            # also set the partition type to BIOS boot. So, the plugin
+            # may embed core.img there.
+
+            # Replicates what grub-install does to boot.img
+            # Found by comparing xxd output of generated boot.img
+            # to boot.img after grub-install.
+            dd_cmd = "echo -ne '\\x00\\x08' | dd of=%s conv=notrunc bs=1 count=2 seek=92" % (full_path)
+            exec_native_cmd(dd_cmd, native_sysroot)
+
+            dd_cmd = "echo -ne '\\x90\\x90' | dd of=%s conv=notrunc bs=1 count=2 seek=102" % (full_path)
+            exec_native_cmd(dd_cmd, native_sysroot)
+
+            for part in creator.parts:
+                if part.part_type == '21686148-6449-6E6F-744E-656564454649':
+                    part_start_byte = part.start * disk.sector_size
+
+                    # Install core.img or grub stage 1.5
+                    dd_cmd = "dd if=%s of=%s conv=notrunc bs=1 seek=%s" % \
+                        (core_img, full_path, part_start_byte)
+                    exec_cmd(dd_cmd, native_sysroot)
+
+                    # Replicates what grub-install does to core.img
+                    # Found by comparing xxd output of generated
+                    # core.img to core.img ater grub install.
+                    dd_cmd = "echo -ne '\\x01\\x08' | dd of=%s conv=notrunc bs=1 count=2 seek=%d" % \
+                            (full_path, part_start_byte + 500)
+                    exec_native_cmd(dd_cmd, native_sysroot)
+
+                    dd_cmd = "echo -ne '\\x2f\\x02' | dd of=%s conv=notrunc bs=1 count=2 seek=%d" % \
+                            (full_path, part_start_byte + 508)
+                    exec_native_cmd(dd_cmd, native_sysroot)
+
+                    break
         else:
             raise WicError("Unsupported partition table: %s" %
                            creator.ptable_format)
-- 
2.43.0

[PATCH 2/7] bootimg_efi: support grub hybrid boot

[Vincent Davis Jr]

If bootimg_biosplusefi wics plugin is
used check for loader-bios in sourceparam.

The grub config is copied by the bootimg_pcbios
wic's plugin. The reason is it's autogenerated
grub config is generic enough to support both
legacy boot and hybrid boot. Thus, no need
to update it.

If partition type marked as EFI System only
copy in UEFI application boot*.efi (core.img).

It appears the EFI directory resides after
initial population of the EFI System partition.
If bootimg_efi called again and the partiton
type not marked as EFI System remove any
unwanted files.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 scripts/lib/wic/plugins/source/bootimg_efi.py | 28 +++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg_efi.py b/scripts/lib/wic/plugins/source/bootimg_efi.py
index cf16705a28..98be12160c 100644
--- a/scripts/lib/wic/plugins/source/bootimg_efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg_efi.py
@@ -223,7 +223,13 @@ class BootimgEFIPlugin(SourcePlugin):
         exec_cmd(install_cmd)
 
         try:
-            if source_params['loader'] == 'grub-efi':
+            # In grub hybrid boot case bootimg_pcbios generates/copies the grub.cfg
+            # that will be stored on same partitions as the grub modules.
+            if 'loader-bios' in source_params and \
+               source_params['loader-bios'] == 'grub' and \
+               source_params['loader'] == 'grub-efi':
+                pass
+            elif source_params['loader'] == 'grub-efi':
                 cls.do_configure_grubefi(hdddir, creator, cr_workdir, source_params)
             elif source_params['loader'] == 'systemd-boot':
                 cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params)
@@ -328,7 +334,25 @@ class BootimgEFIPlugin(SourcePlugin):
                 logger.debug("Installed IMAGE_EFI_BOOT_FILES:\n%s" % out)
 
         try:
-            if source_params['loader'] == 'grub-efi':
+            if 'loader-bios' in source_params and \
+               source_params['loader-bios'] == 'grub' and \
+               source_params['loader'] == 'grub-efi':
+
+                # If partition type is Bios Boot return
+                if part.part_type == '21686148-6449-6E6F-744E-656564454649':
+                       return 0
+
+                # The grub config in the hybrid grub boot setup is
+                # copied by the bootimg_pcbios wics plugin.
+                if part.part_type == 'C12A7328-F81F-11D2-BA4B-00A0C93EC93B':
+                    for mod in [x for x in os.listdir(kernel_dir) if x.startswith("grub-efi-")]:
+                        cp_cmd = "cp -v -p %s/%s %s/EFI/BOOT/%s" % \
+                            (kernel_dir, mod, hdddir, mod[9:])
+                        exec_cmd(cp_cmd, True)
+                else:
+                    shutil.rmtree("%s/EFI" % hdddir)
+                    return 0
+            elif source_params['loader'] == 'grub-efi':
                 shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
                                 "%s/grub.cfg" % cr_workdir)
                 for mod in [x for x in os.listdir(kernel_dir) if x.startswith("grub-efi-")]:
-- 
2.43.0

[PATCH 3/7] bootimg_efi: copy grub modules

[Vincent Davis Jr]

Assumming all grub modules weren't embedded
into boot*.efi (core.img) copy them now.

This commit populates the common partition
used to store the grub config and grub modules
for both PCBIOS and EFI.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 scripts/lib/wic/plugins/source/bootimg_efi.py | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/bootimg_efi.py b/scripts/lib/wic/plugins/source/bootimg_efi.py
index 98be12160c..c3254033d6 100644
--- a/scripts/lib/wic/plugins/source/bootimg_efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg_efi.py
@@ -350,7 +350,39 @@ class BootimgEFIPlugin(SourcePlugin):
                             (kernel_dir, mod, hdddir, mod[9:])
                         exec_cmd(cp_cmd, True)
                 else:
+                    # Assumming all grub modules weren't embedded
+                    # into grub.efi or core.img copy them now.
+                    copy_types = [ '*.mod', '*.o', '*.lst' ]
+
+                    # It appears the EFI directory resides after
+                    # initial population of the EFI System partition
+                    # in the if statement above. Remove so that the
+                    # partition doesn't contain any unrequired files.
                     shutil.rmtree("%s/EFI" % hdddir)
+
+                    hdddir = "%s/hdd" % cr_workdir
+
+                    staging_libdir = cls._get_staging_libdir()
+
+                    grub_format = get_bitbake_var('GRUB_MKIMAGE_FORMAT_EFI')
+                    if not grub_format:
+                        grub_format = 'x86_64-efi'
+
+                    grub_prefix_path = get_bitbake_var('GRUB_PREFIX_PATH')
+                    if not grub_prefix_path:
+                        grub_prefix_path = '/boot/grub'
+
+                    # Copy grub modules
+                    install_dir = '%s/%s/%s' % (hdddir, grub_prefix_path, grub_format)
+                    os.makedirs(install_dir, exist_ok=True)
+
+                    for ctype in copy_types:
+                        files = glob('%s/grub/%s/%s' % \
+                            (staging_libdir, grub_format, ctype))
+                        for file in files:
+                            shutil.copy2(file, install_dir, follow_symlinks=True)
+
+                    # bootimg_pcbios calles prepare_rootfs no need to here.
                     return 0
             elif source_params['loader'] == 'grub-efi':
                 shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
@@ -457,3 +489,24 @@ class BootimgEFIPlugin(SourcePlugin):
 
         part.size = int(bootimg_size)
         part.source_file = bootimg
+
+    @classmethod
+    def _get_staging_libdir(cls):
+        """
+        For unknown reasons when running test with poky
+        STAGING_LIBDIR gets unset when wic create is executed.
+        Bellow is a hack to determine what STAGING_LIBDIR should
+        be if not specified.
+        """
+
+        staging_libdir = get_bitbake_var('STAGING_LIBDIR')
+        staging_dir_target = get_bitbake_var('STAGING_DIR_TARGET')
+
+        if not staging_libdir:
+            staging_libdir = '%s/usr/lib64' % staging_dir_target
+            if not os.path.isdir(staging_libdir):
+                staging_libdir = '%s/usr/lib32' % staging_dir_target
+                if not os.path.isdir(staging_libdir):
+                    staging_libdir = '%s/usr/lib' % staging_dir_target
+
+        return staging_libdir
-- 
2.43.0

[PATCH 4/7] bootimg_biosplusefi: add grub only examples

[Vincent Davis Jr]

Updates the comments section to explain
new usage of wic plugins.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 .../wic/plugins/source/bootimg_biosplusefi.py | 32 +++++++++++++++++--
 1 file changed, 29 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py b/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py
index 4279ddded8..ad4c0e1768 100644
--- a/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py
+++ b/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py
@@ -27,7 +27,7 @@ class BootimgBiosPlusEFIPlugin(SourcePlugin):
     """
     Create MBR + EFI boot partition
 
-    This plugin creates a boot partition that contains both
+    This plugin creates a boot partition(s) that contains both
     legacy BIOS and EFI content. It will be able to boot from both.
     This is useful when managing PC fleet with some older machines
     without EFI support.
@@ -50,7 +50,9 @@ class BootimgBiosPlusEFIPlugin(SourcePlugin):
     not turn the rootfs into an initramfs RAM image.
 
     This plugin is made to put everything into a single /boot partition so it
-    does not have the limitations listed above.
+    does not have the limitations listed above. Unless GRUB is the primary
+    bootloader. We have to seperate it into multiple partitions because
+    of core.img.
 
     The plugin is made so it does tries not to reimplement what's already
     been done in other plugins; as such it imports "bootimg_pcbios"
@@ -70,9 +72,33 @@ class BootimgBiosPlusEFIPlugin(SourcePlugin):
     Plugin options, such as "--sourceparams" can still be passed to a
     plugin, as long they does not cause issue in the other plugin.
 
-    Example wic configuration:
+    Example wic configurations:
+
+    ************ Example kickstart GRUB/Syslinux Hybrid Legacy Bios Or Newer UEFI Boot ************
     part /boot --source bootimg_biosplusefi --sourceparams="loader=grub-efi"\\
                --ondisk sda --label os_boot --active --align 1024 --use-uuid
+    ************ Example kickstart GRUB/Syslinux Hybrid Legacy Bios Or Newer UEFI Boot ************
+
+
+    ********************** Example kickstart GRUB Hybrid Legacy Bios Or Newer UEFI Boot **********************
+    # See https://wiki.archlinux.org/title/GPT_fdisk#Partition_type
+
+    part bios_boot --label bios_boot --fstype none --offset 1024 --fixed-size 1M \\
+                   --part-type 21686148-6449-6E6F-744E-656564454649 --source bootimg_biosplusefi \\
+                   --sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"
+
+    part efi_system --label efi_system --fstype vfat --fixed-size 48M \\
+                    --part-type C12A7328-F81F-11D2-BA4B-00A0C93EC93B --source bootimg_biosplusefi \\
+                    --sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"
+
+    part grub_data --label grub_data --fstype ext4 --fixed-size 78M \\
+                   --part-type 0FC63DAF-8483-4772-8E79-3D69D8477DE4 --source bootimg_biosplusefi \\
+                   --sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"
+
+    part roots --label rootfs --fstype ext4 --source rootfs --part-type 0FC63DAF-8483-4772-8E79-3D69D8477DE4
+
+    bootloader --ptable gpt --source bootimg_biosplusefi
+    ********************** Example kickstart GRUB Hybrid Legacy Bios Or Newer UEFI Boot **********************
     """
 
     name = 'bootimg_biosplusefi'
-- 
2.43.0

[PATCH 5/7] grub-efi: support custom embedded grub configs

[Vincent Davis Jr]

Commit:

Adds support for including custom
embedded grub configs. The default one
provided in OE-core is limited to everything
being in the /EFI/BOOT directory.

Adds an embedded grub config that sets
root and prefix based upon the /boot/grub
directory.

Adds probe to the list of default builtin
grub modules. Probe is required by the
bootimg_pcbios wic plugin generated
grub config.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 meta/recipes-bsp/grub/files/cfg-boot-grub | 3 +++
 meta/recipes-bsp/grub/grub-efi_2.12.bb    | 6 ++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-bsp/grub/files/cfg-boot-grub

diff --git a/meta/recipes-bsp/grub/files/cfg-boot-grub b/meta/recipes-bsp/grub/files/cfg-boot-grub
new file mode 100644
index 0000000000..ab370479af
--- /dev/null
+++ b/meta/recipes-bsp/grub/files/cfg-boot-grub
@@ -0,0 +1,3 @@
+search.file /boot/grub/grub.cfg root
+set prefix=($root)
+configfile ($root)/boot/grub/grub.cfg
diff --git a/meta/recipes-bsp/grub/grub-efi_2.12.bb b/meta/recipes-bsp/grub/grub-efi_2.12.bb
index 6354b43989..91748c8d04 100644
--- a/meta/recipes-bsp/grub/grub-efi_2.12.bb
+++ b/meta/recipes-bsp/grub/grub-efi_2.12.bb
@@ -9,6 +9,7 @@ RDEPENDS:${PN} = "grub-common virtual-grub-bootconf"
 
 SRC_URI += " \
            file://cfg \
+           file://cfg-boot-grub \
           "
 
 S = "${UNPACKDIR}/grub-${PV}"
@@ -48,6 +49,7 @@ EXTRA_OECONF += "--enable-efiemu=no"
 
 # Define GRUB_MKIMAGE_OPTS variable for additional grub-mkimage options (e.g., disabling shim lock)
 GRUB_MKIMAGE_OPTS ?= ""
+EMBEDDED_GRUB_CONFIG ?= "${UNPACKDIR}/cfg"
 
 do_mkimage() {
 	cd ${B}
@@ -63,7 +65,7 @@ do_mkimage() {
 
 	# Search for the grub.cfg on the local boot media by using the
 	# built in cfg file provided via this recipe
-	grub-mkimage -v -c ${UNPACKDIR}/cfg -p ${EFIDIR} -d ./grub-core/ \
+	grub-mkimage -v -c ${EMBEDDED_GRUB_CONFIG} -p ${EFIDIR} -d ./grub-core/ \
 	               -O ${GRUB_TARGET}-efi -o ./${GRUB_IMAGE_PREFIX}${GRUB_IMAGE} \
 	               ${GRUB_MKIMAGE_OPTS} ${GRUB_MKIMAGE_MODULES}
 }
@@ -87,7 +89,7 @@ do_install() {
 
 # To include all available modules, add 'all' to GRUB_BUILDIN
 GRUB_BUILDIN ?= "boot linux ext2 fat serial part_msdos part_gpt normal \
-                 efi_gop iso9660 configfile search loadenv test"
+                 efi_gop iso9660 configfile search probe loadenv test"
 
 # 'xen_boot' is a module valid only for aarch64
 GRUB_BUILDIN:append:aarch64 = "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' xen_boot', '', d)}"
-- 
2.43.0

[PATCH 6/7] oe-selftest[wic]: add test_grub_install_biosplusefi

[Vincent Davis Jr]

Commit adds test to check if required files
are found in the EFI directory and that
grub modules for both EFI and BIOS are found
on the same partition as the grub config.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 meta/lib/oeqa/selftest/cases/wic.py | 54 +++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index 44442e402d..10c3497519 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1456,6 +1456,60 @@ run_wic_cmd() {
             out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname))
             self.assertEqual(1, len(out))
 
+    @skipIfNotArch(['x86_64'])
+    def test_grub_install_biosplusefi(self):
+        """
+        Test the installation of the grub in hybrid boot mode.
+        """
+
+        # create a temporary file for the WKS content
+        with NamedTemporaryFile("w", suffix=".wks") as wks:
+            wks.write(
+                'part bios_boot --label bios_boot --fstype none --offset 1024 --fixed-size 1M ' \
+                '--part-type 21686148-6449-6E6F-744E-656564454649 --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'part efi_system --label efi_system --fstype vfat --fixed-size 48M ' \
+                '--part-type C12A7328-F81F-11D2-BA4B-00A0C93EC93B --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'part grub_data --label grub_data --fstype ext4 --fixed-size 78M ' \
+                '--part-type 0FC63DAF-8483-4772-8E79-3D69D8477DE4 --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'bootloader --ptable gpt --source bootimg_biosplusefi\n'
+            )
+            wks.flush()
+
+            img = "core-image-minimal"
+            config = 'DEPENDS:pn-%s += "grub-native grub grub-efi"' % (img)
+
+            self.append_config(config)
+            bitbake(img)
+            self.remove_config(config)
+
+            cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir)
+            runCmd(cmd)
+
+            wksname = os.path.splitext(os.path.basename(wks.name))[0]
+            out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname))
+            self.assertEqual(1, len(out))
+
+            sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools')
+
+            # Check if grub UEFI application (core.img) installed
+            result = runCmd("wic ls %s:2/EFI/BOOT -n %s" % (out[0], sysroot))
+            self.assertIn('bootx64', result.output)
+
+            # Check if grub.cfg is installed
+            result = runCmd("wic ls %s:3/boot/grub -n %s" % (out[0], sysroot))
+            self.assertIn('grub', result.output)
+
+            # Check if normal.mod is installed
+            result = runCmd("wic ls %s:3/boot/grub/i386-pc -n %s" % (out[0], sysroot))
+            self.assertIn('normal', result.output)
+
+            # Check if normal.mod is installed
+            result = runCmd("wic ls %s:3/boot/grub/x86_64-efi -n %s" % (out[0], sysroot))
+            self.assertIn('normal', result.output)
+
     @skipIfNotArch(['i586', 'i686', 'x86_64', 'aarch64'])
     def test_uefi_kernel(self):
         """ Test uefi-kernel in wic """
-- 
2.43.0

[PATCH 7/7] oe-selftest[wic]: add test_grub_install_biosplusefi_qemu

[Vincent Davis Jr]

Test runs qemu given wic file and checks to
see if the resulting wic image boots with either
legacy bios or newer UEFI frimware whilst grub
is the primary bootloader.

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
---
 meta/lib/oeqa/selftest/cases/wic.py | 61 +++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index 10c3497519..efd1f5e6f0 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1510,6 +1510,67 @@ run_wic_cmd() {
             result = runCmd("wic ls %s:3/boot/grub/x86_64-efi -n %s" % (out[0], sysroot))
             self.assertIn('normal', result.output)
 
+    @skipIfNotArch(['x86_64'])
+    @OETestTag("runqemu")
+    def test_grub_install_biosplusefi_qemu(self):
+        """Test biosplusefi plugin grub only in qemu"""
+
+        img = "core-image-minimal"
+
+        # create a temporary file for the WKS content
+        with NamedTemporaryFile("w", suffix=".wks") as wks:
+            wks.write(
+                'part bios_boot --label bios_boot --fstype none --offset 1024 --fixed-size 1M ' \
+                '--part-type 21686148-6449-6E6F-744E-656564454649 --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'part efi_system --label efi_system --fstype vfat --fixed-size 48M ' \
+                '--part-type C12A7328-F81F-11D2-BA4B-00A0C93EC93B --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'part grub_data --label grub_data --fstype ext4 --fixed-size 78M ' \
+                '--part-type 0FC63DAF-8483-4772-8E79-3D69D8477DE4 --source bootimg_biosplusefi ' \
+                '--sourceparams="loader=grub-efi,loader-bios=grub,install-kernel-into-boot-dir=false"\n' \
+                'part roots --label rootfs --fstype ext4 --source rootfs ' \
+                '--part-type 0FC63DAF-8483-4772-8E79-3D69D8477DE4\n' \
+                'bootloader --timeout=1 --ptable gpt --source bootimg_biosplusefi\n'
+            )
+            wks.flush()
+
+            config = 'DEPENDS:pn-%s += "grub-native grub grub-efi ovmf"\n' % (img)
+            config += 'IMAGE_FSTYPES:pn-%s += "wic"\n' % (img)
+            #config += 'MACHINE_FEATURES:append = " efi"\n'
+            config += 'IMAGE_CLASSES += "qemuboot"\n'
+            config += 'WKS_FILE = "%s"\n' % (os.path.basename(wks.name))
+            config += 'WKS_SEARCH_PATH = "%s"\n' % (os.path.dirname(wks.name))
+
+            self.append_config(config)
+            bitbake(img)
+            self.remove_config(config)
+
+            cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir)
+            runCmd(cmd)
+
+            # Test legacy bios boot
+            runqemu_params = get_bb_var('TEST_RUNQEMUPARAMS', img) or ""
+            with runqemu(img, ssh=False, runqemuparams='%s nographic' % (runqemu_params),
+                         image_fstype='wic') as qemu:
+                # Check that we have all four /dev/sda* partitions (/boot and /)
+                cmd = "grep sda. /proc/partitions | wc -l"
+                status, output = qemu.run_serial(cmd)
+                self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
+                self.assertEqual(output, '4')
+
+            # Test UEFI boot
+            # Can't test as it requires swapping the grub-efi*.bb
+            # embedded grub config. With one that works in the
+            # hybrid boot case.
+            #with runqemu(img, ssh=False, runqemuparams='%s ovmf nographic' % (runqemu_params),
+            #             image_fstype='wic') as qemu:
+            # Check that we have all four /dev/sda* partitions (/boot and /)
+            #    cmd = "grep sda. /proc/partitions | wc -l"
+            #    status, output = qemu.run_serial(cmd)
+            #    self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
+            #    self.assertEqual(output, '4')
+
     @skipIfNotArch(['i586', 'i686', 'x86_64', 'aarch64'])
     def test_uefi_kernel(self):
         """ Test uefi-kernel in wic """
-- 
2.43.0

Re: [OE-core] [PATCH 1/7] bootimg_pcbios: support grub hybrid boot

[Mathieu Dubois-Briand]

On Tue Sep 2, 2025 at 3:58 AM CEST, Vincent Davis Jr wrote:
> If caller wants to primarily leverage
> grub as the primary boot loader current
> oe-core doesn't support booting grub when
> the boot firmware is legacy BIOS based and
> the partition table format is GPT based.
>
> Issue GPT header reside where core.img
> should be located (at byte 512).
>
> To navigate around issue core.img was
> moved to a seperate partition.
>
> If disk is a GPT disk caller must specify
> the file system type as none and set the
> partition type as BIOS boot. No filesystem
> will be created on partition. This also
> allows wic plugin to know where to dd
> core.img.
>
> Unfortunately No deep dive into grub-install
> was perform to know the exact bytes changed
> in hybrid boot case. To see the change in
> bytes generated boot.img + core.img was compared
> to boot.img + core.img after grub-install was
> executed using the xxd command.
>
> Signed-off-by: Vincent Davis Jr <vince@underview.tech>
> ---

Hi Vincent,

It looks like this tends to fail some selftest. Autobuilder logs are not
really verbose here, but I can relaunch a build and extract more log
files if you now were you want to look.

runqemu - INFO - Running /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2799227/tmp/work/x86_64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive if=none,id=hd,file=/home/pokybuild/tmp/core-image-minimal-qemux86-64.rootfs.wic.510160,format=raw -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd -usb -device usb-tablet -usb -device usb-kbd   -cpu Skylake-Client -machine q35,i8042=off -smp 4 -enable-kvm -m 256 -serial tcp:127.0.0.1:42697,nodelay=on -serial tcp:127.0.0.1:54201,nodelay=on  -pidfile /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2799227/pidfile_2799227  -S -qmp unix:./.6pnxdkdq,server,wait -qmp unix:./.p0gm_1m0,server,nowait -nographic
...
Target didn't reach login banner in 1000 seconds (09/02/25 18:40:59)
Last 25 lines of all logging (308):
c[?7l[2J[0mSeaBIOS (version rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org)


iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+0EFC6560+0EF26560 CA00
Press Ctrl-B to configure iPXE (PCI 00:02.0)...



Booting from Hard Disk..
...
RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
...
2025-09-03 00:41:04,380 - oe-selftest - INFO - ERROR: wic.Wic2.test_grub_install_biosplusefi_qemu (subunit.RemotedTestCase)

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2256

Can you have a look at the issue please?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 1/7] bootimg_pcbios: support grub hybrid boot

[Vincent Davis]

[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]

O Yeah,

Have a good idea of what the issue is..
Thought I had that fixed on my end.

Vincent

On Thu, Sep 4, 2025, 1:56 AM Mathieu Dubois-Briand <
mathieu.dubois-briand@bootlin.com> wrote:

> On Tue Sep 2, 2025 at 3:58 AM CEST, Vincent Davis Jr wrote:
> > If caller wants to primarily leverage
> > grub as the primary boot loader current
> > oe-core doesn't support booting grub when
> > the boot firmware is legacy BIOS based and
> > the partition table format is GPT based.
> >
> > Issue GPT header reside where core.img
> > should be located (at byte 512).
> >
> > To navigate around issue core.img was
> > moved to a seperate partition.
> >
> > If disk is a GPT disk caller must specify
> > the file system type as none and set the
> > partition type as BIOS boot. No filesystem
> > will be created on partition. This also
> > allows wic plugin to know where to dd
> > core.img.
> >
> > Unfortunately No deep dive into grub-install
> > was perform to know the exact bytes changed
> > in hybrid boot case. To see the change in
> > bytes generated boot.img + core.img was compared
> > to boot.img + core.img after grub-install was
> > executed using the xxd command.
> >
> > Signed-off-by: Vincent Davis Jr <vince@underview.tech>
> > ---
>
> Hi Vincent,
>
> It looks like this tends to fail some selftest. Autobuilder logs are not
> really verbose here, but I can relaunch a build and extract more log
> files if you now were you want to look.
>
> runqemu - INFO - Running
> /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2799227/tmp/work/x86_64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-x86_64
> -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev
> tap,id=net0,ifname=tap0,script=no,downscript=no -object
> rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0
> -drive
> if=none,id=hd,file=/home/pokybuild/tmp/core-image-minimal-qemux86-64.rootfs.wic.510160,format=raw
> -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd -usb -device
> usb-tablet -usb -device usb-kbd   -cpu Skylake-Client -machine
> q35,i8042=off -smp 4 -enable-kvm -m 256 -serial tcp:127.0.0.1:42697,nodelay=on
> -serial tcp:127.0.0.1:54201,nodelay=on  -pidfile
> /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2799227/pidfile_2799227
> -S -qmp unix:./.6pnxdkdq,server,wait -qmp unix:./.p0gm_1m0,server,nowait
> -nographic
> ...
> Target didn't reach login banner in 1000 seconds (09/02/25 18:40:59)
> Last 25 lines of all logging (308):
> c[?7l[2J[0mSeaBIOS (version rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org)
>
>
> iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+0EFC6560+0EF26560 CA00
> Press Ctrl-B to configure iPXE (PCI 00:02.0)...
>
>
>
> Booting from Hard Disk..
> ...
> RuntimeError: core-image-minimal - FAILED to start qemu - check the task
> log and the boot log
> ...
> 2025-09-03 00:41:04,380 - oe-selftest - INFO - ERROR:
> wic.Wic2.test_grub_install_biosplusefi_qemu (subunit.RemotedTestCase)
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2256
>
> Can you have a look at the issue please?
>
> Thanks,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>

[-- Attachment #2: Type: text/html, Size: 4488 bytes --]