[PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[Hongxu Jia]

In ISO live, if the size of efi.img > 32MB, and copy EFI application
(bootx64.efi) to efi.img behind of kernel and initrd, UEFI system
could not find EFI application bootx64.efi

Using QEMU+OVMF to boot ISO live image, press ESC to enter UEFI shell:
...
Shell> ls FS0:\
Directory of: FS0:\
04/05/2011  23:00          12,985,344  bzImage
04/05/2011  23:00 <DIR>         2,048  EFI
04/05/2011  23:00          20,494,696  initrd
04/05/2011  23:00                  26  startup.nsh
          3 File(s)  33,480,066 bytes
          1 Dir(s)

Shell> ls FS0:\EFI
Directory of: FS0:\EFI
          0 File(s)           0 bytes
          0 Dir(s)
...

In following case, add 64MB extra space to bootable image efi.img,
and the partition table of EFI is truncated to 26.3M

$ echo 'IMAGE_FSTYPES:pn-core-image-minimal = " live"' >> conf/local.conf
$ echo 'MACHINE_FEATURES:append = " efi pcbios"' >> conf/local.conf
$ echo '# 64MB extra space to bootable image efi.img' >> conf/local.conf
$ echo 'BOOTIMG_EXTRA_SPACE = "65535"' >> conf/local.conf
$ bitbake core-image-minimal
$ fdisk -l tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso
...
Device                                                                 Boot Start    End Sectors  Size Id Type
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso1 *        0 376831  376832  184M  0 Empty
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso2        120  54079   53960 26.3M ef EFI (FAT-12/16/32)

According to page 11: `Figure 5 - Section Entry' in El Torito Bootable
CD-ROM Format Specification [1]. The sector count takes 2 byte which
means max sector count is 0xffff (65535), for 512-byte sector, the
size of bootable image is no more than 32MB (65536 * 512 / 1024 / 1024)

This commit truncate to 32MB if image size larger than 32MB, and
report a warning, then save the extra image sector count to
vendor unique selection criteria

After apply this commit, the partition table of EFI is truncated to 32M
$ fdisk -l tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso
...
Device                                                                 Boot Start    End Sectors  Size Id Type
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso1 *        0 376831  376832  184M  0 Empty
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso2        120  65654   65535   32M ef EFI (FAT-12/16/32)

[1]https://pdos.csail.mit.edu/6.828/2017/readings/boot-cdrom.pdf

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 .../cdrtools/cdrtools-native_3.01.bb          |  1 +
 ...ix-nsectors-exceeds-0xffff-situation.patch | 69 +++++++++++++++++++
 2 files changed, 70 insertions(+)
 create mode 100644 meta/recipes-devtools/cdrtools/cdrtools/0001-fix-nsectors-exceeds-0xffff-situation.patch

diff --git a/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb b/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb
index 085384064d0..428202c2585 100644
--- a/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb
+++ b/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb
@@ -15,6 +15,7 @@ SRC_URI = " \
 	file://0001-Don-t-set-uid-gid-during-install.patch \
 	file://riscv64-linux-gcc.rul \
 	file://gcc14-fix.patch \
+	file://0001-fix-nsectors-exceeds-0xffff-situation.patch \
 	"
 
 SRC_URI[md5sum] = "7d45c5b7e1f78d85d1583b361aee6e8b"
diff --git a/meta/recipes-devtools/cdrtools/cdrtools/0001-fix-nsectors-exceeds-0xffff-situation.patch b/meta/recipes-devtools/cdrtools/cdrtools/0001-fix-nsectors-exceeds-0xffff-situation.patch
new file mode 100644
index 00000000000..8b0fbb3fe6d
--- /dev/null
+++ b/meta/recipes-devtools/cdrtools/cdrtools/0001-fix-nsectors-exceeds-0xffff-situation.patch
@@ -0,0 +1,69 @@
+From ab6b5ee4c23099bf15ddd145fdf1ff4f5e34e802 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Sat, 26 Apr 2025 03:38:32 +0000
+Subject: [PATCH] fix nsectors exceeds 0xffff situation
+
+According to page 11: `Figure 5 - Section Entry' in El Torito Bootable
+CD-ROM Format Specification [1]. The sector count takes 2 byte which
+means max sector count is 0xffff (65535), for 512-byte sector, the
+size of bootable image is no more than 32MB (65536 * 512 / 1024 / 1024)
+
+If the size of efi.img > 32MB, the partition table will be truncated
+in ISO, which caused UEFI system or grub-efi read efi.img broken
+occasionally.
+
+In this patch, nsectors means sector count, if it exceeds 0xffff,
+truncate to 0xffff and set selection criteria type = 2, then save
+extra nsectors to vendor unique selection criteria
+
+[1]https://pdos.csail.mit.edu/6.828/2017/readings/boot-cdrom.pdf
+
+Upstream-Status: Inappropriate [upstream https://sourceforge.net/projects/cdrtools/ is not alive since 2019]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ mkisofs/eltorito.c | 28 ++++++++++++++++++++++++++++
+ 1 file changed, 28 insertions(+)
+
+diff --git a/mkisofs/eltorito.c b/mkisofs/eltorito.c
+index 5dd04bc..a391003 100644
+--- a/mkisofs/eltorito.c
++++ b/mkisofs/eltorito.c
+@@ -568,6 +568,34 @@ fill_boot_desc(boot_desc_entry, boot_entry)
+ 	fprintf(stderr, "Extent of boot images is %d\n",
+ 				get_733(de->isorec.extent));
+ #endif
++
++	// Boot sectors exceeds 0xffff
++	if (nsectors > 0xffff) {
++		unsigned int extra_nsectors = nsectors - 0xffff;
++
++		fprintf(stderr, "Warning: Boot sectors 0x%x(%u) exceeds 0xffff(65535), save extra %u to pad2\n",
++		    nsectors, nsectors, extra_nsectors);
++
++		// Set nsectors to maximum 0xffff(65535)
++		nsectors = 0xffff;
++
++		// Offset     : 0C byte
++		// Type       : Byte
++		// Description: Selection criteria type. This defines a vendor unique format
++		// for bytes 0D-1F.
++		// The following formats is reserved by Yocto:
++		// 2 - Save extra sector count to vendor unique selection criteria
++		boot_desc_entry->pad2[0] = 2;
++
++
++		// Offset     : 0D-0E-0F-10 byte
++		// Type       : D Word
++		// Description: Save extra sector count to vendor unique selection criteria.
++		// It takes 4 bytes in pad2, starting at pad2[1]
++		set_731(&boot_desc_entry->pad2[1], extra_nsectors);
++
++	}
++
+ 	set_721(boot_desc_entry->nsect, (unsigned int) nsectors);
+ 	set_731(boot_desc_entry->bootoff,
+ 		(unsigned int) get_733(de->isorec.extent));
+-- 
+2.44.1
+
-- 
2.34.1

[PATCH v2 2/2] syslinux: improve isohybrid to process extra sector count for ISO 9660 image

[Hongxu Jia]

Due to commit [cdrtools-native: fix booting EFI ISO live failed]
applied to improve mkisofs to fix nsectors exceeds 0xffff situation
which set selection criteria type = 2 and save extra nsectors to
vendor unique selection criteria

In following case, add 64MB extra space to bootable image efi.img,
and the partition table of EFI is truncated to 32M

$ echo 'IMAGE_FSTYPES:pn-core-image-minimal = " live"' >> conf/local.conf
$ echo 'MACHINE_FEATURES:append = " efi pcbios"' >> conf/local.conf
$ echo '# 64MB extra space to bootable image efi.img' >> conf/local.conf
$ echo 'BOOTIMG_EXTRA_SPACE = "65535"' >> conf/local.conf
$ bitbake core-image-minimal
$ fdisk -l tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso
...
Device                                                                 Boot Start    End Sectors  Size Id Type
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso1 *        0 376831  376832  184M  0 Empty
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso2        120  65654   65535   32M ef EFI (FAT-12/16/32)

After applying this patch to process extra sector count, the partition
table of EFI is 90.3M

$ fdisk -l tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso
...
Device                                                                 Boot Start    End Sectors  Size Id Type
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso1 *        0 376831  376832  184M  0 Empty
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.iso2        120 185151  185032 90.3M ef EFI (FAT-12/16/32)

[1]https://pdos.csail.mit.edu/6.828/2017/readings/boot-cdrom.pdf

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 ...-count-from-section-entry-for-EFI-ca.patch | 104 ++++++++++++++++++
 .../syslinux/syslinux_6.04-pre2.bb            |   1 +
 2 files changed, 105 insertions(+)
 create mode 100644 meta/recipes-devtools/syslinux/syslinux/0001-Add-extra-sector-count-from-section-entry-for-EFI-ca.patch

diff --git a/meta/recipes-devtools/syslinux/syslinux/0001-Add-extra-sector-count-from-section-entry-for-EFI-ca.patch b/meta/recipes-devtools/syslinux/syslinux/0001-Add-extra-sector-count-from-section-entry-for-EFI-ca.patch
new file mode 100644
index 00000000000..5422d9b2dea
--- /dev/null
+++ b/meta/recipes-devtools/syslinux/syslinux/0001-Add-extra-sector-count-from-section-entry-for-EFI-ca.patch
@@ -0,0 +1,104 @@
+From 79a26046d178ae132cb88ba75de7141bd169ff16 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Sat, 26 Apr 2025 10:45:08 +0800
+Subject: [PATCH] Add extra sector count from section entry for EFI catalogue
+
+According to page 11: `Figure 5 - Section Entry' in El Torito Bootable
+CD-ROM Format Specification [1]. The sector count tooks 2 byte which
+means max sector count is 0xffff (65535), for 512-byte sector, the
+size of bootable image is no more than 32MB (65536 * 512 / 1024 / 1024)
+
+If the size of efi.img > 32MB, the partition table will be truncated
+in ISO, which caused UEFI system or grub-efi read efi.img broken
+occasionally.
+
+This patch extend efi_count, mac_count and count to 4 byte int, if
+Yocto defines `Selection criteria type = 2', add extra sector count
+to original sector count as total count; for other situation, still use
+original sector count as usual
+
+[1]https://pdos.csail.mit.edu/6.828/2017/readings/boot-cdrom.pdf
+
+Upstream-Status: Inappropriate [Yocto specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ utils/isohybrid.c | 48 ++++++++++++++++++++++++++++++++++++++++++-----
+ 1 file changed, 43 insertions(+), 5 deletions(-)
+
+diff --git a/utils/isohybrid.c b/utils/isohybrid.c
+index a9e38d4..1bbfd45 100644
+--- a/utils/isohybrid.c
++++ b/utils/isohybrid.c
+@@ -76,7 +76,7 @@ uint32_t de_lba = 0;
+ uint16_t de_seg = 0, de_count = 0, de_mbz2 = 0;
+ uint8_t de_boot = 0, de_media = 0, de_sys = 0, de_mbz1 = 0;
+ uint32_t efi_lba = 0, mac_lba = 0;
+-uint16_t efi_count = 0, mac_count = 0;
++uint32_t efi_count = 0, mac_count = 0;
+ uint8_t efi_boot = 0, efi_media = 0, efi_sys = 0;
+ 
+ int apm_parts = 3;
+@@ -552,17 +552,55 @@ read_efi_section(const uint8_t *buf)
+ }
+ 
+ int
+-read_efi_catalogue(const uint8_t *buf, uint16_t *count, uint32_t *lba)
++read_efi_catalogue(const uint8_t *buf, uint32_t *count, uint32_t *lba)
+ {
++    uint16_t _count = 0;
++
++    // Jump to offset 6 byte
+     buf += 6;
+ 
+-    memcpy(count, buf, 2);
+-    *count = lendian_short(*count);
++    *count = 0;
++
++    // Offset     : 6-7 byte
++    // Type       : Word
++    // Description: Sector Count, the number of virtual/emulated sectors
++    // the system will store at Load Segment during the initial boot procedure
++    memcpy(&_count, buf, 2);
++    _count = lendian_short(_count);
+     buf += 2;
+ 
++    // Offset     : 8-0B byte
++    // Type       : D Word
++    // Description: Load RBA. This is the start address of the virtual disk. CD’s use
++    // Relative/Logical block addressing.
+     memcpy(lba, buf, 4);
+     *lba = lendian_int(*lba);
+-    buf += 6;
++    buf += 4;
++
++    // Offset     : 0C byte
++    // Type       : Byte
++    // Description: Selection criteria type. This defines a vendor unique format
++    // for bytes 0D-1F.
++    // The following formats have currently been assigned:
++    // 0 - No selection criteria
++    // 1 - Language and Version Information (IBM)
++    // 2 - Save extra sector count to vendor unique selection criteria (Yocto)
++    // 3-FF - Reserved
++    unsigned char slection_criteria_type = *buf;
++    buf += 1;
++
++    // Offset     : 0D-0E-0F-10 byte
++    // Type       : D Word
++    // Description: Selection criteria type = 2, reserved by Yocto,
++    // save extra sector count to vendor unique selection criteria
++    if (slection_criteria_type == 2) {
++        memcpy(count, buf, 4);
++        *count = lendian_int(*count);
++        buf += 4;
++    }
++
++    // total count = sector count + extra sector count
++    *count += _count;
+ 
+     return 0;
+ }
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/syslinux/syslinux_6.04-pre2.bb b/meta/recipes-devtools/syslinux/syslinux_6.04-pre2.bb
index 7cf737170c3..2522205fa1c 100644
--- a/meta/recipes-devtools/syslinux/syslinux_6.04-pre2.bb
+++ b/meta/recipes-devtools/syslinux/syslinux_6.04-pre2.bb
@@ -23,6 +23,7 @@ SRC_URI = "https://www.zytor.com/pub/syslinux/Testing/6.04/syslinux-${PV}.tar.xz
            file://0013-remove-clean-script.patch \
            file://0014-Fix-reproducibility-issues.patch \
            file://0001-ext2_fs.h-do-not-carry-an-outdated-copy.patch \
+           file://0001-Add-extra-sector-count-from-section-entry-for-EFI-ca.patch \
            "
 
 SRC_URI[md5sum] = "2b31c78f087f99179feb357da312d7ec"
-- 
2.34.1

Re: [OE-core] [PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[Alexander Kanavin]

On Sat, 26 Apr 2025 at 10:03, hongxu via lists.openembedded.org
<hongxu.jia=eng.windriver.com@lists.openembedded.org> wrote:

> +Upstream-Status: Inappropriate [upstream https://sourceforge.net/projects/cdrtools/ is not alive since 2019]

We have a dedicated status for this situation: Inactive-Upstream.

But, there is an actively maintained fork of the original cdrtools here:
https://codeberg.org/schilytools/schilytools

Can you check if that can be used, and whether this fix is still needed there?

Alex

Re: [OE-core] [PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[Ross Burton]

On 28 Apr 2025, at 10:31, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> 
> On Sat, 26 Apr 2025 at 10:03, hongxu via lists.openembedded.org
> <hongxu.jia=eng.windriver.com@lists.openembedded.org> wrote:
> 
>> +Upstream-Status: Inappropriate [upstream https://sourceforge.net/projects/cdrtools/ is not alive since 2019]
> 
> We have a dedicated status for this situation: Inactive-Upstream.
> 
> But, there is an actively maintained fork of the original cdrtools here:
> https://codeberg.org/schilytools/schilytools
> 
> Can you check if that can be used, and whether this fix is still needed there?

There’s also the libburnia tools (in meta-oe) which is mkisofs-compatible and is a trivial change to image_live to use, see poky-contrib:ross/iso.

Switching cdrtools for schilytools would be a sensible start, even if “actively maintained” might be stretching the definition a little: last commit a year ago and twelve open PRs.  libburnia is a more complex switch and whilst very active in git, they’re in desperate need of actually making new releases.

Ross

Re: [PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

Hi Alex,

I will update to Inactive-Upstream. in v3

About schilytools, it could not be built from cdrtools recipe directly,  the reason is schilytools using a bootstrap 'smake' binary for compiling
which is created in the psmake/ directory from source [1].  We need to create the recipe schilytools from scratch.

Due to cdrtools have license problem[2], we only have cdrtools-native support in Yocto. For schilytools - the fork of cdrtools, I think we should only support schilytools-native also

[1] https://codeberg.org/schilytools/schilytools/commit/a72089c47380426830118d318d59a65e9e1b2aff
[2] https://lwn.net/Articles/195167/

Hi Ross,

For libisoburn[3], it is not alive since 2023

[3] https://files.libburnia-project.org/

Hi RP,

What is your suggestions, is it necessary to use schilytools or libisoburn to instead of cdrtools in oe-core. It beyond the scope of this bug fix patch

//Hongxu

[-- Attachment #2: Type: text/html, Size: 2205 bytes --]

Re: [OE-core] [PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[Alexander Kanavin]

On Mon, 28 Apr 2025 at 18:55, hongxu via lists.openembedded.org
<hongxu.jia=eng.windriver.com@lists.openembedded.org> wrote:
> I will update to Inactive-Upstream. in v3
>
> About schilytools, it could not be built from cdrtools recipe directly,  the reason is schilytools using a bootstrap 'smake' binary for compiling
> which is created in the psmake/ directory from source [1].  We need to create the recipe schilytools from scratch.
>
> Due to cdrtools have license problem[2], we only have cdrtools-native support in Yocto. For schilytools - the fork of cdrtools, I think we should only support schilytools-native also
>
> [1] https://codeberg.org/schilytools/schilytools/commit/a72089c47380426830118d318d59a65e9e1b2aff
> [2] https://lwn.net/Articles/195167/
>
> Hi Ross,
>
> For libisoburn[3], it is not alive since 2023
>
> [3] https://files.libburnia-project.org/
>
> Hi RP,
>
> What is your suggestions, is it necessary to use schilytools or libisoburn to instead of cdrtools in oe-core. It beyond the scope of this bug fix patch

This was discussed in the patch review meeting today. The patches are
okay, and will be merged. Like Ross said, schilytools seems like it's
close to not being maintained either, so I'm not anymore sure if it
makes sense to try to move to it, but if you find a bit of time and
look into it, that'd be appreciated. This whole area covered by
cdrtools/syslinux is an obsolete retro technology that no one really
wants to keep alive.

Alex

Re: [PATCH v2 1/2] cdrtools-native: fix booting EFI ISO live failed

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 76 bytes --]

Copy, thanks for sharing the information of the review meeting

//Hongxu

[-- Attachment #2: Type: text/html, Size: 115 bytes --]