[PATCH v5 1/4] u-boot: disable CONFIG_BLOBLIST on genericarm64 and qemuarm64

[PATCH v5 1/4] u-boot: disable CONFIG_BLOBLIST on genericarm64 and qemuarm64

[Mikko Rapeli]

Booting u-boot on qemu with kvm is currently hanging on aarch64
build host. Root cause is in u-boot and CONFIG_BLOBLIST can be
disabled as a workaround.

To reproduce, build on kvm enabled host where "kvm-ok"
succeeds. For example genericarm64 machine and core-image-base
should then boot with:

$ runqemu slirp nographic novga snapshot kvm

On qemuarm64, default kvm setup will boot directly to kernel
and is not affected by this. If build enables u-boot as bios
then the same issue happens.

Without this config workaround, the boot hangs without
any messages in qemu output but ctrl-a-c to qemu console
can shutdown the emulated machine.

This seems to have regressed after u-boot 2025.04 update.
KVM boot can be detected from speed, for example genericarm64
boots in 550 ms with KVM and without in over 5 seconds.

Fixes: [YOCTO #15872]

Upstream u-boot discussion:
https://lists.denx.de/pipermail/u-boot/2025-May/590101.html

Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/recipes-bsp/u-boot/files/disable-CONFIG_BLOBLIST.cfg | 1 +
 meta/recipes-bsp/u-boot/u-boot-common.inc                 | 4 ++++
 2 files changed, 5 insertions(+)
 create mode 100644 meta/recipes-bsp/u-boot/files/disable-CONFIG_BLOBLIST.cfg

v4, v5: no changes, posted together with selftest for feature

v3: added u-boot discussion link to commit message
    https://lists.openembedded.org/g/openembedded-core/message/217105

v2: applying to genericarm64 and qemuarm64 machines only
    https://lists.openembedded.org/g/openembedded-core/message/217054

v1: https://lists.openembedded.org/g/openembedded-core/message/217030

diff --git a/meta/recipes-bsp/u-boot/files/disable-CONFIG_BLOBLIST.cfg b/meta/recipes-bsp/u-boot/files/disable-CONFIG_BLOBLIST.cfg
new file mode 100644
index 0000000000..d01d3d12d8
--- /dev/null
+++ b/meta/recipes-bsp/u-boot/files/disable-CONFIG_BLOBLIST.cfg
@@ -0,0 +1 @@
+# CONFIG_BLOBLIST is not set
diff --git a/meta/recipes-bsp/u-boot/u-boot-common.inc b/meta/recipes-bsp/u-boot/u-boot-common.inc
index fd1eab5cdd..a77c49cb8b 100644
--- a/meta/recipes-bsp/u-boot/u-boot-common.inc
+++ b/meta/recipes-bsp/u-boot/u-boot-common.inc
@@ -16,6 +16,10 @@ SRCREV = "34820924edbc4ec7803eb89d9852f4b870fa760a"
 
 SRC_URI = "git://source.denx.de/u-boot/u-boot.git;protocol=https;branch=master;tag=v${PV}"
 
+# workaround for aarch64 kvm qemu boot regression
+SRC_URI:append:qemuarm64 = " file://disable-CONFIG_BLOBLIST.cfg"
+SRC_URI:append:genericarm64 = " file://disable-CONFIG_BLOBLIST.cfg"
+
 S = "${WORKDIR}/git"
 B = "${WORKDIR}/build"
 
-- 
2.43.0

[PATCH v5 2/4] qemuarm64.conf: allow overriding QB_OPT_APPEND

[Mikko Rapeli]

It enables qemu USB support which causes a reset
loop currently with u-boot.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/conf/machine/qemuarm64.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
index d310445a34..f086729641 100644
--- a/meta/conf/machine/qemuarm64.conf
+++ b/meta/conf/machine/qemuarm64.conf
@@ -20,7 +20,7 @@ QB_SMP ?= "-smp 4"
 QB_CPU_KVM = "-cpu host -machine gic-version=3"
 # For graphics to work we need to define the VGA device as well as the necessary USB devices
 QB_GRAPHICS = "-device virtio-gpu-pci"
-QB_OPT_APPEND = "-device qemu-xhci -device usb-tablet -device usb-kbd"
+QB_OPT_APPEND ?= "-device qemu-xhci -device usb-tablet -device usb-kbd"
 # Virtio Networking support
 QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
 QB_NETWORK_DEVICE = "-device virtio-net-pci,netdev=net0,mac=@MAC@"
-- 
2.43.0

[PATCH v5 3/4] oeqa decorator/data.py: add skipIfNotBuildArch decorator

[Mikko Rapeli]

To limit tests to specific build host architectures.
For example KVM testing will only work if target and
build architectures are the same.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/core/decorator/data.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
index 5444b2cb75..0daf46334f 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -228,3 +228,15 @@ class skipIfNotArch(OETestDecorator):
         arch = self.case.td['HOST_ARCH']
         if arch not in self.archs:
              self.case.skipTest('Test skipped on %s' % arch)
+
+@registerDecorator
+class skipIfNotBuildArch(OETestDecorator):
+    """
+    Skip test if BUILD_ARCH is not present in the tuple specified.
+    """
+
+    attrs = ('archs',)
+    def setUpDecorator(self):
+        arch = self.case.td['BUILD_ARCH']
+        if arch not in self.archs:
+             self.case.skipTest('Test skipped on %s' % arch)
-- 
2.43.0

[PATCH v5 4/4] oeqa selftest uboot.py: add qemu KVM test case

[Mikko Rapeli]

Add a test case to boot target system via u-boot
using qemu with KVM. This was broken recently
and workaround proposed to u-boot. Test case
works with genericarm64 and qemuarm64 target machines
compiled and tested on aarch64 build host with KVM
support.

Test execution time with full sstate cache is
around 170 seconds. qemu boot itself takes just
a few seconds to full userspace.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/selftest/cases/uboot.py | 62 ++++++++++++++++++++++++++-
 1 file changed, 60 insertions(+), 2 deletions(-)

v5: added WKS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot" to make
    sure EFI loader is compiled before wic runs, fixes for this are queued
    separately, hopefully fixes autobuilder side boot failure

v4: use skipIfNotBuildArch, variable for image name except config where
    bitbake syntax causes python parsing errors when replacing variables,
    check that target host arch and build arch are the same for KVM to work,
    fixed QB_DRIVE_TYPE syntax to include /dev/ path,
    tested on aarch64 build machine and genericarm64 and qemuarm64 target machines

v1: https://lists.openembedded.org/g/openembedded-core/message/217107

diff --git a/meta/lib/oeqa/selftest/cases/uboot.py b/meta/lib/oeqa/selftest/cases/uboot.py
index 96da4efb06..4e203c4741 100644
--- a/meta/lib/oeqa/selftest/cases/uboot.py
+++ b/meta/lib/oeqa/selftest/cases/uboot.py
@@ -6,8 +6,8 @@
 #
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu
-from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.utils.commands import bitbake, runqemu, get_bb_var, get_bb_vars, runCmd
+from oeqa.core.decorator.data import skipIfNotArch, skipIfNotBuildArch
 from oeqa.core.decorator import OETestTag
 
 uboot_boot_patterns = {
@@ -41,3 +41,61 @@ QEMU_USE_KVM = "False"
             status, output = qemu.run_serial(cmd)
             self.assertEqual(status, 1, msg=output)
             self.assertTrue("U-Boot" in output, msg=output)
+
+    @skipIfNotArch(['aarch64'])
+    @skipIfNotBuildArch(['aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_uboot_kvm_to_full_target(self):
+        """
+        Tests building u-boot and booting it with QEMU and KVM.
+        Requires working KVM on build host. See "kvm-ok" output.
+        """
+
+        runCmd("kvm-ok")
+
+        image = "core-image-minimal"
+        vars = get_bb_vars(['HOST_ARCH', 'BUILD_ARCH'], image)
+        host_arch = vars['HOST_ARCH']
+        build_arch = vars['BUILD_ARCH']
+
+        self.assertEqual(host_arch, build_arch, 'HOST_ARCH %s and BUILD_ARCH %s must match for KVM' % (host_arch, build_arch))
+
+        self.write_config("""
+QEMU_USE_KVM = "1"
+
+# Using u-boot in EFI mode, need ESP partition for grub/systemd-boot/kernel etc
+IMAGE_FSTYPES:pn-core-image-minimal:append = " wic"
+
+# easiest to follow genericarm64 setup with wks file, initrd and EFI loader
+INITRAMFS_IMAGE = "core-image-initramfs-boot"
+EFI_PROVIDER = "${@bb.utils.contains("DISTRO_FEATURES", "systemd", "systemd-boot", "grub-efi", d)}"
+WKS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot"
+WKS_FILE = "genericarm64.wks.in"
+
+# use wic image with ESP for u-boot, not ext4
+QB_DEFAULT_FSTYPE = "wic"
+
+PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
+QB_DEFAULT_BIOS = "u-boot.bin"
+
+# let u-boot or EFI loader load kernel from ESP
+QB_DEFAULT_KERNEL = "none"
+
+# virt pci, not scsi because support not in u-boot to find ESP
+QB_DRIVE_TYPE = "/dev/vd"
+# qemu usb causes u-boot reset atm
+QB_OPT_APPEND = ""
+""")
+        bitbake("virtual/bootloader %s" % image)
+
+        runqemu_params = get_bb_var('TEST_RUNQEMUPARAMS', image) or ""
+        with runqemu(image, ssh=False, runqemuparams='nographic kvm %s' % runqemu_params) as qemu:
+
+            # boot to target and login worked, should have been fast with kvm
+            cmd = "dmesg"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            # Machine is qemu
+            self.assertTrue("Machine model: linux,dummy-virt" in output, msg=output)
+            # with KVM enabled
+            self.assertTrue("KVM: hypervisor services detected" in output, msg=output)
-- 
2.43.0

Re: [OE-core] [PATCH v5 2/4] qemuarm64.conf: allow overriding QB_OPT_APPEND

[Richard Purdie]

On Tue, 2025-05-27 at 12:07 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> It enables qemu USB support which causes a reset
> loop currently with u-boot.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  meta/conf/machine/qemuarm64.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> index d310445a34..f086729641 100644
> --- a/meta/conf/machine/qemuarm64.conf
> +++ b/meta/conf/machine/qemuarm64.conf
> @@ -20,7 +20,7 @@ QB_SMP ?= "-smp 4"
>  QB_CPU_KVM = "-cpu host -machine gic-version=3"
>  # For graphics to work we need to define the VGA device as well as the necessary USB devices
>  QB_GRAPHICS = "-device virtio-gpu-pci"
> -QB_OPT_APPEND = "-device qemu-xhci -device usb-tablet -device usb-kbd"
> +QB_OPT_APPEND ?= "-device qemu-xhci -device usb-tablet -device usb-kbd"
>  # Virtio Networking support
>  QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>  QB_NETWORK_DEVICE = "-device virtio-net-pci,netdev=net0,mac=@MAC@"

I am very worried about merging patches specifically to disable usb
(here and in the selftest) since that is something we expect to work to
allow the graphics images to work properly.

Is there an upstream discussion about why usb causes the hang and/or
any estimate on when it might get fixed? I'm partly tempted to revert
the uboot version instead of this.

Cheers,

Richard

Re: [OE-core] [PATCH v5 2/4] qemuarm64.conf: allow overriding QB_OPT_APPEND

[Mikko Rapeli]

Hi,

On Tue, May 27, 2025 at 12:53:20PM +0100, Richard Purdie wrote:
> On Tue, 2025-05-27 at 12:07 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> > It enables qemu USB support which causes a reset
> > loop currently with u-boot.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> > ---
> > �meta/conf/machine/qemuarm64.conf | 2 +-
> > �1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> > index d310445a34..f086729641 100644
> > --- a/meta/conf/machine/qemuarm64.conf
> > +++ b/meta/conf/machine/qemuarm64.conf
> > @@ -20,7 +20,7 @@ QB_SMP ?= "-smp 4"
> > �QB_CPU_KVM = "-cpu host -machine gic-version=3"
> > �# For graphics to work we need to define the VGA device as well as the necessary USB devices
> > �QB_GRAPHICS = "-device virtio-gpu-pci"
> > -QB_OPT_APPEND = "-device qemu-xhci -device usb-tablet -device usb-kbd"
> > +QB_OPT_APPEND ?= "-device qemu-xhci -device usb-tablet -device usb-kbd"
> > �# Virtio Networking support
> > �QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
> > �QB_NETWORK_DEVICE = "-device virtio-net-pci,netdev=net0,mac=@MAC@"
> 
> I am very worried about merging patches specifically to disable usb
> (here and in the selftest) since that is something we expect to work to
> allow the graphics images to work properly.
>
> Is there an upstream discussion about why usb causes the hang and/or
> any estimate on when it might get fixed? I'm partly tempted to revert
> the uboot version instead of this.

Please allow downstream configurations to override this. They may not
care about USB emulation with qemu/runqemu. Several layers try to
build upon qemuarm64 oe-core machine but may need to tweak things
to get to a working test setup.

With Linux kernel AFAIK qemu USB works well, and qemu itself.
u-boot is a different beast but we have developers working
on improving the situation and I've notified them of this
easy to test scenario. u-boot side needs to sync the USB
stack with Linux kernel side changes and this is not a small
task. Sadly no upstream discussion link currently.

Not a perfect answer but better than nothing I hope.

Cheers,

-Mikko

Re: [OE-core] [PATCH v5 2/4] qemuarm64.conf: allow overriding QB_OPT_APPEND

[Mikko Rapeli]

Hi,

On Tue, May 27, 2025 at 12:53:20PM +0100, Richard Purdie wrote:
> On Tue, 2025-05-27 at 12:07 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> > It enables qemu USB support which causes a reset
> > loop currently with u-boot.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> > ---
> > �meta/conf/machine/qemuarm64.conf | 2 +-
> > �1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> > index d310445a34..f086729641 100644
> > --- a/meta/conf/machine/qemuarm64.conf
> > +++ b/meta/conf/machine/qemuarm64.conf
> > @@ -20,7 +20,7 @@ QB_SMP ?= "-smp 4"
> > �QB_CPU_KVM = "-cpu host -machine gic-version=3"
> > �# For graphics to work we need to define the VGA device as well as the necessary USB devices
> > �QB_GRAPHICS = "-device virtio-gpu-pci"
> > -QB_OPT_APPEND = "-device qemu-xhci -device usb-tablet -device usb-kbd"
> > +QB_OPT_APPEND ?= "-device qemu-xhci -device usb-tablet -device usb-kbd"
> > �# Virtio Networking support
> > �QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
> > �QB_NETWORK_DEVICE = "-device virtio-net-pci,netdev=net0,mac=@MAC@"
> 
> I am very worried about merging patches specifically to disable usb
> (here and in the selftest) since that is something we expect to work to
> allow the graphics images to work properly.
> 
> Is there an upstream discussion about why usb causes the hang and/or
> any estimate on when it might get fixed? I'm partly tempted to revert
> the uboot version instead of this.

I've reported this to upstream u-boot list now
https://lists.denx.de/pipermail/u-boot/2025-June/591233.html

The bisection points at enabling USB support for qemu so
the issue seems to have always been there and possibly
quite fundamental.

Would it be acceptable to disable USB support in
qemuarm64/genericarm64 u-boot as a workaround while
keeping USB in qemu?

I tested this and qemu USB devices were still
detected in by Linux kernel and u-boot just ignored them.

So for u-boot aarch64 KVM support under qemu, both
CONFIG_BLOBLIST and CONFIG_USB currently need to be
disabled. Both issues reported to upstream and hopefully
better fixes get developed. We could apply the workarounds
on qemuarm64 and genericarm64 and improve testing to cover
KVM u-boot with qemu. KVM could improve qemu testing times a
lot on aarch64/qemuarm64/genericarm64.

Cheers,

-Mikko