[PATCH v6 0/7] Add barebox bootloader support (and testing)

[PATCH v6 0/7] Add barebox bootloader support (and testing)

[Enrico Jörns]

This adds support for the barebox bootloader (and tools) to oe-core.

In order to have proper testing, this extends oe-selftest to allow
basic testing of bootloaders. While at it, cover both u-boot and barebox.

v6:
* replace barebox.inc by barebox.bbclass (move content of
  barebox-common.inc into .bbclass and .bb files)
* support for UNPACKDIR

Enrico Jorns (5):
  barebox: set default BAREBOX_CONFIG for qemu machines
  oeqa/utils/qemurunner: support ignoring vt100 escape sequences
  oeqa: support passing custom boot patterns to runqemu
  oeqa/selftest/cases: add basic u-boot test
  oeqa/selftest/cases: add basic barebox tests

Marco Felsch (2):
  barebox: add initial support
  barebox-tools: add initial barebox tools support

 meta/classes-recipe/barebox.bbclass           | 160 ++++++++++++++++++
 meta/conf/distro/include/maintainers.inc      |   2 +
 meta/conf/machine/qemuarm.conf                |   1 +
 meta/conf/machine/qemuarm64.conf              |   1 +
 meta/conf/machine/qemux86-64.conf             |   1 +
 meta/lib/oeqa/selftest/cases/barebox.py       |  43 +++++
 meta/lib/oeqa/selftest/cases/uboot.py         |  42 +++++
 meta/lib/oeqa/targetcontrol.py                |   5 +-
 meta/lib/oeqa/utils/commands.py               |   5 +-
 meta/lib/oeqa/utils/qemurunner.py             |   6 +-
 .../barebox/barebox-tools_2024.08.0.bb        |  59 +++++++
 meta/recipes-bsp/barebox/barebox_2024.08.0.bb |  14 ++
 12 files changed, 333 insertions(+), 6 deletions(-)
 create mode 100644 meta/classes-recipe/barebox.bbclass
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb
 create mode 100644 meta/recipes-bsp/barebox/barebox_2024.08.0.bb

-- 
2.39.2

[PATCH v6 1/7] barebox: add initial support

[Enrico Jörns]

From: Marco Felsch <m.felsch@pengutronix.de>

This adds the support for the barebox bootloader to oe-core. The recipe
is inspired by meta-ptx [1] but is a major rework of the one found there.

Barebox comes with a wide range of supported architectures and follows
the concepts of Linux in various aspects like the driver model, the
shell, or the virtual file system.
This not only eases porting Linux drivers but also makes barebox a
developer-friendly and feature-rich bootloader alternative [2].

For barebox (like for the kernel or other bootloaders) it is quite
likely that people will not just build the original recipe but need to
adapt it, point to custom repositories, apply patch stacks,
COMPATIBLE_MACHINE etc. They may also choose to have different recipe
names for different variants.
Having only a single .bb file and requiring to copy or .bbappend it is
inconvenient and results in unnecessary code duplication. Therefore, the
base support for building barebox is encapsulated in barebox.bbclass
(like kernel.bbclass for the kernel).

[1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox
[2] https://www.barebox.org/demo/?graphic=0

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/classes-recipe/barebox.bbclass           | 160 ++++++++++++++++++
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/recipes-bsp/barebox/barebox_2024.08.0.bb |  14 ++
 3 files changed, 175 insertions(+)
 create mode 100644 meta/classes-recipe/barebox.bbclass
 create mode 100644 meta/recipes-bsp/barebox/barebox_2024.08.0.bb

diff --git a/meta/classes-recipe/barebox.bbclass b/meta/classes-recipe/barebox.bbclass
new file mode 100644
index 0000000000..200ba08326
--- /dev/null
+++ b/meta/classes-recipe/barebox.bbclass
@@ -0,0 +1,160 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit kernel-arch deploy cml1 pkgconfig
+
+LICENSE ?= "GPL-2.0-only"
+
+PROVIDES += "virtual/bootloader"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+DEPENDS += "bison-native flex-native"
+
+S = "${WORKDIR}/barebox-${PV}"
+B = "${WORKDIR}/build"
+
+require conf/image-uefi.conf
+
+# For some platforms and configuration, the barebox build process will require
+# additional host tools that can be activated/deactivated here.
+PACKAGECONFIG ??= "openssl libusb fit"
+
+PACKAGECONFIG[openssl] = ",,openssl-native"
+PACKAGECONFIG[libusb] = ",,libusb1-native"
+PACKAGECONFIG[fit] = ",,u-boot-tools-native dtc-native"
+
+export KBUILD_BUILD_USER ?= "oe-user"
+export KBUILD_BUILD_HOST ?= "oe-host"
+
+# unlike the kernel, barebox may build against host tools like openssl
+export HOST_EXTRACFLAGS
+
+def get_layer_rev(path):
+    try:
+        rev, _ = bb.process.run("git describe --match='' --always --dirty --broken", cwd=path)
+    except bb.process.ExecutionError:
+        rev = ""
+    return rev.strip()
+
+BAREBOX_BUILDSYSTEM_VERSION[doc] = "Build system version to add to the barebox image. By default this is the git description of the containing layer."
+BAREBOX_BUILDSYSTEM_VERSION ??= "${@get_layer_rev(os.path.dirname(d.getVar('FILE')))}"
+
+BAREBOX_FIRMWARE_DIR[doc] = "Overwrite barebox' firmware blobs search directory (CONFIG_EXTRA_FIRMWARE_DIR) with this path, default ${B}/firmware"
+BAREBOX_FIRMWARE_DIR ??= "${B}/firmware"
+
+EXTRA_OEMAKE = " \
+    CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
+    BUILDSYSTEM_VERSION=${BAREBOX_BUILDSYSTEM_VERSION} \
+    CONFIG_EXTRA_FIRMWARE_DIR=${BAREBOX_FIRMWARE_DIR} \
+    PKG_CONFIG=pkg-config-native \
+    CROSS_PKG_CONFIG=pkg-config \
+"
+
+BAREBOX_CONFIG[doc] = "The barebox kconfig defconfig file. Not used if a file called defconfig is added to the SRC_URI."
+BAREBOX_CONFIG ?= ""
+
+# set sensible default configs for some of oe-core's QEMU MACHINEs
+BAREBOX_CONFIG:qemuarm = "multi_v7_defconfig"
+BAREBOX_CONFIG:qemuarm64 = "multi_v8_defconfig"
+BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
+
+# prevent from acting as non-buildable provider
+python () {
+    bareboxconfig = d.getVar('BAREBOX_CONFIG')
+    bareboxdefconfig = 'file://defconfig' in d.getVar('SRC_URI')
+
+    if not bareboxconfig and not bareboxdefconfig:
+        raise bb.parse.SkipRecipe("BAREBOX_CONFIG must be set in the %s machine configuration or file://defconfig must be given in SRC_URI." % d.getVar("MACHINE"))
+}
+
+barebox_do_configure() {
+        if [ -e ${UNPACKDIR}/defconfig ]; then
+                cp ${UNPACKDIR}/defconfig ${B}/.config
+        else
+                if [ -n "${BAREBOX_CONFIG}" ]; then
+                        oe_runmake ${BAREBOX_CONFIG}
+                else
+                        bbfatal "No defconfig given. Either add file 'file://defconfig' to SRC_URI or set BAREBOX_CONFIG"
+                fi
+        fi
+
+        ${S}/scripts/kconfig/merge_config.sh -m .config ${@" ".join(find_cfgs(d))}
+        cml1_do_configure
+}
+
+BAREBOX_ENV_DIR[doc] = "Overlay the barebox built-in environment with the environment provided by the BSP if specified."
+BAREBOX_ENV_DIR ??= "${UNPACKDIR}/env/"
+
+barebox_do_compile () {
+        export userccflags="${TARGET_LDFLAGS}${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+        unset LDFLAGS
+        unset CFLAGS
+        unset CPPFLAGS
+        unset CXXFLAGS
+        unset MACHINE
+        # Allow to use ${UNPACKDIR} in kconfig options to include additionally fetched files
+        export UNPACKDIR=${UNPACKDIR}
+
+        if [ -d ${BAREBOX_ENV_DIR} ]; then
+                BAREBOX_DEFAULT_ENV="$(grep ^CONFIG_DEFAULT_ENVIRONMENT_PATH .config | cut -d '=' -f 2 | tr -d '"')"
+                oe_runmake CONFIG_DEFAULT_ENVIRONMENT_PATH="\"${BAREBOX_DEFAULT_ENV} ${BAREBOX_ENV_DIR}\""
+        else
+                oe_runmake
+        fi
+}
+
+BAREBOX_BINARY[doc] = "Specify the barebox binary to install. If not specified all barebox artifacts are installed."
+BAREBOX_BINARY ??= "${@'barebox.efi' if d.getVar('EFI_PROVIDER') == 'barebox' else ''}"
+BAREBOX_SUFFIX[doc] = "Specify the suffix for ${BAREBOX_IMAGE}."
+BAREBOX_SUFFIX ??= "img"
+BAREBOX_IMAGE[doc] = "A unique barebox image name. Unused if ${BAREBOX_BINARY} is not set."
+BAREBOX_IMAGE_DEFAULT ?= "${PN}-${MACHINE}-${PV}-${PR}.${BAREBOX_SUFFIX}"
+BAREBOX_IMAGE ?= "${@'${EFI_BOOT_IMAGE}' if d.getVar('EFI_PROVIDER') == 'barebox' else '${BAREBOX_IMAGE_DEFAULT}'}"
+
+BAREBOX_INSTALL_PATH ?= "${@'${EFI_FILES_PATH}' if d.getVar('EFI_PROVIDER') == 'barebox' else '/boot'}"
+
+barebox_do_install () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        bbfatal "Failed to locate ${BAREBOX_BINARY}"
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_BINARY}
+        else
+                install -d ${D}${BAREBOX_INSTALL_PATH}/
+                for image in $(cat ${B}/barebox-flash-images); do
+                        install -m 644 ${B}/${image} ${D}${BAREBOX_INSTALL_PATH}/
+                done
+        fi
+}
+FILES:${PN} = "${BAREBOX_INSTALL_PATH}"
+
+barebox_do_deploy () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${DEPLOYDIR}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${DEPLOYDIR}/${BAREBOX_BINARY}
+        else
+                for image in $(cat ${B}/barebox-flash-images); do
+                        cp ${B}/${image} ${DEPLOYDIR}
+                done
+        fi
+}
+addtask deploy after do_compile
+
+EXPORT_FUNCTIONS do_configure do_compile do_install do_deploy
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 60f36a9449..1b3d485e48 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -54,6 +54,7 @@ RECIPE_MAINTAINER:pn-automake = "Robert Yang <liezhi.yang@windriver.com>"
 RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <alex.kanavin@gmail.com>"
+RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <yocto@pengutronix.de>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <alejandro@enedino.org>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <anuj.mittal@intel.com>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <anuj.mittal@intel.com>"
diff --git a/meta/recipes-bsp/barebox/barebox_2024.08.0.bb b/meta/recipes-bsp/barebox/barebox_2024.08.0.bb
new file mode 100644
index 0000000000..1afefa76ab
--- /dev/null
+++ b/meta/recipes-bsp/barebox/barebox_2024.08.0.bb
@@ -0,0 +1,14 @@
+SUMMARY = "barebox is a bootloader designed for embedded systems. It runs on a variety of architectures including x86, ARM, MIPS, PowerPC and others."
+DESCRIPTION = "barebox aims to be a versatile and flexible bootloader not only for booting embedded Linux systems, \
+but also for initial hardware bringup and development. \
+Users should feel right at home with a shell with UNIX-like virtual file system access to hardware, \
+Linux kernel driver API for making driver porting easier, \
+and a subset of the POSIX C library for writing more command-line utilities."
+SECTION = "bootloaders"
+
+LIC_FILES_CHKSUM = "file://COPYING;md5=f5125d13e000b9ca1f0d3364286c4192"
+
+inherit barebox
+
+SRC_URI = "https://barebox.org/download/barebox-${PV}.tar.bz2"
+SRC_URI[sha256sum] = "b08a762da8d63dd18b4f2d9f5d0a8da001b6e608d1b3eff6dcebc6a2e575d535"
-- 
2.39.2

[PATCH v6 2/7] barebox-tools: add initial barebox tools support

[Enrico Jörns]

From: Marco Felsch <m.felsch@pengutronix.de>

Add initial support to build the barebox tools for the host and the
target.

Some of the tools are generic barebox utilities (like 'bareboximd' for
image meta data inspection or 'bareboxenv' for accessing the barebox
environment) some are SoC family-specific specific utilities (like
'imx-usb-loader' or 'omap3-usb-loader').

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/conf/distro/include/maintainers.inc      |  1 +
 .../barebox/barebox-tools_2024.08.0.bb        | 59 +++++++++++++++++++
 2 files changed, 60 insertions(+)
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 1b3d485e48..067c4af096 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -55,6 +55,7 @@ RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <yocto@pengutronix.de>"
+RECIPE_MAINTAINER:pn-barebox-tools = "Enrico Jörns <yocto@pengutronix.de>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <alejandro@enedino.org>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <anuj.mittal@intel.com>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <anuj.mittal@intel.com>"
diff --git a/meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb b/meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb
new file mode 100644
index 0000000000..ed49ce5373
--- /dev/null
+++ b/meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb
@@ -0,0 +1,59 @@
+SUMMARY = "barebox bootloader tools"
+HOMEPAGE = "https://barebox.org/"
+SECTION = "bootloaders"
+
+LICENSE = "GPL-2.0-only"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f5125d13e000b9ca1f0d3364286c4192"
+
+DEPENDS = "bison-native flex-native libusb1"
+
+SRC_URI = "https://barebox.org/download/barebox-${PV}.tar.bz2"
+SRC_URI[sha256sum] = "b08a762da8d63dd18b4f2d9f5d0a8da001b6e608d1b3eff6dcebc6a2e575d535"
+
+S = "${WORKDIR}/barebox-${PV}"
+B = "${WORKDIR}/build"
+
+inherit pkgconfig
+
+EXTRA_OEMAKE = " \
+  ARCH=sandbox \
+  CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
+  CROSS_PKG_CONFIG=pkg-config \
+  CC='${CC}' \
+  LD='${LD}' \
+  "
+
+do_compile:class-target () {
+    export userccflags="${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+    export userldflags="${TARGET_LDFLAGS}${TOOLCHAIN_OPTIONS}"
+    oe_runmake targettools_defconfig
+    oe_runmake scripts
+}
+
+do_compile:class-native () {
+    oe_runmake hosttools_defconfig
+    oe_runmake scripts
+}
+
+BAREBOX_TOOLS = " \
+         bareboxenv \
+         bareboxcrc32 \
+         kernel-install \
+         bareboximd \
+         omap3-usb-loader \
+         omap4_usbboot \
+         imx/imx-usb-loader \
+         "
+
+BAREBOX_TOOLS_SUFFIX = ""
+BAREBOX_TOOLS_SUFFIX:class-target = "-target"
+
+do_install () {
+	install -d ${D}${bindir}
+
+	for tool in ${BAREBOX_TOOLS}; do
+		install -m 0755 scripts/${tool}${BAREBOX_TOOLS_SUFFIX} ${D}${bindir}/${tool##*/}
+	done
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.39.2

[PATCH v6 3/7] barebox: set default BAREBOX_CONFIG for qemu machines

[Enrico Jörns]

From: Enrico Jorns <ejo@pengutronix.de>

These are set in the barebox recipe rather in the corresponding machines
(where they would belong otherwise) to keep the impact of barebox to
oe-core minimal for now.

"multi_v7_defconfig" is used for qemuarm since this is the default
barebox armv7 config that just enables all supported platforms.

"multi_v8_defconfig" is used for qemuarm64 sine this is the default
barebox armv8 config that just enables all supported platforms.

"efi_defconfig" is used for qemux86-64 which is the primary platform
where barebox will not be the first stage bootloader but an EFI payload.

Since these changes make barebox a provider for virtual/bootloader,
explicitly default to u-boot in the corresponding MACHINE configs to not
unnecessarily surprise users.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/conf/machine/qemuarm.conf    | 1 +
 meta/conf/machine/qemuarm64.conf  | 1 +
 meta/conf/machine/qemux86-64.conf | 1 +
 3 files changed, 3 insertions(+)

diff --git a/meta/conf/machine/qemuarm.conf b/meta/conf/machine/qemuarm.conf
index 943ce7c16a..af76a53b8e 100644
--- a/meta/conf/machine/qemuarm.conf
+++ b/meta/conf/machine/qemuarm.conf
@@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
 
 KERNEL_IMAGETYPE = "zImage"
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu_arm_defconfig"
 
 SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
index a096d964db..d310445a34 100644
--- a/meta/conf/machine/qemuarm64.conf
+++ b/meta/conf/machine/qemuarm64.conf
@@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
 
 KERNEL_IMAGETYPE = "Image"
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu_arm64_defconfig"
 
 SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
diff --git a/meta/conf/machine/qemux86-64.conf b/meta/conf/machine/qemux86-64.conf
index 14873a3b4f..4a9c6d364b 100644
--- a/meta/conf/machine/qemux86-64.conf
+++ b/meta/conf/machine/qemux86-64.conf
@@ -13,6 +13,7 @@ DEFAULTTUNE ?= "core2-64"
 require conf/machine/include/x86/tune-x86-64-v3.inc
 require conf/machine/include/x86/qemuboot-x86.inc
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
 
 KERNEL_IMAGETYPE = "bzImage"
-- 
2.39.2

[PATCH v6 4/7] oeqa/utils/qemurunner: support ignoring vt100 escape sequences

[Enrico Jörns]

From: Enrico Jorns <ejo@pengutronix.de>

If we talk to terminals that like colors, we need to ignore the vt100
escape sequences when matching strings.

An unprocessed barebox console prompt would e.g. look like:

  ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m

where we cannot match for something like "barebox@ARM QEMU virt64:/".
The same applies to colored Linux terminal output of course.

The "\x1b\[" from the regex catches the standard start of ANSI escape
sequence while the rest catches the actual command code executed.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/utils/qemurunner.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index cda43aad8c..63fc6f6b53 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -30,6 +30,8 @@ control_range = list(range(0,32))+list(range(127,160))
 control_chars = [chr(x) for x in control_range
                 if chr(x) not in string.printable]
 re_control_char = re.compile('[%s]' % re.escape("".join(control_chars)))
+# Regex to remove the ANSI (color) control codes from console strings in order to match the text only
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
 
 def getOutput(o):
     import fcntl
@@ -681,7 +683,7 @@ class QemuRunner:
                     time.sleep(0.1)
                     answer = self.server_socket.recv(1024)
                     if answer:
-                        data += answer.decode('utf-8')
+                        data += re_vt100.sub("", answer.decode('utf-8'))
                         # Search the prompt to stop
                         if re.search(self.boot_patterns['search_cmd_finished'], data):
                             break
-- 
2.39.2

[PATCH v6 5/7] oeqa: support passing custom boot patterns to runqemu

[Enrico Jörns]

From: Enrico Jorns <ejo@pengutronix.de>

This allows defining non-standard patterns from QEMU tests just as they
are already supported by the testimage.bbclass.

Will allow testing non-Linux shells in QEMU, too (e.g. a U-Boot shell or
another bootloader shell).

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/targetcontrol.py    | 5 +++--
 meta/lib/oeqa/utils/commands.py   | 5 +++--
 meta/lib/oeqa/utils/qemurunner.py | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 6e8b781973..cdf382ee21 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -88,7 +88,7 @@ class QemuTarget(BaseTarget):
 
     supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic']
 
-    def __init__(self, d, logger, image_fstype=None):
+    def __init__(self, d, logger, image_fstype=None, boot_patterns=None):
 
         import oe.types
 
@@ -141,7 +141,8 @@ class QemuTarget(BaseTarget):
                             dump_dir = dump_dir,
                             logger = logger,
                             tmpfsdir = d.getVar("RUNQEMU_TMPFS_DIR"),
-                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()))
+                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()),
+                            boot_patterns = boot_patterns)
 
         self.monitor_dumper = MonitorDumper(dump_monitor_cmds, dump_dir, self.runner)
         if (self.monitor_dumper):
diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index bf2f49d0c0..ca22d69f29 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -312,7 +312,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec=
         f.write('LAYERSERIES_COMPAT_%s = "%s"\n' % (templayername, corenames))
 
 @contextlib.contextmanager
-def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True):
+def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, boot_patterns = {}, discard_writes=True):
     """
     Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource.
     The underlying Qemu will be booted into a shell when the generator yields
@@ -330,6 +330,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
         image_fstype (str): IMAGE_FSTYPE to use
         launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation
         overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables
+        boot_patterns (dict): dict of "'<pattern-name>': value" pairs to override default boot patterns, e.g. when not booting Linux
         discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image
     """
 
@@ -361,7 +362,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
 
         logdir = recipedata.getVar("TEST_LOG_DIR")
 
-        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype)
+        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype, boot_patterns=boot_patterns)
     finally:
         # We need to shut down tinfoil early here in case we actually want
         # to run tinfoil-using utilities with the running QEMU instance.
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 63fc6f6b53..5c3a8e5999 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -103,7 +103,7 @@ class QemuRunner:
 
         # Only override patterns that were set e.g. login user TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
         for pattern in accepted_patterns:
-            if not self.boot_patterns[pattern]:
+            if pattern not in self.boot_patterns or not self.boot_patterns[pattern]:
                 self.boot_patterns[pattern] = default_boot_patterns[pattern]
 
     def create_socket(self):
-- 
2.39.2

[PATCH v6 6/7] oeqa/selftest/cases: add basic u-boot test

[Enrico Jörns]

From: Enrico Jorns <ejo@pengutronix.de>

This adds basic tests for qemuarm and qemuarm64.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/selftest/cases/uboot.py | 42 +++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py

diff --git a/meta/lib/oeqa/selftest/cases/uboot.py b/meta/lib/oeqa/selftest/cases/uboot.py
new file mode 100644
index 0000000000..936dfa655f
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/uboot.py
@@ -0,0 +1,42 @@
+# Qemu-based u-boot bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.core.decorator import OETestTag
+
+uboot_boot_patterns = {
+        'search_reached_prompt': "stop autoboot",
+        'search_login_succeeded': "=>",
+        'search_cmd_finished': "=>"
+        }
+
+
+class UBootTest(OESelftestTestCase):
+
+    @skipIfNotArch(['arm', 'aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_uboot(self):
+        """
+        Tests building u-boot and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_BIOS = "u-boot.bin"
+PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
+""")
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=uboot_boot_patterns) as qemu:
+
+            # test if u-boot console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("U-Boot" in output, msg=output)
-- 
2.39.2

[PATCH v6 7/7] oeqa/selftest/cases: add basic barebox tests

[Enrico Jörns]

From: Enrico Jorns <ejo@pengutronix.de>

This adds basic tests for qemuarm and qemuarm64.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/selftest/cases/barebox.py | 43 +++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py

diff --git a/meta/lib/oeqa/selftest/cases/barebox.py b/meta/lib/oeqa/selftest/cases/barebox.py
new file mode 100644
index 0000000000..773cf6a1c8
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/barebox.py
@@ -0,0 +1,43 @@
+# Qemu-based barebox bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.core.decorator import OETestTag
+
+barebox_boot_patterns = {
+        'search_reached_prompt': r"stop autoboot",
+        'search_login_succeeded': r"barebox@[^:]+:[^ ]+ ",
+        'search_cmd_finished': r"barebox@[a-zA-Z0-9\-\s]+:/"
+        }
+
+
+class BareboxTest(OESelftestTestCase):
+
+    @skipIfNotArch(['arm', 'aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_barebox(self):
+        """
+        Tests building barebox and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_KERNEL = "barebox-dt-2nd.img"
+PREFERRED_PROVIDER_virtual/bootloader = "barebox"
+""")
+
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=barebox_boot_patterns) as qemu:
+
+            # test if barebox console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("barebox" in output, msg=output)
-- 
2.39.2

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Richard Purdie]

On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via lists.openembedded.org wrote:
> This adds support for the barebox bootloader (and tools) to oe-core.
> 
> In order to have proper testing, this extends oe-selftest to allow
> basic testing of bootloaders. While at it, cover both u-boot and barebox.
> 
> v6:
> * replace barebox.inc by barebox.bbclass (move content of
>   barebox-common.inc into .bbclass and .bb files)
> * support for UNPACKDIR
> 
> Enrico Jorns (5):
>   barebox: set default BAREBOX_CONFIG for qemu machines
>   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
>   oeqa: support passing custom boot patterns to runqemu
>   oeqa/selftest/cases: add basic u-boot test
>   oeqa/selftest/cases: add basic barebox tests
> 
> Marco Felsch (2):
>   barebox: add initial support
>   barebox-tools: add initial barebox tools support

We had some testing capacity so I ran this through automated testing.
The one issue that showed up so far was:

https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings

which is at least easily fixed! The rest of the build is still going.

Cheers,

Richard

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Enrico Jörns]

Hi Richard,

Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard Purdie:
> On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via lists.openembedded.org wrote:
> > This adds support for the barebox bootloader (and tools) to oe-core.
> > 
> > In order to have proper testing, this extends oe-selftest to allow
> > basic testing of bootloaders. While at it, cover both u-boot and barebox.
> > 
> > v6:
> > * replace barebox.inc by barebox.bbclass (move content of
> >   barebox-common.inc into .bbclass and .bb files)
> > * support for UNPACKDIR
> > 
> > Enrico Jorns (5):
> >   barebox: set default BAREBOX_CONFIG for qemu machines
> >   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
> >   oeqa: support passing custom boot patterns to runqemu
> >   oeqa/selftest/cases: add basic u-boot test
> >   oeqa/selftest/cases: add basic barebox tests
> > 
> > Marco Felsch (2):
> >   barebox: add initial support
> >   barebox-tools: add initial barebox tools support
> 
> We had some testing capacity so I ran this through automated testing.
> The one issue that showed up so far was:
> 
> https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> 
> which is at least easily fixed! The rest of the build is still going.

thanks for the test run and the first result.

Indeed, looks easily fixable 😉
Is it worth a v7 or should I wait for other results to show up?

Regards, Enrico

> Cheers,
> 
> Richard
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Richard Purdie]

On Wed, 2024-09-18 at 12:39 +0200, Enrico Jörns wrote:
> Hi Richard,
> 
> Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard Purdie:
> > On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via
> > lists.openembedded.org wrote:
> > > This adds support for the barebox bootloader (and tools) to oe-
> > > core.
> > > 
> > > In order to have proper testing, this extends oe-selftest to
> > > allow
> > > basic testing of bootloaders. While at it, cover both u-boot and
> > > barebox.
> > > 
> > > v6:
> > > * replace barebox.inc by barebox.bbclass (move content of
> > >   barebox-common.inc into .bbclass and .bb files)
> > > * support for UNPACKDIR
> > > 
> > > Enrico Jorns (5):
> > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > >   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
> > >   oeqa: support passing custom boot patterns to runqemu
> > >   oeqa/selftest/cases: add basic u-boot test
> > >   oeqa/selftest/cases: add basic barebox tests
> > > 
> > > Marco Felsch (2):
> > >   barebox: add initial support
> > >   barebox-tools: add initial barebox tools support
> > 
> > We had some testing capacity so I ran this through automated
> > testing.
> > The one issue that showed up so far was:
> > 
> > https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> > 
> > which is at least easily fixed! The rest of the build is still
> > going.
> 
> thanks for the test run and the first result.
> 
> Indeed, looks easily fixable 😉
> Is it worth a v7 or should I wait for other results to show up?

Most of the tests have run and there were other issues in the build but
there was one other issue I noticed:

https://valkyrie.yoctoproject.org/#/builders/54/builds/149/steps/14/logs/stdio

which is a missing maintainers entry. Probably worth a v7 with those
two things fixed.

Cheers,

Richard

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Enrico Jörns]

Am Mittwoch, dem 18.09.2024 um 12:21 +0100 schrieb Richard Purdie:
> On Wed, 2024-09-18 at 12:39 +0200, Enrico Jörns wrote:
> > Hi Richard,
> > 
> > Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard Purdie:
> > > On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via
> > > lists.openembedded.org wrote:
> > > > This adds support for the barebox bootloader (and tools) to oe-
> > > > core.
> > > > 
> > > > In order to have proper testing, this extends oe-selftest to
> > > > allow
> > > > basic testing of bootloaders. While at it, cover both u-boot and
> > > > barebox.
> > > > 
> > > > v6:
> > > > * replace barebox.inc by barebox.bbclass (move content of
> > > >   barebox-common.inc into .bbclass and .bb files)
> > > > * support for UNPACKDIR
> > > > 
> > > > Enrico Jorns (5):
> > > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > > >   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
> > > >   oeqa: support passing custom boot patterns to runqemu
> > > >   oeqa/selftest/cases: add basic u-boot test
> > > >   oeqa/selftest/cases: add basic barebox tests
> > > > 
> > > > Marco Felsch (2):
> > > >   barebox: add initial support
> > > >   barebox-tools: add initial barebox tools support
> > > 
> > > We had some testing capacity so I ran this through automated
> > > testing.
> > > The one issue that showed up so far was:
> > > 
> > > https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> > > 
> > > which is at least easily fixed! The rest of the build is still
> > > going.
> > 
> > thanks for the test run and the first result.
> > 
> > Indeed, looks easily fixable 😉
> > Is it worth a v7 or should I wait for other results to show up?
> 
> Most of the tests have run and there were other issues in the build but
> there was one other issue I noticed:
> 
> https://valkyrie.yoctoproject.org/#/builders/54/builds/149/steps/14/logs/stdio
> 
> which is a missing maintainers entry. Probably worth a v7 with those
> two things fixed.

The test output says

> Unable to find recipes for the following entries in maintainers.inc:
> barebox

Could you give me a hint what that means? The barebox_2024.08.0.bb and the change in maintainers.inc
are in the same commit. So I am unsure why it does not find the recipe.


Thanks in advance and regards

Enrico

> Cheers,
> 
> Richard
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Richard Purdie]

On Wed, 2024-09-18 at 13:45 +0200, Enrico Jörns wrote:
> Am Mittwoch, dem 18.09.2024 um 12:21 +0100 schrieb Richard Purdie:
> > On Wed, 2024-09-18 at 12:39 +0200, Enrico Jörns wrote:
> > > Hi Richard,
> > > 
> > > Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard
> > > Purdie:
> > > > On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via
> > > > lists.openembedded.org wrote:
> > > > > This adds support for the barebox bootloader (and tools) to
> > > > > oe-
> > > > > core.
> > > > > 
> > > > > In order to have proper testing, this extends oe-selftest to
> > > > > allow
> > > > > basic testing of bootloaders. While at it, cover both u-boot
> > > > > and
> > > > > barebox.
> > > > > 
> > > > > v6:
> > > > > * replace barebox.inc by barebox.bbclass (move content of
> > > > >   barebox-common.inc into .bbclass and .bb files)
> > > > > * support for UNPACKDIR
> > > > > 
> > > > > Enrico Jorns (5):
> > > > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > > > >   oeqa/utils/qemurunner: support ignoring vt100 escape
> > > > > sequences
> > > > >   oeqa: support passing custom boot patterns to runqemu
> > > > >   oeqa/selftest/cases: add basic u-boot test
> > > > >   oeqa/selftest/cases: add basic barebox tests
> > > > > 
> > > > > Marco Felsch (2):
> > > > >   barebox: add initial support
> > > > >   barebox-tools: add initial barebox tools support
> > > > 
> > > > We had some testing capacity so I ran this through automated
> > > > testing.
> > > > The one issue that showed up so far was:
> > > > 
> > > > https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> > > > 
> > > > which is at least easily fixed! The rest of the build is still
> > > > going.
> > > 
> > > thanks for the test run and the first result.
> > > 
> > > Indeed, looks easily fixable 😉
> > > Is it worth a v7 or should I wait for other results to show up?
> > 
> > Most of the tests have run and there were other issues in the build
> > but
> > there was one other issue I noticed:
> > 
> > https://valkyrie.yoctoproject.org/#/builders/54/builds/149/steps/14/logs/stdio
> > 
> > which is a missing maintainers entry. Probably worth a v7 with
> > those
> > two things fixed.
> 
> The test output says
> 
> > Unable to find recipes for the following entries in
> > maintainers.inc:
> > barebox
> 
> Could you give me a hint what that means? The barebox_2024.08.0.bb
> and the change in maintainers.inc
> are in the same commit. So I am unsure why it does not find the
> recipe.

It means the recipe is being skipped in the context of the QA test.
This means there is a maintainers entry but the test can't see any
recipe to match against it due to the skip.

The test is there to ensure when we delete recipes, we don't have stale
entries in the file.

Can we provide a config that makes sense for qemux86-64?

Cheers,

Richard

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Enrico Jörns]

Am Mittwoch, dem 18.09.2024 um 22:37 +0100 schrieb Richard Purdie:
> On Wed, 2024-09-18 at 13:45 +0200, Enrico Jörns wrote:
> > Am Mittwoch, dem 18.09.2024 um 12:21 +0100 schrieb Richard Purdie:
> > > On Wed, 2024-09-18 at 12:39 +0200, Enrico Jörns wrote:
> > > > Hi Richard,
> > > > 
> > > > Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard
> > > > Purdie:
> > > > > On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via
> > > > > lists.openembedded.org wrote:
> > > > > > This adds support for the barebox bootloader (and tools) to
> > > > > > oe-
> > > > > > core.
> > > > > > 
> > > > > > In order to have proper testing, this extends oe-selftest to
> > > > > > allow
> > > > > > basic testing of bootloaders. While at it, cover both u-boot
> > > > > > and
> > > > > > barebox.
> > > > > > 
> > > > > > v6:
> > > > > > * replace barebox.inc by barebox.bbclass (move content of
> > > > > >   barebox-common.inc into .bbclass and .bb files)
> > > > > > * support for UNPACKDIR
> > > > > > 
> > > > > > Enrico Jorns (5):
> > > > > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > > > > >   oeqa/utils/qemurunner: support ignoring vt100 escape
> > > > > > sequences
> > > > > >   oeqa: support passing custom boot patterns to runqemu
> > > > > >   oeqa/selftest/cases: add basic u-boot test
> > > > > >   oeqa/selftest/cases: add basic barebox tests
> > > > > > 
> > > > > > Marco Felsch (2):
> > > > > >   barebox: add initial support
> > > > > >   barebox-tools: add initial barebox tools support
> > > > > 
> > > > > We had some testing capacity so I ran this through automated
> > > > > testing.
> > > > > The one issue that showed up so far was:
> > > > > 
> > > > > https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> > > > > 
> > > > > which is at least easily fixed! The rest of the build is still
> > > > > going.
> > > > 
> > > > thanks for the test run and the first result.
> > > > 
> > > > Indeed, looks easily fixable 😉
> > > > Is it worth a v7 or should I wait for other results to show up?
> > > 
> > > Most of the tests have run and there were other issues in the build
> > > but
> > > there was one other issue I noticed:
> > > 
> > > https://valkyrie.yoctoproject.org/#/builders/54/builds/149/steps/14/logs/stdio
> > > 
> > > which is a missing maintainers entry. Probably worth a v7 with
> > > those
> > > two things fixed.
> > 
> > The test output says
> > 
> > > Unable to find recipes for the following entries in
> > > maintainers.inc:
> > > barebox
> > 
> > Could you give me a hint what that means? The barebox_2024.08.0.bb
> > and the change in maintainers.inc
> > are in the same commit. So I am unsure why it does not find the
> > recipe.
> 
> It means the recipe is being skipped in the context of the QA test.
> This means there is a maintainers entry but the test can't see any
> recipe to match against it due to the skip.

Thanks. I was not sure if the recipes are just evaluated and skipped based on normal bitbake parsing
or if there is a special mechanism that just looks for .bb files.

> The test is there to ensure when we delete recipes, we don't have stale
> entries in the file.
> 
> Can we provide a config that makes sense for qemux86-64?

The reason it is skipped from default parsing is

  PREFERRED_PROVIDER_virtual/bootloader = "u-boot"

I put in meta/conf/machine/qemux86-64.conf since we have two competing providers of
virtaul/bootloader now (and that's what the mechanism is actually made for I guess 😉).

Not sure if there is a way to remove the provider for just that check?

Or should I add it to the exception list in distrodata.py like its done for similar other recipes
that have competing providers (e.g. musl, newlib for virtual/libc; linux-yocto, linux-dummy for
virtual/kernel)?.

Regards, Enrico

> Cheers,
> 
> Richard
> 
> 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [OE-core] [PATCH v6 0/7] Add barebox bootloader support (and testing)

[Richard Purdie]

On Thu, 2024-09-19 at 11:47 +0200, Enrico Jörns wrote:
> Am Mittwoch, dem 18.09.2024 um 22:37 +0100 schrieb Richard Purdie:
> > On Wed, 2024-09-18 at 13:45 +0200, Enrico Jörns wrote:
> > > Am Mittwoch, dem 18.09.2024 um 12:21 +0100 schrieb Richard Purdie:
> > > > On Wed, 2024-09-18 at 12:39 +0200, Enrico Jörns wrote:
> > > > > Hi Richard,
> > > > > 
> > > > > Am Mittwoch, dem 18.09.2024 um 11:11 +0100 schrieb Richard
> > > > > Purdie:
> > > > > > On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via
> > > > > > lists.openembedded.org wrote:
> > > > > > > This adds support for the barebox bootloader (and tools) to
> > > > > > > oe-
> > > > > > > core.
> > > > > > > 
> > > > > > > In order to have proper testing, this extends oe-selftest to
> > > > > > > allow
> > > > > > > basic testing of bootloaders. While at it, cover both u-boot
> > > > > > > and
> > > > > > > barebox.
> > > > > > > 
> > > > > > > v6:
> > > > > > > * replace barebox.inc by barebox.bbclass (move content of
> > > > > > >   barebox-common.inc into .bbclass and .bb files)
> > > > > > > * support for UNPACKDIR
> > > > > > > 
> > > > > > > Enrico Jorns (5):
> > > > > > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > > > > > >   oeqa/utils/qemurunner: support ignoring vt100 escape
> > > > > > > sequences
> > > > > > >   oeqa: support passing custom boot patterns to runqemu
> > > > > > >   oeqa/selftest/cases: add basic u-boot test
> > > > > > >   oeqa/selftest/cases: add basic barebox tests
> > > > > > > 
> > > > > > > Marco Felsch (2):
> > > > > > >   barebox: add initial support
> > > > > > >   barebox-tools: add initial barebox tools support
> > > > > > 
> > > > > > We had some testing capacity so I ran this through automated
> > > > > > testing.
> > > > > > The one issue that showed up so far was:
> > > > > > 
> > > > > > https://valkyrie.yoctoproject.org/#/builders/29/builds/174/steps/13/logs/warnings
> > > > > > 
> > > > > > which is at least easily fixed! The rest of the build is still
> > > > > > going.
> > > > > 
> > > > > thanks for the test run and the first result.
> > > > > 
> > > > > Indeed, looks easily fixable 😉
> > > > > Is it worth a v7 or should I wait for other results to show up?
> > > > 
> > > > Most of the tests have run and there were other issues in the build
> > > > but
> > > > there was one other issue I noticed:
> > > > 
> > > > https://valkyrie.yoctoproject.org/#/builders/54/builds/149/steps/14/logs/stdio
> > > > 
> > > > which is a missing maintainers entry. Probably worth a v7 with
> > > > those
> > > > two things fixed.
> > > 
> > > The test output says
> > > 
> > > > Unable to find recipes for the following entries in
> > > > maintainers.inc:
> > > > barebox
> > > 
> > > Could you give me a hint what that means? The barebox_2024.08.0.bb
> > > and the change in maintainers.inc
> > > are in the same commit. So I am unsure why it does not find the
> > > recipe.
> > 
> > It means the recipe is being skipped in the context of the QA test.
> > This means there is a maintainers entry but the test can't see any
> > recipe to match against it due to the skip.
> 
> Thanks. I was not sure if the recipes are just evaluated and skipped based on normal bitbake parsing
> or if there is a special mechanism that just looks for .bb files.
> 
> > The test is there to ensure when we delete recipes, we don't have stale
> > entries in the file.
> > 
> > Can we provide a config that makes sense for qemux86-64?
> 
> The reason it is skipped from default parsing is
> 
>   PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
> 
> I put in meta/conf/machine/qemux86-64.conf since we have two competing providers of
> virtaul/bootloader now (and that's what the mechanism is actually made for I guess 😉).
> 
> Not sure if there is a way to remove the provider for just that check?
> 
> Or should I add it to the exception list in distrodata.py like its done for similar other recipes
> that have competing providers (e.g. musl, newlib for virtual/libc; linux-yocto, linux-dummy for
> virtual/kernel)?.

I think it will just have to go into the exception list. We really need
to find a better way to do this but you're right, that is how we're
doing it for the other cases.

Cheers,

Richard

Re: [OE-core] [PATCH v6 3/7] barebox: set default BAREBOX_CONFIG for qemu machines

[Richard Purdie]

On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via lists.openembedded.org wrote:
> From: Enrico Jorns <ejo@pengutronix.de>
> 
> These are set in the barebox recipe rather in the corresponding machines
> (where they would belong otherwise) to keep the impact of barebox to
> oe-core minimal for now.
> 
> "multi_v7_defconfig" is used for qemuarm since this is the default
> barebox armv7 config that just enables all supported platforms.
> 
> "multi_v8_defconfig" is used for qemuarm64 sine this is the default
> barebox armv8 config that just enables all supported platforms.
> 
> "efi_defconfig" is used for qemux86-64 which is the primary platform
> where barebox will not be the first stage bootloader but an EFI payload.
> 
> Since these changes make barebox a provider for virtual/bootloader,
> explicitly default to u-boot in the corresponding MACHINE configs to not
> unnecessarily surprise users.
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> ---
>  meta/conf/machine/qemuarm.conf    | 1 +
>  meta/conf/machine/qemuarm64.conf  | 1 +
>  meta/conf/machine/qemux86-64.conf | 1 +
>  3 files changed, 3 insertions(+)

This patch needs the commit message updating as it no longer matches
the changes below!

Things did pass in automated testing so I think we're close with this.

Cheers,

Richard

> diff --git a/meta/conf/machine/qemuarm.conf b/meta/conf/machine/qemuarm.conf
> index 943ce7c16a..af76a53b8e 100644
> --- a/meta/conf/machine/qemuarm.conf
> +++ b/meta/conf/machine/qemuarm.conf
> @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
>  
>  KERNEL_IMAGETYPE = "zImage"
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu_arm_defconfig"
>  
>  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> index a096d964db..d310445a34 100644
> --- a/meta/conf/machine/qemuarm64.conf
> +++ b/meta/conf/machine/qemuarm64.conf
> @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
>  
>  KERNEL_IMAGETYPE = "Image"
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu_arm64_defconfig"
>  
>  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> diff --git a/meta/conf/machine/qemux86-64.conf b/meta/conf/machine/qemux86-64.conf
> index 14873a3b4f..4a9c6d364b 100644
> --- a/meta/conf/machine/qemux86-64.conf
> +++ b/meta/conf/machine/qemux86-64.conf
> @@ -13,6 +13,7 @@ DEFAULTTUNE ?= "core2-64"
>  require conf/machine/include/x86/tune-x86-64-v3.inc
>  require conf/machine/include/x86/qemuboot-x86.inc
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
>  
>  KERNEL_IMAGETYPE = "bzImage"
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#204432): https://lists.openembedded.org/g/openembedded-core/message/204432
> Mute This Topic: https://lists.openembedded.org/mt/108410146/1686473
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [richard.purdie@linuxfoundation.org]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [OE-core] [PATCH v6 3/7] barebox: set default BAREBOX_CONFIG for qemu machines

[Richard Purdie]

On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via lists.openembedded.org wrote:
> From: Enrico Jorns <ejo@pengutronix.de>
> 
> These are set in the barebox recipe rather in the corresponding machines
> (where they would belong otherwise) to keep the impact of barebox to
> oe-core minimal for now.
> 
> "multi_v7_defconfig" is used for qemuarm since this is the default
> barebox armv7 config that just enables all supported platforms.
> 
> "multi_v8_defconfig" is used for qemuarm64 sine this is the default
> barebox armv8 config that just enables all supported platforms.
> 
> "efi_defconfig" is used for qemux86-64 which is the primary platform
> where barebox will not be the first stage bootloader but an EFI payload.
> 
> Since these changes make barebox a provider for virtual/bootloader,
> explicitly default to u-boot in the corresponding MACHINE configs to not
> unnecessarily surprise users.
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> ---
>  meta/conf/machine/qemuarm.conf    | 1 +
>  meta/conf/machine/qemuarm64.conf  | 1 +
>  meta/conf/machine/qemux86-64.conf | 1 +
>  3 files changed, 3 insertions(+)

This patch needs the commit message updating as it no longer matches
the changes below!

Things did pass in automated testing so I think we're close with this.

Cheers,

Richard

> diff --git a/meta/conf/machine/qemuarm.conf b/meta/conf/machine/qemuarm.conf
> index 943ce7c16a..af76a53b8e 100644
> --- a/meta/conf/machine/qemuarm.conf
> +++ b/meta/conf/machine/qemuarm.conf
> @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
>  
>  KERNEL_IMAGETYPE = "zImage"
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu_arm_defconfig"
>  
>  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> index a096d964db..d310445a34 100644
> --- a/meta/conf/machine/qemuarm64.conf
> +++ b/meta/conf/machine/qemuarm64.conf
> @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
>  
>  KERNEL_IMAGETYPE = "Image"
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu_arm64_defconfig"
>  
>  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> diff --git a/meta/conf/machine/qemux86-64.conf b/meta/conf/machine/qemux86-64.conf
> index 14873a3b4f..4a9c6d364b 100644
> --- a/meta/conf/machine/qemux86-64.conf
> +++ b/meta/conf/machine/qemux86-64.conf
> @@ -13,6 +13,7 @@ DEFAULTTUNE ?= "core2-64"
>  require conf/machine/include/x86/tune-x86-64-v3.inc
>  require conf/machine/include/x86/qemuboot-x86.inc
>  
> +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
>  
>  KERNEL_IMAGETYPE = "bzImage"
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#204432): https://lists.openembedded.org/g/openembedded-core/message/204432
> Mute This Topic: https://lists.openembedded.org/mt/108410146/1686473
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [richard.purdie@linuxfoundation.org]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [OE-core] [PATCH v6 3/7] barebox: set default BAREBOX_CONFIG for qemu machines

[Enrico Jörns]

Am Freitag, dem 11.10.2024 um 12:08 +0100 schrieb Richard Purdie:
> On Thu, 2024-09-12 at 11:18 +0200, Enrico Jörns via lists.openembedded.org wrote:
> > From: Enrico Jorns <ejo@pengutronix.de>
> > 
> > These are set in the barebox recipe rather in the corresponding machines
> > (where they would belong otherwise) to keep the impact of barebox to
> > oe-core minimal for now.
> > 
> > "multi_v7_defconfig" is used for qemuarm since this is the default
> > barebox armv7 config that just enables all supported platforms.
> > 
> > "multi_v8_defconfig" is used for qemuarm64 sine this is the default
> > barebox armv8 config that just enables all supported platforms.
> > 
> > "efi_defconfig" is used for qemux86-64 which is the primary platform
> > where barebox will not be the first stage bootloader but an EFI payload.
> > 
> > Since these changes make barebox a provider for virtual/bootloader,
> > explicitly default to u-boot in the corresponding MACHINE configs to not
> > unnecessarily surprise users.
> > 
> > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > ---
> >  meta/conf/machine/qemuarm.conf    | 1 +
> >  meta/conf/machine/qemuarm64.conf  | 1 +
> >  meta/conf/machine/qemux86-64.conf | 1 +
> >  3 files changed, 3 insertions(+)
> 
> This patch needs the commit message updating as it no longer matches
> the changes below!

Indeed. This hunk accidentally moved to "barebox: add initial support"
when merging the .inc and the .bb file.
I'd simply restore the original split so that the commit messages matches
again and we preserve the explicit explanation of defconfigs that Alex
requested.

> Things did pass in automated testing so I think we're close with this.

This sounds really, really great!


Thanks a lot for your effort!

Regards, Enrico

> Cheers,
> 
> Richard
> 
> > diff --git a/meta/conf/machine/qemuarm.conf b/meta/conf/machine/qemuarm.conf
> > index 943ce7c16a..af76a53b8e 100644
> > --- a/meta/conf/machine/qemuarm.conf
> > +++ b/meta/conf/machine/qemuarm.conf
> > @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
> >  
> >  KERNEL_IMAGETYPE = "zImage"
> >  
> > +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
> >  UBOOT_MACHINE ?= "qemu_arm_defconfig"
> >  
> >  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> > diff --git a/meta/conf/machine/qemuarm64.conf b/meta/conf/machine/qemuarm64.conf
> > index a096d964db..d310445a34 100644
> > --- a/meta/conf/machine/qemuarm64.conf
> > +++ b/meta/conf/machine/qemuarm64.conf
> > @@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
> >  
> >  KERNEL_IMAGETYPE = "Image"
> >  
> > +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
> >  UBOOT_MACHINE ?= "qemu_arm64_defconfig"
> >  
> >  SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> > diff --git a/meta/conf/machine/qemux86-64.conf b/meta/conf/machine/qemux86-64.conf
> > index 14873a3b4f..4a9c6d364b 100644
> > --- a/meta/conf/machine/qemux86-64.conf
> > +++ b/meta/conf/machine/qemux86-64.conf
> > @@ -13,6 +13,7 @@ DEFAULTTUNE ?= "core2-64"
> >  require conf/machine/include/x86/tune-x86-64-v3.inc
> >  require conf/machine/include/x86/qemuboot-x86.inc
> >  
> > +PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
> >  UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
> >  
> >  KERNEL_IMAGETYPE = "bzImage"
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#204432): https://lists.openembedded.org/g/openembedded-core/message/204432
> > Mute This Topic: https://lists.openembedded.org/mt/108410146/1686473
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe:
> > https://lists.openembedded.org/g/openembedded-core/unsub [richard.purdie@linuxfoundation.org]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |