Re: [OE-core] [PATCH v3 00/11] systemd based initrd and modular kernel support

[Mikko Rapeli <mikko.rapeli@linaro.org>]

Hi,

On Mon, Apr 07, 2025 at 11:52:01AM +0200, Mathieu Dubois-Briand wrote:
> On Mon Apr 7, 2025 at 11:24 AM CEST, Mikko Rapeli wrote:
> > On Mon, Apr 07, 2025 at 10:51:43AM +0200, Mathieu Dubois-Briand wrote:
> >> On Mon Apr 7, 2025 at 10:10 AM CEST, Mikko Rapeli wrote:
> >> > Hi,
> >> >
> >> > On Mon, Apr 07, 2025 at 09:53:45AM +0200, Mathieu Dubois-Briand wrote:
> >> >> On Fri Apr 4, 2025 at 6:29 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> >> >> > systemd based initrd supports more security, encryption etc features
> >> >> > than custom shell script ones but oe-core lacks support for it.
> >> >> > Convert core-image-initramfs-boot to create systemd based initrd
> >> >> > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test
> >> >> > for this initrd to mount rootfs via Unified Kernel Image uki.py.
> >> >> >
> >> >> > Sadly the boot time is really slow, e.g. over 30 seconds with
> >> >> > systemd based initrd. One of the major reasons is the large
> >> >> > amount of kernel modules built into the linux-yocto kernel and
> >> >> > processing of their udev events in userspace. Thus add
> >> >> > kernel-initrd-modules meta package to easily install
> >> >> > subset of all kernel modules to initrd. The subset
> >> >> > supports "mount rootfs from any block device" but
> >> >> > doesn't include graphics, UBS etc support. Without this
> >> >> > generated meta package, initrd recipe needs to manually define
> >> >> > which exact kernel modules to include which breaks if
> >> >> > those modules are built into the kernel. Thus the initrd
> >> >> > becomes machine and kernel config specific and hard to maintain.
> >> >> > With this meta package a generic initrd can be created.
> >> >> >
> >> >> > Separate changes to linux-yocto compile a lot more drivers
> >> >> > as modules but they depend on these initrd changes.
> >> >> >
> >> >> 
> >> >> Hi Mikko,
> >> >> 
> >> >> Thanks for your patch.
> >> >> 
> >> >> I believe one of these wic related patches is responsible of some
> >> >> oe-selftest failures we can see on the autobuilder:
> >> >> 
> >> >> 2025-04-06 15:51:49,317 - oe-selftest - INFO - wic.Wic.test_permissions (subunit.RemotedTestCase)
> >> >> 2025-04-06 15:51:49,318 - oe-selftest - INFO -  ... FAIL
> >> >> 
> >> >> ...
> >> >> 
> >> >> ERROR: When reparsing /tmp/selftest-fetchsrayuxoq/test.bb:do_checkuri, the basehash value changed from 84b9e23a83a764e7a5d8a96ee523199e2b1c604f4280a45030e248de0ff52962 to 4123da244f465feffecb2b4fd2b132b64851981ec2fb70bf9445df8b0ea6a69e. The metadata is not deterministic and this needs to be fixed.
> >> >> ERROR: The following commands may help:
> >> >> ERROR: $ bitbake test -cdo_checkuri -Snone
> >> >> ERROR: Then:
> >> >> ERROR: $ bitbake test -cdo_checkuri -Sprintdiff
> >> >> 
> >> >> ERROR: When reparsing /tmp/selftest-fetchsrayuxoq/test.bb:do_fetch, the basehash value changed from 6d55d6743729b615749dc016857d7e5f9c884a8b92f0d57f68e743f4910333d3 to fa7d6ea0563b60362418d2a6c5e41a6684183d01ec578b65abd0c64a9cfba7cf. The metadata is not deterministic and this needs to be fixed.
> >> >> ERROR: The following commands may help:
> >> >> ERROR: $ bitbake test -cdo_fetch -Snone
> >> >> ERROR: Then:
> >> >> ERROR: $ bitbake test -cdo_fetch -Sprintdiff
> >> >> 
> >> >> ERROR: When reparsing /tmp/selftest-fetchsrayuxoq/test.bb:do_unpack, the basehash value changed from ae9d94aafc721d891f8754df82ed90da8191357178f3db64221a49d5c1de3bcd to 1ac8f87a71db151b8eb2ae8d13f9a6ea1a0c884ef98555d61d2c2fe07a476a8a. The metadata is not deterministic and this needs to be fixed.
> >> >> ERROR: The following commands may help:
> >> >> ERROR: $ bitbake test -cdo_unpack -Snone
> >> >> ERROR: Then:
> >> >> ERROR: $ bitbake test -cdo_unpack -Sprintdiff
> >> >> 
> >> >> ...
> >> >> 
> >> >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1391
> >> >> 
> >> >> Can you have a look at this please?
> >> >
> >> > I can have a look but how can these patches change the basehash'es when testing?
> >> >
> >> > I've seen some of the wic selftets failing on x86_64 build machine with zfs
> >> > but changes to fix this was rejected multiple times.
> >> >
> >> > Then I've seen some of these failing with odd pseudo failures on aarch64
> >> > build machine.
> >> >
> >> > With my patches the end result was the same.
> >> >
> >> > I ran wic and uki selftests on x86_64 and aarch64 build machines for targets qemuarm64,
> >> > qemuarm and genericx86-64.
> >> >
> >> > The wic patches have actually nothing to do with this series except I ran into bugs
> >> > in wic which resulted in unbootable images, e.g. the UEFI bootloader binaries
> >> > missing from ESP partition. These were impacting some builds but not all
> >> > which was really troubling.
> >> >
> >> > Cheers,
> >> >
> >> > -Mikko
> >> 
> >> Ok, I am running some git bisect locally anyway, I will keep you updated
> >> a bit later today.
> >
> > I ran wic.Wic.test_permissions and it passed without this series
> > on aarch64 build machine and genericarm64 target machine:
> >
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Retrying server connection (#1)... (08:13:45.320086)
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Retrying server connection (#1)... (08:13:45.320086)
> > NOTE: Retrying server connection (#1)... (08:13:45.320086)
> > NOTE: Starting bitbake server...
> > hint: Using 'master' as the name for the initial branch. This default branch name
> > hint: is subject to change. To configure the initial branch name to use in all
> > hint: of your new repositories, which will suppress this warning, call:
> > hint: 
> > hint:   git config --global init.defaultBranch <name>
> > hint: 
> > hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
> > hint: 'development'. The just-created branch can be renamed via this command:
> > hint: 
> > hint:   git branch -m <name>
> > 2025-04-07 08:13:46,418 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/mcfrisk/src/base/poky/build_genericarm64-st/conf/local.conf
> > 2025-04-07 08:13:46,418 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
> > 2025-04-07 08:13:46,418 - oe-selftest - INFO - test_permissions (wic.Wic.test_permissions)
> > 2025-04-07 08:24:47,941 - oe-selftest - INFO -  ... ok
> > 2025-04-07 08:24:54,376 - oe-selftest - INFO - ----------------------------------------------------------------------
> > 2025-04-07 08:24:54,376 - oe-selftest - INFO - Ran 1 test in 668.167s
> > 2025-04-07 08:24:54,376 - oe-selftest - INFO - OK
> > 2025-04-07 08:25:00,978 - oe-selftest - INFO - RESULTS:
> > 2025-04-07 08:25:00,978 - oe-selftest - INFO - RESULTS - wic.Wic.test_permissions: PASSED (661.52s)
> > 2025-04-07 08:25:00,979 - oe-selftest - INFO - SUMMARY:
> > 2025-04-07 08:25:00,979 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 668.168s
> > 2025-04-07 08:25:00,979 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
> >
> > Then with patches from this series (and a few more which I sent out separately,
> > should be unrelated) it failed with pseudo errors:
> >
> > 2025-04-07 08:32:12,107 - oe-selftest - INFO - Changing cwd to /home/mcfrisk/src/base/poky/build_genericarm64
> > 2025-04-07 08:32:12,107 - oe-selftest - INFO - Adding layer libraries:
> > 2025-04-07 08:32:12,107 - oe-selftest - INFO -  /home/mcfrisk/src/base/poky/meta/lib
> > 2025-04-07 08:32:12,107 - oe-selftest - INFO -  /home/mcfrisk/src/base/poky/meta-yocto-bsp/lib
> > 2025-04-07 08:32:12,107 - oe-selftest - INFO -  /home/mcfrisk/src/base/poky/meta-selftest/lib
> > 2025-04-07 08:32:12,108 - oe-selftest - INFO - Checking base configuration is valid/parsable
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Retrying server connection (#1)... (08:32:12.728357)
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Reconnecting to bitbake server...
> > NOTE: Retrying server connection (#1)... (08:32:12.728357)
> > NOTE: Retrying server connection (#1)... (08:32:12.728357)
> > NOTE: Starting bitbake server...
> > hint: Using 'master' as the name for the initial branch. This default branch name
> > hint: is subject to change. To configure the initial branch name to use in all
> > hint: of your new repositories, which will suppress this warning, call:
> > hint: 
> > hint:   git config --global init.defaultBranch <name>
> > hint: 
> > hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
> > hint: 'development'. The just-created branch can be renamed via this command:
> > hint: 
> > hint:   git branch -m <name>
> > 2025-04-07 08:32:13,824 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/mcfrisk/src/base/poky/build_genericarm64-st/conf/local.conf
> > 2025-04-07 08:32:13,824 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
> > 2025-04-07 08:32:13,824 - oe-selftest - INFO - test_permissions (wic.Wic.test_permissions)
> > 2025-04-07 09:10:05,405 - oe-selftest - INFO -  ... FAIL
> > 2025-04-07 09:10:05,488 - oe-selftest - INFO - Traceback (most recent call last):
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/selftest/cases/wic.py", line 677, in test_permissions
> >     bitbake('core-image-minimal')
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/utils/commands.py", line 236, in bitbake
> >     return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
> >            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
> >     raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
> > AssertionError: Command 'bitbake  core-image-minimal' returned non-zero exit status 1:
> > Loading cache...done.
> > Loaded 1976 entries from dependency cache.
> > NOTE: Resolving any missing task queue dependencies
> >
> > Build Configuration:
> > BB_VERSION           = "2.12.0"
> > BUILD_SYS            = "aarch64-linux"
> > NATIVELSBSTRING      = "universal"
> > TARGET_SYS           = "aarch64-poky-linux"
> > MACHINE              = "genericarm64"
> > DISTRO               = "poky-altcfg"
> > DISTRO_VERSION       = "5.2"
> > TUNE_FEATURES        = "aarch64 armv8a crc"
> > TARGET_FPU           = ""
> > meta                 
> > meta-poky            
> > meta-yocto-bsp       = "HEAD:a75117f79e4771927b89125e464e38acdc9c556e"
> > meta-selftest        = "master:fd26cf3cc0e6e590b56e331971cdf884d1b881b9"
> >
> > Initialising tasks...Sstate summary: Wanted 539 Local 536 Mirrors 0 Missed 3 Current 1968 (99% match, 99% complete)
> > Removing 3 stale sstate objects for arch genericarm64...done.
> > NOTE: Executing Tasks
> > NOTE: Running task 5157 of 5259 (/home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic)
> > NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Started
> > ERROR: core-image-minimal-1.0-r0 do_image_wic: Execution of '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/run.do_image_wic.117565' failed with exit code 134
> > ERROR: Logfile of failure stored in: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/log.do_image_wic.117565
> > Log data follows:
> > | DEBUG: Executing python function extend_recipe_sysroot
> > | NOTE: Direct dependencies are ['/home/mcfrisk/src/base/poky/meta/recipes-bsp/grub/grub-efi_2.12.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-core/gli
> > bc/cross-localedef-native_2.41.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-core/systemd/systemd-boot_257.4.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/binutils/binutils-cross_2.44.bb:do_popu
> > late_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/btrfs-tools
> > /btrfs-tools_6.14.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/dosfstools/dosfstools_4.2.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.47.1.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/er
> > ofs-utils/erofs-utils_1.8.5.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.10.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/mtools/mtools_4.0.48.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.7.0.bb:do_populate_sysroot', 'virtua
> > l:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/opkg/opkg_0.7.0.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb:do_populate_sysroot', 'virtual:nat
> > ive:/home/mcfrisk/src/base/poky/meta/recipes-extended/parted/parted_3.6.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-extended/pigz/pigz_
> > 2.8.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-support/bmaptool/bmaptool_git.bb:do_populate_sysroot']
> > | NOTE: Installed into sysroot: []
> > | NOTE: Skipping as already exists in sysroot: ['grub-efi', 'gettext-minimal-native', 'cross-localedef-native', 'glibc', 'ldconfig-native', 'libxcrypt', 'ncurses', 'systemd-
> > boot', 'systemd-systemctl-native', 'util-linux-libuuid', 'util-linux', 'zlib', 'binutils-cross-aarch64', 'cdrtools-native', 'cmake-native', 'gcc-runtime', 'libgcc', 'libtool-native', 'm4-native', 'opkg-utils', 'qemu-helper-native', 'qemu-native', 'qemu-system-native', 'qemuwrapper-cross', 'libpam', 'texinfo-dummy-native', 'depmodwrapper-cross', 'linux-libc-headers', 'bash-completion', 'libcap-ng', 'libcap', 'sqlite3', 'openssl-native', 'libslirp-native', 'coreutils-native', 'expat-native', 'gettext-native', 'glib-
> > 2.0-native', 'libxml2-native', 'ncurses-native', 'readline-native', 'update-rc.d-native', 'util-linux-libuuid-native', 'util-linux-native', 'zlib-native', 'btrfs-tools-nativ
> > e', 'chrpath-native', 'dosfstools-native', 'e2fsprogs-native', 'elfutils-native', 'erofs-utils-native', 'gptfdisk-native', 'flex-native', 'gnu-config-native', 'json-c-native
> > ', 'libedit-native', 'llvm-native', 'make-native', 'makedevs-native', 'mtools-native', 'opkg-utils-native', 'opkg-native', 'perl-native', 'pseudo-native', 'python3-build-native', 'python3-calver-native', 'python3-editables-native', 'python3-flit-core-native', 'python3-hatchling-native', 'python3-installer-native', 'python3-jinja2-native', 'pyth
> > on3-mako-native', 'python3-markupsafe-native', 'python3-packaging-native', 'python3-pathspec-native', 'python3-pip-native', 'python3-pluggy-native', 'python3-pygments-native
> > ', 'python3-pyparsing-native', 'python3-pyproject-hooks-native', 'python3-setuptools-scm-native', 'python3-setuptools-native', 'python3-six-native', 'python3-trove-classifiers-native', 'python3-typing-extensions-native', 'python3-wheel-native', 'python3-native', 'squashfs-tools-native', 'unfs3-native', 'bzip2-native', 'cpio-native', 'diffutils-
> > native', 'libarchive-native', 'libidn2-native', 'libnsl2-native', 'libsolv-native', 'libtirpc-native', 'parted-native', 'pigz-native', 'shadow-native', 'unzip-native', 'xz-native', 'zstd-native', 'libdrm-native', 'libepoxy-native', 'libsdl2-native', 'mesa-native', 'virglrenderer-native', 'wayland-protocols-native', 'wayland-native', 'xrandr-nat
> > ive', 'libpciaccess-native', 'libpthread-stubs-native', 'libx11-native', 'libxau-native', 'libxcb-native', 'libxdamage-native', 'libxdmcp-native', 'libxext-native', 'libxfixes-native', 'libxrandr-native', 'libxrender-native', 'libxshmfence-native', 'libxxf86vm-native', 'pixman-native', 'xtrans-native', 'xcb-proto-native', 'xorgproto-native', 'makedepend-native', 'util-macros-native', 'dtc-native', 'kmod-native', 'alsa-lib-native', 'libpng-native', 'acl-native', 'attr-native', 'bmaptool-native', 'curl-native', 'deb
> > ianutils-native', 'gdbm-native', 'gmp-native', 'gnutls-native', 'libtasn1-native', 'libbsd-native', 'libcap-ng-native', 'libcap-native', 'libffi-native', 'libgcrypt-native', 'libgpg-error-native', 'libmd-native', 'libmicrohttpd-native', 'libpcre2-native', 'libunistring-native', 'lz4-native', 'lzo-native', 'nettle-native', 'popt-native', 'shared
> > -mime-info-native', 'sqlite3-native']
> > | DEBUG: Python function extend_recipe_sysroot finished
> > | DEBUG: Executing python function set_image_size
> > | DEBUG: 159546.400000 = 122728 * 1.300000
> > | DEBUG: 163642.400000 = max(159546.400000, 8192)[159546.400000] + 4096
> > | DEBUG: 163643.000000 = int(163642.400000)
> > | DEBUG: 163643 = aligned(163643)
> > | DEBUG: returning 163643
> > | DEBUG: Python function set_image_size finished
> > | DEBUG: Executing shell function do_image_wic
> > | abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
> > | Check logfile: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/pseudo//pseudo.log
> > | WARNING: exit code 134 from a shell command.
> > | Aborted
> > NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Failed
> > ERROR: Task (/home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic) failed with exit code '1' 
> > Pseudo log:
> > path mismatch [2 links]: ino 43965780 db '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/rootfs/boot/loader/loader.conf' req '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/tmp-wic/rootfs1/boot/loader/loader.conf'.
> > Setup complete, sending SIGUSR1 to pid 117389.
> >
> > NOTE: Tasks Summary: Attempted 5254 tasks of which 5253 didn't need to be rerun and 1 failed.
> >
> > Summary: 1 task failed:
> >   /home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic
> >     log: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/log.do_image_wic.117565
> > Summary: There was 1 ERROR message, returning a non-zero exit code.
> >
> > 2025-04-07 09:10:05,488 - oe-selftest - INFO - ======================================================================
> > 2025-04-07 09:10:05,488 - oe-selftest - INFO - FAIL: test_permissions (wic.Wic.test_permissions)
> > 2025-04-07 09:10:05,489 - oe-selftest - INFO - ----------------------------------------------------------------------
> > 2025-04-07 09:10:05,489 - oe-selftest - INFO - Traceback (most recent call last):
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/selftest/cases/wic.py", line 677, in test_permissions
> >     bitbake('core-image-minimal')
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/utils/commands.py", line 236, in bitbake
> >     return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
> >            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >   File "/home/mcfrisk/src/base/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
> >     raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
> > AssertionError: Command 'bitbake  core-image-minimal' returned non-zero exit status 1:
> > Loading cache...done.
> > Loaded 1976 entries from dependency cache.
> > NOTE: Resolving any missing task queue dependencies
> >
> > Build Configuration:
> > BB_VERSION           = "2.12.0"
> > BUILD_SYS            = "aarch64-linux"
> > NATIVELSBSTRING      = "universal"
> > TARGET_SYS           = "aarch64-poky-linux"
> > MACHINE              = "genericarm64"
> > DISTRO               = "poky-altcfg"
> > DISTRO_VERSION       = "5.2"
> > TUNE_FEATURES        = "aarch64 armv8a crc"
> > TARGET_FPU           = ""
> > meta                 
> > meta-poky            
> > meta-yocto-bsp       = "HEAD:a75117f79e4771927b89125e464e38acdc9c556e"
> > meta-selftest        = "master:fd26cf3cc0e6e590b56e331971cdf884d1b881b9"
> >
> > Initialising tasks...Sstate summary: Wanted 539 Local 536 Mirrors 0 Missed 3 Current 1968 (99% match, 99% complete)
> > Removing 3 stale sstate objects for arch genericarm64...done.
> > NOTE: Executing Tasks
> > NOTE: Running task 5157 of 5259 (/home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic)
> > NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Started
> > ERROR: core-image-minimal-1.0-r0 do_image_wic: Execution of '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/r
> > un.do_image_wic.117565' failed with exit code 134
> > ERROR: Logfile of failure stored in: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/log.do_image_wic.117565
> > Log data follows:
> > | DEBUG: Executing python function extend_recipe_sysroot
> > | NOTE: Direct dependencies are ['/home/mcfrisk/src/base/poky/meta/recipes-bsp/grub/grub-efi_2.12.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-core/glibc/cross-localedef-native_2.41.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', '/home/mcfrisk/s
> > rc/base/poky/meta/recipes-core/systemd/systemd-boot_257.4.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/binutils/binutils-cross_2.44.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', '/home/mcfrisk/src/base/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/ho
> > me/mcfrisk/src/base/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/btrfs-tools/btrfs-tools_6.14.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/dosfstools/dosfstools_4.2.bb:do_populate_sysroot', 'virtual:nati
> > ve:/home/mcfrisk/src/base/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.47.1.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/erofs-utils/erofs-utils_1.8.5.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.10.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/mtools/mtools_4.0.48.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.7.0.bb:do_populate_sysroot', 'virtua
> > l:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/opkg/opkg_0.7.0.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/pseudo/
> > pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-extended/parted/parted_3.6.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-extended/pigz/pigz_2.8.bb:do_populate_sysroot', 'virtual:native:/home/mcfrisk/src/base/poky/meta/recipes-support/bmaptool/bmaptool_git.bb:do_populate_sysroot']
> > | NOTE: Installed into sysroot: []
> > | NOTE: Skipping as already exists in sysroot: ['grub-efi', 'gettext-minimal-native', 'cross-localedef-native', 'glibc', 'ldconfig-native', 'libxcrypt', 'ncurses', 'systemd-boot', 'systemd-systemctl-native', 'util-linux-libuuid', 'util-linux', 'zlib', 'binutils-cross-aarch64', 'cdrtools-native', 'cmake-native', 'gcc-runtime', 'libgcc', 'libtool
> > -native', 'm4-native', 'opkg-utils', 'qemu-helper-native', 'qemu-native', 'qemu-system-native', 'qemuwrapper-cross', 'libpam', 'texinfo-dummy-native', 'depmodwrapper-cross', 'linux-libc-headers', 'bash-completion', 'libcap-ng', 'libcap', 'sqlite3', 'openssl-native', 'libslirp-native', 'coreutils-native', 'expat-native', 'gettext-native', 'glib-
> > 2.0-native', 'libxml2-native', 'ncurses-native', 'readline-native', 'update-rc.d-native', 'util-linux-libuuid-native', 'util-linux-native', 'zlib-native', 'btrfs-tools-native', 'chrpath-native', 'dosfstools-native', 'e2fsprogs-native', 'elfutils-native', 'erofs-utils-native', 'gptfdisk-native', 'flex-native', 'gnu-config-native', 'json-c-native', 'libedit-native', 'llvm-native', 'make-native', 'makedevs-native', 'mtools-native', 'opkg-utils-native', 'opkg-native', 'perl-native', 'pseudo-native', 'python3-build-nat
> > ive', 'python3-calver-native', 'python3-editables-native', 'python3-flit-core-native', 'python3-hatchling-native', 'python3-installer-native', 'python3-jinja2-native', 'python3-mako-native', 'python3-markupsafe-native', 'python3-packaging-native', 'python3-pathspec-native', 'python3-pip-native', 'python3-pluggy-native', 'python3-pygments-native
> > ', 'python3-pyparsing-native', 'python3-pyproject-hooks-native', 'python3-setuptools-scm-native', 'python3-setuptools-native', 'python3-six-native', 'python3-trove-classifiers-native', 'python3-typing-extensions-native', 'python3-wheel-native', 'python3-native', 'squashfs-tools-native', 'unfs3-native', 'bzip2-native', 'cpio-native', 'diffutils-native', 'libarchive-native', 'libidn2-native', 'libnsl2-native', 'libsolv-native', 'libtirpc-native', 'parted-native', 'pigz-native', 'shadow-native', 'unzip-native', 'xz-n
> > ative', 'zstd-native', 'libdrm-native', 'libepoxy-native', 'libsdl2-native', 'mesa-native', 'virglrenderer-native', 'wayland-protocols-native', 'wayland-native', 'xrandr-native', 'libpciaccess-native', 'libpthread-stubs-native', 'libx11-native', 'libxau-native', 'libxcb-native', 'libxdamage-native', 'libxdmcp-native', 'libxext-native', 'libxfix
> > es-native', 'libxrandr-native', 'libxrender-native', 'libxshmfence-native', 'libxxf86vm-native', 'pixman-native', 'xtrans-native', 'xcb-proto-native', 'xorgproto-native', 'makedepend-native', 'util-macros-native', 'dtc-native', 'kmod-native', 'alsa-lib-native', 'libpng-native', 'acl-native', 'attr-native', 'bmaptool-native', 'curl-native', 'debianutils-native', 'gdbm-native', 'gmp-native', 'gnutls-native', 'libtasn1-native', 'libbsd-native', 'libcap-ng-native', 'libcap-native', 'libffi-native', 'libgcrypt-native',
> >  'libgpg-error-native', 'libmd-native', 'libmicrohttpd-native', 'libpcre2-native', 'libunistring-native', 'lz4-native', 'lzo-native', 'nettle-native', 'popt-native', 'shared-mime-info-native', 'sqlite3-native']
> > | DEBUG: Python function extend_recipe_sysroot finished
> > | DEBUG: Executing python function set_image_size
> > | DEBUG: 159546.400000 = 122728 * 1.300000
> > | DEBUG: 163642.400000 = max(159546.400000, 8192)[159546.400000] + 4096
> > | DEBUG: 163643.000000 = int(163642.400000)
> > | DEBUG: 163643 = aligned(163643)
> > | DEBUG: returning 163643
> > | DEBUG: Python function set_image_size finished
> > | DEBUG: Executing shell function do_image_wic
> > | abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
> > | Check logfile: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/pseudo//pseudo.log
> > | WARNING: exit code 134 from a shell command.
> > | Aborted
> > NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Failed
> > ERROR: Task (/home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic) failed with exit code '1' 
> > Pseudo log:
> > path mismatch [2 links]: ino 43965780 db '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/rootfs/boot/loader/loader.conf' req '/home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/tmp-wic/rootfs1/boot/loader/loader.conf'.
> > Setup complete, sending SIGUSR1 to pid 117389.
> >
> > NOTE: Tasks Summary: Attempted 5254 tasks of which 5253 didn't need to be rerun and 1 failed.
> >
> > Summary: 1 task failed:
> >   /home/mcfrisk/src/base/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic
> >     log: /home/mcfrisk/src/base/poky/build_genericarm64-st/tmp/work/genericarm64-poky-linux/core-image-minimal/1.0/temp/log.do_image_wic.117565
> > Summary: There was 1 ERROR message, returning a non-zero exit code.
> > ----------------------------------------------------------------------
> > 2025-04-07 09:10:05,489 - oe-selftest - INFO - Ran 1 test in 2271.863s
> > 2025-04-07 09:10:05,489 - oe-selftest - INFO - FAILED
> > 2025-04-07 09:10:05,489 - oe-selftest - INFO -  (failures=1)
> > 2025-04-07 09:10:11,706 - oe-selftest - INFO - RESULTS:
> > 2025-04-07 09:10:11,706 - oe-selftest - INFO - RESULTS - wic.Wic.test_permissions: FAILED (2271.66s)
> > 2025-04-07 09:10:11,708 - oe-selftest - INFO - SUMMARY:
> > 2025-04-07 09:10:11,708 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2271.863s
> > 2025-04-07 09:10:11,708 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0)
> >
> > I think something in this test is faulty. For example building an image
> > in yocto do_image with fakeroot/pseudo and then modifying it outside of build
> > env with wic is likely going to fail like this.
> >
> > I will try with other machine targets and cleaner local.conf
> > but I suspect I will run into this same issue again.
> >
> > And the error from your CI buidl was completely different... :/
> >
> > Cheers,
> >
> > -Mikko
> 
> I confirm, localally I see the same thing as you do: just some pseudo
> errors. They seem to come from "image_types_wic.bbclass: capture verbose
> wic output by default" patch.
> 
> So yes, different than on the autobuilder, but yet if I drop the series,
> autobuilder builds do succeed :(
> 
> So we can hope there is an obscure link between both failures, but
> that's all I've got so far.

With:

--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -674,6 +674,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc
 
                 config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "%s"\n' % wks_file
                 self.append_config(config)
+                bitbake('-c clean core-image-minimal')
                 bitbake('core-image-minimal')
                 tmpdir = os.path.join(get_bb_var('WORKDIR', 'core-image-minimal'),'build-wic')
 

the test is passing for me now without pseudo errors.

The debug logging patch is IMO unrelated. It's just capturing logs from wic by default.
It could trigger rebuilds vs sstate cache if testing is done without and then with
the patch applied. Is full world build done with this patch applied before
selftests are run? I think this may impact the test as well.

Cheers,

-Mikko