Re: [OE-core] [PATCH v3 03/11] kernel.bbclass: add kernel-initrd-modules meta package

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2025-04-04 at 19:29 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> At the moment linux-yocto kernels for various architectures
> are not very modular and a lot of drivers are built into the kernel
> even when they are not needed at runtime. These make the main kernel
> binary big and slow to boot. This also impacts udev in userspace
> which takes a long time processing events from all these built in drivers,
> for example when udev runs in initrd.
> 
> Then constructing the initrd is very device and kernel configuration specific.
> initrd image needs explicitly define which binary packages to install
> to avoid pulling in complex dependencies. A full set of kernel modules
> via kernel-modules meta package is too big for initrd and most of the
> drivers are not needed for use cases like "just load modules to mount
> main rootfs". Then the initrd configuration breaks if kernel driver
> is built into the kernel since the binary package doesn't exist.
> 
> Introduce kernel-initrd-modules meta package to solve these problems.
> The meta package adds dependencies to real kernel modules based on
> the kernel module file paths so that it will include several
> kernel subsystems and their drivers which are often needed to find
> main rootfs from some block device. This works when drivers are built
> as modules but does not break if drivers are built into the kernel.
> 
> The resulting initrd is also smaller since only a subset of drivers
> are needed for "mount the rootfs" usecase. Tested on genericarm64
> kernel and qemu and AMD KV260 HW.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  .../kernel-module-split.bbclass               | 48 +++++++++++++++++++
>  meta/classes-recipe/kernel.bbclass            |  5 +-
>  meta/classes-recipe/module.bbclass            | 37 ++++++++++++++
>  3 files changed, 89 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/kernel-module-split.bbclass b/meta/classes-recipe/kernel-module-split.bbclass
> index 9487365eb7..101c5cd959 100644
> --- a/meta/classes-recipe/kernel-module-split.bbclass
> +++ b/meta/classes-recipe/kernel-module-split.bbclass
> @@ -42,6 +42,40 @@ KERNEL_MODULE_PACKAGE_PREFIX ?= ""
>  KERNEL_MODULE_PACKAGE_SUFFIX ?= "-${KERNEL_VERSION}"
>  KERNEL_MODULE_PROVIDE_VIRTUAL ?= "1"
>  
> +# subset of kernel modules needed in initrd, to e.g. mount rootfs from block device
> +KERNEL_INITRD_MODULES_META_PACKAGE ?= "${@ d.getVar("KERNEL_PACKAGE_NAME") or "kernel" }-initrd-modules"
> +
> +# match regex to path or file name. E.g. include all drivers with files in path /drivers/ata/
> +KERNEL_INITRD_MODULES_REGEX ?= "(.*)(\
> +/drivers/acpi/|\
> +/drivers/ata/|\
> +/drivers/block/|\
> +/drivers/cdrom/|\
> +/drivers/char/hw_random/|\
> +/drivers/char/tpm/|\
> +/drivers/char/|\
> +/drivers/crypto/|\
> +/drivers/dax/|\
> +/drivers/firmware/arm_scmi/|\
> +/drivers/gpu/drm/|\
> +/drivers/md/|\
> +/drivers/mmc/|\
> +/drivers/mtd/|\
> +/drivers/nvdimm/|\
> +/drivers/nvme/|\
> +/drivers/pci/|\
> +/drivers/scsi/|\
> +/drivers/tee/|\
> +/drivers/tty/serial/|\
> +/drivers/virtio/|\
> +/drivers/watchdog/|\
> +/kernel/arch/|\
> +/kernel/block/|\
> +/kernel/crypto/|\
> +/kernel/fs/|\
> +/kernel/lib/\
> +)(.*)"
> +
>  python split_kernel_module_packages () {
>      import re
>  
> @@ -183,6 +217,20 @@ python split_kernel_module_packages () {
>      modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', file_regex=module_regex, output_pattern=module_pattern, description='%s kernel module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name, kernel_version))
>      if modules:
>          d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
> +
> +    initrd_metapkg = d.getVar('KERNEL_INITRD_MODULES_META_PACKAGE') or ""
> +    initrd_module_regex = d.getVar('KERNEL_INITRD_MODULES_REGEX') or ""
> +    if (initrd_metapkg != "") and (initrd_module_regex != ""):
> +        initrd_module_regex = re.compile(initrd_module_regex)
> +        initrd_modules = []
> +        for module in modules:
> +            files = d.getVar('FILES:' + module)
> +            m = re.match(initrd_module_regex, files)
> +            if m:
> +                initrd_modules.append(module)
> +
> +        if initrd_modules:
> +            d.appendVar('RDEPENDS:' + initrd_metapkg, ' '+' '.join(initrd_modules))
>  }
>  
>  do_package[vardeps] += '${@" ".join(map(lambda s: "module_conf_" + s, (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()))}'
> diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
> index 36ce659762..3dcaebcaed 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -695,13 +695,14 @@ EXPORT_FUNCTIONS do_compile do_transform_kernel do_transform_bundled_initramfs d
>  
>  # kernel-base becomes kernel-${KERNEL_VERSION}
>  # kernel-image becomes kernel-image-${KERNEL_VERSION}
> -PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-dbg"
> +PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-initrd-modules ${KERNEL_PACKAGE_NAME}-dbg"
>  FILES:${PN} = ""
>  FILES:${KERNEL_PACKAGE_NAME}-base = "${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.order ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.builtin ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.builtin.modinfo"
>  FILES:${KERNEL_PACKAGE_NAME}-image = ""
>  FILES:${KERNEL_PACKAGE_NAME}-dev = "/${KERNEL_IMAGEDEST}/System.map* /${KERNEL_IMAGEDEST}/Module.symvers* /${KERNEL_IMAGEDEST}/config* ${KERNEL_SRC_PATH} ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build"
>  FILES:${KERNEL_PACKAGE_NAME}-vmlinux = "/${KERNEL_IMAGEDEST}/vmlinux-${KERNEL_VERSION_NAME}"
>  FILES:${KERNEL_PACKAGE_NAME}-modules = ""
> +FILES:${KERNEL_PACKAGE_NAME}-initrd-modules = ""
>  FILES:${KERNEL_PACKAGE_NAME}-dbg = "/usr/lib/debug /usr/src/debug"
>  RDEPENDS:${KERNEL_PACKAGE_NAME} = "${KERNEL_PACKAGE_NAME}-base (= ${EXTENDPKGV})"
>  # Allow machines to override this dependency if kernel image files are
> @@ -716,7 +717,9 @@ ALLOW_EMPTY:${KERNEL_PACKAGE_NAME} = "1"
>  ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-base = "1"
>  ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-image = "1"
>  ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-modules = "1"
> +ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-initrd-modules = "1"
>  DESCRIPTION:${KERNEL_PACKAGE_NAME}-modules = "Kernel modules meta package"
> +DESCRIPTION:${KERNEL_PACKAGE_NAME}-initrd-modules = "Kernel initrd modules meta package"
>  
>  pkg_postinst:${KERNEL_PACKAGE_NAME}-base () {
>  	if [ ! -e "$D/lib/modules/${KERNEL_VERSION}" ]; then
> diff --git a/meta/classes-recipe/module.bbclass b/meta/classes-recipe/module.bbclass
> index f2f0b25a2d..51f864f1f9 100644
> --- a/meta/classes-recipe/module.bbclass
> +++ b/meta/classes-recipe/module.bbclass
> @@ -86,3 +86,40 @@ EXPORT_FUNCTIONS do_compile do_install
>  KERNEL_MODULES_META_PACKAGE = "${PN}"
>  FILES:${PN} = ""
>  ALLOW_EMPTY:${PN} = "1"
> +
> +# subset of kernel modules needed in initrd, to e.g. mount rootfs from block device
> +KERNEL_INITRD_MODULES_META_PACKAGE ?= "${@ d.getVar("KERNEL_PACKAGE_NAME") or "kernel" }-initrd-modules"
> +
> +# match regex to path or file name. E.g. include all drivers with files in path /drivers/ata/
> +KERNEL_INITRD_MODULES_REGEX ?= "(.*)(\
> +/drivers/acpi/|\
> +/drivers/ata/|\
> +/drivers/block/|\
> +/drivers/cdrom/|\
> +/drivers/char/hw_random/|\
> +/drivers/char/tpm/|\
> +/drivers/char/|\
> +/drivers/crypto/|\
> +/drivers/dax/|\
> +/drivers/firmware/arm_scmi/|\
> +/drivers/gpu/drm/|\
> +/drivers/md/|\
> +/drivers/mmc/|\
> +/drivers/mtd/|\
> +/drivers/nvdimm/|\
> +/drivers/nvme/|\
> +/drivers/pci/|\
> +/drivers/scsi/|\
> +/drivers/tee/|\
> +/drivers/tty/serial/|\
> +/drivers/virtio/|\
> +/drivers/watchdog/|\
> +/kernel/arch/|\
> +/kernel/block/|\
> +/kernel/crypto/|\
> +/kernel/fs/|\
> +/kernel/lib/\
> +)(.*)"
> +
> +FILES:${PN}-initrd = ""
> +ALLOW_EMPTY:${PN}-initrd = "1"

What is the difference between the variable defined in kernel-module-
split.bbclass and this one in module.bbclass? Do we need/want to
separate but seemingly similar definitions?

Cheers,

Richard