[OE-core][RFC PATCH 0/8] enhance kernel -devsrc output options

[OE-core][RFC PATCH 0/8] enhance kernel -devsrc output options

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Hi all,

This RFC follows on from the "Fix kernel complementary package
installation" thread with Adrian, and covers the use cases listed there:
debugging a kernel built for a specific MACHINE, from a debug rootfs on
the host (TRACE32 and similar), or on the target.

kernel-devsrc today is what is needed to build out of tree modules on
the target ('make scripts prepare'): headers, Kconfig/Makefiles and
scripts. It has almost no .c files, and installing it pulls in the build
tools (bc, bison, flex, gcc-plugins, openssl-dev, ...), which aren't
wanted in a rootfs-dbg.

Adding another source package just means more maintenance and
complexity.  We can keep the tightly curated kernel-devsrc as the
deafult, and add to it via packageconfigs. kernel-devsrc stays one
package and one tree at /usr/src/kernel, where the debug info points,
and PACKAGECONFIG chooses what goes in it:

  build:   what kernel-devsrc is today, with its RDEPENDS. The default,
           so nothing changes for existing users.
  source:  a curated spine (core directories plus the machine's arch),
           and the rest of the files the kernel build read, from a
           manifest of kbuild's .cmd files that kernel.bbclass now
           stages. No RDEPENDS.
  full:    the whole tree.
  tarball: the whole tree as a tarball, to unpack elsewhere.

They can be combined: "source build" gives both the source and what is
needed to build modules on target. With 'build' and the whole tree, the
tools to build the kernel itself are added to the RDEPENDS.

The package can optionally be versioned (kernel-devsrc-<release>, with
kernel-devsrc holding just the /usr/src/kernel link), so the source of
more than one kernel can be installed side by side.

The patches:

  1:   maps the kernel's build directory to KERNEL_SRC_PATH, so all of
       the kernel's debug source paths are under /usr/src/kernel.
  2:   stages the manifest of the source files the build read.
  3-6: the kernel-devsrc levels, versioning, and a check that at least
       one level is set.
  7-8: two general oe-core fixes, for issues I noticed in the debug
       rootfs while working on this support. They aren't kernel
       specific and don't depend on the rest of the series:
       postinsts of debug only packages being registered against the
       image, and IMAGE_INSTALL_DEBUGFS packages with broken
       dependencies being silently skipped.

With:

  PACKAGECONFIG:pn-kernel-devsrc = "source"
  IMAGE_INSTALL_DEBUGFS = "kernel-dbg kernel-vmlinux kernel-devsrc"

on qemux86-64 (7.2.13), rootfs-dbg gets the symbols and 254MB of source
at /usr/src/kernel, and no build tools. All 9016 source files named in
the kernel's debug info are there. The only ones missing are the 592
generated files from the build directory. The full tree is 1.5GB.

One thing to know: if the image itself already installs kernel-devsrc,
IMAGE_INSTALL_DEBUGFS won't put it in rootfs-dbg. rootfs-dbg starts from
the image's package database, so it is seen as already installed, and
only the debug files are copied over. That's how the debug rootfs works
in general, but it will need documenting.

Tested on qemux86-64 (7.2.13), core-image-minimal:

  build (default): same content and RDEPENDS as before.
  source build:    the build content plus the source.
  build full:      the image builds, and on target both
                   'make scripts prepare' and a full kernel build work
                   from /usr/src/kernel.
  build tarball:   the tarball unpacks on target, 94915 files.
  source:          in rootfs-dbg via IMAGE_INSTALL_DEBUGFS, as above,
                   with no build tools in either rootfs.
  versioned:       kernel-devsrc-<release> holds the tree, and
                   kernel-devsrc only the /usr/src/kernel link.
  no level set:    the recipe is skipped with an error.

kernel-devsrc package sizes, same build (installed size, and the rpm):

  level             files  installed     rpm  RDEPENDS
  build (default)   13623      79 MB   20 MB  module build tools
  source            17590     254 MB   61 MB  none
  source build      24022     283 MB   69 MB  module build tools
  full              94915    1618 MB  251 MB  none
  build full        94935    1630 MB  253 MB  + kernel build tools
  build tarball     13623     278 MB  217 MB  + kernel build tools,
                  + tarball  (199 MB)           bzip2
  versioned source  17590     254 MB   61 MB  none
                  (kernel-devsrc itself is just the 43 byte link)

'build' has the same files and size (79302094 bytes) as kernel-devsrc
before the series.

Bruce

openembedded-core (8):
  kernel.bbclass: map the kernel build directory to KERNEL_SRC_PATH
  kernel.bbclass: stage a manifest of the source files the build read
  kernel-devsrc: add build and source PACKAGECONFIG levels
  kernel-devsrc: add full and tarball PACKAGECONFIG levels
  kernel-devsrc: optionally version the package
  kernel-devsrc: skip the recipe if no PACKAGECONFIG level is set
  rootfs: don't carry debug rootfs postinst registrations into the image
  rootfs: fail if an IMAGE_INSTALL_DEBUGFS package can't be installed

 openembedded-core:
 meta/classes-recipe/kernel.bbclass         |  47 +++++++
 meta/lib/oe/rootfs.py                      |  19 ++-
 meta/recipes-kernel/linux/kernel-devsrc.bb | 195 ++++++++++++++++++++++++-----
 3 files changed, 231 insertions(+), 30 deletions(-)

-- 
2.43.0

[OE-core][RFC PATCH 01/08] kernel.bbclass: map the kernel build directory to KERNEL_SRC_PATH

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

kernel-arch's KERNEL_CC maps STAGING_KERNEL_DIR and
STAGING_KERNEL_BUILDDIR to KERNEL_SRC_PATH, but the kernel itself builds
in ${B}, which isn't mapped. Sources generated there (*.mod.c, asn1
parsers, data tables, etc) fall through to the generic debug prefix map,
so the kernel's debug info is split across two roots, and those files
are the only content of the kernel's ${PN}-src package.

Map ${B} to KERNEL_SRC_PATH too. All of the kernel's debug source paths
are then under /usr/src/kernel, and no ${PN}-src package is created
for the kernel.

The generated sources aren't packaged anywhere. They aren't needed to
debug the kernel, and those that are of any use can be regenerated from
the kernel source, where they land at the path the debug info expects.

This is only done in kernel.bbclass, since other kernel-arch users
(module-base, perf, make-mod-scripts) have their own mapping.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/classes-recipe/kernel.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 46da867a63..ab064bb085 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -246,6 +246,12 @@ UBOOT_LOADADDRESS ?= "${UBOOT_ENTRYPOINT}"
 # Some Linux kernel configurations need additional parameters on the command line
 KERNEL_EXTRA_ARGS ?= ""
 
+# The kernel builds out of tree in ${B}, which kernel-arch does not remap.
+# Map it to KERNEL_SRC_PATH as well, so generated sources are recorded under
+# the same root as the rest of the kernel and are not collected into a
+# ${PN}-src package. The last matching prefix map wins.
+KERNEL_CC:append = " -ffile-prefix-map=${B}=${KERNEL_SRC_PATH}"
+
 EXTRA_OEMAKE += ' ARCH="${@oe.kernel.map_kernel_arch(d)}"'
 EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"'
 EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"'
-- 
2.43.0

[OE-core][RFC PATCH 02/08] kernel.bbclass: stage a manifest of the source files the build read

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

kernel-devsrc packages from the shared work directories and can't see the
kernel's ${B}, so it has no way to know which parts of the tree the
kernel for a given MACHINE was built from. kbuild already records that:
every .cmd file lists the source and headers the compiler read for its
object.

Collect those paths (relative to ${S}) in do_shared_workdir, after the
build, into kernel-source-files in STAGING_KERNEL_BUILDDIR. kernel-devsrc
can then package the source needed to debug a kernel built for a specific
MACHINE, without copying the whole tree.

The new kernel_source_manifest function does it in three steps, and leaves
each intermediate list in the recipe's temporary directory, so a wrong
manifest can be traced back:

  - the .cmd files whose savedcmd starts with ${HOST_PREFIX} (the target
    compiler, optionally through ccache). Host programs (scripts/mod,
    scripts/kconfig, objtool) are built with the host compiler, and are
    not part of the kernel.
  - every path under ${S} those files name.
  - the paths normalized with realpath, since some sources are included
    with ../ (kernel/bpf includes tools/lib/bpf), and kept only if they
    are regular files in ${S}.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/classes-recipe/kernel.bbclass | 41 ++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index ab064bb085..f7910fbdee 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -538,6 +538,44 @@ emit_depmod_pkgdata() {
 
 PACKAGEFUNCS += "emit_depmod_pkgdata"
 
+# Print, relative to ${S}, the source files the target compiler read, from
+# kbuild's .cmd files. The intermediate lists are left in ${T}.
+kernel_source_manifest() {
+	(
+		cd ${B}
+		# The .cmd files hold millions of lines: a UTF-8 locale is ~40x slower
+		export LC_ALL=C
+
+		cmds=${T}/kernel-source-cmds
+		paths=${T}/kernel-source-paths
+
+		# The .cmd files of objects built with the target compiler, matching:
+		#   savedcmd_kernel/fork.o := x86_64-poky-linux-gcc -fuse-ld=bfd ...
+		#   savedcmd_kernel/fork.o := ccache x86_64-poky-linux-gcc ...
+		target_cc="^savedcmd_[^ ]+ := ([^ ]*ccache )?${HOST_PREFIX}"
+		find . -name '.*.cmd' -exec grep -l -E "$target_cc" {} + > $cmds
+
+		# Every path under ${S} they name, matching (dedup before the regex):
+		#   source_kernel/fork.o := ${S}/kernel/fork.c
+		#     ${S}/include/linux/types.h
+		#   ... -I${S}/include -include ${S}/include/linux/kconfig.h ...
+		# Directories and other non-files are dropped below.
+		xargs -r grep -hF "${S}/" < $cmds | sort -u | \
+			grep -o "${S}/[^ ]*" | sort -u > $paths
+
+		# Normalized relative to ${S}, for example:
+		#   ${S}/arch/x86/boot/compressed/../../../../lib/ctype.c -> lib/ctype.c
+		# and kept only if they are regular files inside ${S}.
+		xargs -r realpath -m --relative-to=${S} < $paths | \
+			grep -v '^\.\./' | sort -u | \
+			while read f; do
+				if [ -f "${S}/$f" ]; then
+					echo "$f"
+				fi
+			done
+	)
+}
+
 do_shared_workdir[cleandirs] += " ${STAGING_KERNEL_BUILDDIR}"
 do_shared_workdir () {
 	cd ${B}
@@ -605,6 +643,9 @@ do_shared_workdir () {
 		fi
 	fi
 
+	# The source files this kernel was built from, for kernel-devsrc
+	kernel_source_manifest > $kerneldir/kernel-source-files
+
 	# When building with CONFIG_MODVERSIONS=y and CONFIG_RANDSTRUCT=y we need
 	# to copy the build assets generated for the randstruct seed to
 	# STAGING_KERNEL_BUILDDIR, otherwise the out-of-tree modules build will
-- 
2.43.0

[OE-core][RFC PATCH 03/08] kernel-devsrc: add build and source PACKAGECONFIG levels

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Debugging the kernel from a debug rootfs needs its source at
/usr/src/kernel, where the debug info points, but kernel-devsrc only
carries what is needed to build modules, and installing it pulls in the
build tools (bc, bison, flex, gcc-plugins, openssl-dev, ...).

Split the content of the one kernel-devsrc tree into PACKAGECONFIG
levels instead of adding a second source package:

  build:  today's content and RDEPENDS ('make scripts prepare' on
          target). The default, so nothing changes for existing users.
  source: a curated spine (KERNELDEV_SOURCE_SPINE plus arch/${ARCH})
          and the drivers/sound files the kernel build read, taken from
          the kernel's kernel-source-files manifest. Only source files
          are copied, never scripts, so there are no runtime or file
          dependencies.

The RDEPENDS are split out, so they are only added when the 'build'
level is enabled.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 107 +++++++++++++++------
 1 file changed, 80 insertions(+), 27 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index 90bdbfc534..3b83fd145c 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -28,30 +28,31 @@ B = "${STAGING_KERNEL_BUILDDIR}"
 
 PACKAGE_ARCH = "${MACHINE_ARCH}"
 
+# The flags choose what goes in the kernel-devsrc tree:
+#   build:  what is needed to build out of tree modules against the kernel
+#           ('make scripts prepare' on target). The default, and what
+#           kernel-devsrc has always been.
+#   source: the source needed to debug a kernel built for a specific
+#           MACHINE, see KERNELDEV_SOURCE_SPINE. No runtime dependencies,
+#           so it can go in a debug rootfs without the build tools.
+PACKAGECONFIG ?= "build"
+PACKAGECONFIG[build] = ",,,${KERNELDEV_BUILD_RDEPENDS}"
+PACKAGECONFIG[source] = ",,"
+
+# The curated part of the 'source' level, copied in full along with
+# arch/${ARCH}. The rest (drivers, sound, samples, ...) is not curated: the
+# files the kernel build read are discovered from the manifest the kernel
+# stages in do_shared_workdir. The manifest could find the spine's files too,
+# but the spine is listed so that 'source' always has a sane default.
+KERNELDEV_SOURCE_SPINE ?= "block certs crypto fs include init io_uring ipc kernel lib mm net security virt"
+
 KERNEL_BUILD_ROOT = "${nonarch_base_libdir}/modules/"
 
-do_install() {
+# The 'build' level: what is needed to build out of tree modules against the
+# kernel ('make scripts prepare' on target), and what kernel-devsrc has
+# always contained.
+kernel_devsrc_install_build() {
     kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
-    install -d $kerneldir
-
-    # create the directory structure
-    rm -f $kerneldir/build
-    rm -f $kerneldir/source
-    mkdir -p $kerneldir/build
-
-    # for compatibility with some older variants of this package, we
-    # create  a /usr/src/kernel symlink to /lib/modules/<version>/source
-    mkdir -p ${D}/usr/src
-    (
-        cd ${D}/usr/src
-        ln -rs ${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}/source kernel
-    )
-
-    # for on target purposes, we unify build and source
-    (
-        cd $kerneldir
-        ln -s build source
-    )
 
     # first copy everything
     (
@@ -448,7 +449,58 @@ do_install() {
         sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "$ss"
         sed -i 's,/usr/bin/python$,/usr/bin/env python3,' "$ss"
     done
+}
+
+# The 'source' level: the curated spine plus the files the kernel build read,
+# from the manifest the kernel stages in do_shared_workdir. Only source files
+# are copied, never scripts, so the package gets no interpreter dependencies.
+kernel_devsrc_install_source() {
+    kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
 
+    if [ ! -s ${B}/kernel-source-files ]; then
+        bbfatal "${B}/kernel-source-files is missing, the kernel did not stage its source manifest"
+    fi
+    (
+        cd ${S}
+        {
+            find ${KERNELDEV_SOURCE_SPINE} arch/${ARCH} -type f
+            cat ${B}/kernel-source-files
+        } | grep -E '\.([chS]|rs)$' | sort -u | cpio --quiet -pdlu $kerneldir/build
+    )
+}
+
+do_install() {
+    kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
+    install -d $kerneldir
+
+    # create the directory structure
+    rm -f $kerneldir/build
+    rm -f $kerneldir/source
+    mkdir -p $kerneldir/build
+
+    # for compatibility with some older variants of this package, we
+    # create  a /usr/src/kernel symlink to /lib/modules/<version>/source
+    mkdir -p ${D}/usr/src
+    (
+        cd ${D}/usr/src
+        ln -rs ${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}/source kernel
+    )
+
+    # for on target purposes, we unify build and source
+    (
+        cd $kerneldir
+        ln -s build source
+    )
+
+    if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'build', '1', '', d)}" ]; then
+        kernel_devsrc_install_build
+    fi
+
+    # The 'source' level goes in the same tree, at /usr/src/kernel, which is
+    # where the kernel's debug info points.
+    if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'source', '1', '', d)}" ]; then
+        kernel_devsrc_install_source
+    fi
     chown -R root:root ${D}
 }
 
@@ -457,14 +509,15 @@ do_install[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
 
 FILES:${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH}"
 
-RDEPENDS:${PN} = "bc python3-core flex bison ${TCLIBC}-utils gawk"
+# Added to RDEPENDS:${PN} by the 'build' PACKAGECONFIG
+KERNELDEV_BUILD_RDEPENDS = "bc python3-core flex bison ${TCLIBC}-utils gawk"
 # 4.15+ needs these next two RDEPENDS
-RDEPENDS:${PN} += "openssl-dev util-linux"
+KERNELDEV_BUILD_RDEPENDS += "openssl-dev util-linux"
 # and x86 needs a bit more for 4.15+
-RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-dev', '', d)}"
+KERNELDEV_BUILD_RDEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-dev', '', d)}"
 # powerpc needs elfutils on 6.3+
-RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'powerpc', 'elfutils-dev', '', d)}"
+KERNELDEV_BUILD_RDEPENDS += "${@bb.utils.contains('ARCH', 'powerpc', 'elfutils-dev', '', d)}"
 # 5.8+ needs gcc-plugins libmpc-dev
-RDEPENDS:${PN} += "gcc-plugins libmpc-dev"
+KERNELDEV_BUILD_RDEPENDS += "gcc-plugins libmpc-dev"
 # 5.13+ needs grep for powerpc
-RDEPENDS:${PN}:append:powerpc = " grep"
+KERNELDEV_BUILD_RDEPENDS:append:powerpc = " grep"
-- 
2.43.0

[OE-core][RFC PATCH 04/08] kernel-devsrc: add full and tarball PACKAGECONFIG levels

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

The 'source' PACKAGECONFIG packages what a debugger needs to debug a
kernel built for a specific MACHINE. Other use cases need the entire
tree, either installed in a rootfs or as a package on a feed.

Add two more levels to the same kernel-devsrc tree:

  full:    the whole source tree (without the git metadata), a
           superset of 'source', uncompressed at /usr/src/kernel.
  tarball: the whole tree as /usr/src/kernel-source-<version>.tar.bz2
           instead of uncompressed. It can't be debugged or built
           against, it is a copy to unpack elsewhere.

The scripts in a 'full' tree are not run from it, so unless 'build' is
also enabled, file dependencies are skipped for the package. Otherwise
it pulls in perl, python3, make, etc, and can fail to install when an
interpreter path isn't provided (/bin/awk on usrmerge).

With 'build', the file dependencies are kept, so a few scripts in the
tree that use #!/bin/awk or #!/bin/env (arm64 and x86 awk tools,
selftests) would make kernel-devsrc uninstallable: no package provides
those paths. Point them at ${bindir}, as the 'build' level already does
for the arm64 awk tools. The package QA file-rdeps check doesn't catch
this, since it is skipped for module-base recipes.

With 'build' and the whole tree ('full', or 'tarball' unpacked on the
target), more than 'make scripts prepare' can be built from it (the
kernel itself, modules_install, compressed images and modules, module
signing), so add the tools those run that aren't found by file
dependencies: perl, kmod, cpio, tar, gzip, xz, zstd, lz4 and openssl.
This is a fixed list, not derived from the kernel's configuration.
pahole (for BTF) isn't in oe-core, so it isn't included.

'tarball' adds bzip2 on its own: unpacking the tarball is the only
use of it, and tar runs bzip2 to do that.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 70 ++++++++++++++++++++--
 1 file changed, 66 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index 3b83fd145c..46337bd79b 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -35,9 +35,20 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
 #   source: the source needed to debug a kernel built for a specific
 #           MACHINE, see KERNELDEV_SOURCE_SPINE. No runtime dependencies,
 #           so it can go in a debug rootfs without the build tools.
+#   full:   the whole source tree (a superset of 'source').
+#   tarball: the whole source tree as /usr/src/kernel-source-<version>.tar.bz2,
+#           in place of the uncompressed tree from 'full'. Not usable by
+#           debuggers or for building, it is a copy to unpack elsewhere.
 PACKAGECONFIG ?= "build"
 PACKAGECONFIG[build] = ",,,${KERNELDEV_BUILD_RDEPENDS}"
 PACKAGECONFIG[source] = ",,"
+PACKAGECONFIG[full] = ",,"
+PACKAGECONFIG[tarball] = ",,,bzip2"
+
+# The scripts in a 'full' tree are not run from it, so unless 'build' needs
+# them, don't generate runtime dependencies from their interpreters (perl,
+# python3, make, ...).
+SKIP_FILEDEPS:${PN} = "${@'1' if bb.utils.contains('PACKAGECONFIG', 'full', True, False, d) and not bb.utils.contains_any('PACKAGECONFIG', 'build tarball', True, False, d) else '0'}"
 
 # The curated part of the 'source' level, copied in full along with
 # arch/${ARCH}. The rest (drivers, sound, samples, ...) is not curated: the
@@ -469,6 +480,45 @@ kernel_devsrc_install_source() {
     )
 }
 
+# The files of the whole source tree, NUL separated, for 'full' and
+# 'tarball'. Run from ${S}. The git metadata and kernel-meta are left out.
+kernel_devsrc_tree_files() {
+    find . -type d -name '.git*' -prune -o -type d -name '.kernel-meta' -prune -o -type f -print0
+}
+
+# The 'full' level: the whole source tree, without the git metadata.
+kernel_devsrc_install_full() {
+    kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
+
+    # With 'build' also enabled, this runs after it and copies over the
+    # source files it installed (cpio -u). Most are identical, but the
+    # 'build' level's customizations are lost: scripts/package/debian/rules
+    # comes back, and the arm64 awk and python shebang fixups are undone (the
+    # awk ones are redone below). Generated files from ${B} aren't in ${S}, so
+    # they are kept. Skipping the files 'build' installed isn't worth the
+    # complexity: 'full' is meant to be the whole tree.
+    (
+        cd ${S}
+        kernel_devsrc_tree_files | cpio --quiet --null -pdlu $kerneldir/build
+    )
+
+    # The whole tree brings in scripts the 'build' level doesn't copy, and
+    # some use #!/bin/awk or #!/bin/env, which no package provides. Point them
+    # at ${bindir}, keeping their arguments (awk -f). sed -i writes a new file,
+    # so the hardlinks into ${S} are not modified.
+    grep -rlZE '^#! */bin/(awk|env)( |$)' $kerneldir/build | \
+        xargs -0 -r sed -i -E -e '1s,^#! */bin/(awk|env)( |$),#!${bindir}/\1\2,'
+}
+
+# The 'tarball' level: the whole source tree, compressed.
+kernel_devsrc_install_tarball() {
+    (
+        cd ${S}
+        kernel_devsrc_tree_files | \
+            tar --null --no-recursion -cjf ${D}/usr/src/kernel-source-${KERNEL_VERSION}.tar.bz2 --files-from -
+    )
+}
+
 do_install() {
     kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
     install -d $kerneldir
@@ -496,18 +546,25 @@ do_install() {
         kernel_devsrc_install_build
     fi
 
-    # The 'source' level goes in the same tree, at /usr/src/kernel, which is
-    # where the kernel's debug info points.
-    if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'source', '1', '', d)}" ]; then
+    # The source levels go in the same tree, at /usr/src/kernel, which is
+    # where the kernel's debug info points. 'full' is a superset of 'source',
+    # and 'tarball' replaces the uncompressed 'full' tree.
+    if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'full', '1', '', d)}" ] &&
+       [ -z "${@bb.utils.contains('PACKAGECONFIG', 'tarball', '1', '', d)}" ]; then
+        kernel_devsrc_install_full
+    elif [ -n "${@bb.utils.contains('PACKAGECONFIG', 'source', '1', '', d)}" ]; then
         kernel_devsrc_install_source
     fi
+    if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'tarball', '1', '', d)}" ]; then
+        kernel_devsrc_install_tarball
+    fi
     chown -R root:root ${D}
 }
 
 # Ensure we don't race against "make scripts" during cpio
 do_install[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
 
-FILES:${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH}"
+FILES:${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH} /usr/src/kernel-source-*.tar.bz2"
 
 # Added to RDEPENDS:${PN} by the 'build' PACKAGECONFIG
 KERNELDEV_BUILD_RDEPENDS = "bc python3-core flex bison ${TCLIBC}-utils gawk"
@@ -521,3 +578,8 @@ KERNELDEV_BUILD_RDEPENDS += "${@bb.utils.contains('ARCH', 'powerpc', 'elfutils-d
 KERNELDEV_BUILD_RDEPENDS += "gcc-plugins libmpc-dev"
 # 5.13+ needs grep for powerpc
 KERNELDEV_BUILD_RDEPENDS:append:powerpc = " grep"
+# With the whole tree as well ('full', or 'tarball' unpacked on target), more
+# than 'make scripts prepare' can be built: a whole kernel, modules_install,
+# compressed images and modules, module signing.
+KERNELDEV_FULL_BUILD_RDEPENDS = "perl kmod cpio tar gzip xz zstd lz4 openssl"
+KERNELDEV_BUILD_RDEPENDS += "${@bb.utils.contains_any('PACKAGECONFIG', 'full tarball', '${KERNELDEV_FULL_BUILD_RDEPENDS}', '', d)}"
-- 
2.43.0

[OE-core][RFC PATCH 05/08] kernel-devsrc: optionally version the package

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

kernel-devsrc is one package name for every kernel version, so the trees
of two kernel versions can't be installed side by side (an upgrade
replaces the old one).

The tree itself is already versioned (/lib/modules/<version>/build), only
the /usr/src/kernel link is shared. So add KERNELDEV_VERSIONED_PACKAGE.
When it is "1", the tree is packaged as kernel-devsrc-<version>, and
kernel-devsrc keeps only the link and depends on the version it points
at, the way the kernel's kernel-image and kernel-modules packages depend
on their versioned packages.

The link stays a packaged file, so installing or upgrading kernel-devsrc
moves it with no postinst or postrm, and nothing has to run at rootfs
time. The debug info always records /usr/src/kernel, so the link pointing
at the current kernel is what debuggers expect; an older version installed
directly is reached through its own /lib/modules/<version>/source.
Anything that looks for kernel-devsrc by name (OEHasPackage in the
kernelmodule and stap runtime tests, IMAGE_INSTALL, the SDK) still finds
it.

The 'build' RDEPENDS go on the versioned package too, so a tree installed
on its own still has what it needs to build modules.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index 46337bd79b..c2021002d4 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -59,6 +59,14 @@ KERNELDEV_SOURCE_SPINE ?= "block certs crypto fs include init io_uring ipc kerne
 
 KERNEL_BUILD_ROOT = "${nonarch_base_libdir}/modules/"
 
+# Set to "1" to package the tree as kernel-devsrc-<kernel version>, so the
+# trees of two kernel versions can be installed side by side. kernel-devsrc
+# then only holds the /usr/src/kernel link and depends on the version it
+# points at, like the kernel's own kernel-image/kernel-modules packages.
+# Installing or upgrading kernel-devsrc moves the link to that version, and
+# anything that looks for kernel-devsrc by name still finds it.
+KERNELDEV_VERSIONED_PACKAGE ?= ""
+
 # The 'build' level: what is needed to build out of tree modules against the
 # kernel ('make scripts prepare' on target), and what kernel-devsrc has
 # always contained.
@@ -566,6 +574,15 @@ do_install[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
 
 FILES:${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH} /usr/src/kernel-source-*.tar.bz2"
 
+# The versioned tree (KERNELDEV_VERSIONED_PACKAGE). It is ahead of ${PN} in
+# PACKAGES, so it takes everything but the /usr/src/kernel link.
+PACKAGES =+ "${@'${PN}-tree' if d.getVar('KERNELDEV_VERSIONED_PACKAGE') == '1' else ''}"
+PKG:${PN}-tree = "${PN}-${@legitimize_package_name(d.getVar('KERNEL_VERSION') or '')}"
+FILES:${PN}-tree = "${KERNEL_BUILD_ROOT} /usr/src/kernel-source-*.tar.bz2"
+RDEPENDS:${PN}-tree = "${@bb.utils.contains('PACKAGECONFIG', 'build', '${KERNELDEV_BUILD_RDEPENDS}', '', d)}"
+SKIP_FILEDEPS:${PN}-tree = "${SKIP_FILEDEPS:${PN}}"
+RDEPENDS:${PN} += "${@'${PN}-tree (= ${EXTENDPKGV})' if d.getVar('KERNELDEV_VERSIONED_PACKAGE') == '1' else ''}"
+
 # Added to RDEPENDS:${PN} by the 'build' PACKAGECONFIG
 KERNELDEV_BUILD_RDEPENDS = "bc python3-core flex bison ${TCLIBC}-utils gawk"
 # 4.15+ needs these next two RDEPENDS
-- 
2.43.0

[OE-core][RFC PATCH 06/08] kernel-devsrc: skip the recipe if no PACKAGECONFIG level is set

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

With none of build, source, full or tarball in PACKAGECONFIG,
kernel-devsrc still builds and installs, but the package is only the
/usr/src/kernel link to an empty tree. Nothing fails until someone
tries to use it on target, and then it is not obvious why.

Skip the recipe with a message instead, so an image that installs
kernel-devsrc fails at parse time and says which PACKAGECONFIG
is missing.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index c2021002d4..621afb04de 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -50,6 +50,13 @@ PACKAGECONFIG[tarball] = ",,,bzip2"
 # python3, make, ...).
 SKIP_FILEDEPS:${PN} = "${@'1' if bb.utils.contains('PACKAGECONFIG', 'full', True, False, d) and not bb.utils.contains_any('PACKAGECONFIG', 'build tarball', True, False, d) else '0'}"
 
+# Without a level, kernel-devsrc would only be a /usr/src/kernel link to an
+# empty tree.
+python () {
+    if not bb.utils.contains_any('PACKAGECONFIG', 'build source full tarball', True, False, d):
+        raise bb.parse.SkipRecipe("PACKAGECONFIG needs at least one of build, source, full or tarball")
+}
+
 # The curated part of the 'source' level, copied in full along with
 # arch/${ARCH}. The rest (drivers, sound, samples, ...) is not curated: the
 # files the kernel build read are discovered from the manifest the kernel
-- 
2.43.0

[OE-core][RFC PATCH 07/08] rootfs: don't carry debug rootfs postinst registrations into the image

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

The debug rootfs is created in the image's rootfs directory, with the
same package manager, so its package installs use the image's intercept
scripts. Postinsts register their packages in those scripts, and the
intercepts are only run once, for the image, after the debug rootfs has
been moved aside.

So a package that is only installed in the debug rootfs (from
IMAGE_INSTALL_DEBUGFS) and has an on-target postinst is registered with
delay_to_first_boot, and do_rootfs then tries to save its postinst from
the image, where it isn't installed:

  ERROR: core-image-minimal-1.0-r0 do_rootfs: Could not invoke rpm. Command
  '.../rpm -q --root=.../rootfs --queryformat %{postin} <pkg>' returned 1:
  package <pkg> is not installed

Other intercepts get the debug packages added to their package lists in
the same way.

The debug rootfs is a debug sysroot, it isn't booted, so there is nothing
to defer its postinsts to. Save the image's intercept scripts before the
debug packages are installed, and put them back afterwards, so the
intercepts only see what the image installed.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/lib/oe/rootfs.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index e0365da285..aac93dc42e 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -145,6 +145,16 @@ class Rootfs(object, metaclass=ABCMeta):
                     shutil.copy(os.path.join(root, f),
                                 self.image_rootfs + relative_dir)
 
+        # The debug rootfs is installed with the image's intercept scripts, and
+        # postinsts register their packages in them (delay_to_first_boot, for
+        # example). The intercepts are only run for the image, so a package
+        # installed only in the debug rootfs would have its postinst saved
+        # against an image that doesn't contain it. Put the image's intercepts
+        # back once the debug packages are installed.
+        intercepts_dir = self.pm.intercepts_dir
+        bb.utils.remove(intercepts_dir + '-orig', True)
+        shutil.copytree(intercepts_dir, intercepts_dir + '-orig', symlinks=True)
+
         bb.note("  Install complementary '*-dbg' packages...")
         self.pm.install_complementary('*-dbg')
 
@@ -162,6 +172,9 @@ class Rootfs(object, metaclass=ABCMeta):
             bb.note("  Install extra debug packages...")
             self.pm.install(extra_debug_pkgs.split(), True)
 
+        bb.utils.remove(intercepts_dir, True)
+        bb.utils.rename(intercepts_dir + '-orig', intercepts_dir)
+
         bb.note("  Removing package database...")
         for path in package_paths:
             if os.path.isdir(self.image_rootfs + path):
-- 
2.43.0

[OE-core][RFC PATCH 08/08] rootfs: fail if an IMAGE_INSTALL_DEBUGFS package can't be installed

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

IMAGE_INSTALL_DEBUGFS packages are installed in attempt-only mode, so a
package whose dependencies can't be met is skipped (dnf --skip-broken),
the image builds, and the package is silently missing from the debug
rootfs. The only trace is in log.do_rootfs:

  - nothing provides /usr/bin/foo needed by <pkg> from oe-repo
  Skipping packages with broken dependencies:
   <pkg>

These packages are asked for by name, the same as IMAGE_INSTALL, which
fails the build in this case. (A missing package already fails earlier,
since the image RDEPENDS on them.) Install them normally, so a broken
dependency is reported as an error rather than leaving a debug rootfs
without the package that was requested.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/lib/oe/rootfs.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index aac93dc42e..a28c888070 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -165,12 +165,14 @@ class Rootfs(object, metaclass=ABCMeta):
         """
         Install additional debug packages. Possibility to install additional packages,
         which are not automatically installed as complementary package of
-        standard one, e.g. debug package of static libraries.
+        standard one, e.g. debug package of static libraries. They are asked
+        for by name, so like IMAGE_INSTALL, it is an error if they can't be
+        installed.
         """
         extra_debug_pkgs = self.d.getVar('IMAGE_INSTALL_DEBUGFS')
         if extra_debug_pkgs:
             bb.note("  Install extra debug packages...")
-            self.pm.install(extra_debug_pkgs.split(), True)
+            self.pm.install(extra_debug_pkgs.split())
 
         bb.utils.remove(intercepts_dir, True)
         bb.utils.rename(intercepts_dir + '-orig', intercepts_dir)
-- 
2.43.0

Re: [OE-core][RFC PATCH 0/8] enhance kernel -devsrc output options

[Richard Purdie]

On Fri, 2026-11-06 at 12:32 -0500, bruce.ashfield@gmail.com wrote:
>   7-8: two general oe-core fixes, for issues I noticed in the debug
>        rootfs while working on this support. They aren't kernel
>        specific and don't depend on the rest of the series:
>        postinsts of debug only packages being registered against the
>        image, and IMAGE_INSTALL_DEBUGFS packages with broken
>        dependencies being silently skipped.

Do you have an example of something which has a postinst which this
pulls in? I'm wondering why debug src packages have postinsts :/

Cheers,

Richard

Re: [OE-core][RFC PATCH 0/8] enhance kernel -devsrc output options

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

On Fri, 2026-11-06 at 14:08 -0500, Richard Purdie wrote:
> On Fri, 2026-11-06 at 12:32 -0500, bruce.ashfield@gmail.com wrote:
> >   7-8: two general oe-core fixes, for issues I noticed in the debug
> >        rootfs while working on this support. They aren't kernel
> >        specific and don't depend on the rest of the series:
> >        postinsts of debug only packages being registered against the
> >        image, and IMAGE_INSTALL_DEBUGFS packages with broken
> >        dependencies being silently skipped.
>
> Do you have an example of something which has a postinst which this
> pulls in? I'm wondering why debug src packages have postinsts :/

They don't :) Nothing in the -devsrc part of the series has a postinst,
and neither do the complementary -dbg packages.

I hit it while prototyping the versioned package, which originally used
a postinst for the /usr/src symlink (dropped, the link is now a plain
file owned by kernel-devsrc). I then reproduced it with a test recipe
using pkg_postinst_ontarget. I could dig up that reproducer and send
it along. I should have mentioned that in the series send.

The general case is that IMAGE_INSTALL_DEBUGFS takes any package. If
one that is only in the debug rootfs has a postinst that defers to
first boot, it gets registered in the intercepts the image shares, and
do_rootfs fails trying to save a postinst for a package that isn't in
the image. In oe-core, gnutls-fips and openssl-ossl-module-fips are the
obvious candidates, though I haven't built either into a debug rootfs.

So it's not needed by this series anymore. I'm happy to split 7-8 out
and send them separately, or chuck it in the bin if it isn't necessary.
I'll run it locally, since I don't throw much away!

Bruce

>
> Cheers,
>
> Richard