Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH v2 0/3] Fix kernel complementary package installation
@ 2026-09-23 20:51 AdrianF
  2026-09-23 20:51 ` [PATCH v2 1/3] kernel.bbclass: make the -src package installable on purpose AdrianF
                   ` (3 more replies)
  0 siblings, 4 replies; 7+ messages in thread
From: AdrianF @ 2026-09-23 20:51 UTC (permalink / raw)
  To: openembedded-core; +Cc: Adrian Freihofer

From: Adrian Freihofer <adrian.freihofer@siemens.com>

Changes in comparison to v1: Completely different approach
- Complementary package installation for kernel sources is disabled
- The linux-yocto-src package can now be installed as a regular package which
  was not possible before.
- Additionally, the kernel source files are now correctly included in the
  linux-yocto-src package.

Adrian Freihofer (3):
  kernel.bbclass: make the -src package installable on purpose
  core-image.bbclass: do not pull the kernel -src package in via
    complementary packages
  oe/package.py: copydebugsources: add all kernel source files

 meta/classes-recipe/core-image.bbclass        |  4 +-
 meta/classes-recipe/kernel.bbclass            |  3 +-
 meta/lib/oe/package.py                        | 25 ++++++-
 meta/lib/oeqa/selftest/cases/oelib/package.py | 74 ++++++++++++++++++-
 4 files changed, 98 insertions(+), 8 deletions(-)

-- 
2.55.0



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v2 1/3] kernel.bbclass: make the -src package installable on purpose
  2026-09-23 20:51 [PATCH v2 0/3] Fix kernel complementary package installation AdrianF
@ 2026-09-23 20:51 ` AdrianF
  2026-09-23 20:51 ` [PATCH v2 2/3] core-image.bbclass: do not pull the kernel -src package in via complementary packages AdrianF
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 7+ messages in thread
From: AdrianF @ 2026-09-23 20:51 UTC (permalink / raw)
  To: openembedded-core; +Cc: Adrian Freihofer

From: Adrian Freihofer <adrian.freihofer@siemens.com>

The kernel-source-with-debug-src debug-with-srcpkg split style stashes the
kernel's source under the -src package, but that package was never added
to PACKAGES. It therefore does not exist as an ordinary, installable
package: the only way it ever ends up on an image is through the package
complementary mechanism, which is not suitable for the kernel. Not even
kernel-dbg, which the complementary mechanism is meant for, can be pulled
in that way for the kernel, since IMAGE_INSTALL_COMPLEMENTARY on
*-dbg matches against PN, and the kernel recipe's packages are named after
KERNEL_PACKAGE_NAME rather than PN.

The result was that -src got installed on debug images most likely by
accident, as a side effect of the complementary glob, rather than because
anyone asked for it.

Add -src to PACKAGES when PACKAGE_DEBUG_SPLIT_STYLE is
"debug-with-srcpkg", so it becomes a regular package that can be
requested through IMAGE_INSTALL or RDEPENDS like any other.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 0a6d754108..3c887a5cc5 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -678,7 +678,8 @@ EXPORT_FUNCTIONS do_compile do_transform_kernel do_transform_bundled_initramfs d
 
 # kernel-base becomes kernel-${KERNEL_VERSION}
 # kernel-image becomes kernel-image-${KERNEL_VERSION}
-PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-dbg"
+PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-dbg \
+    ${@d.getVar('PN') + '-src' if d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-with-srcpkg' else ''}"
 FILES:${PN} = ""
 FILES:${KERNEL_PACKAGE_NAME}-base = "${KERNEL_MODULE_INSTALL_PREFIX}/modules.order ${KERNEL_MODULE_INSTALL_PREFIX}/modules.builtin ${KERNEL_MODULE_INSTALL_PREFIX}/modules.builtin.modinfo"
 FILES:${KERNEL_PACKAGE_NAME}-image = ""
-- 
2.55.0



^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH v2 2/3] core-image.bbclass: do not pull the kernel -src package in via complementary packages
  2026-09-23 20:51 [PATCH v2 0/3] Fix kernel complementary package installation AdrianF
  2026-09-23 20:51 ` [PATCH v2 1/3] kernel.bbclass: make the -src package installable on purpose AdrianF
@ 2026-09-23 20:51 ` AdrianF
  2026-09-23 20:51 ` [PATCH v2 3/3] oe/package.py: copydebugsources: add all kernel source files AdrianF
  2026-10-29 23:09 ` [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation Bruce Ashfield
  3 siblings, 0 replies; 7+ messages in thread
From: AdrianF @ 2026-09-23 20:51 UTC (permalink / raw)
  To: openembedded-core; +Cc: Adrian Freihofer

From: Adrian Freihofer <adrian.freihofer@siemens.com>

IMAGE_INSTALL_COMPLEMENTARY matches complementary packages such as *-src
and *-dbg against PN, but the kernel recipe names its packages after
KERNEL_PACKAGE_NAME rather than PN. That mismatch means kernel-dbg is
never actually installed by the complementary mechanism, while -src
still got pulled in, since it is matched by the generic *-src glob rather
than one keyed off KERNEL_PACKAGE_NAME. So a debug image ended up with
the kernel's -src package in rootfs-dbg, most likely unintentionally,
without the -dbg package it would normally come with.

Add ${PREFERRED_PROVIDER_virtual/kernel}-src to
PACKAGE_EXCLUDE_COMPLEMENTARY so the kernel -src package is no longer
installed automatically like this. Getting kernel debug symbols onto an
image most likely needs a different approach than pulling them into
rootfs or rootfs-dbg via IMAGE_INSTALL_COMPLEMENTARY, which is left for
follow-up work.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/core-image.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/core-image.bbclass b/meta/classes-recipe/core-image.bbclass
index 167fa97fd9..a3afca5ebb 100644
--- a/meta/classes-recipe/core-image.bbclass
+++ b/meta/classes-recipe/core-image.bbclass
@@ -74,7 +74,9 @@ IMAGE_FEATURES_REPLACES_ssh-server-openssh = "ssh-server-dropbear"
 # Do not install openssh complementary packages if either packagegroup-core-ssh-dropbear or dropbear
 # is installed # to avoid openssh-dropbear conflict
 # see [Yocto #14858] for more information
-PACKAGE_EXCLUDE_COMPLEMENTARY:append = "${@bb.utils.contains_any('PACKAGE_INSTALL', 'packagegroup-core-ssh-dropbear dropbear', ' openssh', '' , d)}"
+# Do not automatically install e.g. linux-yocto-src but not kernel-dbg
+PACKAGE_EXCLUDE_COMPLEMENTARY:append = "${@bb.utils.contains_any('PACKAGE_INSTALL', 'packagegroup-core-ssh-dropbear dropbear', ' openssh', '' , d)}\
+    ${PREFERRED_PROVIDER_virtual/kernel}-src"
 
 # IMAGE_FEATURES_CONFLICTS_foo = 'bar1 bar2'
 # An error exception would be raised if both image features foo and bar1(or bar2) are included
-- 
2.55.0



^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH v2 3/3] oe/package.py: copydebugsources: add all kernel source files
  2026-09-23 20:51 [PATCH v2 0/3] Fix kernel complementary package installation AdrianF
  2026-09-23 20:51 ` [PATCH v2 1/3] kernel.bbclass: make the -src package installable on purpose AdrianF
  2026-09-23 20:51 ` [PATCH v2 2/3] core-image.bbclass: do not pull the kernel -src package in via complementary packages AdrianF
@ 2026-09-23 20:51 ` AdrianF
  2026-10-29 23:09 ` [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation Bruce Ashfield
  3 siblings, 0 replies; 7+ messages in thread
From: AdrianF @ 2026-09-23 20:51 UTC (permalink / raw)
  To: openembedded-core; +Cc: Adrian Freihofer

From: Adrian Freihofer <adrian.freihofer@siemens.com>

Add all kernel source files to the linux-yocto-src package, not only
the sources from STAGING_KERNEL_DIR.

KERNEL_CC (kernel-arch.bbclass) remaps STAGING_KERNEL_DIR and
STAGING_KERNEL_BUILDDIR to KERNEL_SRC_PATH, overriding CFLAGS own
-ffile-prefix-map for the kernel. copydebugsources() only looks at
CFLAGS to build its prefixmap, so kernel source files recorded under
KERNEL_SRC_PATH were silently dropped. Add prefixmap entries for both
kernel dirs, still destined for TARGET_DBGSRC_DIR.

Also fix oeqa FakeDataStore, which did not support the getVar(name,
expand) signature used by bb.data.inherits_class(), and add a test
covering the kernel-arch recovery path.

Verified by building linux-yocto for qemux86-64: without this fix,
linux-yocto-src only contains 545 files (510 auto-generated *.mod.c
module stubs plus 35 misc .c/.h files compiled outside KERNEL_CC's
remap); with the fix, all 8356 real kernel .c/.h sources recorded
under KERNEL_SRC_PATH are recovered too, for 8901 files total,
including files that only exist under STAGING_KERNEL_BUILDDIR (e.g.
arch/x86/include/generated/asm/syscalls_32.h).

Having all kernel files available is useful for example when debugging
a user space application with the Lauterbach debugger which supports
seamless stepping between user space and kernel space.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/lib/oe/package.py                        | 25 ++++++-
 meta/lib/oeqa/selftest/cases/oelib/package.py | 74 ++++++++++++++++++-
 2 files changed, 93 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index b7030643d2..477fa5d5a7 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -979,6 +979,10 @@ def copydebugsources(debugsrcdir, sources, d):
         sdir = d.getVar("S")
         cflags = d.expand("${CFLAGS}")
 
+        # prefixmap maps a host directory to a (match, dest) pair: "match" is
+        # the DWARF-recorded path prefix to select from debugsources.list,
+        # "dest" is the subdirectory under PKGD to copy matches to. For plain
+        # ${CFLAGS}-derived entries these are the same string.
         prefixmap = {}
         for flag in cflags.split():
             if not flag.startswith("-ffile-prefix-map"):
@@ -986,7 +990,20 @@ def copydebugsources(debugsrcdir, sources, d):
             if "recipe-sysroot" in flag:
                 continue
             flag = flag.split("=")
-            prefixmap[flag[1]] = flag[2]
+            prefixmap[flag[1]] = (flag[2], flag[2])
+
+        # kernel-arch.bbclass's KERNEL_CC remaps STAGING_KERNEL_DIR and
+        # STAGING_KERNEL_BUILDDIR to KERNEL_SRC_PATH ("/usr/src/kernel"),
+        # overriding CFLAGS' own mapping for the kernel, so its source files
+        # need a dedicated prefixmap entry to be found, still destined for
+        # TARGET_DBGSRC_DIR like everything else.
+        if bb.data.inherits_class('kernel-arch', d):
+            kernel_src_path = d.getVar('KERNEL_SRC_PATH')
+            target_dbgsrc_dir = d.getVar('TARGET_DBGSRC_DIR')
+            for kernel_dir_var in ('STAGING_KERNEL_DIR', 'STAGING_KERNEL_BUILDDIR'):
+                kernel_dir = d.getVar(kernel_dir_var)
+                if kernel_dir and kernel_src_path and target_dbgsrc_dir:
+                    prefixmap[kernel_dir] = (kernel_src_path, target_dbgsrc_dir)
 
         nosuchdir = []
         basepath = dvar
@@ -1008,9 +1025,9 @@ def copydebugsources(debugsrcdir, sources, d):
                        and not path.endswith((b"<internal>", b"<built-in>"))
                        and b"recipe-sysroot" not in os.path.dirname(path)}
 
-        for pmap, prefix in prefixmap.items():
-            dstroot = dvar + prefix
-            prefix_slash = os.fsencode(prefix) + b"/"
+        for pmap, (match, dest) in prefixmap.items():
+            dstroot = dvar + dest
+            prefix_slash = os.fsencode(match) + b"/"
             relpaths = [path.removeprefix(prefix_slash) for path in sourcepaths
                         if path.startswith(prefix_slash)]
 
diff --git a/meta/lib/oeqa/selftest/cases/oelib/package.py b/meta/lib/oeqa/selftest/cases/oelib/package.py
index 16e13aece9..e78320e6d0 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/package.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/package.py
@@ -17,12 +17,13 @@ class FakeDataStore:
     def __init__(self, values):
         self.values = values
 
-    def getVar(self, name):
+    def getVar(self, name, expand=True):
         return self.values.get(name)
 
     def expand(self, value):
         for name, replacement in self.values.items():
-            value = value.replace("${%s}" % name, replacement)
+            if isinstance(replacement, str):
+                value = value.replace("${%s}" % name, replacement)
         return value
 
 
@@ -270,3 +271,72 @@ class TestCopyDebugSources(TestCase):
             with open(copied_source) as f:
                 self.assertEqual(f.read(), "real\n")
             self.assertFalse(os.path.exists(relocation))
+
+    def test_copydebugsources_recovers_kernel_source_files(self):
+        """Recovers kernel-arch source files recorded outside debugsrcdir.
+
+        kernel-arch recipes remap STAGING_KERNEL_DIR/BUILDDIR to
+        KERNEL_SRC_PATH (typically "/usr/src/kernel") via KERNEL_CC, bypassing
+        CFLAGS entirely, so debugsources.list records kernel files under
+        KERNEL_SRC_PATH rather than under debugsrcdir (typically
+        "/usr/src/debug/...") like every other CFLAGS-derived entry.
+
+        Simulates a kernel-arch recipe with one file coming from each of the
+        two remapped directories (a source file under STAGING_KERNEL_DIR and a
+        generated header under STAGING_KERNEL_BUILDDIR) and asserts
+        copydebugsources() recovers both into the normal debugsrcdir, proving
+        the dedicated kernel-arch prefixmap entries are used instead of (or in
+        addition to) the plain CFLAGS-derived ones.
+        """
+        with tempfile.TemporaryDirectory(prefix="oe-test-package-") as tmpdir:
+            kernel_src_dir = os.path.join(tmpdir, "kernel-source")
+            kernel_build_dir = os.path.join(tmpdir, "kernel-build-artifacts")
+            workdir = os.path.join(tmpdir, "work")
+            pkgd = os.path.join(tmpdir, "pkgd")
+            debugsrcdir = "/usr/src/debug/kernel/1.0"
+            kernel_src_path = "/usr/src/kernel"
+
+            src_rel = os.path.join("arch", "main.c")
+            build_rel = os.path.join("include", "generated", "autoconf.h")
+
+            os.makedirs(os.path.dirname(os.path.join(kernel_src_dir, src_rel)))
+            os.makedirs(os.path.dirname(os.path.join(kernel_build_dir, build_rel)))
+            os.makedirs(workdir)
+            os.makedirs(pkgd)
+
+            src_file = os.path.join(kernel_src_dir, src_rel)
+            build_file = os.path.join(kernel_build_dir, build_rel)
+            with open(src_file, "w") as f:
+                f.write("main\n")
+            with open(build_file, "w") as f:
+                f.write("autoconf\n")
+
+            sources = [
+                os.path.join(kernel_src_path, src_rel),
+                os.path.join(kernel_src_path, build_rel),
+            ]
+            d = FakeDataStore({
+                "WORKDIR": workdir,
+                "PKGD": pkgd,
+                "STRIP": "strip",
+                "OBJCOPY": "objcopy",
+                "S": kernel_src_dir,
+                # KERNEL_CC's own -ffile-prefix-map overrides this for the
+                # kernel, so CFLAGS carries no entry for kernel_src_path.
+                "CFLAGS": "",
+                "__inherit_cache": ["/layer/classes-recipe/kernel-arch.bbclass"],
+                "STAGING_KERNEL_DIR": kernel_src_dir,
+                "STAGING_KERNEL_BUILDDIR": kernel_build_dir,
+                "KERNEL_SRC_PATH": kernel_src_path,
+                "TARGET_DBGSRC_DIR": debugsrcdir,
+            })
+
+            copydebugsources(debugsrcdir, sources, d)
+
+            copied_src = oe.path.join(pkgd, debugsrcdir, src_rel)
+            copied_build = oe.path.join(pkgd, debugsrcdir, build_rel)
+
+            with open(copied_src) as f:
+                self.assertEqual(f.read(), "main\n")
+            with open(copied_build) as f:
+                self.assertEqual(f.read(), "autoconf\n")
-- 
2.55.0



^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation
  2026-09-23 20:51 [PATCH v2 0/3] Fix kernel complementary package installation AdrianF
                   ` (2 preceding siblings ...)
  2026-09-23 20:51 ` [PATCH v2 3/3] oe/package.py: copydebugsources: add all kernel source files AdrianF
@ 2026-10-29 23:09 ` Bruce Ashfield
  2026-10-30 10:40   ` Freihofer, Adrian
  3 siblings, 1 reply; 7+ messages in thread
From: Bruce Ashfield @ 2026-10-29 23:09 UTC (permalink / raw)
  To: adrian.freihofer; +Cc: openembedded-core, richard.purdie

On Wed, Sep 23, 2026 at 4:51 PM Adrian Freihofer wrote:
>
> Changes in comparison to v1: Completely different approach
> - Complementary package installation for kernel sources is disabled
> - The linux-yocto-src package can now be installed as a regular package which
>   was not possible before.
> - Additionally, the kernel source files are now correctly included in the
>   linux-yocto-src package.

Sorry for the delay on this, I wanted to actually build and measure
before replying, since this is in master-next.

First, to make sure I have the use case right: this is TRACE32 (or
cross-gdb to qemu, it's a similar pattern) running on the host, with
rootfs-dbg extracted on the host as the debug sysroot. Nothing here runs
gdb on the target. Correct ?

As I said on IRC, I don't want a kernel -src package. It's a second copy
of the kernel tree that you can't build against, and the generic -src
machinery is the wrong tool for the kernel. But I'm not saying no to the
goal. I looked at why linux-yocto-src exists at all, and I think there's
a simpler way to get you there.

Where the kernel's debug info points today (7.2.13, qemux86-64):

  recorded root                      files   what they are
  /usr/src/kernel/                    9016   real source
  /usr/src/debug/linux-yocto/<PV>/     592   generated in ${B}

kernel-arch's KERNEL_CC maps STAGING_KERNEL_DIR and
STAGING_KERNEL_BUILDDIR to /usr/src/kernel, but not ${B}, where the
kernel actually builds. ${B} falls through to the generic debug prefix
map, and those 592 generated files are the only content of
linux-yocto-src. Everything real is already recorded under
/usr/src/kernel.

Are the 592 needed for debugging ? I went through them (with
grep and some AI help):

  files                                     count  content                  regenerable from source?
  *.mod.c                                     557  modinfo boilerplate      no (needs module objects + modpost)
  .vmlinux.export.c                             1  export table             no (needs vmlinux.o)
  vdso32/64-image.c                             2  vdso byte arrays         no (needs the vdso build)
  *.asn1.[ch]                                  14  ASN.1 decoder tables     yes
  capflags.c, inat-tables.c, crc32table.h,
  raid6 tables.c, consolemap_deftbl.c,
  defkeymap.c, ucs_*_table.h,
  oid_registry_data.c, shipped-certs.c,
  gen-kdb_cmds.c, syscalls_{32,64}.h           14  data tables              yes
  raid6 int{1,2,4,8}.c                          4  code (awk-unrolled)      yes

Only the raid6 int*.c files are code that you'd ever step through. A
debugger missing a file just shows no source for that frame. What's
useful can be regenerated by kbuild from the source (as a prerequisite
of the .o), and what can't be regenerated isn't useful. So I don't want
to package any of them. Shipping build output that can be regenerated
can lead to QA issues with mixed arches.

So, what I'm prototyping instead (and have tested):

1) kernel.bbclass:

     KERNEL_CC:append = " -ffile-prefix-map=${B}=${KERNEL_SRC_PATH}"

   All 9608 debug source paths are now under /usr/src/kernel.
   copydebugsources() finds nothing for the -src package, so
   linux-yocto-src isn't created at all. There's no special case in
   package.py. A generated file regenerated in the tree lands exactly
   where the debug info expects it.

   I did an A/B of the kernel built with and without it. In the loaded
   vmlinux sections and the 557 modules, the only differing bytes are
   the build-ids (vmlinux, the embedded vdso note, the modules) and the
   .gnu_debuglink CRC. That is expected, since both hash the debug info,
   which is what changed. There are no code or data changes.

2) For the source itself, I have been carrying an optional linux-source
   package in kernel-devsrc for a long time. It's off by default
   (PACKAGECONFIG), a full source tree at /usr/src/linux-source-<kver>,
   and it doesn't overlap with devsrc (which stays the curated, buildable
   subset). With:

     IMAGE_INSTALL_DEBUGFS = "kernel-dbg kernel-vmlinux linux-source"

   rootfs-dbg has the vmlinux and module .debug files plus the source.
   The debugger maps /usr/src/kernel to /usr/src/linux-source (TRACE32
   has source path translation for this).

   I'm not completely sold on sending this one yet, but I could be
   convinced to go that way rather than a -src package. If it covers
   your use case, that's the argument that would do it.

On the series:

 - 1/3: I'd drop it. With (1) there's no kernel -src package to make
   installable. Assuming I haven't screwed up the logic or misunderstood
   something.

 - 2/3: fine with me. Keeping the kernel out of the complementary
   install is what I asked for in v1, and with (1) it is a no-op.

 - 3/3: I'd drop it. It selects files by their recorded /usr/src/kernel
   path, but installs them under /usr/src/debug/linux-yocto/<PV>, so the
   debugger still needs a path substitution. That's the same cost as
   mapping to a source package. It also duplicates the whole tree (your
   8901 files) into a package that can't be built against. And it
   applies to every kernel-arch recipe (module-base, so out-of-tree
   modules, perf and make-mod-scripts), not just the kernel, so they all
   start carrying kernel sources in their -src packages.

   The linux-source package in (2) is what I'd propose in its place.
   It covers the same need (the full kernel source in rootfs-dbg), but
   it's opt-in, it's a single copy for the kernel only, it carries no
   generated files, and it isn't a -src package at all.

Richard: since 1/3 and 3/3 are in master-next, I'd ask that they be held
while we sort this out.

Two oe-core issues I hit along the way, which I'll report separately:

 - IMAGE_INSTALL_DEBUGFS installs with attempt_only, so a package that
   can't be installed is silently skipped. I hit this with a file
   dependency on a usrmerge distro.

 - An on-target postinst from a debugfs-only package leaks into the main
   rootfs postinst handling and fails do_rootfs.

If this direction works for you, I can send (1) and the linux-source
package as an RFC so you can try it with TRACE32.

Bruce

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation
  2026-10-29 23:09 ` [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation Bruce Ashfield
@ 2026-10-30 10:40   ` Freihofer, Adrian
  2026-10-30 15:17     ` Bruce Ashfield
  0 siblings, 1 reply; 7+ messages in thread
From: Freihofer, Adrian @ 2026-10-30 10:40 UTC (permalink / raw)
  To: Bruce Ashfield
  Cc: openembedded-core@lists.openembedded.org,
	richard.purdie@linuxfoundation.org, Marko, Peter

Hi Bruce

Thank you very much for the detailed response.

On Thu, 2026-10-29 at 16:09 -0700, Bruce Ashfield wrote:
> On Wed, Sep 23, 2026 at 4:51 PM Adrian Freihofer wrote:
> > 
> > Changes in comparison to v1: Completely different approach
> > - Complementary package installation for kernel sources is disabled
> > - The linux-yocto-src package can now be installed as a regular
> > package which
> >   was not possible before.
> > - Additionally, the kernel source files are now correctly included
> > in the
> >   linux-yocto-src package.
> 
> Sorry for the delay on this, I wanted to actually build and measure
> before replying, since this is in master-next.
> 
> First, to make sure I have the use case right: this is TRACE32 (or
> cross-gdb to qemu, it's a similar pattern) running on the host, with
> rootfs-dbg extracted on the host as the debug sysroot. Nothing here
> runs
> gdb on the target. Correct ?

Yes, that's the use case which triggered this discussion.

But I can think about more use cases which could benefit from a
complete -src package (or different package name) for the kernel as
there are also more use cases for any other -src package.

> 
> As I said on IRC, I don't want a kernel -src package. It's a second
> copy
> of the kernel tree that you can't build against, and the generic -src
> machinery is the wrong tool for the kernel. But I'm not saying no to
> the
> goal. I looked at why linux-yocto-src exists at all, and I think
> there's
> a simpler way to get you there.
> 
> Where the kernel's debug info points today (7.2.13, qemux86-64):
> 
>   recorded root                      files   what they are
>   /usr/src/kernel/                    9016   real source
>   /usr/src/debug/linux-yocto/<PV>/     592   generated in ${B}
> 
> kernel-arch's KERNEL_CC maps STAGING_KERNEL_DIR and
> STAGING_KERNEL_BUILDDIR to /usr/src/kernel, but not ${B}, where the
> kernel actually builds. ${B} falls through to the generic debug
> prefix
> map, and those 592 generated files are the only content of
> linux-yocto-src. Everything real is already recorded under
> /usr/src/kernel.

Yes, I can confirm that.

> 
> Are the 592 needed for debugging ? I went through them (with
> grep and some AI help):
> 
>   files                                     count 
> content                  regenerable from source?
>   *.mod.c                                     557  modinfo
> boilerplate      no (needs module objects + modpost)
>   .vmlinux.export.c                             1  export
> table             no (needs vmlinux.o)
>   vdso32/64-image.c                             2  vdso byte
> arrays         no (needs the vdso build)
>   *.asn1.[ch]                                  14  ASN.1 decoder
> tables     yes
>   capflags.c, inat-tables.c, crc32table.h,
>   raid6 tables.c, consolemap_deftbl.c,
>   defkeymap.c, ucs_*_table.h,
>   oid_registry_data.c, shipped-certs.c,
>   gen-kdb_cmds.c, syscalls_{32,64}.h           14  data
> tables              yes
>   raid6 int{1,2,4,8}.c                          4  code (awk-
> unrolled)      yes
> 
> Only the raid6 int*.c files are code that you'd ever step through. A
> debugger missing a file just shows no source for that frame. What's
> useful can be regenerated by kbuild from the source (as a
> prerequisite
> of the .o), and what can't be regenerated isn't useful. So I don't
> want
> to package any of them. Shipping build output that can be regenerated
> can lead to QA issues with mixed arches.
> 
> So, what I'm prototyping instead (and have tested):
> 
> 1) kernel.bbclass:
> 
>      KERNEL_CC:append = " -ffile-prefix-map=${B}=${KERNEL_SRC_PATH}"
> 
>    All 9608 debug source paths are now under /usr/src/kernel.
>    copydebugsources() finds nothing for the -src package, so
>    linux-yocto-src isn't created at all. There's no special case in
>    package.py. A generated file regenerated in the tree lands exactly
>    where the debug info expects it.
> 
>    I did an A/B of the kernel built with and without it. In the
> loaded
>    vmlinux sections and the 557 modules, the only differing bytes are
>    the build-ids (vmlinux, the embedded vdso note, the modules) and
> the
>    .gnu_debuglink CRC. That is expected, since both hash the debug
> info,
>    which is what changed. There are no code or data changes.

Sounds very good.

> 
> 2) For the source itself, I have been carrying an optional linux-
> source
>    package in kernel-devsrc for a long time. It's off by default
>    (PACKAGECONFIG), a full source tree at /usr/src/linux-source-
> <kver>,
>    and it doesn't overlap with devsrc (which stays the curated,
> buildable
>    subset). With:
> 
>      IMAGE_INSTALL_DEBUGFS = "kernel-dbg kernel-vmlinux linux-source"
> 
>    rootfs-dbg has the vmlinux and module .debug files plus the
> source.
>    The debugger maps /usr/src/kernel to /usr/src/linux-source
> (TRACE32
>    has source path translation for this).
> 
>    I'm not completely sold on sending this one yet, but I could be
>    convinced to go that way rather than a -src package. If it covers
>    your use case, that's the argument that would do it.

I think this sounds very good and it would address at least the remote
debugging case and make it much more consistent than it is now.


> 
> On the series:
> 
>  - 1/3: I'd drop it. With (1) there's no kernel -src package to make
>    installable. Assuming I haven't screwed up the logic or
> misunderstood
>    something.
> 
>  - 2/3: fine with me. Keeping the kernel out of the complementary
>    install is what I asked for in v1, and with (1) it is a no-op.

Should we then not just drop this patch as well? It's ugly to have this
kernel specific case handled outside of the kernel recipe.

What the kernel recipe could provide instead:
A: an empty -src package (just to behave like all other packages).
B: an empty meta package which RDEPENDS on all the real kernel source
packages. The kernel would no longer be something special. The sources
are also included in the rootfs-dbg by default as for all user space
packages. If the user prefers to have the kernel excluded from rootfs-
dbg he can do what the patch does.

I think B would be the most consistent approach and therefore also the
most self-explanatory implementation.

Thank you very much!
Regards,
Adrian

> 
>  - 3/3: I'd drop it. It selects files by their recorded
> /usr/src/kernel
>    path, but installs them under /usr/src/debug/linux-yocto/<PV>, so
> the
>    debugger still needs a path substitution. That's the same cost as
>    mapping to a source package. It also duplicates the whole tree
> (your
>    8901 files) into a package that can't be built against. And it
>    applies to every kernel-arch recipe (module-base, so out-of-tree
>    modules, perf and make-mod-scripts), not just the kernel, so they
> all
>    start carrying kernel sources in their -src packages.
> 
>    The linux-source package in (2) is what I'd propose in its place.
>    It covers the same need (the full kernel source in rootfs-dbg),
> but
>    it's opt-in, it's a single copy for the kernel only, it carries no
>    generated files, and it isn't a -src package at all.
> 
> Richard: since 1/3 and 3/3 are in master-next, I'd ask that they be
> held
> while we sort this out.
> 
> Two oe-core issues I hit along the way, which I'll report separately:
> 
>  - IMAGE_INSTALL_DEBUGFS installs with attempt_only, so a package
> that
>    can't be installed is silently skipped. I hit this with a file
>    dependency on a usrmerge distro.
> 
>  - An on-target postinst from a debugfs-only package leaks into the
> main
>    rootfs postinst handling and fails do_rootfs.
> 
> If this direction works for you, I can send (1) and the linux-source
> package as an RFC so you can try it with TRACE32.
> 
> Bruce

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation
  2026-10-30 10:40   ` Freihofer, Adrian
@ 2026-10-30 15:17     ` Bruce Ashfield
  0 siblings, 0 replies; 7+ messages in thread
From: Bruce Ashfield @ 2026-10-30 15:17 UTC (permalink / raw)
  To: adrian.freihofer; +Cc: openembedded-core, richard.purdie, Peter.Marko

On Thu, Oct 30, 2026 at 6:40 AM Freihofer, Adrian wrote:
>
> > First, to make sure I have the use case right: this is TRACE32 (or
> > cross-gdb to qemu, it's a similar pattern) running on the host, with
> > rootfs-dbg extracted on the host as the debug sysroot. Nothing here
> > runs gdb on the target. Correct ?
>
> Yes, that's the use case which triggered this discussion.
>
> But I can think about more use cases which could benefit from a
> complete -src package (or different package name) for the kernel as
> there are also more use cases for any other -src package.

Thanks for confirming. I'd like to hear what those other use cases are,
since they're what would drive a full source package (more on that below).

For the host-side case, though, I don't think we need a source package
at all. The debugger runs on the host, so rootfs-dbg only has to supply
the symbols (kernel-dbg, kernel-vmlinux). The source can come from any
tree on the host that matches the build: the build's
work-shared/<machine>/kernel-source, or a linux-yocto checkout at the
recipe's SRCREV. With the prefix map change, every path in the kernel's
debug info is under /usr/src/kernel, so it's a single mapping in the
debugger to that tree.

> >    I'm not completely sold on sending this one yet, but I could be
> >    convinced to go that way rather than a -src package. If it covers
> >    your use case, that's the argument that would do it.
>
> I think this sounds very good and it would address at least the remote
> debugging case and make it much more consistent than it is now.

Having thought about it more, I'm going to keep linux-source local for
now. The problem is that a full source tree plus the ${B} artifacts is a
superset of kernel-devsrc. Everything devsrc does can be done with it,
just with a bigger footprint. I don't want two copies of the kernel source
to maintain, one curated and one not, and that's what linux-source is.

If a use case arrives that needs the source inside a rootfs (and not just
on the host), I'd want it done inside kernel-devsrc: one tree at
/usr/src/kernel, curated by default as it is today, with an opt-in to
widen it to the full source. No second copy, and no path mapping at all,
since the debug info already points there. The source would need to be
its own package without devsrc's toolchain RDEPENDS, so rootfs-dbg can
take it without pulling in a compiler. And it stays opt-in: nobody should
get ~1.6GB of source by accident.

It also doesn't have to be the whole tree. devsrc is already machine
specific, so the likely way I'd go is a larger devsrc: a curated spine
(core plus the machine's arch), with the drivers it needs discovered
from what the build actually compiled. On qemux86-64 that's about 330MB,
versus 1.5GB for the full tree.

> >  - 2/3: fine with me. Keeping the kernel out of the complementary
> >    install is what I asked for in v1, and with (1) it is a no-op.
>
> Should we then not just drop this patch as well? It's ugly to have this
> kernel specific case handled outside of the kernel recipe.

Agreed, drop it. With the prefix map, linux-yocto-src isn't created at
all, so there's nothing for core-image to exclude. That makes it all three.

The way I see it now, the 592 generated files in linux-yocto-src are the
actual bug today. They're there only because ${B} isn't mapped, and
they're the only reason the kernel gets a -src package. Complementary
install then pulls that package into rootfs-dbg without kernel-dbg, which
is the inconsistency you started from. Fixing the map removes the package,
and so the inconsistency, without any kernel special case in the image
classes or package.py.

> What the kernel recipe could provide instead:
> A: an empty -src package (just to behave like all other packages).
> B: an empty meta package which RDEPENDS on all the real kernel source
> packages. The kernel would no longer be something special. The sources
> are also included in the rootfs-dbg by default as for all user space
> packages. If the user prefers to have the kernel excluded from rootfs-
> dbg he can do what the patch does.
>
> I think B would be the most consistent approach and therefore also the
> most self-explanatory implementation.

I don't think we should do either.

A is an empty package that exists only so the kernel matches a pattern.
It doesn't give anyone anything.

For B, the real source package it would depend on doesn't exist (see
above). Even if it did, B puts the kernel back on the generic
complementary path, which is what I pushed back on in v1:

 - Complementary install maps every installed kernel-module-* package to
   linux-yocto-src, so almost every debug image would get the full kernel
   source by default, about 1.6GB unpacked.

 - It ties linux-yocto to whichever recipe ends up providing the source.

 - kernel-dbg still isn't pulled in by complementary install
   (KERNEL_PACKAGE_NAME vs PN), so you'd still get source without symbols.

The kernel is special here, in that it isn't usually a rootfs package.
I'd rather that be explicit (IMAGE_INSTALL_DEBUGFS for the symbols, and
a mapping to the source on the host) than hidden behind a meta-package.

So my plan is:

 - drop 1/3, 2/3 and 3/3
 - I'll send the kernel.bbclass prefix map change on its own, once I've
   checked SPDX's compiled-sources filter is unaffected (it's the only
   other consumer of the kernel's debugsources list)
 - revisit a full source package, inside kernel-devsrc, when there's a
   concrete use case that needs it in a rootfs

Thougts ?

Bruce

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-10-30 15:17 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-23 20:51 [PATCH v2 0/3] Fix kernel complementary package installation AdrianF
2026-09-23 20:51 ` [PATCH v2 1/3] kernel.bbclass: make the -src package installable on purpose AdrianF
2026-09-23 20:51 ` [PATCH v2 2/3] core-image.bbclass: do not pull the kernel -src package in via complementary packages AdrianF
2026-09-23 20:51 ` [PATCH v2 3/3] oe/package.py: copydebugsources: add all kernel source files AdrianF
2026-10-29 23:09 ` [OE-core] [PATCH v2 0/3] Fix kernel complementary package installation Bruce Ashfield
2026-10-30 10:40   ` Freihofer, Adrian
2026-10-30 15:17     ` Bruce Ashfield

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox