[PATCH] classes-global/license: create -lic package from PKGD, not D

[PATCH] classes-global/license: create -lic package from PKGD, not D

[roman.nazarenko]

From: Roman Nazarenko <roman.nazarenko@leica-geosystems.com>

do_package and do_populate_sysroot are siblings - both are only ordered
"after do_install" - so they run concurrently. With
LICENSE_CREATE_PACKAGE=1, perform_packagecopy:prepend wrote the
LIC_FILES_CHKSUM texts straight into ${D}${datadir}/licenses/${PN}/,
mutating the very tree do_populate_sysroot stages with find|cpio:

  | DEBUG: Executing shell function sysroot_stage_all
  | cpio: ./licenses/libmodule/LICENSE: Cannot stat: No such file or directory
  | WARNING: exit code 2 from a shell command.

copy_license_files() removes and re-links an already existing
destination, so a file can be listed by find and gone by the time cpio
stats it.

Write into ${PKGD} from a perform_packagecopy:append instead. By then
${D} is complete and untouched, and ${PKGD} is a private copy that
do_populate_sysroot never reads. Package contents, on-target paths and
ownership are unchanged - do_package is a fakeroot task, so the
os.chown() in copy_license_files() still applies. As a side effect the
license texts stop being staged into every target recipe's sysroot,
which they had no business being in.

This changes perform_packagecopy, so do_package basehashes change for
every recipe; expect a one-time repackage.

Signed-off-by: Roman Nazarenko <roman.nazarenko@leica-geosystems.com>
---
 meta/classes-global/license.bbclass | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/meta/classes-global/license.bbclass b/meta/classes-global/license.bbclass
index 86c9ee5878..1d266dd25d 100644
--- a/meta/classes-global/license.bbclass
+++ b/meta/classes-global/license.bbclass
@@ -38,14 +38,15 @@ python do_populate_lic() {
     oe.qa.exit_if_errors(d)
 }
 
-# it would be better to copy them in do_install:append, but find_license_files is python
-python perform_packagecopy:prepend () {
+# Write into PKGD, not D: do_populate_sysroot stages D concurrently and races
+# with the remove-then-relink in copy_license_files().
+python perform_packagecopy:append () {
     enabled = oe.data.typed_value('LICENSE_CREATE_PACKAGE', d)
     if d.getVar('CLASSOVERRIDE') == 'class-target' and enabled:
         lic_files_paths = find_license_files(d)
 
-        # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join D and LICENSE_FILES_DIRECTORY
-        destdir = d.getVar('D') + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY'), d.getVar('PN'))
+        # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join PKGD and LICENSE_FILES_DIRECTORY
+        destdir = d.getVar('PKGD') + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY'), d.getVar('PN'))
         copy_license_files(lic_files_paths, destdir)
         add_package_and_files(d)
 }
-- 
2.43.0

Re: [OE-core] [PATCH] classes-global/license: create -lic package from PKGD, not D

[Richard Purdie]

On Thu, 2026-08-27 at 17:52 +0200, roman.nazarenko via lists.openembedded.org wrote:
> From: Roman Nazarenko <roman.nazarenko@leica-geosystems.com>
> 
> do_package and do_populate_sysroot are siblings - both are only ordered
> "after do_install" - so they run concurrently. With
> LICENSE_CREATE_PACKAGE=1, perform_packagecopy:prepend wrote the
> LIC_FILES_CHKSUM texts straight into ${D}${datadir}/licenses/${PN}/,
> mutating the very tree do_populate_sysroot stages with find|cpio:
> 
>   | DEBUG: Executing shell function sysroot_stage_all
>   | cpio: ./licenses/libmodule/LICENSE: Cannot stat: No such file or directory
>   | WARNING: exit code 2 from a shell command.
> 
> copy_license_files() removes and re-links an already existing
> destination, so a file can be listed by find and gone by the time cpio
> stats it.
> 
> Write into ${PKGD} from a perform_packagecopy:append instead. By then
> ${D} is complete and untouched, and ${PKGD} is a private copy that
> do_populate_sysroot never reads. Package contents, on-target paths and
> ownership are unchanged - do_package is a fakeroot task, so the
> os.chown() in copy_license_files() still applies. As a side effect the
> license texts stop being staged into every target recipe's sysroot,
> which they had no business being in.
> 
> This changes perform_packagecopy, so do_package basehashes change for
> every recipe; expect a one-time repackage.
> 
> Signed-off-by: Roman Nazarenko <roman.nazarenko@leica-geosystems.com>
> ---
>  meta/classes-global/license.bbclass | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes-global/license.bbclass b/meta/classes-global/license.bbclass
> index 86c9ee5878..1d266dd25d 100644
> --- a/meta/classes-global/license.bbclass
> +++ b/meta/classes-global/license.bbclass
> @@ -38,14 +38,15 @@ python do_populate_lic() {
>      oe.qa.exit_if_errors(d)
>  }
>  
> -# it would be better to copy them in do_install:append, but find_license_files is python
> -python perform_packagecopy:prepend () {
> +# Write into PKGD, not D: do_populate_sysroot stages D concurrently and races
> +# with the remove-then-relink in copy_license_files().
> +python perform_packagecopy:append () {
>      enabled = oe.data.typed_value('LICENSE_CREATE_PACKAGE', d)
>      if d.getVar('CLASSOVERRIDE') == 'class-target' and enabled:
>          lic_files_paths = find_license_files(d)
>  
> -        # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join D and LICENSE_FILES_DIRECTORY
> -        destdir = d.getVar('D') + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY'), d.getVar('PN'))
> +        # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join PKGD and LICENSE_FILES_DIRECTORY
> +        destdir = d.getVar('PKGD') + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY'), d.getVar('PN'))
>          copy_license_files(lic_files_paths, destdir)
>          add_package_and_files(d)
>  }

The comment you removed does still apply. Could you instead try making
this a do_install postfunc? I suspect those didn't exist when this was
written.

Cheers,

Richard

Re: FILLME-exact-v1-subject

[roman.nazarenko]

From: Roman Nazarenko <roman.nazarenko@leica-geosystems.com>

I tried the do_install postfunc and it breaks packagegroups.

packagegroup.bbclass does "deltask do_install" but keeps do_package, and
a deleted task never runs its postfuncs. So with
LICENSE_CREATE_PACKAGE=1 add_package_and_files() still creates ${PN}-lic
in perform_packagecopy, nothing fills it, and ALLOW_EMPTY is set for
packagegroup packages - a silently empty package. Those -lic packages do
have content today, as find_license_files() returns generic_<LICENSE>
even with an empty LIC_FILES_CHKSUM. Packagegroups also deltask
do_populate_sysroot, so they were never racy to begin with.

It is also wider than needed: do_install[postfuncs] has to be added
unconditionally, so do_install's basehash changes for every recipe and
sstate is invalidated from there downwards.

${PKGD} avoids both - it is private to do_package, so the race is gone
by construction, and it works for recipes without a do_install. It does
stop staging license texts into recipe-sysroot, but that looks like the
fix rather than a side effect, given the failure came out of
sysroot_stage_all.

Ok to keep ${PKGD}, or is there something about ${D} I am missing?

Re: FILLME-exact-v1-subject

[Richard Purdie]

On Tue, 2026-09-01 at 13:58 +0200, roman.nazarenko@leica-geosystems.com wrote:
> I tried the do_install postfunc and it breaks packagegroups.
> 
> packagegroup.bbclass does "deltask do_install" but keeps do_package, and
> a deleted task never runs its postfuncs. So with
> LICENSE_CREATE_PACKAGE=1 add_package_and_files() still creates ${PN}-lic
> in perform_packagecopy, nothing fills it, and ALLOW_EMPTY is set for
> packagegroup packages - a silently empty package. Those -lic packages do
> have content today, as find_license_files() returns generic_<LICENSE>
> even with an empty LIC_FILES_CHKSUM. Packagegroups also deltask
> do_populate_sysroot, so they were never racy to begin with.

I suspect we should fix packagegroups differently, either disabling the
license packages, allowing them to be empty in that case or allowing
the postfunc to run. Might just be easiest to disable there.

> It is also wider than needed: do_install[postfuncs] has to be added
> unconditionally, so do_install's basehash changes for every recipe and
> sstate is invalidated from there downwards.

It would be a one off basehash change so I don't think this is a good
reason to base an architectural decision off.

> ${PKGD} avoids both - it is private to do_package, so the race is gone
> by construction, and it works for recipes without a do_install. It does
> stop staging license texts into recipe-sysroot, but that looks like the
> fix rather than a side effect, given the failure came out of
> sysroot_stage_all.
> 
> Ok to keep ${PKGD}, or is there something about ${D} I am missing?

The intent has always been to have the packaged files and the sysroots
as similar as we can keep them, just to reduce complexity and potential
bugs. This is why I'm leaning to make this more in line with that
rather than making it worse...

Think about this from the other perspective of debugging where files
are coming from. They *supposed* to come from do_install and be in ${D}
. Even where there are differences between sysroots and ${D}, files are
pretty much always in ${D}. The current behaviour will therefore take
people by surprise. We have enough usability issues without creating
more differences without a good reason.

Cheers,

Richard