[PATCH] package: disable renamed dependency error if allarch is overridden

[PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

allarch code is only triggered if PACKAGE_ARCH remains set to "all".
This is not the case under some circumstances, like in the case of
nativesdk class which overrides this variable. This error then breaks
nativesdk package groups which depend on renamed packages yet aren't
affected by allarch.

Signed-off-by: Oleksandr Hnatiuk <ohnatiuk@cisco.com>
---
 meta/lib/oe/packagedata.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/packagedata.py b/meta/lib/oe/packagedata.py
index 2d1d6ddeb75e..2bc57b8c343c 100644
--- a/meta/lib/oe/packagedata.py
+++ b/meta/lib/oe/packagedata.py
@@ -138,7 +138,7 @@ def get_package_mapping(pkg, basepkg, d, depversions=None):
     key = "PKG:%s" % pkg
 
     if key in data:
-        if bb.data.inherits_class('allarch', d) and bb.data.inherits_class('packagegroup', d) and pkg != data[key]:
+        if d.getVar("PACKAGE_ARCH") == "all" and bb.data.inherits_class('packagegroup', d) and pkg != data[key]:
             bb.error("An allarch packagegroup shouldn't depend on packages which are dynamically renamed (%s to %s)" % (pkg, data[key]))
         # Have to avoid undoing the write_extra_pkgs(global_variants...)
         if bb.data.inherits_class('allarch', d) and not d.getVar('MULTILIB_VARIANTS') \
-- 
2.35.6

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Fri, 9 May 2025 at 23:19, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> allarch code is only triggered if PACKAGE_ARCH remains set to "all".
> This is not the case under some circumstances, like in the case of
> nativesdk class which overrides this variable. This error then breaks
> nativesdk package groups which depend on renamed packages yet aren't
> affected by allarch.

Can you show steps to observe the issue please? Why isn't it seen in
the test matrix?

Alex

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

There was a report of this on yocto mailing list with detailed info 2 years ago, but nobody responded to it.
See: https://lists.yoctoproject.org/g/yocto/topic/nativesdk_packagegroup_sdk_host/98365178

[-- Attachment #2: Type: text/html, Size: 374 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Mon, 12 May 2025 at 11:31, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> There was a report of this on yocto mailing list with detailed info 2 years ago, but nobody responded to it.
> See: https://lists.yoctoproject.org/g/yocto/topic/nativesdk_packagegroup_sdk_host/98365178

Adding library-only packages to packagegroups is simply not currently
supported. What is the use case for it?

Alex

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

I will have to check why such packages are included.

However, consider the following. Here are all the parts of this issue:
* a packagegroup explicitly inherits packagegroup and nativesdk classes
* same packagegroup depends on a package which is subject to dynamic package renaming
* packagegroup.bbclass unconditionally (because of immediate expansion) inherits allarch.bbclass
* nativesdk.bbclass overrides PACKAGE_ARCH with correct value, preventing code in allarch.bbclass from executing (because allarch.bbclass tests this variable before doing anything)
* bitbake raises an allarch-specific error by testing whether allarch was ever inherited instead of whether any of its code was executed (see 4)

Regardless of whether 2 is justified, the behavior in 3 and/or 5 is seems incorrect.
allarch.bbclass is unique in that we cannot test if it is really used without checking the value of PACKAGE_ARCH.
If we simply check via bb.data.inherits_class, we get false positives (again, see step 4).
Thus, it is incorrect to issue an error for packagegroups that inherit allarch without also verifying that they are actually allarch via PACKAGE_ARCH.
Without this extra check, the error is produced for packagegroups for which code in allarch.bbclass was never and will never be executed.

[-- Attachment #2: Type: text/html, Size: 1433 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Tue, 13 May 2025 at 12:30, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> a packagegroup explicitly inherits packagegroup and nativesdk classes

Why? I have to ask what is the use case for inheriting nativesdk in a
packagegroup then :)

Alex

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]

You have the same approach in openembedded-core layer.
See meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb

> 
> SUMMARY = "Host packages for the standalone SDK or external toolchain"
> inherit packagegroup
> inherit_defer nativesdk
>

[-- Attachment #2: Type: text/html, Size: 339 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Wed, 14 May 2025 at 17:35, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> You have the same approach in openembedded-core layer.
> See meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
>
> SUMMARY = "Host packages for the standalone SDK or external toolchain"
> inherit packagegroup
> inherit_defer nativesdk

That does not answer the question, and oe-core doing it doesn't mean
it's correct or necessary. Oe-core makes mistakes or forgets to clean
up obsolete code too :)

In this case, this packagegroup is bundled together with other
packagegroups in the SDK code, and those other packagegroups don't
inherit nativesdk:

TOOLCHAIN_HOST_TASK ?= " \
    nativesdk-packagegroup-sdk-host \
    packagegroup-cross-canadian-${MACHINE} \
    ${@bb.utils.contains('SDK_TOOLCHAIN_LANGS', 'go',
'packagegroup-go-cross-canadian-${MACHINE}', '', d)} \
    ${@bb.utils.contains('SDK_TOOLCHAIN_LANGS', 'rust',
'packagegroup-rust-cross-canadian-${MACHINE}', '', d)} \
"
Alex

Alex

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

How would building packages for SDK work without inheriting nativesdk?

All other packagegroups in this variable use $MACHINE to parametrize themselves, so they are not plain packagegroups either.
Furthermore, they inherit cross-canadian which sets PACKAGE_ARCH in the same way nativesdk.bbclass does.

I cannot know every single detail and history behind why oe-core works the way that it does.
What I do know is that I cannot add certain packages to SDK packagegroup because an unrelated error check is producing a false positive because the possibility of an allarch override is not being accounted for.

[-- Attachment #2: Type: text/html, Size: 703 bytes --]

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

Also, consider this

* packagegroup.bbclass inherits allarch.bbclass - except when PACKAGE_ARCH is set before inheriting packagegroup
* in order to do so, a nativesdk packagegroup must inherit nativesdk first
* there is a native-last QA check which strongly suggests inheriting nativesdk last
* nativesdk is inherited last via inherit_defer in nativesdk-packagegroup-sdk-host

I see all of this as either very misleading at best or a mistake at worst.

[-- Attachment #2: Type: text/html, Size: 519 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Mon, 19 May 2025 at 12:24, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
>
> Also, consider this
>
> packagegroup.bbclass inherits allarch.bbclass - except when PACKAGE_ARCH is set before inheriting packagegroup
> in order to do so, a nativesdk packagegroup must inherit nativesdk first
> there is a native-last QA check which strongly suggests inheriting nativesdk last
> nativesdk is inherited last via inherit_defer in nativesdk-packagegroup-sdk-host
>
> I see all of this as either very misleading at best or a mistake at worst.

This is probably true. It does look like a pile of ad hoc fixups
rather than an overall design and structure. Maybe it's possible to
clean and simplify?

 I'd say if you resubmit the patch with all the extra research you did
in the commit message, that would help review. I don't have objections
anymore.

Alex

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Richard Purdie]

On Tue, 2025-05-20 at 15:06 +0200, Alexander Kanavin via lists.openembedded.org wrote:
> On Mon, 19 May 2025 at 12:24, Oleksandr Hnatiuk via
> lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
> wrote:
> > 
> > Also, consider this
> > 
> > packagegroup.bbclass inherits allarch.bbclass - except when PACKAGE_ARCH is set before inheriting packagegroup
> > in order to do so, a nativesdk packagegroup must inherit nativesdk first
> > there is a native-last QA check which strongly suggests inheriting nativesdk last
> > nativesdk is inherited last via inherit_defer in nativesdk-packagegroup-sdk-host
> > 
> > I see all of this as either very misleading at best or a mistake at worst.
> 
> This is probably true. It does look like a pile of ad hoc fixups
> rather than an overall design and structure. Maybe it's possible to
> clean and simplify?
> 
>  I'd say if you resubmit the patch with all the extra research you did
> in the commit message, that would help review. I don't have objections
> anymore.

There is a lot of history here. I'm tempted to suggest we remove the
allarch inherit from packagegroup and add it to the recipes explicitly
as it is all getting far too convoluted and I'd like to see things
simplified, not complicated.

I'd like to see the conditional code in allarch removed too FWIW if we
can work out how to do it.

Cheers,

Richard

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

Hello Alex,

Could you please clarify - which additional information is required in the commit message so that I can resubmit it?
Alternatively, it is OK by me if you amend the commit how you see fit and pass it along.
How should we proceed?

[-- Attachment #2: Type: text/html, Size: 308 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Thu, 10 Jul 2025 at 17:02, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> Could you please clarify - which additional information is required in the commit message so that I can resubmit it?
> Alternatively, it is OK by me if you amend the commit how you see fit and pass it along.
> How should we proceed?

I guess you should first try to do what RP suggests rather than
resubmit the original patch?

Alex

Re: [PATCH] package: disable renamed dependency error if allarch is overridden

[Oleksandr Hnatiuk]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

Alex,

Do you mean removing the conditional or removing allarch inherit?
I don't think I can find the time to work on either of those as it would involve additional research and testing to make sure nothing breaks in the process.
While I would like to help, doing major changes or large refactoring to address an issue already solved is likely outside the scope of my job responsibilities.
As downstream users of the code, we were hoping the patch could be merged, as we would then not have to maintain it on our side.

I suggest to merge it as-is and then have somebody else work on proposed changes.
Otherwise, I would like to open a bug report so that when the issue is eventually resolved, we could drop the patch.

[-- Attachment #2: Type: text/html, Size: 839 bytes --]

Re: [OE-core] [PATCH] package: disable renamed dependency error if allarch is overridden

[Alexander Kanavin]

On Fri, 25 Jul 2025 at 21:24, Oleksandr Hnatiuk via
lists.openembedded.org <ohnatiuk=cisco.com@lists.openembedded.org>
wrote:
> Do you mean removing the conditional or removing allarch inherit?
> I don't think I can find the time to work on either of those as it would involve additional research and testing to make sure nothing breaks in the process.
> While I would like to help, doing major changes or large refactoring to address an issue already solved is likely outside the scope of my job responsibilities.
> As downstream users of the code, we were hoping the patch could be merged, as we would then not have to maintain it on our side.
>
> I suggest to merge it as-is and then have somebody else work on proposed changes.
> Otherwise, I would like to open a bug report so that when the issue is eventually resolved, we could drop the patch.

Filing a bug report is fine. But do not expect that the issue is
somehow 'eventually resolved' just because a bug for it exists. This
is not how open source works.

'Merging as-is', on the other hand, is not fine. We have learned from
hard experience that merging 'quick fixes' means all incentive to
solve the broader problem properly is lost, and nobody would find time
or interest to look at it again. Or can you name that 'somebody', and
did they agree to do the work?

I'd politely suggest you talk to your managers, and explain to them
that using open source is not 'free' and sometimes you do need time
for making major changes in it that don't directly benefit your
company's products. Even better, convince them to let you maintain
something in yocto. I'd be happy to supply a long list of
possibilities.

Alex