Openembedded Bitbake Development
 help / color / mirror / Atom feed
* [PATCH 0/1] data_smart: fix key expansion in conditional overrides
@ 2026-07-25 14:30 Nguyen Minh Tien
  2026-07-25 14:30 ` [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable Nguyen Minh Tien
  0 siblings, 1 reply; 6+ messages in thread
From: Nguyen Minh Tien @ 2026-07-25 14:30 UTC (permalink / raw)
  To: bitbake-devel; +Cc: zizuzacker

An :append whose override name still contains a variable is silently
dropped. From the report:

    RDEPENDS:${PN}:append:pn-packagegroup-cross-canadian-${MACHINE} = " ..."

Still reproduces on master (c251833d2). The mechanism is in the patch;
briefly, renameVar() rebuilds dependent override keys with a string
replace, so renaming RDEPENDS:${PN} leaves the ${MACHINE} behind and the
append lands on a key no override can ever match.

The part worth flagging is the '${' not in newkey guard. Expanding
unconditionally looks like the obvious fix but regresses parsing:
native.bbclass renames to a key that is itself still unexpanded, before
expandKeys() runs, and expanding the dependent keys there turns the later
rename into a no-op. oe-core then parses with three warnings master does
not have, in dbus and python3-psutil. The second of the two added tests
covers this and fails without the guard.

Tested against bitbake c251833d2 and oe-core 9d89b3b802:

 - bitbake-selftest: 790 tests, no failures. Six bb.tests.fetch errors
   remain, host issues that occur identically on master
 - all 952 oe-core recipes parse with no warnings, same as master
 - bitbake -e for dbus-native and python3-psutil-native, the two recipes
   that hit this path, identical to master
 - same parse with multilib enabled, exercising multilib_global.bbclass
 - core-image-minimal builds for qemux86-64, 5275 tasks

Nguyen Minh Tien (1):
  data_smart: fix operations lost when an override name contains a
    variable

 lib/bb/data_smart.py | 11 +++++++++--
 lib/bb/tests/data.py | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 2 deletions(-)

-- 
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable
  2026-07-25 14:30 [PATCH 0/1] data_smart: fix key expansion in conditional overrides Nguyen Minh Tien
@ 2026-07-25 14:30 ` Nguyen Minh Tien
  2026-07-26 12:21   ` [bitbake-devel] " Richard Purdie
  0 siblings, 1 reply; 6+ messages in thread
From: Nguyen Minh Tien @ 2026-07-25 14:30 UTC (permalink / raw)
  To: bitbake-devel; +Cc: zizuzacker

An operation whose override name needs key expansion is silently dropped:

    RDEPENDS:${PN}:append:pn-foo-${MACHINE} = " bar"

renameVar() rebuilds the dependent override keys with a plain string
replace, so renaming RDEPENDS:${PN} leaves the append attached to
RDEPENDS:foo:append:pn-foo-${MACHINE}, which never matches an active
override. expandKeys() cannot fix that up afterwards either, as it works
from a list of keys collected before any renaming happened.

Expand the derived name before renaming it. Only do so once newkey is
itself expanded, otherwise expandKeys() has still to rename newkey and
handles the dependent keys along with it.

Add regression tests for both cases.

Fixes [YOCTO #14867]

Signed-off-by: Nguyen Minh Tien <zizuzacker@gmail.com>
---
 lib/bb/data_smart.py | 11 +++++++++--
 lib/bb/tests/data.py | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 9961269a3..5738aff95 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -697,8 +697,15 @@ class DataSmart(MutableMapping):
             found = True
             self.overridedata[newkey] = []
             for (v, o) in self.overridedata[key]:
-                self.overridedata[newkey].append([v.replace(key, newkey), o])
-                self.renameVar(v, v.replace(key, newkey))
+                newv = v.replace(key, newkey)
+                # The derived name may still hold a variable reference which
+                # expandKeys() will never revisit, so expand it here. Only once
+                # newkey is expanded though, otherwise expandKeys() has still to
+                # rename newkey and handles the dependent keys along with it.
+                if '${' in newv and '${' not in newkey:
+                    newv = self.expand(newv)
+                self.overridedata[newkey].append([newv, o])
+                self.renameVar(v, newv)
 
         if not found:
             # No variable to rename so not worth the work in writing extra
diff --git a/lib/bb/tests/data.py b/lib/bb/tests/data.py
index fd690a9e2..5ef159eae 100644
--- a/lib/bb/tests/data.py
+++ b/lib/bb/tests/data.py
@@ -406,6 +406,31 @@ class TestOverrides(unittest.TestCase):
         bb.data.expandKeys(self.d)
         self.assertEqual(self.d.getVar("VERSION"), "2")
 
+    # Test an :append whose override name is only resolved by key expansion
+    def test_append_in_expanded_override(self):
+        self.d.setVar("MACHINE", "qemux86")
+        self.d.setVar("PN", "gizmo")
+        self.d.setVar("OVERRIDES", "gizmo:pn-gizmo-qemux86")
+        self.d.setVar("TEST:${PN}", "base")
+        self.d.setVar("TEST:${PN}:append:pn-gizmo-${MACHINE}", " appended")
+        bb.data.expandKeys(self.d)
+        self.assertEqual(self.d.getVar("TEST"), "base appended")
+
+    # Test renaming to a key which is itself not expanded yet, as native.bbclass
+    # does. The dependent override keys must be left for expandKeys() to rename.
+    def test_rename_to_unexpanded_key_with_override(self):
+        self.d.setVar("BPN", "gizmo")
+        self.d.setVar("PN", "gizmo-native")
+        self.d.setVar("OVERRIDES", "class-target")
+        self.d.setVar("TEST:${PN}-lib", "base")
+        self.d.setVar("TEST:${PN}-lib:class-target", "target")
+        with LogRecord() as logs:
+            self.d.renameVar("TEST:${PN}-lib", "TEST:${BPN}-lib-native")
+            bb.data.expandKeys(self.d)
+            self.assertFalse(logContains("renameVar with equivalent keys", logs))
+            self.assertFalse(logContains("replaces original key", logs))
+        self.assertEqual(self.d.getVar("TEST:gizmo-lib-native"), "target")
+
     def test_remove_with_override(self):
         self.d.setVar("TEST:bar", "testvalue2")
         self.d.setVar("TEST:some_val", "testvalue3 testvalue5")
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [bitbake-devel] [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable
  2026-07-25 14:30 ` [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable Nguyen Minh Tien
@ 2026-07-26 12:21   ` Richard Purdie
  2026-07-26 14:53     ` Minh Tiến Nguyễn
  0 siblings, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2026-07-26 12:21 UTC (permalink / raw)
  To: zizuzacker, bitbake-devel

On Sat, 2026-07-25 at 21:30 +0700, Zk47T via lists.openembedded.org wrote:
> An operation whose override name needs key expansion is silently dropped:
> 
>     RDEPENDS:${PN}:append:pn-foo-${MACHINE} = " bar"
> 
> renameVar() rebuilds the dependent override keys with a plain string
> replace, so renaming RDEPENDS:${PN} leaves the append attached to
> RDEPENDS:foo:append:pn-foo-${MACHINE}, which never matches an active
> override. expandKeys() cannot fix that up afterwards either, as it works
> from a list of keys collected before any renaming happened.
> 
> Expand the derived name before renaming it. Only do so once newkey is
> itself expanded, otherwise expandKeys() has still to rename newkey and
> handles the dependent keys along with it.
> 
> Add regression tests for both cases.
> 
> Fixes [YOCTO #14867]
> 
> Signed-off-by: Nguyen Minh Tien <zizuzacker@gmail.com>
> ---
>  lib/bb/data_smart.py | 11 +++++++++--
>  lib/bb/tests/data.py | 25 +++++++++++++++++++++++++
>  2 files changed, 34 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index 9961269a3..5738aff95 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -697,8 +697,15 @@ class DataSmart(MutableMapping):
>              found = True
>              self.overridedata[newkey] = []
>              for (v, o) in self.overridedata[key]:
> -                self.overridedata[newkey].append([v.replace(key, newkey), o])
> -                self.renameVar(v, v.replace(key, newkey))
> +                newv = v.replace(key, newkey)
> +                # The derived name may still hold a variable reference which
> +                # expandKeys() will never revisit, so expand it here. Only once
> +                # newkey is expanded though, otherwise expandKeys() has still to
> +                # rename newkey and handles the dependent keys along with it.
> +                if '${' in newv and '${' not in newkey:
> +                    newv = self.expand(newv)
> +                self.overridedata[newkey].append([newv, o])
> +                self.renameVar(v, newv)
>  

Thanks for the patch and the test cases. I had a deeper look at this as
I was a bit puzzled how this would work without your patch:

XYZ = "yocto"
ABC = "123"
ABC:append:pn-linux-${XYZ} = " 456"
$ bitbake-getvar -r linux-yocto ABC
ABC="123 456"

as if the code can handle that, it should be able to handle the other
case too. I think this is because expandKeys() calls renameVar and then
renameVar itself also calls renameVar() on the same element, which
breaks things. That means that if you do:

diff --git a/lib/bb/data.py b/lib/bb/data.py
index 5fdcdb04a..e3af12a35 100644
--- a/lib/bb/data.py
+++ b/lib/bb/data.py
@@ -94,7 +94,7 @@ def expandKeys(alterdata, readdata = None):
             val = alterdata.getVar(key, False)
             if val is not None:
                 bb.warn("Variable key %s (%s) replaces original key %s (%s)." % (key, val, ekey, newval))
-        alterdata.renameVar(key, ekey)
+        alterdata.renameVar(key, ekey, recurse=False)
 
 def inheritFromOS(d, savedenv, permitted):
     """Inherit variables from the initial environment."""
diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 110dfa111..78ff9b961 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -661,7 +661,7 @@ class DataSmart(MutableMapping):
     def getVar(self, var, expand=True, noweakdefault=False, parsing=False):
         return self.getVarFlag(var, "_content", expand, noweakdefault, parsing)
 
-    def renameVar(self, key, newkey, **loginfo):
+    def renameVar(self, key, newkey, recurse=True, **loginfo):
         """
         Rename the variable key to newkey
         """
@@ -693,7 +693,8 @@ class DataSmart(MutableMapping):
             self.overridedata[newkey] = []
             for (v, o) in self.overridedata[key]:
                 self.overridedata[newkey].append([v.replace(key, newkey), o])
-                self.renameVar(v, v.replace(key, newkey))
+                if recurse:
+                    self.renameVar(v, v.replace(key, newkey))
 
         if ':' in newkey and val is None:
             self._setvar_update_overrides(newkey, **loginfo)



then I think that resolve the issue? It would also perhaps resolve an
issue where a variable with three different key expansions in it might
not work! I'm less sure that three different key variables would work
with plain renameVar, that may also need to set the "no recurse"
option, I'm not sure.

Could you see if this makes sense to you?

Cheers,

Richard






^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [bitbake-devel] [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable
  2026-07-26 12:21   ` [bitbake-devel] " Richard Purdie
@ 2026-07-26 14:53     ` Minh Tiến Nguyễn
  2026-07-26 20:11       ` Richard Purdie
  2026-08-04 21:12       ` Richard Purdie
  0 siblings, 2 replies; 6+ messages in thread
From: Minh Tiến Nguyễn @ 2026-07-26 14:53 UTC (permalink / raw)
  To: Richard Purdie; +Cc: bitbake-devel

> I think this is because expandKeys() calls renameVar and then
> renameVar itself also calls renameVar() on the same element, which
> breaks things. [...] then I think that resolve the issue?

Yes, it does, and it is the better fix. Both keys are already in the
todolist with their expanded names, so expandKeys() can rename them
both. My patch only worked around the second rename with expand().

I tested your diff on a fresh clone at c251833d2. It fixes the exact
form from the bug report, :prepend and :remove, a conditional override
with no operation keyword, several variables in one override name, and a
variable in the key as well as in the override name, all of which fail
on master. bb.tests.data passes, oe-core parses clean at 952 recipes
with no new warnings, and both v1 test cases pass on your diff unchanged.

> I'm less sure that three different key variables would work with plain
> renameVar, that may also need to set the "no recurse" option.

It does not need it, and it must not have it. In the native.bbclass
shape a key is renamed to another unexpanded key before expandKeys()
runs. recurse=True gives the right answer with no warnings. I tried
recurse=False there and the override value is lost: the dependent key is
left behind, and expandKeys() then expands it from the old parent name,
so it no longer matches.

Two things before I send a v2.

First, the new argument breaks recipe_sanity.bbclass, which replaces
DataSmart.renameVar with a wrapper taking a fixed set of arguments. Any
build using INHERIT += "recipe_sanity" now dies before parsing starts:

    TypeError: myrename() got an unexpected keyword argument 'recurse'

A one line oe-core patch passing *args/**kwargs through fixes it, it
just has to go in at the same time.

Second, and this is why I have not sent a v2 yet. There is an ordering
bug here and it is not yours. Master already has it, in the case where
nothing gets dropped:

    ABC = "123"
    ABC:append:pn-linux-${XYZ} = " 456"
    ABC:append:cfg-${SFX} = " 789"

    master gives   "123 789 456"
    should be      "123 456 789"

Both appends are applied, just in the wrong order, and your diff does
not change that case at all.

It matters because of the case your diff does fix. Master drops those
operations, so nobody ever sees the order being wrong. Once they stop
being dropped, it shows:

    PN = "packagegroup-cross-canadian-qemux86"

    RDEPENDS:${PN} = "base"
    RDEPENDS:${PN}:append:pn-packagegroup-cross-canadian-${MACHINE} = " a"
    RDEPENDS:${PN}:append:libc-${TCLIBC} = " b"
    RDEPENDS:${PN}:append:class-target = " c"

    master      "base c"      a and b dropped
    your diff   "base c b a"
    my v1       "base c a b"
    should be   "base a b c"

pn-${PN}, libc-glibc and class-target are all in OVERRIDES at once, so
all three appends are meant to apply.

The reason seems to be __setvar_regexp__, which ends in
(:(?P<add>[^A-Z]*))?$. That came from 6eb56624e, where you stopped
capitalised overrides being processed. ${MACHINE} has uppercase in it,
so an override name holding it fails that test and is stored as a
variable of its own, while a plain override name passes and becomes an
entry in the :append flag on the base variable. The operations for one
variable then live in two places, and the flag group is applied first
regardless of what was written first. With your diff all six orderings
of a, b and c give "base c b a". Mine keeps the order within the second
group but still puts c first, so it is wrong as well.

That line leaves a second gap neither patch closes. A lowercase variable
name passes the regexp, so an append written with ${machine} rather than
${MACHINE} becomes a flag whose override name is still
pn-gizmo-${machine}, and that name is never expanded when overrides are
matched. Master, your diff and mine all return "base" for it. Nobody
writes ${machine} in practice, but it is another way to lose the same
operation.

Would you like me to fix the drop now and report the ordering as its own
bug, or look at these together? I did not want to send a v2 that changes
ordering without asking first.

Best regards,
Tien


Vào CN, 26 thg 7, 2026 vào lúc 19:21 Richard Purdie
<richard.purdie@linuxfoundation.org> đã viết:

>
> On Sat, 2026-07-25 at 21:30 +0700, Zk47T via lists.openembedded.org wrote:
> > An operation whose override name needs key expansion is silently dropped:
> >
> >     RDEPENDS:${PN}:append:pn-foo-${MACHINE} = " bar"
> >
> > renameVar() rebuilds the dependent override keys with a plain string
> > replace, so renaming RDEPENDS:${PN} leaves the append attached to
> > RDEPENDS:foo:append:pn-foo-${MACHINE}, which never matches an active
> > override. expandKeys() cannot fix that up afterwards either, as it works
> > from a list of keys collected before any renaming happened.
> >
> > Expand the derived name before renaming it. Only do so once newkey is
> > itself expanded, otherwise expandKeys() has still to rename newkey and
> > handles the dependent keys along with it.
> >
> > Add regression tests for both cases.
> >
> > Fixes [YOCTO #14867]
> >
> > Signed-off-by: Nguyen Minh Tien <zizuzacker@gmail.com>
> > ---
> >  lib/bb/data_smart.py | 11 +++++++++--
> >  lib/bb/tests/data.py | 25 +++++++++++++++++++++++++
> >  2 files changed, 34 insertions(+), 2 deletions(-)
> >
> > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > index 9961269a3..5738aff95 100644
> > --- a/lib/bb/data_smart.py
> > +++ b/lib/bb/data_smart.py
> > @@ -697,8 +697,15 @@ class DataSmart(MutableMapping):
> >              found = True
> >              self.overridedata[newkey] = []
> >              for (v, o) in self.overridedata[key]:
> > -                self.overridedata[newkey].append([v.replace(key, newkey), o])
> > -                self.renameVar(v, v.replace(key, newkey))
> > +                newv = v.replace(key, newkey)
> > +                # The derived name may still hold a variable reference which
> > +                # expandKeys() will never revisit, so expand it here. Only once
> > +                # newkey is expanded though, otherwise expandKeys() has still to
> > +                # rename newkey and handles the dependent keys along with it.
> > +                if '${' in newv and '${' not in newkey:
> > +                    newv = self.expand(newv)
> > +                self.overridedata[newkey].append([newv, o])
> > +                self.renameVar(v, newv)
> >
>
> Thanks for the patch and the test cases. I had a deeper look at this as
> I was a bit puzzled how this would work without your patch:
>
> XYZ = "yocto"
> ABC = "123"
> ABC:append:pn-linux-${XYZ} = " 456"
> $ bitbake-getvar -r linux-yocto ABC
> ABC="123 456"
>
> as if the code can handle that, it should be able to handle the other
> case too. I think this is because expandKeys() calls renameVar and then
> renameVar itself also calls renameVar() on the same element, which
> breaks things. That means that if you do:
>
> diff --git a/lib/bb/data.py b/lib/bb/data.py
> index 5fdcdb04a..e3af12a35 100644
> --- a/lib/bb/data.py
> +++ b/lib/bb/data.py
> @@ -94,7 +94,7 @@ def expandKeys(alterdata, readdata = None):
>              val = alterdata.getVar(key, False)
>              if val is not None:
>                  bb.warn("Variable key %s (%s) replaces original key %s (%s)." % (key, val, ekey, newval))
> -        alterdata.renameVar(key, ekey)
> +        alterdata.renameVar(key, ekey, recurse=False)
>
>  def inheritFromOS(d, savedenv, permitted):
>      """Inherit variables from the initial environment."""
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index 110dfa111..78ff9b961 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -661,7 +661,7 @@ class DataSmart(MutableMapping):
>      def getVar(self, var, expand=True, noweakdefault=False, parsing=False):
>          return self.getVarFlag(var, "_content", expand, noweakdefault, parsing)
>
> -    def renameVar(self, key, newkey, **loginfo):
> +    def renameVar(self, key, newkey, recurse=True, **loginfo):
>          """
>          Rename the variable key to newkey
>          """
> @@ -693,7 +693,8 @@ class DataSmart(MutableMapping):
>              self.overridedata[newkey] = []
>              for (v, o) in self.overridedata[key]:
>                  self.overridedata[newkey].append([v.replace(key, newkey), o])
> -                self.renameVar(v, v.replace(key, newkey))
> +                if recurse:
> +                    self.renameVar(v, v.replace(key, newkey))
>
>          if ':' in newkey and val is None:
>              self._setvar_update_overrides(newkey, **loginfo)
>
>
>
> then I think that resolve the issue? It would also perhaps resolve an
> issue where a variable with three different key expansions in it might
> not work! I'm less sure that three different key variables would work
> with plain renameVar, that may also need to set the "no recurse"
> option, I'm not sure.
>
> Could you see if this makes sense to you?
>
> Cheers,
>
> Richard
>
>
>
>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [bitbake-devel] [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable
  2026-07-26 14:53     ` Minh Tiến Nguyễn
@ 2026-07-26 20:11       ` Richard Purdie
  2026-08-04 21:12       ` Richard Purdie
  1 sibling, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2026-07-26 20:11 UTC (permalink / raw)
  To: Minh Tiến Nguyễn; +Cc: bitbake-devel

On Sun, 2026-07-26 at 21:53 +0700, Minh Tiến Nguyễn wrote:
> > I think this is because expandKeys() calls renameVar and then
> > renameVar itself also calls renameVar() on the same element, which
> > breaks things. [...] then I think that resolve the issue?
> 
> Yes, it does, and it is the better fix. Both keys are already in the
> todolist with their expanded names, so expandKeys() can rename them
> both. My patch only worked around the second rename with expand().
> 
> I tested your diff on a fresh clone at c251833d2. It fixes the exact
> form from the bug report, :prepend and :remove, a conditional override
> with no operation keyword, several variables in one override name, and a
> variable in the key as well as in the override name, all of which fail
> on master. bb.tests.data passes, oe-core parses clean at 952 recipes
> with no new warnings, and both v1 test cases pass on your diff unchanged.

I have to wonder whether my reply is just being fed to AI? :/

The project's policy is that AI use isn't prohibited but it's use
should be made clear...

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [bitbake-devel] [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable
  2026-07-26 14:53     ` Minh Tiến Nguyễn
  2026-07-26 20:11       ` Richard Purdie
@ 2026-08-04 21:12       ` Richard Purdie
  1 sibling, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2026-08-04 21:12 UTC (permalink / raw)
  To: Minh Tiến Nguyễn; +Cc: bitbake-devel

Hi Tien,

On Sun, 2026-07-26 at 21:53 +0700, Minh Tiến Nguyễn wrote:
> > I think this is because expandKeys() calls renameVar and then
> > renameVar itself also calls renameVar() on the same element, which
> > breaks things. [...] then I think that resolve the issue?
> 
> Yes, it does, and it is the better fix. Both keys are already in the
> todolist with their expanded names, so expandKeys() can rename them
> both. My patch only worked around the second rename with expand().
> 
> I tested your diff on a fresh clone at c251833d2. It fixes the exact
> form from the bug report, :prepend and :remove, a conditional override
> with no operation keyword, several variables in one override name, and a
> variable in the key as well as in the override name, all of which fail
> on master. bb.tests.data passes, oe-core parses clean at 952 recipes
> with no new warnings, and both v1 test cases pass on your diff unchanged.
> 
> > I'm less sure that three different key variables would work with plain
> > renameVar, that may also need to set the "no recurse" option.
> 
> It does not need it, and it must not have it. In the native.bbclass
> shape a key is renamed to another unexpanded key before expandKeys()
> runs. recurse=True gives the right answer with no warnings. I tried
> recurse=False there and the override value is lost: the dependent key is
> left behind, and expandKeys() then expands it from the old parent name,
> so it no longer matches.
> 
> Two things before I send a v2.
> 
> First, the new argument breaks recipe_sanity.bbclass, which replaces
> DataSmart.renameVar with a wrapper taking a fixed set of arguments. Any
> build using INHERIT += "recipe_sanity" now dies before parsing starts:
> 
>     TypeError: myrename() got an unexpected keyword argument 'recurse'
> 
> A one line oe-core patch passing *args/**kwargs through fixes it, it
> just has to go in at the same time.
> 
> Second, and this is why I have not sent a v2 yet. There is an ordering
> bug here and it is not yours. Master already has it, in the case where
> nothing gets dropped:
> 
>     ABC = "123"
>     ABC:append:pn-linux-${XYZ} = " 456"
>     ABC:append:cfg-${SFX} = " 789"
> 
>     master gives   "123 789 456"
>     should be      "123 456 789"
> 
> Both appends are applied, just in the wrong order, and your diff does
> not change that case at all.
> 
> It matters because of the case your diff does fix. Master drops those
> operations, so nobody ever sees the order being wrong. Once they stop
> being dropped, it shows:
> 
>     PN = "packagegroup-cross-canadian-qemux86"
> 
>     RDEPENDS:${PN} = "base"
>     RDEPENDS:${PN}:append:pn-packagegroup-cross-canadian-${MACHINE} = " a"
>     RDEPENDS:${PN}:append:libc-${TCLIBC} = " b"
>     RDEPENDS:${PN}:append:class-target = " c"
> 
>     master      "base c"      a and b dropped
>     your diff   "base c b a"
>     my v1       "base c a b"
>     should be   "base a b c"
> 
> pn-${PN}, libc-glibc and class-target are all in OVERRIDES at once, so
> all three appends are meant to apply.
> 
> The reason seems to be __setvar_regexp__, which ends in
> (:(?P<add>[^A-Z]*))?$. That came from 6eb56624e, where you stopped
> capitalised overrides being processed. ${MACHINE} has uppercase in it,
> so an override name holding it fails that test and is stored as a
> variable of its own, while a plain override name passes and becomes an
> entry in the :append flag on the base variable. The operations for one
> variable then live in two places, and the flag group is applied first
> regardless of what was written first. With your diff all six orderings
> of a, b and c give "base c b a". Mine keeps the order within the second
> group but still puts c first, so it is wrong as well.
> 
> That line leaves a second gap neither patch closes. A lowercase variable
> name passes the regexp, so an append written with ${machine} rather than
> ${MACHINE} becomes a flag whose override name is still
> pn-gizmo-${machine}, and that name is never expanded when overrides are
> matched. Master, your diff and mine all return "base" for it. Nobody
> writes ${machine} in practice, but it is another way to lose the same
> operation.
> 
> Would you like me to fix the drop now and report the ordering as its own
> bug, or look at these together? I did not want to send a v2 that changes
> ordering without asking first.

Were you going to send an updated patch? I think the issues are
separate, we should fix the operations in one patch. I'm not convinced
the ordering issue is "real" in that we've not commited to any
particular order, only that for a given bitbake version, it should
always be consistent. It would be good to close out the first issue.
I'm happy to write some patches if you're not able to.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-08-04 21:12 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-25 14:30 [PATCH 0/1] data_smart: fix key expansion in conditional overrides Nguyen Minh Tien
2026-07-25 14:30 ` [PATCH 1/1] data_smart: fix operations lost when an override name contains a variable Nguyen Minh Tien
2026-07-26 12:21   ` [bitbake-devel] " Richard Purdie
2026-07-26 14:53     ` Minh Tiến Nguyễn
2026-07-26 20:11       ` Richard Purdie
2026-08-04 21:12       ` Richard Purdie

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox