[PATCH] data_smart: Fix remove operator and its interaction with data expansion

[PATCH] data_smart: Fix remove operator and its interaction with data expansion

[Richard Purdie]

If you have:

FOO = "${bindir}/X Y"
FOO_remove = "${bindir}/X"

the expected result is "Y". Currently this doesn't work since the removed
expressions are not expanded first. This patch adjusts things so the
expressions are expanded before being processed for removal.

Also add a test to ensure this case continues to work.

[YOCTO #6624]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index 7b2032a..16a85bc 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -642,7 +642,8 @@ class DataSmart(MutableMapping):
                 cachename = var + "[" + flag + "]"
             value = self.expand(value, cachename)
         if value and flag == "_content" and local_var is not None and "_removeactive" in local_var:
-            filtered = filter(lambda v: v not in local_var["_removeactive"],
+            removes = [self.expand(r) for r in local_var["_removeactive"]]
+            filtered = filter(lambda v: v not in removes,
                               value.split(" "))
             value = " ".join(filtered)
             if expand:
diff --git a/bitbake/lib/bb/tests/data.py b/bitbake/lib/bb/tests/data.py
index 944a906..9b09ff4 100644
--- a/bitbake/lib/bb/tests/data.py
+++ b/bitbake/lib/bb/tests/data.py
@@ -259,6 +259,13 @@ class TestConcatOverride(unittest.TestCase):
         bb.data.update_data(self.d)
         self.assertEqual(self.d.getVar("TEST", True), "")
 
+    def test_remove_expansion(self):
+        self.d.setVar("BAR", "Z")
+        self.d.setVar("TEST", "${BAR}/X Y")
+        self.d.setVar("TEST_remove", "${BAR}/X")
+        bb.data.update_data(self.d)
+        self.assertEqual(self.d.getVar("TEST", True), "Y")
+
 class TestOverrides(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()

Re: [PATCH] data_smart: Fix remove operator and its interaction with data expansion

[Burton, Ross]

On 13 September 2014 08:50, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> If you have:
>
> FOO = "${bindir}/X Y"
> FOO_remove = "${bindir}/X"
>
> the expected result is "Y". Currently this doesn't work since the removed
> expressions are not expanded first. This patch adjusts things so the
> expressions are expanded before being processed for removal.

Whilst the actual expansion works as expected now (consider this a
tested-by), the output in bitbake -e doesn't make sense:

i.e. with PACKAGES_remove = "${PN}-staticdev ${PN}-locale" in a distro config:

# $PACKAGES [5 operations]
#   set /home/ross/foo/poky/meta/conf/bitbake.conf:264
#     "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
${PACKAGE_BEFORE_PN} ${PN}"
#   _remove /home/ross/foo/meta-foo/conf/distro/foo.conf:201
#     "${PN}-staticdev ${PN}-locale"
#   set /home/ross/foo/poky/meta/conf/documentation.conf:308
#     [doc] "The list of packages to be created from the recipe."
#   prepend /home/ross/foo/poky/meta/recipes-support/curl/curl_7.37.1.bb:43
#     "lib${BPN} lib${BPN}-dev lib${BPN}-staticdev lib${BPN}-doc"
#   _remove /home/ross/foo/meta-foo/recipes-support/curl/curl_%.bbappend:1
#     "${PN}-dev"
# computed:
#   "lib${BPN} lib${BPN}-dev lib${BPN}-staticdev lib${BPN}-doc
${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
${PACKAGE_BEFORE_PN} ${PN}"
PACKAGES="libcurl libcurl-dev libcurl-staticdev libcurl-doc curl-dbg
curl-doc  curl"

The PACKAGES value is right, but the "computed" value displayed by -e isn't.

Ross

Re: [PATCH] data_smart: Fix remove operator and its interaction with data expansion

[Richard Purdie]

On Sun, 2014-09-14 at 21:17 +0100, Burton, Ross wrote:
> On 13 September 2014 08:50, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > If you have:
> >
> > FOO = "${bindir}/X Y"
> > FOO_remove = "${bindir}/X"
> >
> > the expected result is "Y". Currently this doesn't work since the removed
> > expressions are not expanded first. This patch adjusts things so the
> > expressions are expanded before being processed for removal.
> 
> Whilst the actual expansion works as expected now (consider this a
> tested-by), the output in bitbake -e doesn't make sense:
> 
> i.e. with PACKAGES_remove = "${PN}-staticdev ${PN}-locale" in a distro config:
> 
> # $PACKAGES [5 operations]
> #   set /home/ross/foo/poky/meta/conf/bitbake.conf:264
> #     "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
> ${PACKAGE_BEFORE_PN} ${PN}"
> #   _remove /home/ross/foo/meta-foo/conf/distro/foo.conf:201
> #     "${PN}-staticdev ${PN}-locale"
> #   set /home/ross/foo/poky/meta/conf/documentation.conf:308
> #     [doc] "The list of packages to be created from the recipe."
> #   prepend /home/ross/foo/poky/meta/recipes-support/curl/curl_7.37.1.bb:43
> #     "lib${BPN} lib${BPN}-dev lib${BPN}-staticdev lib${BPN}-doc"
> #   _remove /home/ross/foo/meta-foo/recipes-support/curl/curl_%.bbappend:1
> #     "${PN}-dev"
> # computed:
> #   "lib${BPN} lib${BPN}-dev lib${BPN}-staticdev lib${BPN}-doc
> ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
> ${PACKAGE_BEFORE_PN} ${PN}"
> PACKAGES="libcurl libcurl-dev libcurl-staticdev libcurl-doc curl-dbg
> curl-doc  curl"
> 
> The PACKAGES value is right, but the "computed" value displayed by -e isn't.

This is the "pre expansion" value, so its right but the comment needs
improving...

Cheers,

Richard