[PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[Otavio Salvador]

This allow the addition and removal of distro features easily. To add
a feature, use:

 EXTRA_DISTRO_FEATURES += "wayland"

and to remove, use '~' prefix, as:

 EXTRA_DISTRO_FEATURES += "~x11"

This code has been mostly copied from Mentor Graphics public layer but
changed the variable name for a more descriptive name. The original
code can be seen at user_feature.bbclass at:

 http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
---
Changes for v3:
- Drop _prepend/_append flags from DISTRO_FEATURES to avoid the re-add

 meta/classes/base.bbclass |  4 ++++
 meta/lib/oe/utils.py      | 25 +++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 9c92e0b..83f6458 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -298,6 +298,9 @@ def buildcfg_neededvars(d):
     if pesteruser:
         bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
 
+EXTRA_DISTRO_FEATURES ?= ""
+EXTRA_DISTRO_FEATURES[type] = "list"
+
 addhandler base_eventhandler
 base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted"
 python base_eventhandler() {
@@ -307,6 +310,7 @@ python base_eventhandler() {
         preferred_ml_updates(e.data)
         oe.utils.features_backfill("DISTRO_FEATURES", e.data)
         oe.utils.features_backfill("MACHINE_FEATURES", e.data)
+        oe.utils.extra_distro_features(e.data)
 
     if isinstance(e, bb.event.BuildStarted):
         localdata = bb.data.createCopy(e.data)
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 82987e8..db468cb 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -117,6 +117,31 @@ def features_backfill(var,d):
     if addfeatures:
         d.appendVar(var, " " + " ".join(addfeatures))
 
+def extra_distro_features(d):
+    import oe.data
+
+    l = d.createCopy()
+    l.finalize()
+
+    extra_features = oe.data.typed_value('EXTRA_DISTRO_FEATURES', l)
+    if not extra_features:
+        return
+
+    distro_features = l.getVar('DISTRO_FEATURES', True).split()
+    for feature in extra_features:
+        if feature.startswith('~'):
+            feature = feature[1:]
+            if feature in distro_features:
+                distro_features.remove(feature)
+        else:
+            if feature not in distro_features:
+                distro_features.append(feature)
+
+    # Avoid readding the removed features later
+    d.delVarFlag("DISTRO_FEATURES", "_append")
+    d.delVarFlag("DISTRO_FEATURES", "_prepend")
+
+    d.setVar('DISTRO_FEATURES', ' '.join(distro_features))
 
 def packages_filter_out_system(d):
     """
-- 
1.8.3.2

Re: [PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[Richard Purdie]

On Wed, 2013-07-31 at 12:13 -0300, Otavio Salvador wrote:
> This allow the addition and removal of distro features easily. To add
> a feature, use:
> 
>  EXTRA_DISTRO_FEATURES += "wayland"
> 
> and to remove, use '~' prefix, as:
> 
>  EXTRA_DISTRO_FEATURES += "~x11"
> 
> This code has been mostly copied from Mentor Graphics public layer but
> changed the variable name for a more descriptive name. The original
> code can be seen at user_feature.bbclass at:
> 
>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor
> 
> Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
> ---
> Changes for v3:
> - Drop _prepend/_append flags from DISTRO_FEATURES to avoid the re-add
> 
>  meta/classes/base.bbclass |  4 ++++
>  meta/lib/oe/utils.py      | 25 +++++++++++++++++++++++++
>  2 files changed, 29 insertions(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 9c92e0b..83f6458 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -298,6 +298,9 @@ def buildcfg_neededvars(d):
>      if pesteruser:
>          bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
>  
> +EXTRA_DISTRO_FEATURES ?= ""
> +EXTRA_DISTRO_FEATURES[type] = "list"
> +
>  addhandler base_eventhandler
>  base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted"
>  python base_eventhandler() {
> @@ -307,6 +310,7 @@ python base_eventhandler() {
>          preferred_ml_updates(e.data)
>          oe.utils.features_backfill("DISTRO_FEATURES", e.data)
>          oe.utils.features_backfill("MACHINE_FEATURES", e.data)
> +        oe.utils.extra_distro_features(e.data)
>  
>      if isinstance(e, bb.event.BuildStarted):
>          localdata = bb.data.createCopy(e.data)
> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
> index 82987e8..db468cb 100644
> --- a/meta/lib/oe/utils.py
> +++ b/meta/lib/oe/utils.py
> @@ -117,6 +117,31 @@ def features_backfill(var,d):
>      if addfeatures:
>          d.appendVar(var, " " + " ".join(addfeatures))
>  
> +def extra_distro_features(d):
> +    import oe.data
> +
> +    l = d.createCopy()
> +    l.finalize()
> +
> +    extra_features = oe.data.typed_value('EXTRA_DISTRO_FEATURES', l)
> +    if not extra_features:
> +        return
> +
> +    distro_features = l.getVar('DISTRO_FEATURES', True).split()
> +    for feature in extra_features:
> +        if feature.startswith('~'):
> +            feature = feature[1:]
> +            if feature in distro_features:
> +                distro_features.remove(feature)
> +        else:
> +            if feature not in distro_features:
> +                distro_features.append(feature)
> +
> +    # Avoid readding the removed features later
> +    d.delVarFlag("DISTRO_FEATURES", "_append")
> +    d.delVarFlag("DISTRO_FEATURES", "_prepend")

I quite clearly said this was a hack to allow you to see what the
problem was with the code and that it was *not* an acceptable change to
go into the core. I don't see any mention of that now and that is a
little frustrating to see this being proposed as a patch.

Cheers,

Richard

Re: [PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[Otavio Salvador]

On Thu, Aug 1, 2013 at 4:37 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2013-07-31 at 12:13 -0300, Otavio Salvador wrote:
...
>> +    # Avoid readding the removed features later
>> +    d.delVarFlag("DISTRO_FEATURES", "_append")
>> +    d.delVarFlag("DISTRO_FEATURES", "_prepend")
>
> I quite clearly said this was a hack to allow you to see what the
> problem was with the code and that it was *not* an acceptable change to
> go into the core. I don't see any mention of that now and that is a
> little frustrating to see this being proposed as a patch.

I forgot to add it to commit log, sorry. The idea was to send the
working code so people using it could use a valid one. I understand we
need to find a better solution for this but I am clueless about where
to look.

I can work in another solution if someone can tell me what needs to be
done. I looked at bb.data_smart code and didn't find any other way to
fix this.

The finalize code would be the other place I can think about changing. Any hint?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[Richard Purdie]

On Thu, 2013-08-01 at 09:19 -0300, Otavio Salvador wrote:
> On Thu, Aug 1, 2013 at 4:37 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Wed, 2013-07-31 at 12:13 -0300, Otavio Salvador wrote:
> ...
> >> +    # Avoid readding the removed features later
> >> +    d.delVarFlag("DISTRO_FEATURES", "_append")
> >> +    d.delVarFlag("DISTRO_FEATURES", "_prepend")
> >
> > I quite clearly said this was a hack to allow you to see what the
> > problem was with the code and that it was *not* an acceptable change to
> > go into the core. I don't see any mention of that now and that is a
> > little frustrating to see this being proposed as a patch.
> 
> I forgot to add it to commit log, sorry. The idea was to send the
> working code so people using it could use a valid one. I understand we
> need to find a better solution for this but I am clueless about where
> to look.
> 
> I can work in another solution if someone can tell me what needs to be
> done. I looked at bb.data_smart code and didn't find any other way to
> fix this.
> 
> The finalize code would be the other place I can think about changing. Any hint?

Using existing API, did you try just d.delVar("DISTRO_FEATURES") and
then setting it again? Unlike the above, that doesn't assume any
knowledge of bitbake internals.

Cheers,

Richard

Re: [PATCH v3] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

[Otavio Salvador]

On Thu, Aug 1, 2013 at 10:08 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2013-08-01 at 09:19 -0300, Otavio Salvador wrote:
>> On Thu, Aug 1, 2013 at 4:37 AM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Wed, 2013-07-31 at 12:13 -0300, Otavio Salvador wrote:
>> ...
>> >> +    # Avoid readding the removed features later
>> >> +    d.delVarFlag("DISTRO_FEATURES", "_append")
>> >> +    d.delVarFlag("DISTRO_FEATURES", "_prepend")
>> >
>> > I quite clearly said this was a hack to allow you to see what the
>> > problem was with the code and that it was *not* an acceptable change to
>> > go into the core. I don't see any mention of that now and that is a
>> > little frustrating to see this being proposed as a patch.
>>
>> I forgot to add it to commit log, sorry. The idea was to send the
>> working code so people using it could use a valid one. I understand we
>> need to find a better solution for this but I am clueless about where
>> to look.
>>
>> I can work in another solution if someone can tell me what needs to be
>> done. I looked at bb.data_smart code and didn't find any other way to
>> fix this.
>>
>> The finalize code would be the other place I can think about changing. Any hint?
>
> Using existing API, did you try just d.delVar("DISTRO_FEATURES") and
> then setting it again? Unlike the above, that doesn't assume any
> knowledge of bitbake internals.

Works like a charm. I will send v4 with this one.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750