[PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Patrick Ohly]

Having a .bbappend without corresponding .bb file triggers an error or
at least warning, depending on the global BB_DANGLINGAPPENDS_WARNONLY.

Some layers (for example, meta-freescale) avoid that message by only
adding .bbappends to the BBFILES when the layers they apply to are
present. Others (like intel-iot-refkit) avoid such .bbappends by
falling back to global assignments with _pn-<recipe> as override. Both
is complicated.

Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
all bbappends which match a file pattern in that list. This is an
easier way to have bbappends which may or may not apply to an
existing recipe.

Example usage:

  # We have recipes-* directories, add to BBFILES
  BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
	${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
	${LAYERDIR}/recipes-*/*/*.bbappend"

  # Several of our *.bbappends might be for layers that are not
  # guaranteed to be present. Don't warn or even error out because
  # of those.
  BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend"

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 lib/bb/cooker.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index bc8574a..687fbfd 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -45,6 +45,7 @@ import pyinotify
 import json
 import pickle
 import codecs
+import fnmatch
 
 logger      = logging.getLogger("BitBake")
 collectlog  = logging.getLogger("BitBake.Collection")
@@ -1008,8 +1009,11 @@ class BBCooker:
             applied_appends.extend(self.collection.get_file_appends(fn))
 
         appends_without_recipes = []
+        appends_whitelist = (self.data.getVar("BBAPPENDS_DANGLING_WHITELIST") or "").split()
+        bb.note('Whitelist: %s' % appends_whitelist)
         for _, appendfn in self.collection.bbappends:
-            if not appendfn in applied_appends:
+            if not appendfn in applied_appends and \
+               not any(map(lambda whitelist_entry: fnmatch.fnmatch(appendfn, whitelist_entry), appends_whitelist)):
                 appends_without_recipes.append(appendfn)
 
         if appends_without_recipes:

base-commit: 362f6044fcaafe51ab4377af8f2606165b112717
-- 
git-series 0.9.1

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]

On Mon, May 22, 2017 at 3:50 AM, Patrick Ohly <patrick.ohly@intel.com>
wrote:

> Having a .bbappend without corresponding .bb file triggers an error or
> at least warning, depending on the global BB_DANGLINGAPPENDS_WARNONLY.
>
> Some layers (for example, meta-freescale) avoid that message by only
> adding .bbappends to the BBFILES when the layers they apply to are
> present. Others (like intel-iot-refkit) avoid such .bbappends by
> falling back to global assignments with _pn-<recipe> as override. Both
> is complicated.
>
> Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
> all bbappends which match a file pattern in that list. This is an
> easier way to have bbappends which may or may not apply to an
> existing recipe.
>

IMHO this makes it too easy to miss legitimate problems. By adding them
only when the layers are present, you ensure that if that layer removes the
recipe you’re appending, you’ll immediately know it. This hides such actual
problems by suppressing for all optional appends across the board.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1712 bytes --]

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Patrick Ohly]

On Thu, 2017-05-25 at 08:50 -0700, Christopher Larson wrote:
> 
> On Mon, May 22, 2017 at 3:50 AM, Patrick Ohly <patrick.ohly@intel.com>
> wrote:
>         Having a .bbappend without corresponding .bb file triggers an
>         error or
>         at least warning, depending on the global
>         BB_DANGLINGAPPENDS_WARNONLY.
>         
>         Some layers (for example, meta-freescale) avoid that message
>         by only
>         adding .bbappends to the BBFILES when the layers they apply to
>         are
>         present. Others (like intel-iot-refkit) avoid such .bbappends
>         by
>         falling back to global assignments with _pn-<recipe> as
>         override. Both
>         is complicated.
>         
>         Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and
>         ignores
>         all bbappends which match a file pattern in that list. This is
>         an
>         easier way to have bbappends which may or may not apply to an
>         existing recipe.
> 
> IMHO this makes it too easy to miss legitimate problems. By adding
> them only when the layers are present, you ensure that if that layer
> removes the recipe you’re appending, you’ll immediately know it. This
> hides such actual problems by suppressing for all optional appends
> across the board.

I understand that it is a double-edged sword. It's meant for layer
maintainers who know what they are doing. If a layer maintainer prefers
to be warned about missing recipes, they can simply ignore the whitelist
feature. Layer maintainers who use it need to ensure that they have
tests in place which will fail when the bbappend is no longer used.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Patrick Ohly]

On Thu, 2017-05-25 at 22:01 +0200, Patrick Ohly wrote:
> On Thu, 2017-05-25 at 08:50 -0700, Christopher Larson wrote:
> > 
> > On Mon, May 22, 2017 at 3:50 AM, Patrick Ohly <patrick.ohly@intel.com>
> > wrote:
> >         Having a .bbappend without corresponding .bb file triggers an
> >         error or
> >         at least warning, depending on the global
> >         BB_DANGLINGAPPENDS_WARNONLY.
> >         
> >         Some layers (for example, meta-freescale) avoid that message
> >         by only
> >         adding .bbappends to the BBFILES when the layers they apply to
> >         are
> >         present. Others (like intel-iot-refkit) avoid such .bbappends
> >         by
> >         falling back to global assignments with _pn-<recipe> as
> >         override. Both
> >         is complicated.
> >         
> >         Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and
> >         ignores
> >         all bbappends which match a file pattern in that list. This is
> >         an
> >         easier way to have bbappends which may or may not apply to an
> >         existing recipe.
> > 
> > IMHO this makes it too easy to miss legitimate problems. By adding
> > them only when the layers are present, you ensure that if that layer
> > removes the recipe you’re appending, you’ll immediately know it. This
> > hides such actual problems by suppressing for all optional appends
> > across the board.
> 
> I understand that it is a double-edged sword. It's meant for layer
> maintainers who know what they are doing. If a layer maintainer prefers
> to be warned about missing recipes, they can simply ignore the whitelist
> feature. Layer maintainers who use it need to ensure that they have
> tests in place which will fail when the bbappend is no longer used.

Also note that the alternative solution currently used by layers like
meta-freescale is brittle (depends on ordering of entries in
bblayers.conf). This is not just a theoretic problem; there's currently
a mail thread on the Yocto mailing list ("dynamic-layers") where someone
had the order wrong and had to spend time on debugging the problem.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Mark Hatle]

On 5/25/17 10:50 AM, Christopher Larson wrote:
> 
> On Mon, May 22, 2017 at 3:50 AM, Patrick Ohly <patrick.ohly@intel.com
> <mailto:patrick.ohly@intel.com>> wrote:
> 
>     Having a .bbappend without corresponding .bb file triggers an error or
>     at least warning, depending on the global BB_DANGLINGAPPENDS_WARNONLY.
> 
>     Some layers (for example, meta-freescale) avoid that message by only
>     adding .bbappends to the BBFILES when the layers they apply to are
>     present. Others (like intel-iot-refkit) avoid such .bbappends by
>     falling back to global assignments with _pn-<recipe> as override. Both
>     is complicated.
> 
>     Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
>     all bbappends which match a file pattern in that list. This is an
>     easier way to have bbappends which may or may not apply to an
>     existing recipe.
> 
> 
> IMHO this makes it too easy to miss legitimate problems. By adding them only
> when the layers are present, you ensure that if that layer removes the recipe
> you’re appending, you’ll immediately know it. This hides such actual problems by
> suppressing for all optional appends across the board.

I agree.  I prefer the Freescale approach.  Only add the bbappend when the thing
it should be appending is present.

Avoids the problem of 'hidden' errors (things missing), but does cause an issue
with more combinations needing to be tested, but at least this is more explicit.

--Mark

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Richard Purdie]

On Mon, 2017-05-22 at 12:50 +0200, Patrick Ohly wrote:
> Having a .bbappend without corresponding .bb file triggers an error
> or
> at least warning, depending on the global
> BB_DANGLINGAPPENDS_WARNONLY.
> 
> Some layers (for example, meta-freescale) avoid that message by only
> adding .bbappends to the BBFILES when the layers they apply to are
> present. Others (like intel-iot-refkit) avoid such .bbappends by
> falling back to global assignments with _pn-<recipe> as override.
> Both
> is complicated.
> 
> Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
> all bbappends which match a file pattern in that list. This is an
> easier way to have bbappends which may or may not apply to an
> existing recipe.
> 
> Example usage:
> 
>   # We have recipes-* directories, add to BBFILES
>   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
> 	${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
> 	${LAYERDIR}/recipes-*/*/*.bbappend"
> 
>   # Several of our *.bbappends might be for layers that are not
>   # guaranteed to be present. Don't warn or even error out because
>   # of those.
>   BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-
> bbappends/recipes-*/*/*.bbappend"

I've been giving this some thought. The freescale layer approach does
have ordering issues which I don't like. Equally, Chris' comment about
this is also a valid concern. Experience suggests saying the layer
maintainer needs to be careful doesn't work in practise.

I'm wondering if there is a hybrid solution if we extend this syntax a
little, e.g.:

BBFILES_DYNAMIC += "\
    XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
    YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
"

so that the code only applies these BBFILES entries if the layer named
in the first parameter is present?

This would happen at the end of parsing so ordering becomes a moot
point. The direction you stack BBFILES_DYANMIC in would determin
bbappend application order.

How does that sound to people?

Cheers,

Richard

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Patrick Ohly]

On Fri, 2017-06-02 at 13:58 +0100, Richard Purdie wrote:
> I'm wondering if there is a hybrid solution if we extend this syntax a
> little, e.g.:
> 
> BBFILES_DYNAMIC += "\
>     XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
>     YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
> "
> 
> so that the code only applies these BBFILES entries if the layer named
> in the first parameter is present?
> 
> This would happen at the end of parsing so ordering becomes a moot
> point. The direction you stack BBFILES_DYANMIC in would determin
> bbappend application order.
> 
> How does that sound to people?

Sounds good to me.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Mark Hatle]

On 6/2/17 7:58 AM, Richard Purdie wrote:
> On Mon, 2017-05-22 at 12:50 +0200, Patrick Ohly wrote:
>> Having a .bbappend without corresponding .bb file triggers an error
>> or
>> at least warning, depending on the global
>> BB_DANGLINGAPPENDS_WARNONLY.
>>
>> Some layers (for example, meta-freescale) avoid that message by only
>> adding .bbappends to the BBFILES when the layers they apply to are
>> present. Others (like intel-iot-refkit) avoid such .bbappends by
>> falling back to global assignments with _pn-<recipe> as override.
>> Both
>> is complicated.
>>
>> Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
>> all bbappends which match a file pattern in that list. This is an
>> easier way to have bbappends which may or may not apply to an
>> existing recipe.
>>
>> Example usage:
>>
>>   # We have recipes-* directories, add to BBFILES
>>   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
>> 	${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
>> 	${LAYERDIR}/recipes-*/*/*.bbappend"
>>
>>   # Several of our *.bbappends might be for layers that are not
>>   # guaranteed to be present. Don't warn or even error out because
>>   # of those.
>>   BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-
>> bbappends/recipes-*/*/*.bbappend"
> 
> I've been giving this some thought. The freescale layer approach does
> have ordering issues which I don't like. Equally, Chris' comment about
> this is also a valid concern. Experience suggests saying the layer
> maintainer needs to be careful doesn't work in practise.
> 
> I'm wondering if there is a hybrid solution if we extend this syntax a
> little, e.g.:
> 
> BBFILES_DYNAMIC += "\
>     XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
>     YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
> "
> 
> so that the code only applies these BBFILES entries if the layer named
> in the first parameter is present?
> 
> This would happen at the end of parsing so ordering becomes a moot
> point. The direction you stack BBFILES_DYANMIC in would determin
> bbappend application order.
> 
> How does that sound to people?

That sounds reasonable to me, and has a precedent in the package_dynamic as well.

--Mark

> Cheers,
> 
> Richard
> 

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 2608 bytes --]

On Fri, Jun 2, 2017 at 6:26 AM, Mark Hatle <mark.hatle@windriver.com> wrote:

> On 6/2/17 7:58 AM, Richard Purdie wrote:
> > On Mon, 2017-05-22 at 12:50 +0200, Patrick Ohly wrote:
> >> Having a .bbappend without corresponding .bb file triggers an error
> >> or
> >> at least warning, depending on the global
> >> BB_DANGLINGAPPENDS_WARNONLY.
> >>
> >> Some layers (for example, meta-freescale) avoid that message by only
> >> adding .bbappends to the BBFILES when the layers they apply to are
> >> present. Others (like intel-iot-refkit) avoid such .bbappends by
> >> falling back to global assignments with _pn-<recipe> as override.
> >> Both
> >> is complicated.
> >>
> >> Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
> >> all bbappends which match a file pattern in that list. This is an
> >> easier way to have bbappends which may or may not apply to an
> >> existing recipe.
> >>
> >> Example usage:
> >>
> >>   # We have recipes-* directories, add to BBFILES
> >>   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
> >>      ${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
> >>      ${LAYERDIR}/recipes-*/*/*.bbappend"
> >>
> >>   # Several of our *.bbappends might be for layers that are not
> >>   # guaranteed to be present. Don't warn or even error out because
> >>   # of those.
> >>   BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-
> >> bbappends/recipes-*/*/*.bbappend"
> >
> > I've been giving this some thought. The freescale layer approach does
> > have ordering issues which I don't like. Equally, Chris' comment about
> > this is also a valid concern. Experience suggests saying the layer
> > maintainer needs to be careful doesn't work in practise.
> >
> > I'm wondering if there is a hybrid solution if we extend this syntax a
> > little, e.g.:
> >
> > BBFILES_DYNAMIC += "\
> >     XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
> >     YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
> > "
> >
> > so that the code only applies these BBFILES entries if the layer named
> > in the first parameter is present?
> >
> > This would happen at the end of parsing so ordering becomes a moot
> > point. The direction you stack BBFILES_DYANMIC in would determin
> > bbappend application order.
> >
> > How does that sound to people?
>
> That sounds reasonable to me, and has a precedent in the package_dynamic
> as well.


Agreed.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3553 bytes --]

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Patrick Ohly]

On Fri, 2017-06-02 at 13:58 +0100, Richard Purdie wrote:
> On Mon, 2017-05-22 at 12:50 +0200, Patrick Ohly wrote:
> > Having a .bbappend without corresponding .bb file triggers an error
> > or
> > at least warning, depending on the global
> > BB_DANGLINGAPPENDS_WARNONLY.
> > 
> > Some layers (for example, meta-freescale) avoid that message by only
> > adding .bbappends to the BBFILES when the layers they apply to are
> > present. Others (like intel-iot-refkit) avoid such .bbappends by
> > falling back to global assignments with _pn-<recipe> as override.
> > Both
> > is complicated.
> > 
> > Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
> > all bbappends which match a file pattern in that list. This is an
> > easier way to have bbappends which may or may not apply to an
> > existing recipe.
> > 
> > Example usage:
> > 
> >   # We have recipes-* directories, add to BBFILES
> >   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
> > 	${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
> > 	${LAYERDIR}/recipes-*/*/*.bbappend"
> > 
> >   # Several of our *.bbappends might be for layers that are not
> >   # guaranteed to be present. Don't warn or even error out because
> >   # of those.
> >   BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-
> > bbappends/recipes-*/*/*.bbappend"
> 
> I've been giving this some thought. The freescale layer approach does
> have ordering issues which I don't like. Equally, Chris' comment about
> this is also a valid concern. Experience suggests saying the layer
> maintainer needs to be careful doesn't work in practise.
> 
> I'm wondering if there is a hybrid solution if we extend this syntax a
> little, e.g.:
> 
> BBFILES_DYNAMIC += "\
>     XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
>     YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
> "
> 
> so that the code only applies these BBFILES entries if the layer named
> in the first parameter is present?
>
> This would happen at the end of parsing so ordering becomes a moot
> point. The direction you stack BBFILES_DYANMIC in would determin
> bbappend application order.

While the usecase above was for .bbappends, would the same also make
sense for .bb files, and would that change the implementation? Perhaps
not.

The use case would be a .bb which depends on something in a layer which
may or may not be present. It's a bit harder, though, because it's not
always as clear as for the .bbappend case where something comes from
that a recipe depends on.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH] cooker.py: allow dangling bbappends if explicitly whitelisted

[Mark Hatle]

On 6/2/17 11:14 AM, Patrick Ohly wrote:
> On Fri, 2017-06-02 at 13:58 +0100, Richard Purdie wrote:
>> On Mon, 2017-05-22 at 12:50 +0200, Patrick Ohly wrote:
>>> Having a .bbappend without corresponding .bb file triggers an error
>>> or
>>> at least warning, depending on the global
>>> BB_DANGLINGAPPENDS_WARNONLY.
>>>
>>> Some layers (for example, meta-freescale) avoid that message by only
>>> adding .bbappends to the BBFILES when the layers they apply to are
>>> present. Others (like intel-iot-refkit) avoid such .bbappends by
>>> falling back to global assignments with _pn-<recipe> as override.
>>> Both
>>> is complicated.
>>>
>>> Now the warning code checks BBAPPENDS_DANGLING_WHITELIST and ignores
>>> all bbappends which match a file pattern in that list. This is an
>>> easier way to have bbappends which may or may not apply to an
>>> existing recipe.
>>>
>>> Example usage:
>>>
>>>   # We have recipes-* directories, add to BBFILES
>>>   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
>>> 	${LAYERDIR}/optional-bbappends/recipes-*/*/*.bbappend \
>>> 	${LAYERDIR}/recipes-*/*/*.bbappend"
>>>
>>>   # Several of our *.bbappends might be for layers that are not
>>>   # guaranteed to be present. Don't warn or even error out because
>>>   # of those.
>>>   BBAPPENDS_DANGLING_WHITELIST += "${LAYERDIR}/optional-
>>> bbappends/recipes-*/*/*.bbappend"
>>
>> I've been giving this some thought. The freescale layer approach does
>> have ordering issues which I don't like. Equally, Chris' comment about
>> this is also a valid concern. Experience suggests saying the layer
>> maintainer needs to be careful doesn't work in practise.
>>
>> I'm wondering if there is a hybrid solution if we extend this syntax a
>> little, e.g.:
>>
>> BBFILES_DYNAMIC += "\
>>     XXXX:${LAYERDIR}/optional-bbappends-XXX/recipes-*/*/*.bbappend
>>     YYYY:${LAYERDIR}/optional-bbappends-YYY/recipes-*/*/*.bbappend
>> "
>>
>> so that the code only applies these BBFILES entries if the layer named
>> in the first parameter is present?
>>
>> This would happen at the end of parsing so ordering becomes a moot
>> point. The direction you stack BBFILES_DYANMIC in would determin
>> bbappend application order.
> 
> While the usecase above was for .bbappends, would the same also make
> sense for .bb files, and would that change the implementation? Perhaps
> not.
> 
> The use case would be a .bb which depends on something in a layer which
> may or may not be present. It's a bit harder, though, because it's not
> always as clear as for the .bbappend case where something comes from
> that a recipe depends on.
> 

My assumption is the (new) BBFILES_DYNAMIC should have the same format as the
existing BBFILES.  So it will work for both bbappend and bb files.

--Mark

[PATCH] cookerdata: Add support for BBFILES_DYNAMIC

[Patrick Ohly]

BBFILES_DYNAMIC can be used to activate content only when some other
layers are present. The other layers are identified by the collections
that they define.

The main use case is to avoid .bbappends without the corresponding .bb
file in layers that want to modify other layers via .bbappends without
introducing a hard dependency on those other layers. .bb files could
also be handled via BBFILES_DYNAMIC.

Entries in BBFILES_DYNAMIC must have the form <collection
name>:<filename pattern>. Example usage:
 BBFILES_DYNAMIC += " \
     clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \
     core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \
 "

Parsing is aborted when invalid entries are found with an error
message like this:

 ERROR: BBFILES_DYNAMIC entries must be of the form <collection name>:<filename pattern>, not:
     /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend
     /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend

Based on a patch by Richard Purdie.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 lib/bb/cookerdata.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 722d860..aa96f54 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -346,6 +346,20 @@ class CookerDataBuilder(object):
             data.delVar('LAYERDIR_RE')
             data.delVar('LAYERDIR')
 
+            bbfiles_dynamic = (data.getVar('BBFILES_DYNAMIC') or "").split()
+            collections = (data.getVar('BBFILE_COLLECTIONS') or "").split()
+            invalid = []
+            for entry in bbfiles_dynamic:
+                parts = entry.split(":", 1)
+                if len(parts) != 2:
+                    invalid.append(entry)
+                    continue
+                l, f = parts
+                if l in collections:
+                    data.appendVar("BBFILES", " " + f)
+            if invalid:
+                bb.fatal("BBFILES_DYNAMIC entries must be of the form <collection name>:<filename pattern>, not:\n    %s" % "\n    ".join(invalid))
+
         if not data.getVar("BBPATH"):
             msg = "The BBPATH variable is not set"
             if not layerconf:

base-commit: 705ab252e631903e6d2e46202b419a9e8adcd861
-- 
git-series 0.9.1