Re: Layer Priority with Wildcard .bbappend Files

[Bob Cochran <yocto@mindchasers.com>]

On 11/25/2014 06:12 AM, Paul Eggleton wrote:
> Hi Bob / Nick,
>
> On Monday 24 November 2014 16:25:58 Bob Cochran wrote:
>> On 11/24/2014 03:22 PM, Stevens, Nick wrote:
>>> I think I've encountered a bug with how multiple bbappend files are
>>> processed when one of the bbappends contains a filename wildcard, but I
>>> want to make sure there's not something I'm missing before filing a bug
>>> report.>
>>> I have a BSP layer and a customization layer that are based on the OE/poky
>>> Daisy release. The output of `bitbake-layers show-layers` looks like:
>>>       layer                 path                 priority
>>>       =====================================================
>>>       meta                  poky/meta            5
>>>       meta-yocto            poky/meta-yocto      5
>>>       meta-yocto-bsp        poky/meta-yocto-bsp  5
>>>       ...ellided...
>>>       meta-bsp              meta-bsp             6
>>>       meta-custom           meta-custom          8
>>>
>>> In meta-bsp there is a bbappend for base-files named base-
>>> files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -
>>> pretty straightforward:
>>>       FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>>>       SRC_URI += "file://sysctl.conf"
>>>       do_install_append() {
>>>
>>>           install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
>>>
>>>       }
>>>
>>> Now what I want to do is add a file called base-files_%.bbappend to meta-
>>> custom. It has its own version of sysctl.conf, and the recipe looks like
>>> this:
>>>       FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>>>
>>> Here's where things get weird. If I run `bitbake -e base-files` and pull
>>> out the section for FILESEXTRAPATHS, this is what I get (note that I've
>>> stripped out the huge absolute paths to make this easier to read):
>>>       # $FILESEXTRAPATHS [5 operations]
>>>       #   set poky/meta/conf/documentation.conf:172
>>>       #     [doc] "Extends the search path the OpenEmbedded build system
>>>       uses when looking for files and patches as it processes recipes and
>>>       append files." #   _prepend
>>>       meta-custom/recipes-core/base-files/base-files_%.bbappend:6 #
>>>       "meta-custom/recipes-core/base-files/base-files:"
>>>       #   _prepend
>>>       meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
>>>       #     "meta-bsp/recipes-core/base-files/base-files:"
>>>       #   set data_smart.py:432 [finalize]
>>>       #     "meta-custom/recipes-core/base-files/base-files:"
>>>       #   set data_smart.py:432 [finalize]
>>>       #
>>>       "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
>>>       e/base-files/base-files:" # computed:
>>>       #
>>>       "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
>>>       e/base-files/base-files:"
>>>       FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-cu
>>>       stom/recipes-core/base-files/base-files:">
>>> For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS,
>>> even though meta-custom has a higher priority!
>>
>> I also have some questions about how FILESEXTRAPATHS is supposed to work
>> with append files and layer priorities.
>>
>> Does a higher layer priority mean that the FILESEXTRAPATHS operation
>> should occur first?
>
> No. bbappends are parsed in ascending layer priority order. It wouldn't make
> sense to do it in the reverse - you want the higher priority layer's settings
> to take precedence
>
>> If this is the case, then a lower priority layer prepend will appear to the
>> left of the higher layer prepend since it runs later, and this will probably
>> not provide the result you want.
>
> The highest priority layer's bbappend will prepend last, thus anything it
> prepends will be first in the list.
>
>> The prepend and layer logic seems messy to me.  I would like to have a
>> way to write my custom layer and specify that what I include in my
>> SRC_URI in each recipe is definitive and can not be overwritten.
>> Suggestions on how to best accomplish this will be greatly appreciated.
>
> I think this is pretty much already the case. Is the problem that you can't
> follow the current behaviour, or that you don't completely trust the layers
> you are pulling in?


Thanks Paul.  Something is amiss when I try to prepend my recipes. I 
suspect it has something to do with OVERRIDE interactions.    I'm 
digging and will report later.



>
>> I have been trying to accomplish this with FILESOVERRIDES, but this
>> logic seems counterintuitive since  DISTROOVERRIDES take precedence over
>> MACHINEOVERRIDES in building the file search path during the unpack task.
>>
>> It seems to me that the file search path should be built using
>> FILESOVERRIDES from left to right:  TRANSLATED_TARGET_ARCH,
>> MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but
>> this isn't what I'm seeing.  Still investigating how it's all put
>> together...
>>
>>
>> I verified this by building an image - the sysctl.conf that ends up in
>> the final image is the one from meta-bsp, not the one from meta-custom.
>> But if I switch the name of base-files_%.bbappend in meta-custom to
>> base-files_3.0.14.bbappend, this is what I get:
>>>       # $FILESEXTRAPATHS [5 operations]
>>>       #   set poky/meta/conf/documentation.conf:172
>>>       #     [doc] "Extends the search path the OpenEmbedded build system
>>>       uses when looking for files and patches as it processes recipes and
>>>       append files." #   _prepend
>>>       meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3 #
>>>       "meta-bsp/recipes-core/base-files/base-files:"
>>>       #   _prepend
>>>       meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6 #
>>>         "meta-custom/recipes-core/base-files/base-files:"
>>>       #   set data_smart.py:432 [finalize]
>>>       #     "meta-bsp/recipes-core/base-files/base-files:"
>>>       #   set data_smart.py:432 [finalize]
>>>       #
>>>       "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
>>>       e/base-files/base-files:" # computed:
>>>       #
>>>       "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
>>>       e/base-files/base-files:"
>>>       FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta
>>>       -bsp/recipes-core/base-files/base-files:">
>>> And now the sysctl.conf file is being pulled from meta-custom.
>
> That absolutely should not happen. If you change it back does it revert to the
> version from meta-bsp? The wildcarding should not change the order in which
> the files are applied. The only corner case I can think of would be if the
> layer priority values are the same number - this isn't the case is it?
>
> Cheers,
> Paul
>