New warning

New warning

[Gary Thomas]

For a while now, I've been seeing warnings like this:
   WARNING: Unable to get checksum for some-pkg SRC_URI entry some.patch: file could not be found
This happens when I have two intertwined layers - I have a recipe in layer1
which references a file satisfied only by layer2.  This is how I handle common
recipes with many targeted [BSP] layers in addition.

Why is this warning happening?
What can I do to suppress it?

I suppose that someday, these warnings might get elevated to errors
so I'd really like to know how to fix them.

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Khem Raj]

On Wed, Jun 20, 2012 at 12:12 PM, Gary Thomas <gary@mlbassoc.com> wrote:
> This happens when I have two intertwined layers - I have a recipe in layer1
> which references a file satisfied only by layer2.

why not move the reference from layer1 to layer2 as well ?

Re: New warning

[Gary Thomas]

On 2012-06-20 15:55, Khem Raj wrote:
> On Wed, Jun 20, 2012 at 12:12 PM, Gary Thomas<gary@mlbassoc.com>  wrote:
>> This happens when I have two intertwined layers - I have a recipe in layer1
>> which references a file satisfied only by layer2.
>
> why not move the reference from layer1 to layer2 as well ?

Layer1 holds the recipe and I find it more informative to have the
reference there - that way the .bbappends in layer2 _must_ define
it and the dependency is obvious since the recipe will fail without
this file being specified.

My use case is layer1 has a generic kernel recipe and layer2 (BSP)
has a platform specific patch for that kernel.  I have other use
cases, but this is indicative of how I view the layering.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Andrea Adami]

On Thu, Jun 21, 2012 at 12:51 AM, Gary Thomas <gary@mlbassoc.com> wrote:
> On 2012-06-20 15:55, Khem Raj wrote:
>>
>> On Wed, Jun 20, 2012 at 12:12 PM, Gary Thomas<gary@mlbassoc.com>  wrote:
>>>
>>> This happens when I have two intertwined layers - I have a recipe in
>>> layer1
>>> which references a file satisfied only by layer2.
>>
>>
>> why not move the reference from layer1 to layer2 as well ?
>
>
> Layer1 holds the recipe and I find it more informative to have the
> reference there - that way the .bbappends in layer2 _must_ define
> it and the dependency is obvious since the recipe will fail without
> this file being specified.
>
> My use case is layer1 has a generic kernel recipe and layer2 (BSP)
> has a platform specific patch for that kernel.  I have other use
> cases, but this is indicative of how I view the layering.
>

It seems this happens as well with recipes self-contained in a single layer.
I see 2 warnings about SRC_URI chksum of  defconfig so I suspect about
two recipes [1]

The defconfigs are nested in subdirs, one per $MACHINE.

Regards

Andrea


[1] http://cgit.openembedded.org/meta-handheld/tree/recipes-kernel/linux/linux_3.2.bb
 and 3.1

>
> --
> ------------------------------------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------------------------------------
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky

Re: New warning

[Paul Eggleton]

On Wednesday 20 June 2012 16:51:20 Gary Thomas wrote:
> On 2012-06-20 15:55, Khem Raj wrote:
> > On Wed, Jun 20, 2012 at 12:12 PM, Gary Thomas<gary@mlbassoc.com>  wrote:
> >> This happens when I have two intertwined layers - I have a recipe in
> >> layer1
> >> which references a file satisfied only by layer2.
> > 
> > why not move the reference from layer1 to layer2 as well ?
> 
> Layer1 holds the recipe and I find it more informative to have the
> reference there - that way the .bbappends in layer2 _must_ define
> it and the dependency is obvious since the recipe will fail without
> this file being specified.
>
> My use case is layer1 has a generic kernel recipe and layer2 (BSP)
> has a platform specific patch for that kernel.  I have other use
> cases, but this is indicative of how I view the layering.

Well, that's not unreasonable I suppose. However I've just tested a trivial 
example of cross-layer file references here and it does not produce a warning. 
In any case, the code that does the checksums uses the same code as do_fetch 
when it locates the files, so I'm a bit puzzled as to how it would not be able 
to find the file in the first instance and it would when it came to fetch time.

FWIW, Andrea and I figured out that his reported case was caused by some 
recipes that only provided a file for specific machines which did not include 
the current MACHINE, but the recipes were not marked with COMPATIBLE_MACHINE, 
so the warning there was legitimate. Gary, is it possible something similar is 
happening for your layer structure? If not, can you provide some more 
information that might help me reproduce the warning?

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: New warning

[Gary Thomas]

On 2012-06-22 03:44, Paul Eggleton wrote:
> On Wednesday 20 June 2012 16:51:20 Gary Thomas wrote:
>> On 2012-06-20 15:55, Khem Raj wrote:
>>> On Wed, Jun 20, 2012 at 12:12 PM, Gary Thomas<gary@mlbassoc.com>   wrote:
>>>> This happens when I have two intertwined layers - I have a recipe in
>>>> layer1
>>>> which references a file satisfied only by layer2.
>>>
>>> why not move the reference from layer1 to layer2 as well ?
>>
>> Layer1 holds the recipe and I find it more informative to have the
>> reference there - that way the .bbappends in layer2 _must_ define
>> it and the dependency is obvious since the recipe will fail without
>> this file being specified.
>>
>> My use case is layer1 has a generic kernel recipe and layer2 (BSP)
>> has a platform specific patch for that kernel.  I have other use
>> cases, but this is indicative of how I view the layering.
>
> Well, that's not unreasonable I suppose. However I've just tested a trivial
> example of cross-layer file references here and it does not produce a warning.
> In any case, the code that does the checksums uses the same code as do_fetch
> when it locates the files, so I'm a bit puzzled as to how it would not be able
> to find the file in the first instance and it would when it came to fetch time.
>
> FWIW, Andrea and I figured out that his reported case was caused by some
> recipes that only provided a file for specific machines which did not include
> the current MACHINE, but the recipes were not marked with COMPATIBLE_MACHINE,
> so the warning there was legitimate. Gary, is it possible something similar is
> happening for your layer structure? If not, can you provide some more
> information that might help me reproduce the warning?

I see now what's causing the warning, but I don't see a good way to fix it.

I have my own DISTRO, similar to meta-yocto, plus a number of BSP layers.  Only
one BSP layer will be selected in bblayers.conf.

My meta tree looks something like this:
   /local/poky-multi/
     meta/
     meta-yocto/
     meta-ti/
     meta-MYDISTRO/
     meta-BSPa/
     meta-BSPb/
     ...
     meta-BSPn/
A typical build will have this for bblayers.conf:
   BBLAYERS = " \
     /local/poky-multi/meta \
     /local/poky-multi/meta-MYDISTRO \
     /local/poky-multi/meta-BSPf \
     /local/poky-multi/meta-ti \
   "

The problem is that my generic layer1 has a main recipe which is used by only
a few of the BSP layers (layerBSPa, layerBSPb, etc).  For example, if my configuration
selects meta-BSPf, that BSP may not need to use the recipe in question, but it's
still in meta-MYDISTRO.  More concretely, meta-MYDISTRO defines a recipe 'fpga-tool'
which needs BSP specific setup files like this:
   % find /local/poky-multi -name "fpga-tools*.bb*"
   /local/poky-multi/meta-MYDISTRO/packages/fpga/fpga-tools_0.0.bb
   /local/poky-multi/meta-BSPc/packages/fpga-tools_0.0.bbappend
   /local/poky-multi/meta-BSPd/packages/fpga-tools_0.0.bbappend
Notice that there is no fpga-tools setup in meta-BSPa.  If I'm building for
BSPa, I'll get this warning:
   WARNING: Unable to get checksum for fpga-tool SRC_URI entry fpga_gpio_layout.h: file could not be found
   WARNING: Unable to get checksum for fpga-tool SRC_URI entry video_mux.vme: file could not be found

I can see how to fix this warning by defining those files with dummy versions in meta-MYDISTRO

It gets more complicated as my kernel recipe (in meta-MYDISTRO) references files like
   SRC_URI += "file://${MACHINE}.patch"
I don't see any way to define a dummy version of this.

Is this clear now?  Any ideas how to clean it up?

Thanks for any ideas

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Tomas Frydrych]

On 22/06/12 12:49, Gary Thomas wrote:
> It gets more complicated as my kernel recipe (in meta-MYDISTRO)
> references files like
>   SRC_URI += "file://${MACHINE}.patch"
> I don't see any way to define a dummy version of this.
> 
> Is this clear now?  Any ideas how to clean it up?

Seems to me the basic problem is the circular dependency between your
distro and bsp layers; would it not work if you removed the SRC_URI +=
from the distro recipes and add a bbappend to each of your bsb layers?

Tomas

Re: New warning

[Paul Eggleton]

On Friday 22 June 2012 05:49:55 Gary Thomas wrote:
> I see now what's causing the warning, but I don't see a good way to fix it.
> 
> I have my own DISTRO, similar to meta-yocto, plus a number of BSP layers. 
> Only one BSP layer will be selected in bblayers.conf.
> 
> My meta tree looks something like this:
>    /local/poky-multi/
>      meta/
>      meta-yocto/
>      meta-ti/
>      meta-MYDISTRO/
>      meta-BSPa/
>      meta-BSPb/
>      ...
>      meta-BSPn/
> A typical build will have this for bblayers.conf:
>    BBLAYERS = " \
>      /local/poky-multi/meta \
>      /local/poky-multi/meta-MYDISTRO \
>      /local/poky-multi/meta-BSPf \
>      /local/poky-multi/meta-ti \
>    "
> 
> The problem is that my generic layer1 has a main recipe which is used by
> only a few of the BSP layers (layerBSPa, layerBSPb, etc).  For example, if
> my configuration selects meta-BSPf, that BSP may not need to use the recipe
> in question, but it's still in meta-MYDISTRO.  More concretely,
> meta-MYDISTRO defines a recipe 'fpga-tool' which needs BSP specific setup
> files like this:
>    % find /local/poky-multi -name "fpga-tools*.bb*"
>    /local/poky-multi/meta-MYDISTRO/packages/fpga/fpga-tools_0.0.bb
>    /local/poky-multi/meta-BSPc/packages/fpga-tools_0.0.bbappend
>    /local/poky-multi/meta-BSPd/packages/fpga-tools_0.0.bbappend
> Notice that there is no fpga-tools setup in meta-BSPa.  If I'm building for
> BSPa, I'll get this warning:
>    WARNING: Unable to get checksum for fpga-tool SRC_URI entry
> fpga_gpio_layout.h: file could not be found WARNING: Unable to get checksum
> for fpga-tool SRC_URI entry video_mux.vme: file could not be found
> 
> I can see how to fix this warning by defining those files with dummy
> versions in meta-MYDISTRO

I guess if I were doing it I would not make the BSP layers depend upon the 
distro layer, as Tomas was suggesting. In any case though I suspect the best 
way to handle this is to use COMPATIBLE_MACHINE which you set to some invalid 
value in your distro layer recipe and then append to that value with the 
actual MACHINE in the bbappend in each BSP, e.g.


meta-MYDISTRO recipes/fpga/fpga-tools_0.0.bb:
...
COMPATIBLE_MACHINE = "not-valid"
...


meta-BSPc recipes/fpga/fpga-tools_0.0.bbappend:
...
COMPATIBLE_MACHINE .= "|machinec"
...


meta-BSPd recipes/fpga/fpga-tools_0.0.bbappend:
...
COMPATIBLE_MACHINE .= "|machined"
...


Then, the recipe will be skipped if it is not available for the current 
MACHINE.

> It gets more complicated as my kernel recipe (in meta-MYDISTRO) references
> files like SRC_URI += "file://${MACHINE}.patch"
> I don't see any way to define a dummy version of this.

Rather than ${MACHINE}.patch, the traditional way to do this is to have a 
generic name for the file and then use a MACHINE-named subdirectory within your 
BSP layers to define the machine-specific version. You can always make the dummy 
patch not apply or specifically check for some aspect added by the dummy in 
do_configure.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: New warning

[Paul Eggleton]

On Friday 22 June 2012 14:01:39 Paul Eggleton wrote:
> Rather than ${MACHINE}.patch, the traditional way to do this is to have a
> generic name for the file and then use a MACHINE-named subdirectory within
> your BSP layers to define the machine-specific version. You can always make
> the dummy patch not apply or specifically check for some aspect added by
> the dummy in do_configure.

(Assuming you use a dummy file that is - with the COMPATIBLE_MACHINE fix I 
suggested this is not necessary; this was from an earlier edit of my message.)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: New warning

[Gary Thomas]

On 2012-06-22 07:05, Paul Eggleton wrote:
> On Friday 22 June 2012 14:01:39 Paul Eggleton wrote:
>> Rather than ${MACHINE}.patch, the traditional way to do this is to have a
>> generic name for the file and then use a MACHINE-named subdirectory within
>> your BSP layers to define the machine-specific version. You can always make
>> the dummy patch not apply or specifically check for some aspect added by
>> the dummy in do_configure.
>
> (Assuming you use a dummy file that is - with the COMPATIBLE_MACHINE fix I
> suggested this is not necessary; this was from an earlier edit of my message.)

I like the idea of using COMPATIBLE_MACHINE - I think I can make that work well.

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Khem Raj]

On Fri, Jun 22, 2012 at 4:49 AM, Gary Thomas <gary@mlbassoc.com> wrote:
>
>
> It gets more complicated as my kernel recipe (in meta-MYDISTRO) references
> files like
>  SRC_URI += "file://${MACHINE}.patch"
> I don't see any way to define a dummy version of this.
>
> Is this clear now?  Any ideas how to clean it up?

well you should be fine if file://${MACHINE}.patch is available for
every machine you build.

You can also call it same name e.g. platform.patch and then put
platform.patch in files/<MACHINE>/ directory


But If I were you I would put these patches in a bbappend in same
layer where they come from

New warning

[Gary Thomas]

I'm getting these new warnings, even with a build from scratch.  This is from
poky master rev 7c39c87d52c20e47cf90275a16e4517a296c8388

WARNING: The recipe is trying to install files into a shared area when those files already exist. Those files are:
    /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/include/python2.7/pyconfig.h
    /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so
    /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so.1.0
    /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/python2.7/config/Makefile

Do they hurt anything?  Why am I getting them now, what's changed, etc?

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Richard Purdie]

On Tue, 2012-10-02 at 09:34 -0600, Gary Thomas wrote:
> I'm getting these new warnings, even with a build from scratch.  This is from
> poky master rev 7c39c87d52c20e47cf90275a16e4517a296c8388
> 
> WARNING: The recipe is trying to install files into a shared area when those files already exist. Those files are:
>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/include/python2.7/pyconfig.h
>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so
>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so.1.0
>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/python2.7/config/Makefile
> 
> Do they hurt anything?  Why am I getting them now, what's changed, etc?

I fixed the code that was supposed to be generate these messages and we
started seeing them. Its a new type of check added relatively recently
but was accidentally disabled in its original commit.

The python recipe does something particularly evil, the do_compile pokes
file into STAGING_DIR_* directly for various reasons.

The do_install then does save off the files correctly but there is an
overwrite situation occurring there and I don't consider the recipe to
be doing something particularly nice.

So there is an issue there, it won't break your build but we should
really do something better.

Cheers,

Richard

Re: New warning

[Khem Raj]

On Tue, Oct 2, 2012 at 8:43 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Tue, 2012-10-02 at 09:34 -0600, Gary Thomas wrote:
>> I'm getting these new warnings, even with a build from scratch.  This is from
>> poky master rev 7c39c87d52c20e47cf90275a16e4517a296c8388
>>
>> WARNING: The recipe is trying to install files into a shared area when those files already exist. Those files are:
>>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/include/python2.7/pyconfig.h
>>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so
>>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/libpython2.7.so.1.0
>>     /home/local/p81_build/tmp/sysroots/cobra3530p81/usr/lib/python2.7/config/Makefile
>>
>> Do they hurt anything?  Why am I getting them now, what's changed, etc?
>
> I fixed the code that was supposed to be generate these messages and we
> started seeing them. Its a new type of check added relatively recently
> but was accidentally disabled in its original commit.
>
> The python recipe does something particularly evil, the do_compile pokes
> file into STAGING_DIR_* directly for various reasons.
>
> The do_install then does save off the files correctly but there is an
> overwrite situation occurring there and I don't consider the recipe to
> be doing something particularly nice.
>
> So there is an issue there, it won't break your build but we should
> really do something better.
>

yeah probably we need two stage approach may be a python-initial and
then real python.

> Cheers,
>
> Richard
>
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky

New warning

[Gary Thomas]

I'm seeing a lot of these:
   NOTE: /local/poky-cutting-edge/meta-xyz/.../some-pkg_2.38.0.bb: base_contains is deprecated, please use 
bb.utils.contains instead.

This is all well and good and I'll clean them up as I go.  That
said, why is the same NOTE printed [at least] twice for every recipe
even when the file contains 'base_contains' only once?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: New warning

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

On Tue, Jun 7, 2016 at 7:55 PM, Gary Thomas <gary@mlbassoc.com> wrote:

> I'm seeing a lot of these:
>   NOTE: /local/poky-cutting-edge/meta-xyz/.../some-pkg_2.38.0.bb:
> base_contains is deprecated, please use bb.utils.contains instead.
>
> This is all well and good and I'll clean them up as I go.  That
> said, why is the same NOTE printed [at least] twice for every recipe
> even when the file contains 'base_contains' only once?
>

I haven't checked the code, but the most likely explanation is the error is
a parse or expansion time one, and bitbake parses the recipe anew when
running each task, after the up front parse which is used to generate the
runqueue. *shrug*. It's a downside of showing messages in anything but task
execution context.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1452 bytes --]