sstate black hole?

sstate black hole?

[Gary Thomas]

I'm building for multiple ARM i.MX6 platforms.  These have
the same SoC, but slightly different peripherals. As far as
I can tell, they should be able to share everything except
for a few ${MACHINE} specific packages, e.g. the kernel and
u-boot.

Sadly, that doesn't seem to be the case.  The architecture
specific packages are being split into two categories - plain
ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
For example, after building a complete image (on the order of
core-image-sato), I have this split:
$ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
attr                 gstreamer                  libtheora              ncurses               startup-notification
base-passwd          gstreamer1.0               libtirpc               neon                  strace
    ...
gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
gst-meta-base        libpthread-stubs           mkfontscale            shadow

$ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio

It's the second category that is causing problems.  They do not
seem to end up in any shareable sstate at all.  If I try to rebuild
using only sstate, i.e. build my complete image to success, then
remove 'tmp' and rebuild, using the sstate-cache from the first go,
all of the above packages (alsa-lib, ..., xserver-xorg) are all
rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
but they are never reused from it.

What would make this happen?  How can I prevent it?

As is, sstate is not really shareable between these i.MX6 targets
as so much is being rebuilt all the time...

Any ideas or pointers gladly welcomed.

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 3044 bytes --]

On Tue, Apr 7, 2015 at 7:52 AM, Gary Thomas <gary@mlbassoc.com> wrote:

> I'm building for multiple ARM i.MX6 platforms.  These have
> the same SoC, but slightly different peripherals. As far as
> I can tell, they should be able to share everything except
> for a few ${MACHINE} specific packages, e.g. the kernel and
> u-boot.
>
> Sadly, that doesn't seem to be the case.  The architecture
> specific packages are being split into two categories - plain
> ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
> For example, after building a complete image (on the order of
> core-image-sato), I have this split:
> $ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
> acl                  gst-player                 libsamplerate0
>  modutils-initscripts  shadow-sysroot
> alsa-utils           gst-plugins-bad            libsm
> mpeg2dec              shared-mime-info
> apmd                 gst-plugins-good           libsndfile1
> mplayer2              speex
> atk                  gst-plugins-ugly           libsoup-2.4
> mtdev                 sqlite3
> attr                 gstreamer                  libtheora
> ncurses               startup-notification
> base-passwd          gstreamer1.0               libtirpc
>  neon                  strace
>    ...
> gst-ffmpeg           libpostproc                matchbox-wm
> scrnsaverproto        zlib
> gst-fluendo-mpegmux  libproxy                   mkfontdir
> settings-daemon
> gst-meta-base        libpthread-stubs           mkfontscale
> shadow
>
> $ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
> alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser
>  libsdl      xf86-video-imxfb-vivante
> cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa
>       xserver-xorg
> firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu
>  pulseaudio
>
> It's the second category that is causing problems.  They do not
> seem to end up in any shareable sstate at all.  If I try to rebuild
> using only sstate, i.e. build my complete image to success, then
> remove 'tmp' and rebuild, using the sstate-cache from the first go,
> all of the above packages (alsa-lib, ..., xserver-xorg) are all
> rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
> but they are never reused from it.
>
> What would make this happen?  How can I prevent it?
>
> As is, sstate is not really shareable between these i.MX6 targets
> as so much is being rebuilt all the time...
>

bitbake -S printdiff <target> (e.g. a specific recipe, your image,
whatever) will tell you why the sstate wasn't used, by finding the ones
that are in SSTATE_DIR which most closely match the current configuration,
and displaying a detailed delta between the two. You might need
https://gist.github.com/kergoth/3713d779c14dc8b98f36.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 4038 bytes --]

Re: sstate black hole?

[Gary Thomas]

On 2015-04-07 09:27, Christopher Larson wrote:
>
> On Tue, Apr 7, 2015 at 7:52 AM, Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>> wrote:
>
>     I'm building for multiple ARM i.MX6 platforms.  These have
>     the same SoC, but slightly different peripherals. As far as
>     I can tell, they should be able to share everything except
>     for a few ${MACHINE} specific packages, e.g. the kernel and
>     u-boot.
>
>     Sadly, that doesn't seem to be the case.  The architecture
>     specific packages are being split into two categories - plain
>     ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
>     For example, after building a complete image (on the order of
>     core-image-sato), I have this split:
>     $ ls tmp/work/cortexa9hf-vfp-neon-__amltd-linux-gnueabi/
>     acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
>     alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
>     apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
>     atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
>     attr                 gstreamer                  libtheora              ncurses               startup-notification
>     base-passwd          gstreamer1.0               libtirpc               neon                  strace
>         ...
>     gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
>     gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
>     gst-meta-base        libpthread-stubs           mkfontscale            shadow
>
>     $ ls tmp/work/cortexa9hf-vfp-neon-__mx6qdl-amltd-linux-gnueabi/
>     alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
>     cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
>     firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio
>
>     It's the second category that is causing problems.  They do not
>     seem to end up in any shareable sstate at all.  If I try to rebuild
>     using only sstate, i.e. build my complete image to success, then
>     remove 'tmp' and rebuild, using the sstate-cache from the first go,
>     all of the above packages (alsa-lib, ..., xserver-xorg) are all
>     rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
>     but they are never reused from it.
>
>     What would make this happen?  How can I prevent it?
>
>     As is, sstate is not really shareable between these i.MX6 targets
>     as so much is being rebuilt all the time...
>
>
> bitbake -S printdiff <target> (e.g. a specific recipe, your image, whatever) will tell you why the sstate wasn't used, by finding the ones that are in SSTATE_DIR which most closely
> match the current configuration, and displaying a detailed delta between the two. You might need https://gist.github.com/kergoth/3713d779c14dc8b98f36.

Sorry, that didn't seem to do anything (I didn't apply your
patch since I'm not looking at any native or sdk packages).

Here's what I got:
   $ bitbake -S printdiffs xserver-xorg
   ...
   NOTE: Preparing RunQueue
   NOTE: Reparsing files to collect dependency data
   Writing locked sigs to /local/imx6_2015-03-26/locked-sigs.inc
   NOTE: Tasks Summary: Attempted 0 tasks of which 0 didn't need to be rerun and all succeeded.

No other output.

I can see [manually] that there are a number of cache entries, e.g.

$ find sstate-cache/ -name "*xserver-xorg*package.tgz"
sstate-cache/48/sstate:xserver-xorg:cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi:1.16.3:r0:cortexa9hf-vfp-neon-mx6qdl:3:487ec958840dd9ed48ff3da86e2dabdb_package.tgz
sstate-cache/4f/sstate:xserver-xorg:cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi:1.16.3:r0:cortexa9hf-vfp-neon-mx6qdl:3:4f7a493630c7f29762714430ea6be4d1_package.tgz
sstate-cache/80/sstate:xserver-xorg:cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi:1.16.3:r0:cortexa9hf-vfp-neon-mx6qdl:3:8041b9f7bda788038e011829939e2049_package.tgz
sstate-cache/2d/sstate:xserver-xorg:cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi:1.16.3:r0:cortexa9hf-vfp-neon-mx6qdl:3:2de173b1c4875889f124ce4560f42ed7_package.tgz
sstate-cache/b4/sstate:xserver-xorg:cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi:1.16.3:r0:cortexa9hf-vfp-neon-mx6qdl:3:b492f53c05ca6cbeb51b2e574dd60495_package.tgz

So I would think that something would be printed.  Do I need to do something
else to make this work, e.g. cleaning the target package, etc?

Note: I looked at the 'locked-sigs.inc' that was generated and I noticed
that there is no section for the recipes I'm interested in:
   $ grep ^SIG locked-sigs.inc
   SIGGEN_LOCKEDSIGS_t-imx6qsabresd = "\
   SIGGEN_LOCKEDSIGS_t-x86-64 = "\
   SIGGEN_LOCKEDSIGS_t-x86-64-arm = "\
   SIGGEN_LOCKEDSIGS_t-cortexa9hf-vfp-neon = "\
   SIGGEN_LOCKEDSIGS_TYPES_imx6qsabresd = "t-imx6qsabresd t-x86-64 t-x86-64-arm t-cortexa9hf-vfp-neon"

I would think there would need to be a "SIGGEN_LOCKEDSIGS_t-cortexa9hf-vfp-neon-mx6qdl" section?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2794 bytes --]

On Tue, Apr 07, 2015 at 08:52:36AM -0600, Gary Thomas wrote:
> I'm building for multiple ARM i.MX6 platforms.  These have
> the same SoC, but slightly different peripherals. As far as
> I can tell, they should be able to share everything except
> for a few ${MACHINE} specific packages, e.g. the kernel and
> u-boot.
> 
> Sadly, that doesn't seem to be the case.  The architecture
> specific packages are being split into two categories - plain
> ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
> For example, after building a complete image (on the order of
> core-image-sato), I have this split:
> $ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
> acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
> alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
> apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
> atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
> attr                 gstreamer                  libtheora              ncurses               startup-notification
> base-passwd          gstreamer1.0               libtirpc               neon                  strace
>     ...
> gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
> gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
> gst-meta-base        libpthread-stubs           mkfontscale            shadow
> 
> $ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
> alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
> cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
> firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio
> 
> It's the second category that is causing problems.  They do not
> seem to end up in any shareable sstate at all.  If I try to rebuild
> using only sstate, i.e. build my complete image to success, then
> remove 'tmp' and rebuild, using the sstate-cache from the first go,
> all of the above packages (alsa-lib, ..., xserver-xorg) are all
> rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
> but they are never reused from it.
> 
> What would make this happen?  How can I prevent it?
> 
> As is, sstate is not really shareable between these i.MX6 targets
> as so much is being rebuilt all the time...
> 
> Any ideas or pointers gladly welcomed.

Try openembedded-core/scripts/sstate-diff-machines.sh
to see why.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: sstate black hole?

[Gary Thomas]

On 2015-04-07 10:19, Martin Jansa wrote:
> On Tue, Apr 07, 2015 at 08:52:36AM -0600, Gary Thomas wrote:
>> I'm building for multiple ARM i.MX6 platforms.  These have
>> the same SoC, but slightly different peripherals. As far as
>> I can tell, they should be able to share everything except
>> for a few ${MACHINE} specific packages, e.g. the kernel and
>> u-boot.
>>
>> Sadly, that doesn't seem to be the case.  The architecture
>> specific packages are being split into two categories - plain
>> ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
>> For example, after building a complete image (on the order of
>> core-image-sato), I have this split:
>> $ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
>> acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
>> alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
>> apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
>> atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
>> attr                 gstreamer                  libtheora              ncurses               startup-notification
>> base-passwd          gstreamer1.0               libtirpc               neon                  strace
>>      ...
>> gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
>> gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
>> gst-meta-base        libpthread-stubs           mkfontscale            shadow
>>
>> $ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
>> alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
>> cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
>> firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio
>>
>> It's the second category that is causing problems.  They do not
>> seem to end up in any shareable sstate at all.  If I try to rebuild
>> using only sstate, i.e. build my complete image to success, then
>> remove 'tmp' and rebuild, using the sstate-cache from the first go,
>> all of the above packages (alsa-lib, ..., xserver-xorg) are all
>> rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
>> but they are never reused from it.
>>
>> What would make this happen?  How can I prevent it?
>>
>> As is, sstate is not really shareable between these i.MX6 targets
>> as so much is being rebuilt all the time...
>>
>> Any ideas or pointers gladly welcomed.
>
> Try openembedded-core/scripts/sstate-diff-machines.sh
> to see why.
>

Can this work if I build for the two machines in separate trees?
Also, I'm really trying to find out why a build for the same machine
doesn't reuse sstate, in the same build tree, back-to-back builds
(no metadata changes)

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3498 bytes --]

On Tue, Apr 07, 2015 at 10:29:14AM -0600, Gary Thomas wrote:
> On 2015-04-07 10:19, Martin Jansa wrote:
> > On Tue, Apr 07, 2015 at 08:52:36AM -0600, Gary Thomas wrote:
> >> I'm building for multiple ARM i.MX6 platforms.  These have
> >> the same SoC, but slightly different peripherals. As far as
> >> I can tell, they should be able to share everything except
> >> for a few ${MACHINE} specific packages, e.g. the kernel and
> >> u-boot.
> >>
> >> Sadly, that doesn't seem to be the case.  The architecture
> >> specific packages are being split into two categories - plain
> >> ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
> >> For example, after building a complete image (on the order of
> >> core-image-sato), I have this split:
> >> $ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
> >> acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
> >> alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
> >> apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
> >> atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
> >> attr                 gstreamer                  libtheora              ncurses               startup-notification
> >> base-passwd          gstreamer1.0               libtirpc               neon                  strace
> >>      ...
> >> gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
> >> gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
> >> gst-meta-base        libpthread-stubs           mkfontscale            shadow
> >>
> >> $ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
> >> alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
> >> cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
> >> firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio
> >>
> >> It's the second category that is causing problems.  They do not
> >> seem to end up in any shareable sstate at all.  If I try to rebuild
> >> using only sstate, i.e. build my complete image to success, then
> >> remove 'tmp' and rebuild, using the sstate-cache from the first go,
> >> all of the above packages (alsa-lib, ..., xserver-xorg) are all
> >> rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
> >> but they are never reused from it.
> >>
> >> What would make this happen?  How can I prevent it?
> >>
> >> As is, sstate is not really shareable between these i.MX6 targets
> >> as so much is being rebuilt all the time...
> >>
> >> Any ideas or pointers gladly welcomed.
> >
> > Try openembedded-core/scripts/sstate-diff-machines.sh
> > to see why.
> >
> 
> Can this work if I build for the two machines in separate trees?

Yes, you can generate the report in each tree separately and then
compare them in one of them.

> Also, I'm really trying to find out why a build for the same machine
> doesn't reuse sstate, in the same build tree, back-to-back builds
> (no metadata changes)

Sorry, is "the same build tree" something else than "the two machines in
separate trees" or are you testing both?

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: sstate black hole?

[Gary Thomas]

On 2015-04-07 10:33, Martin Jansa wrote:
> On Tue, Apr 07, 2015 at 10:29:14AM -0600, Gary Thomas wrote:
>> On 2015-04-07 10:19, Martin Jansa wrote:
>>> On Tue, Apr 07, 2015 at 08:52:36AM -0600, Gary Thomas wrote:
>>>> I'm building for multiple ARM i.MX6 platforms.  These have
>>>> the same SoC, but slightly different peripherals. As far as
>>>> I can tell, they should be able to share everything except
>>>> for a few ${MACHINE} specific packages, e.g. the kernel and
>>>> u-boot.
>>>>
>>>> Sadly, that doesn't seem to be the case.  The architecture
>>>> specific packages are being split into two categories - plain
>>>> ARM/Cortex-A9 and those that have i.MX6 specific optimizations.
>>>> For example, after building a complete image (on the order of
>>>> core-image-sato), I have this split:
>>>> $ ls tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/
>>>> acl                  gst-player                 libsamplerate0         modutils-initscripts  shadow-sysroot
>>>> alsa-utils           gst-plugins-bad            libsm                  mpeg2dec              shared-mime-info
>>>> apmd                 gst-plugins-good           libsndfile1            mplayer2              speex
>>>> atk                  gst-plugins-ugly           libsoup-2.4            mtdev                 sqlite3
>>>> attr                 gstreamer                  libtheora              ncurses               startup-notification
>>>> base-passwd          gstreamer1.0               libtirpc               neon                  strace
>>>>       ...
>>>> gst-ffmpeg           libpostproc                matchbox-wm            scrnsaverproto        zlib
>>>> gst-fluendo-mpegmux  libproxy                   mkfontdir              settings-daemon
>>>> gst-meta-base        libpthread-stubs           mkfontscale            shadow
>>>>
>>>> $ ls tmp/work/cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi/
>>>> alsa-lib      gst-plugins-base           imx-gpu-viv  libfslparser   libsdl      xf86-video-imxfb-vivante
>>>> cairo         gstreamer1.0-plugins-bad   libdrm       libfslvpuwrap  mesa        xserver-xorg
>>>> firmware-imx  gstreamer1.0-plugins-base  libfslcodec  libglu         pulseaudio
>>>>
>>>> It's the second category that is causing problems.  They do not
>>>> seem to end up in any shareable sstate at all.  If I try to rebuild
>>>> using only sstate, i.e. build my complete image to success, then
>>>> remove 'tmp' and rebuild, using the sstate-cache from the first go,
>>>> all of the above packages (alsa-lib, ..., xserver-xorg) are all
>>>> rebuilt from scratch.  Those recipes do seem to end in my sstate-cache,
>>>> but they are never reused from it.
>>>>
>>>> What would make this happen?  How can I prevent it?
>>>>
>>>> As is, sstate is not really shareable between these i.MX6 targets
>>>> as so much is being rebuilt all the time...
>>>>
>>>> Any ideas or pointers gladly welcomed.
>>>
>>> Try openembedded-core/scripts/sstate-diff-machines.sh
>>> to see why.
>>>
>>
>> Can this work if I build for the two machines in separate trees?
>
> Yes, you can generate the report in each tree separately and then
> compare them in one of them.
>
>> Also, I'm really trying to find out why a build for the same machine
>> doesn't reuse sstate, in the same build tree, back-to-back builds
>> (no metadata changes)
>
> Sorry, is "the same build tree" something else than "the two machines in
> separate trees" or are you testing both?
>

Both actually, but at the moment the most interesting problem
is for a given machine that sstate is not being [re]used for
these recipes.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

sstate black hole?

[Gary Thomas]

I'm working with i.MX6 targets (meta-fsl-arm*).  For these
targets, some packages are "special" in that they use i.MX6
specific graphics support.  This ends up with an additional
layer of stratification, so my tmp/work tree has:
   all-amltd-linux
   cortexa9hf-vfp-neon-amltd-linux-gnueabi
   cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
   teton_p0382-amltd-linux-gnueabi
   x86_64-amltd-linux-gnueabi
   x86_64-linux

The packages that are built in tmp/work/cortex* are architecture
specific, not target specific, hence my question:

   If I build for two i.MX6 targets, identical in every way
   except for the ${MACHINE} name, if I use sstate to share
   the builds from target A when building for target B, why
   are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
   not shared by sstate?  I can see that they are present in
   the sstate cache, but they are always rebuilt for target B.
   I consider this incorrect behaviour as these are the same
   architecture and so they should be sharable via sstate.

Am I missing something here?  How can I determine why the
package from target A (sstate cache) is not usable by target B?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Nikolay Dimitrov]

Hi Gary,

On 06/15/2015 04:35 PM, Gary Thomas wrote:
> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
> targets, some packages are "special" in that they use i.MX6
> specific graphics support.  This ends up with an additional
> layer of stratification, so my tmp/work tree has:
>    all-amltd-linux
>    cortexa9hf-vfp-neon-amltd-linux-gnueabi
>    cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>    teton_p0382-amltd-linux-gnueabi
>    x86_64-amltd-linux-gnueabi
>    x86_64-linux
>
> The packages that are built in tmp/work/cortex* are architecture
> specific, not target specific, hence my question:
>
>    If I build for two i.MX6 targets, identical in every way
>    except for the ${MACHINE} name, if I use sstate to share
>    the builds from target A when building for target B, why
>    are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>    not shared by sstate?  I can see that they are present in
>    the sstate cache, but they are always rebuilt for target B.
>    I consider this incorrect behaviour as these are the same
>    architecture and so they should be sharable via sstate.
>
> Am I missing something here?  How can I determine why the
> package from target A (sstate cache) is not usable by target B?

Are these packages (the ones that are getting rebuilt) depending on
machine-specific headers (kernel)?

Regards,
Nikolay

Re: sstate black hole?

[Martin Jansa]

On Mon, Jun 15, 2015 at 07:35:20AM -0600, Gary Thomas wrote:
> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
> targets, some packages are "special" in that they use i.MX6
> specific graphics support.  This ends up with an additional
> layer of stratification, so my tmp/work tree has:
>    all-amltd-linux
>    cortexa9hf-vfp-neon-amltd-linux-gnueabi
>    cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>    teton_p0382-amltd-linux-gnueabi
>    x86_64-amltd-linux-gnueabi
>    x86_64-linux
> 
> The packages that are built in tmp/work/cortex* are architecture
> specific, not target specific, hence my question:
> 
>    If I build for two i.MX6 targets, identical in every way
>    except for the ${MACHINE} name, if I use sstate to share
>    the builds from target A when building for target B, why
>    are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>    not shared by sstate?  I can see that they are present in
>    the sstate cache, but they are always rebuilt for target B.
>    I consider this incorrect behaviour as these are the same
>    architecture and so they should be sharable via sstate.
> 
> Am I missing something here?  How can I determine why the
> package from target A (sstate cache) is not usable by target B?

Use openembedded-core/scripts/sstate-diff-machines.sh to check if the
signatures of the recipes you expect to be re-used are the same.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: sstate black hole?

[Gary Thomas]

On 2015-06-15 08:21, Martin Jansa wrote:
> On Mon, Jun 15, 2015 at 07:35:20AM -0600, Gary Thomas wrote:
>> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
>> targets, some packages are "special" in that they use i.MX6
>> specific graphics support.  This ends up with an additional
>> layer of stratification, so my tmp/work tree has:
>>     all-amltd-linux
>>     cortexa9hf-vfp-neon-amltd-linux-gnueabi
>>     cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>     teton_p0382-amltd-linux-gnueabi
>>     x86_64-amltd-linux-gnueabi
>>     x86_64-linux
>>
>> The packages that are built in tmp/work/cortex* are architecture
>> specific, not target specific, hence my question:
>>
>>     If I build for two i.MX6 targets, identical in every way
>>     except for the ${MACHINE} name, if I use sstate to share
>>     the builds from target A when building for target B, why
>>     are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>     not shared by sstate?  I can see that they are present in
>>     the sstate cache, but they are always rebuilt for target B.
>>     I consider this incorrect behaviour as these are the same
>>     architecture and so they should be sharable via sstate.
>>
>> Am I missing something here?  How can I determine why the
>> package from target A (sstate cache) is not usable by target B?
>
> Use openembedded-core/scripts/sstate-diff-machines.sh to check if the
> signatures of the recipes you expect to be re-used are the same.
>

How can I use this if the two targets have their own tmp/ tree?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Martin Jansa]

On Mon, Jun 15, 2015 at 11:41:47AM -0600, Gary Thomas wrote:
> On 2015-06-15 08:21, Martin Jansa wrote:
> > On Mon, Jun 15, 2015 at 07:35:20AM -0600, Gary Thomas wrote:
> >> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
> >> targets, some packages are "special" in that they use i.MX6
> >> specific graphics support.  This ends up with an additional
> >> layer of stratification, so my tmp/work tree has:
> >>     all-amltd-linux
> >>     cortexa9hf-vfp-neon-amltd-linux-gnueabi
> >>     cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
> >>     teton_p0382-amltd-linux-gnueabi
> >>     x86_64-amltd-linux-gnueabi
> >>     x86_64-linux
> >>
> >> The packages that are built in tmp/work/cortex* are architecture
> >> specific, not target specific, hence my question:
> >>
> >>     If I build for two i.MX6 targets, identical in every way
> >>     except for the ${MACHINE} name, if I use sstate to share
> >>     the builds from target A when building for target B, why
> >>     are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
> >>     not shared by sstate?  I can see that they are present in
> >>     the sstate cache, but they are always rebuilt for target B.
> >>     I consider this incorrect behaviour as these are the same
> >>     architecture and so they should be sharable via sstate.
> >>
> >> Am I missing something here?  How can I determine why the
> >> package from target A (sstate cache) is not usable by target B?
> >
> > Use openembedded-core/scripts/sstate-diff-machines.sh to check if the
> > signatures of the recipes you expect to be re-used are the same.
> >
> 
> How can I use this if the two targets have their own tmp/ tree?

call it twice (once in each tmp tree) and compare resulting list.M files

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: sstate black hole?

[Gary Thomas]

On 2015-06-15 11:46, Martin Jansa wrote:
> On Mon, Jun 15, 2015 at 11:41:47AM -0600, Gary Thomas wrote:
>> On 2015-06-15 08:21, Martin Jansa wrote:
>>> On Mon, Jun 15, 2015 at 07:35:20AM -0600, Gary Thomas wrote:
>>>> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
>>>> targets, some packages are "special" in that they use i.MX6
>>>> specific graphics support.  This ends up with an additional
>>>> layer of stratification, so my tmp/work tree has:
>>>>      all-amltd-linux
>>>>      cortexa9hf-vfp-neon-amltd-linux-gnueabi
>>>>      cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>>>      teton_p0382-amltd-linux-gnueabi
>>>>      x86_64-amltd-linux-gnueabi
>>>>      x86_64-linux
>>>>
>>>> The packages that are built in tmp/work/cortex* are architecture
>>>> specific, not target specific, hence my question:
>>>>
>>>>      If I build for two i.MX6 targets, identical in every way
>>>>      except for the ${MACHINE} name, if I use sstate to share
>>>>      the builds from target A when building for target B, why
>>>>      are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>>>      not shared by sstate?  I can see that they are present in
>>>>      the sstate cache, but they are always rebuilt for target B.
>>>>      I consider this incorrect behaviour as these are the same
>>>>      architecture and so they should be sharable via sstate.
>>>>
>>>> Am I missing something here?  How can I determine why the
>>>> package from target A (sstate cache) is not usable by target B?
>>>
>>> Use openembedded-core/scripts/sstate-diff-machines.sh to check if the
>>> signatures of the recipes you expect to be re-used are the same.
>>>
>>
>> How can I use this if the two targets have their own tmp/ tree?
>
> call it twice (once in each tmp tree) and compare resulting list.M files
>

Sadly, that script seems to destroy all of the .sigdata files which
are needed to actually track down the culprit(s).

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Gary Thomas]

On 2015-06-15 13:33, Gary Thomas wrote:
> On 2015-06-15 11:46, Martin Jansa wrote:
>> On Mon, Jun 15, 2015 at 11:41:47AM -0600, Gary Thomas wrote:
>>> On 2015-06-15 08:21, Martin Jansa wrote:
>>>> On Mon, Jun 15, 2015 at 07:35:20AM -0600, Gary Thomas wrote:
>>>>> I'm working with i.MX6 targets (meta-fsl-arm*).  For these
>>>>> targets, some packages are "special" in that they use i.MX6
>>>>> specific graphics support.  This ends up with an additional
>>>>> layer of stratification, so my tmp/work tree has:
>>>>>      all-amltd-linux
>>>>>      cortexa9hf-vfp-neon-amltd-linux-gnueabi
>>>>>      cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>>>>      teton_p0382-amltd-linux-gnueabi
>>>>>      x86_64-amltd-linux-gnueabi
>>>>>      x86_64-linux
>>>>>
>>>>> The packages that are built in tmp/work/cortex* are architecture
>>>>> specific, not target specific, hence my question:
>>>>>
>>>>>      If I build for two i.MX6 targets, identical in every way
>>>>>      except for the ${MACHINE} name, if I use sstate to share
>>>>>      the builds from target A when building for target B, why
>>>>>      are the packages built in cortexa9hf-vfp-neon-mx6qdl-amltd-linux-gnueabi
>>>>>      not shared by sstate?  I can see that they are present in
>>>>>      the sstate cache, but they are always rebuilt for target B.
>>>>>      I consider this incorrect behaviour as these are the same
>>>>>      architecture and so they should be sharable via sstate.
>>>>>
>>>>> Am I missing something here?  How can I determine why the
>>>>> package from target A (sstate cache) is not usable by target B?
>>>>
>>>> Use openembedded-core/scripts/sstate-diff-machines.sh to check if the
>>>> signatures of the recipes you expect to be re-used are the same.
>>>>
>>>
>>> How can I use this if the two targets have their own tmp/ tree?
>>
>> call it twice (once in each tmp tree) and compare resulting list.M files
>>
>
> Sadly, that script seems to destroy all of the .sigdata files which
> are needed to actually track down the culprit(s).
>

I followed this down to an i.MX6 specific package which is
in DEPENDS:
   Variable PROVIDES value changed from
   '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1 virtual/libgles2'
    to
   '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1 virtual/libgles2 virtual/libgl virtual/libgles1 
virtual/libgles2'

Why aren't these considered the same (duplicate elements and/or order)
should not matter for PROVIDES.  Is there some way we could get bitbake
to collapse this and remove the duplicates?

I found that this happens in meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc
which contains these lines:
   PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl virtual/libopenvg virtual/libg2d"
   PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1 virtual/libgles2"
   PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1 virtual/libgles2"
   PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1 virtual/libgles2"

Some i.MX6 targets, nitrogen6x in particular, has overrides for both mx6q and mx6dl,
hence the duplication.

Note: I removed the extra override (for testing) and found that
now the packages built by target A can be shared via sstate with
target B (at least the few I tested)

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Paul Eggleton]

On Monday 15 June 2015 18:19:21 Gary Thomas wrote:
> I followed this down to an i.MX6 specific package which is
> in DEPENDS:
>    Variable PROVIDES value changed from
>    '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> virtual/libgles2' to
>    '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> virtual/libgles2 virtual/libgl virtual/libgles1 virtual/libgles2'
> 
> Why aren't these considered the same (duplicate elements and/or order)
> should not matter for PROVIDES.  Is there some way we could get bitbake
> to collapse this and remove the duplicates?
> 
> I found that this happens in
> meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc which contains
> these lines:
>    PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
> virtual/libopenvg virtual/libg2d" 
> PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> 
> Some i.MX6 targets, nitrogen6x in particular, has overrides for both mx6q
> and mx6dl, hence the duplication.
> 
> Note: I removed the extra override (for testing) and found that
> now the packages built by target A can be shared via sstate with
> target B (at least the few I tested)

I agree this would be nice, but for this to work we'd probably need to 
introduce a typing system within bitbake so that it can understand that this 
variable is a space-separated list. That is something we will probably do at 
some point for a variety of reasons, but I don't know that it's something we 
will get to soon.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: sstate black hole?

[Gary Thomas]

On 2015-06-16 02:37, Paul Eggleton wrote:
> On Monday 15 June 2015 18:19:21 Gary Thomas wrote:
>> I followed this down to an i.MX6 specific package which is
>> in DEPENDS:
>>     Variable PROVIDES value changed from
>>     '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
>> virtual/libgles2' to
>>     '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
>> virtual/libgles2 virtual/libgl virtual/libgles1 virtual/libgles2'
>>
>> Why aren't these considered the same (duplicate elements and/or order)
>> should not matter for PROVIDES.  Is there some way we could get bitbake
>> to collapse this and remove the duplicates?
>>
>> I found that this happens in
>> meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc which contains
>> these lines:
>>     PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
>> virtual/libopenvg virtual/libg2d"
>> PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1 virtual/libgles2"
>> PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1 virtual/libgles2"
>> PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1 virtual/libgles2"
>>
>> Some i.MX6 targets, nitrogen6x in particular, has overrides for both mx6q
>> and mx6dl, hence the duplication.
>>
>> Note: I removed the extra override (for testing) and found that
>> now the packages built by target A can be shared via sstate with
>> target B (at least the few I tested)
>
> I agree this would be nice, but for this to work we'd probably need to
> introduce a typing system within bitbake so that it can understand that this
> variable is a space-separated list. That is something we will probably do at
> some point for a variety of reasons, but I don't know that it's something we
> will get to soon.

Is there some way to solve this in the recipe itself?  I can see that
it needs to make sure that those values are provided for each of the
[sub]class of processor, but is there a way to write it so that they
are only added once?  Perhaps a little bit of Python magic?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Paul Eggleton]

On Tuesday 16 June 2015 04:47:53 Gary Thomas wrote:
> On 2015-06-16 02:37, Paul Eggleton wrote:
> > On Monday 15 June 2015 18:19:21 Gary Thomas wrote:
> >> I followed this down to an i.MX6 specific package which is
> >> 
> >> in DEPENDS:
> >>     Variable PROVIDES value changed from
> >>     '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >> 
> >> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> >> virtual/libgles2' to
> >> 
> >>     '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >> 
> >> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> >> virtual/libgles2 virtual/libgl virtual/libgles1 virtual/libgles2'
> >> 
> >> Why aren't these considered the same (duplicate elements and/or order)
> >> should not matter for PROVIDES.  Is there some way we could get bitbake
> >> to collapse this and remove the duplicates?
> >> 
> >> I found that this happens in
> >> meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc which contains
> >> 
> >> these lines:
> >>     PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >> 
> >> virtual/libopenvg virtual/libg2d"
> >> PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1
> >> virtual/libgles2"
> >> PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1
> >> virtual/libgles2"
> >> PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1
> >> virtual/libgles2"
> >> 
> >> Some i.MX6 targets, nitrogen6x in particular, has overrides for both mx6q
> >> and mx6dl, hence the duplication.
> >> 
> >> Note: I removed the extra override (for testing) and found that
> >> now the packages built by target A can be shared via sstate with
> >> target B (at least the few I tested)
> > 
> > I agree this would be nice, but for this to work we'd probably need to
> > introduce a typing system within bitbake so that it can understand that
> > this variable is a space-separated list. That is something we will
> > probably do at some point for a variety of reasons, but I don't know that
> > it's something we will get to soon.
> 
> Is there some way to solve this in the recipe itself?  I can see that
> it needs to make sure that those values are provided for each of the
> [sub]class of processor, but is there a way to write it so that they
> are only added once?  Perhaps a little bit of Python magic?

Sure, you could use an anonymous python function with actual conditional 
statements instead of overrides to set PROVIDES here.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: sstate black hole?

[Gary Thomas]

On 2015-06-16 06:36, Paul Eggleton wrote:
> On Tuesday 16 June 2015 04:47:53 Gary Thomas wrote:
>> On 2015-06-16 02:37, Paul Eggleton wrote:
>>> On Monday 15 June 2015 18:19:21 Gary Thomas wrote:
>>>> I followed this down to an i.MX6 specific package which is
>>>>
>>>> in DEPENDS:
>>>>      Variable PROVIDES value changed from
>>>>      '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
>>>>
>>>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
>>>> virtual/libgles2' to
>>>>
>>>>      '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
>>>>
>>>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
>>>> virtual/libgles2 virtual/libgl virtual/libgles1 virtual/libgles2'
>>>>
>>>> Why aren't these considered the same (duplicate elements and/or order)
>>>> should not matter for PROVIDES.  Is there some way we could get bitbake
>>>> to collapse this and remove the duplicates?
>>>>
>>>> I found that this happens in
>>>> meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc which contains
>>>>
>>>> these lines:
>>>>      PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
>>>>
>>>> virtual/libopenvg virtual/libg2d"
>>>> PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1
>>>> virtual/libgles2"
>>>> PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1
>>>> virtual/libgles2"
>>>> PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1
>>>> virtual/libgles2"
>>>>
>>>> Some i.MX6 targets, nitrogen6x in particular, has overrides for both mx6q
>>>> and mx6dl, hence the duplication.
>>>>
>>>> Note: I removed the extra override (for testing) and found that
>>>> now the packages built by target A can be shared via sstate with
>>>> target B (at least the few I tested)
>>>
>>> I agree this would be nice, but for this to work we'd probably need to
>>> introduce a typing system within bitbake so that it can understand that
>>> this variable is a space-separated list. That is something we will
>>> probably do at some point for a variety of reasons, but I don't know that
>>> it's something we will get to soon.
>>
>> Is there some way to solve this in the recipe itself?  I can see that
>> it needs to make sure that those values are provided for each of the
>> [sub]class of processor, but is there a way to write it so that they
>> are only added once?  Perhaps a little bit of Python magic?
>
> Sure, you could use an anonymous python function with actual conditional
> statements instead of overrides to set PROVIDES here.

Thanks.  I found a much simpler way - I replaced the previous version by:
   EXTRA_PROVIDES = ""
   EXTRA_PROVIDES_mx6q  = " virtual/libgl virtual/libgles1 virtual/libgles2"
   EXTRA_PROVIDES_mx6dl  = " virtual/libgl virtual/libgles1 virtual/libgles2"
   EXTRA_PROVIDES_mx6sx  = " virtual/libgl virtual/libgles1 virtual/libgles2"
   PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl virtual/libopenvg virtual/libg2d ${EXTRA_PROVIDES}"

If the additions were unique, i.e. _mx6q was somehow different from _mx6dl,
I would have had to use the python function, but since they are all the same
this simpler way works just fine.

Thanks for the pointers

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate black hole?

[Paul Eggleton]

On Tuesday 16 June 2015 07:21:05 Gary Thomas wrote:
> On 2015-06-16 06:36, Paul Eggleton wrote:
> > On Tuesday 16 June 2015 04:47:53 Gary Thomas wrote:
> >> On 2015-06-16 02:37, Paul Eggleton wrote:
> >>> On Monday 15 June 2015 18:19:21 Gary Thomas wrote:
> >>>> I followed this down to an i.MX6 specific package which is
> >>>> 
> >>>> in DEPENDS:
> >>>>      Variable PROVIDES value changed from
> >>>>      '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >>>> 
> >>>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> >>>> virtual/libgles2' to
> >>>> 
> >>>>      '${PN}  virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >>>> 
> >>>> virtual/libopenvg virtual/libg2d virtual/libgl virtual/libgles1
> >>>> virtual/libgles2 virtual/libgl virtual/libgles1 virtual/libgles2'
> >>>> 
> >>>> Why aren't these considered the same (duplicate elements and/or order)
> >>>> should not matter for PROVIDES.  Is there some way we could get bitbake
> >>>> to collapse this and remove the duplicates?
> >>>> 
> >>>> I found that this happens in
> >>>> meta-fsl-arm:recipes-graphics/imx-gpu-viv/imx-gpu-viv.inc which
> >>>> contains
> >>>> 
> >>>> these lines:
> >>>>      PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
> >>>> 
> >>>> virtual/libopenvg virtual/libg2d"
> >>>> PROVIDES_append_mx6q  = " virtual/libgl virtual/libgles1
> >>>> virtual/libgles2"
> >>>> PROVIDES_append_mx6dl  = " virtual/libgl virtual/libgles1
> >>>> virtual/libgles2"
> >>>> PROVIDES_append_mx6sx  = " virtual/libgl virtual/libgles1
> >>>> virtual/libgles2"
> >>>> 
> >>>> Some i.MX6 targets, nitrogen6x in particular, has overrides for both
> >>>> mx6q
> >>>> and mx6dl, hence the duplication.
> >>>> 
> >>>> Note: I removed the extra override (for testing) and found that
> >>>> now the packages built by target A can be shared via sstate with
> >>>> target B (at least the few I tested)
> >>> 
> >>> I agree this would be nice, but for this to work we'd probably need to
> >>> introduce a typing system within bitbake so that it can understand that
> >>> this variable is a space-separated list. That is something we will
> >>> probably do at some point for a variety of reasons, but I don't know
> >>> that
> >>> it's something we will get to soon.
> >> 
> >> Is there some way to solve this in the recipe itself?  I can see that
> >> it needs to make sure that those values are provided for each of the
> >> [sub]class of processor, but is there a way to write it so that they
> >> are only added once?  Perhaps a little bit of Python magic?
> > 
> > Sure, you could use an anonymous python function with actual conditional
> > statements instead of overrides to set PROVIDES here.
> 
> Thanks.  I found a much simpler way - I replaced the previous version by:
>    EXTRA_PROVIDES = ""
>    EXTRA_PROVIDES_mx6q  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> EXTRA_PROVIDES_mx6dl  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> EXTRA_PROVIDES_mx6sx  = " virtual/libgl virtual/libgles1 virtual/libgles2"
> PROVIDES += "virtual/wayland-egl virtual/libgal-x11 virtual/egl
> virtual/libopenvg virtual/libg2d ${EXTRA_PROVIDES}"
> 
> If the additions were unique, i.e. _mx6q was somehow different from _mx6dl,
> I would have had to use the python function, but since they are all the same
> this simpler way works just fine.

Yep, that will work too - good job :)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre