Re: Usage of yocto on different (production vs debug) scenarios

[Uwe Geuder <jrswdnan22@snkmail.com>]

Thanks Ross for your answers.  I'm still working on fully understanding
shared state so I appreciate your help.  Let me follow up on both your
answers in a single message.

On Mon, Apr 23, 2018 at 6:38 PM, Burton, Ross <ross.burton@intel.com> wrote:
> On 23 April 2018 at 16:23, Iván Castell <icastell@nayarsystems.com> wrote:
>>
[...]
>>
>> Related with that shared state cache, I found some information on a e-book
>> (search the text on google to find the source):
>>
>> "Sharing a shared state cache is possible; however, it needs to be
>> approached with care. Not all changes are detected by the shared state cache
>> implementation, and when this happens, some or all of the cache needs to be
>> invalidated. This can cause problems when the state cache is being shared.
[...]
> Very curious as to what book said that, because *any* example of that
> happening is a bug in the recipe itself.  I wouldn't listen to it: the YP
> autobuilder has a shared sstate for three distributions * four architectures
> * two libc implementations and doesn't have problems.
>
> Ross
>

I think when talking about shared state there can be 2 aspects of sharing

1.) sharing over time: Several invocations of bitbake in the same environment
  (DISTRO, MACHINE etc)
2.) sharing over distros etc.

Number 1 is an incremental build (as opposed to clean or full
build).  Without any doubt that is useful (or really mandatory) for
development work.  You build, you make little changes, you build, you
test, you make more changes...  You share state from the previous build,
because making a full clean build every time would lead to completely
inacceptable cycle times.

But when you are "done" with your changes you make an integration build
and to my experience that should be a clean (aka full) build.  You
intentionally do not share any state from a previous build, you don't
make an incremental build, because there is always the risk of broken
recipes.  I mean if there were no broken software, we would all be
unemployed by now...

Let me give a simple example I experienced my self recently:

I added creation of a new file in some task.  In my incremental
developer build it all worked fine.  When I put into a clean integration
build the task failed.  It turned out that the directory I wrote my new
file to in task A was created by a different task B.  In my development
build task B had already been executed long before I even started to
make the recipe modificationto to create a new file, so task A always
succeeded.  In the clean build task A happened to run first and failed.
A simple missing dependency / incorrect task ordering.  With parallel
clean builds you might not always find it, but with incremental builds
it can go undetected forever.

It's not only "some book" that mentions the risk of "bad" state, but
also the Yocto documentaton:

https://www.yoctoproject.org/docs/2.5/mega-manual/mega-manual.html#concepts-tips-and-tricks

(Especially 4.5.4.2. Invalidating Shared State.  Of course the
invalidating is easy, but finding what you need to invalidate is hard.)

Of course the less something is tested the bigger the risk is.  So the
bitbake system should be rather fine, commonly used Poky recipes
probably too.  But when it comes to random layers and our own local
recipes and bbappends I would expect the risk to grow a lot.

So do you really tell us in the message above that above that Yocto
project runs incremental integration builds?

Another way to think about it: Shared state is used to to avoid
rebuilds.  So how could I benefit from having state information in some
SSTATE_DIR already?  I have to build it anyway, because my build area is
empty to begin with.

The 2nd aspect, sharing over distros was covered in the other message.

On Mon, Apr 23, 2018 at 6:10 PM, Burton, Ross <ross.burton@intel.com> wrote:
> On 20 April 2018 at 11:47, Uwe Geuder <jrswdnan22@snkmail.com> wrote:
>> But can you share state between distros? Isn't the purpose of distros to
>> use different options (variable settings) so the state would always be
>> different?
>
> If the input to the recipe is different then the hashes would be
> different so the content won't conflict.  You can definitely share
> sstate between DISTROs.
>
> Ross

Under the assumption that all recepies were perfect (which they probably
aren't in every point of history, see above) I fully believe that the
build system would work perfectly when I share the SSTATE_DIR over
DISTRO borders (or diferent MACHINE settings).  But do I really have any
commmon data that would lead to any saving? As you say the hashes would
be different.  In my thinking all "expensive" state info should be
different, because a different DISTRO (or different MACHINE) would lead
to different hashes for bascially every item.  So in the best case I end
up with several distinct sets of state data stored under the same top
directory, but that doesn't seem to result in any saving.  In the worst
case an item goes into the same file, so I end up with "cache
thrashing".  Build A calculates the state and stores it, build B finds
the hashes not matching and puts its new info there.  Build A finds the
hashes not matching and...

Or does the saving come on the native part of the build? There I would
guess that common state could occur more commonly.

I think it would really help to have a hello world type of Yocto build,
where you could manually read through the sstate data to understand how
it works exactly.  In a real world sized build the amount of data does
not really help to understand the concepts.  Does anybody have a
recommendation what would be the smallest possible Yocto build that
makes some sense?

Regards,

Uwe Geuder
Neuro Event Labs Oy
Tampere, Finland
uwe.gexder@neuroeventlabs.com (Bot check: fix one obvious typo)