Why did my build need to redo so many tasks?

Why did my build need to redo so many tasks?

[Gary Thomas]

Yesterday, I built a large image using Poky/Yocto 870323cac1e
Today I updated my Poky repo to 231d4a9d3

When I rebuilt the same image, bitbake kicked off more than
5200 tasks, building virtually every package from scratch.

How can I tell why this happened?  I don't have buildhistory
enabled, but I do still have the tmp & sstate-cache trees
from the process.  Is there enough info laying around that
I can tell, e.g. why the GCC toolchain had to be rebuilt
completely?

Am I correct in assuming that bitbake-whatchanged is really
a Quija board and can only tell me [in advance] what will need
to be done if my layers change?  When I ran it after the long
build above, it didn't really tell me much :-(

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Why did my build need to redo so many tasks?

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

On 12 February 2015 at 21:25, Gary Thomas <gary@mlbassoc.com> wrote:

> Yesterday, I built a large image using Poky/Yocto 870323cac1e
> Today I updated my Poky repo to 231d4a9d3
>
> When I rebuilt the same image, bitbake kicked off more than
> 5200 tasks, building virtually every package from scratch.
>
> How can I tell why this happened?  I don't have buildhistory
> enabled, but I do still have the tmp & sstate-cache trees
> from the process.  Is there enough info laying around that
> I can tell, e.g. why the GCC toolchain had to be rebuilt
> completely?
>
> Am I correct in assuming that bitbake-whatchanged is really
> a Quija board and can only tell me [in advance] what will need
> to be done if my layers change?  When I ran it after the long
> build above, it didn't really tell me much :-(
>

If you checkout the first commit, do a build, then the second commit and
run bitbake-whatchanged, it should tell you the difference.

Looking at the git log shows cc5f80, which is a change to libtool, so
everything inheriting autotools will need a rebuild.

Ross

[-- Attachment #2: Type: text/html, Size: 1605 bytes --]

Re: Why did my build need to redo so many tasks?

[Gary Thomas]

On 2015-02-12 14:33, Burton, Ross wrote:
>
> On 12 February 2015 at 21:25, Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>> wrote:
>
>     Yesterday, I built a large image using Poky/Yocto 870323cac1e
>     Today I updated my Poky repo to 231d4a9d3
>
>     When I rebuilt the same image, bitbake kicked off more than
>     5200 tasks, building virtually every package from scratch.
>
>     How can I tell why this happened?  I don't have buildhistory
>     enabled, but I do still have the tmp & sstate-cache trees
>     from the process.  Is there enough info laying around that
>     I can tell, e.g. why the GCC toolchain had to be rebuilt
>     completely?
>
>     Am I correct in assuming that bitbake-whatchanged is really
>     a Quija board and can only tell me [in advance] what will need
>     to be done if my layers change?  When I ran it after the long
>     build above, it didn't really tell me much :-(
>
>
> If you checkout the first commit, do a build, then the second commit and run bitbake-whatchanged, it should tell you the difference.
>
> Looking at the git log shows cc5f80, which is a change to libtool, so everything inheriting autotools will need a rebuild.

Ouch, thanks.  Next time I'll know a bit more on how to diagnose this.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Why did my build need to redo so many tasks?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]

On Thu, Feb 12, 2015 at 02:25:32PM -0700, Gary Thomas wrote:
> Yesterday, I built a large image using Poky/Yocto 870323cac1e
> Today I updated my Poky repo to 231d4a9d3
> 
> When I rebuilt the same image, bitbake kicked off more than
> 5200 tasks, building virtually every package from scratch.
> 
> How can I tell why this happened?  I don't have buildhistory
> enabled, but I do still have the tmp & sstate-cache trees
> from the process.  Is there enough info laying around that
> I can tell, e.g. why the GCC toolchain had to be rebuilt
> completely?
> 
> Am I correct in assuming that bitbake-whatchanged is really
> a Quija board and can only tell me [in advance] what will need
> to be done if my layers change?  When I ran it after the long
> build above, it didn't really tell me much :-(

I'm using
openembedded-core/scripts/sstate-diff-machines.sh
to create sstate signature snaphosts for interesting builds, then
when I'm surprised by number of re-executions, I create another snapshot
and compare them.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Why did my build need to redo so many tasks?

[Barros Pena, Belen]

On 12/02/2015 22:19, "Martin Jansa" <martin.jansa@gmail.com> wrote:

>On Thu, Feb 12, 2015 at 02:25:32PM -0700, Gary Thomas wrote:
>> Yesterday, I built a large image using Poky/Yocto 870323cac1e
>> Today I updated my Poky repo to 231d4a9d3
>> 
>> When I rebuilt the same image, bitbake kicked off more than
>> 5200 tasks, building virtually every package from scratch.
>> 
>> How can I tell why this happened?  I don't have buildhistory
>> enabled, but I do still have the tmp & sstate-cache trees
>> from the process.  Is there enough info laying around that
>> I can tell, e.g. why the GCC toolchain had to be rebuilt
>> completely?
>> 
>> Am I correct in assuming that bitbake-whatchanged is really
>> a Quija board and can only tell me [in advance] what will need
>> to be done if my layers change?  When I ran it after the long
>> build above, it didn't really tell me much :-(
>
>I'm using
>openembedded-core/scripts/sstate-diff-machines.sh
>to create sstate signature snaphosts for interesting builds, then
>when I'm surprised by number of re-executions, I create another snapshot
>and compare them.

And there is Toaster, which also reports on sstate use, although after the
build is done and it might not give you the same granularity as the other
suggestions. It looks nice, though ;)

Belén

>
>-- 
>Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com