Further build profiling info

Further build profiling info

[Richard Purdie]

I'm sending these notes to the list since they might be useful to others
and it archives them.

Timing a build with license.bbclass:
consisting of 4847 tasks:

real 107m23.857s
user 376m31.640s
sys 46m51.020s

Without license.bbclass:
4409 tasks:

real 107m46.881s
user 369m6.670s
sys 46m19.890s

Without the setscene task and without license.bbclass:

3970 tasks:

real	105m36.940s
user	364m6.900s
sys	45m33.950s

which makes sense as setscene is a critical path dependency whilst the
license checks are not.

So a 120 second reduction for a drop of 439 tasks so a 0.27 second
overhead per task assuming they were all critical path. Numbers from the
graph suggested a 0.5 second task overhead so perhaps 50% were critical
path.

Interesting numbers and it suggests the situation isn't as bad as the
graph makes out though.

Cheers,

Richard

Re: Further build profiling info

[Richard Purdie]

On Tue, 2011-02-15 at 22:05 +0000, Richard Purdie wrote:
> I'm sending these notes to the list since they might be useful to others
> and it archives them.
> 
> Timing a build with license.bbclass:
> consisting of 4847 tasks:
> 
> real 107m23.857s
> user 376m31.640s
> sys 46m51.020s
> 
> Without license.bbclass:
> 4409 tasks:
> 
> real 107m46.881s
> user 369m6.670s
> sys 46m19.890s
> 
> Without the setscene task and without license.bbclass:
> 
> 3970 tasks:
> 
> real	105m36.940s
> user	364m6.900s
> sys	45m33.950s
> 
> which makes sense as setscene is a critical path dependency whilst the
> license checks are not.

We've had some fixes into master which:

* improve the pseudo-native build time (worth 2 minutes)
* shorten the rpm rootfs creation times
* enabling parallel builds for some packages
* Improve the task processing/creation latency (worth 1-2 minutes)

The time for the same build as above is now taking:

real	97m1.204s
user	385m24.460s
sys	48m30.990s
(4848 tasks)

and this is despite adding in some extra documentation generation pieces
too (which accounts for the extra user time) so quite a nice
improvement. Nice work but I think there is further low hanging fruit to
gain a little more time back too! :)

Cheers,

Richard

Re: Further build profiling info

[Richard Purdie]

On Wed, 2011-02-16 at 20:49 +0000, Richard Purdie wrote:
> We've had some fixes into master which:
> 
> * improve the pseudo-native build time (worth 2 minutes)
> * shorten the rpm rootfs creation times
> * enabling parallel builds for some packages
> * Improve the task processing/creation latency (worth 1-2 minutes)
> 
> The time for the same build as above is now taking:
> 
> real	97m1.204s
> user	385m24.460s
> sys	48m30.990s
> (4848 tasks)
> 
> and this is despite adding in some extra documentation generation pieces
> too (which accounts for the extra user time) so quite a nice
> improvement. Nice work but I think there is further low hanging fruit to
> gain a little more time back too! :)

I've just run a test of Poky master and we've regressed to:

real	106m46.524s
user	385m28.050s
sys	47m30.990s
(4948 tasks)

At this point I can only think its the extra documentation dependencies
that have changed things. I'm more than a little disappointed in
this :(.

Cheers,

Richard