Re: [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote:
> We discussed this 2.3 months ago.
> Did some studies on my dual hex-core machine (24 H/W treads) while 
> building a cloud9-gnome-image derivative.
> This did about 7500 tasks.
> 
> Enabled the CPU supervisors in the panel.
> 
> Everything seems to be ok with BB_NUMBER_THREADS = "24" for about 4-4500 
> tasks.
> 
> Then the CPUs are mostly inactive and only 1-2 running for ~500 tasks.
> Then parallellism  is resumed until about task 7000, and again
> only a few CPUs are active.

This is likely whilst the lib and toolchain is getting built.

> I believe that some tools use "make" within the Makefile,
> and they are written badly, and do not use "-j <n>" for
> that part of the build.

Which recipes were building at this point? It would be interesting to
track them down.

> Got my build down to 83 minutes.
> 
> Since I have 96 GB of RAM, I tried creating an 80 GB tmpfs for the build,
> and copied the download and the recipes to the ram.
> 
> That shaved only 2 monutes from the build, and some stuff,
> still built using only a single CPU.

There are certainly dependency bottlenecks in the build such as the
toolchain, compiler, gettext, gtk+ and so where large numbers of things
need those dependencies to get built before they can proceed. Not sure
what we can do to help this though.

Cheers,

Richard