Re: [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

[Mark Hatle <mark.hatle@windriver.com>]

On 2/4/14, 10:13 AM, Enrico Scholz wrote:
> Koen Kooi <koen-QLwJDigV5abLmq1fohREcCpxlwaOVQ5f@public.gmane.org>
> writes:
>
>>> +# Default to setting automatically based on cpu count
>>> +BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
>>
>> I've noticed that after 4 threads IO becomes a big bottleneck when you
>> have things like webkit, qt, asio etc in the buildqueue. Combine that
>> with issues like every make -j thread taking >2GB ram with asio and
>> webkit this default seems a bit high. I'd use 0.5*numcpu with a lower
>> bound of 2.
>
> limitting the load mitigates this (high i/o increases it); e.g.
>
> PARALLEL_MAKE = "\
>    ...
>    -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100} \
> "

FYI, I think this points out the variability in system performance, between CPU, 
RAM and I/o.

As it stands the patch gives my machine the best performance.  So I like it as 
it is.  But my machine (dual 8-core w/ HT, 64 GB of RAM, and hardware raid). 
But on hardware with less RAM, slower disk, it may not perform optimally.

So the catch is what is the proper optimal setting?  As I see it, assuming that 
the system has enough ram and I/O to fill the CPUs is the best approach (what 
was implemented.)  And then in the comments document that this may not be the 
best setting for all systems, and the user should adjust it as necessary.  Even 
suggesting some of the alternative systems such as the 150/100 above.

No setting is going to make everyone happy, but something has to be better then 
defaulting to '1'.

--Mark

> Enrico
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>