Re: [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Tue, 2014-01-28 at 11:08 +0100, Koen Kooi wrote:
> Op 27 jan. 2014, om 15:39 heeft Richard Purdie <richard.purdie@linuxfoundation.org> het volgende geschreven:
> 
> > Its rather sad that people don't appear to read local.conf and then complain
> > about slow builds when they're just using a single thread. Most systems have
> > more than one core now so we might as well use a more automatic default
> > for these values. This may lead to better experiences for new users.
> > 
> > [YOCTO #2528]
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
> > index 71856b8..36d33e1 100644
> > --- a/meta/conf/local.conf.sample
> > +++ b/meta/conf/local.conf.sample
> > @@ -18,12 +18,18 @@
> > # option determines how many tasks bitbake should run in parallel:
> > #
> > #BB_NUMBER_THREADS ?= "4"
> > +#
> > +# Default to setting automatically based on cpu count
> > +BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
> 
> I've noticed that after 4 threads IO becomes a big bottleneck when you
> have things like webkit, qt, asio etc in the buildqueue. Combine that
> with issues like every make -j thread taking >2GB ram with asio and
> webkit this default seems a bit high. I'd use 0.5*numcpu with a lower
> bound of 2.

This is one of those things I think we'll not find a perfect answer to
and its why I left the defaults alone for as long.

Personally, I run with 48 threads enabled and I've yet to see it do
anything too crazy. At the end of the day it all comes down to the
hardware you're running it on though.

webkit is a particular case where whilst its building you'd probably
want different values. Which numbers you use depends on how often you're
building it...

I'm not sure we can win. If the build appears to take over a system, I'm
hoping the user will look at the conf file and figure out how to make it
use less of the system...

Cheers,

Richard