[Buildroot] [PATCH v4 1/2] packages: fix and improve support for top-level parallel makefile

[Thomas Petazzoni <thomas.petazzoni@free-electrons.com>]

Hello Fabio,

On Fri, 26 Jun 2015 21:39:43 +0200, Fabio Porcedda wrote:

> > On Fri, 26 Jun 2015 05:58:53 +0200, Fabio Porcedda wrote:
> >> To fix packages that fail to build when PARALLEL_JOBS is empty instead
> >
> > Which packages specifically?
> 
> At least the boost and jack2 packages fail.

Ok, this should probably been indicated in the commit log, in order to
make it more specific than "To fix 'packages'".


> The good thing about GNU Make jobs is that the number of jobs
> available are shared between each instance of make so if you want to
> increase the number of jobs you just need to increase the <jobs> value
> in  the top make -j<jobs> command.
> 
> e.g. 2 in top-level and 2 in each package, the actual cpu utilization
> start from 1 to 4.
> Using 4 in top-level and let make share the jobs for each packages the
> cpu utilization start from 1 to 4 but you have a higher chance to use
> 4 processor because in this case you can have:
> 
> - 4 packages that use make1
> or
> - 1 package that use make but it actually use 4 processor himself
> or
> - 3 extract operation and 1 package that use make1
> 
> 
> Another example, you have 8 cores, you use top-make -j8 and sub-make -j8.
> This time the number of processes goes up to 64 that it's not good,
> too much process take more time to be completed.
> In my tests on a cpu with 8 cores -j16 is slower than -j9.
> 
> So I think it's better to let gnu make handle the jobs distribution.

Ah, I didn't know this. This is a very good explanation! Why isn't it
part of the commit log? It very clearly explains why we want to not
pass any -j option to the sub-makes used to build the packages.

However, I don't see how it properly fixes the packages that are
directly using PARALLEL_JOBS. Yes, now, PARALLEL_JOBS is no longer
empty. But it's going to be number of CPU + 1, which is not good when
we're doing top-level parallel job, as you explained very well. So why
would all make-based packages be built with "make", and packages using
special build systems (midori, qt, qt5, jack2, cmake, boost) would
still use a PARALLEL_JOBS number higher than 1. What is your plan to
solve this problem?

Thanks a lot!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com