Re: PARALLEL_MAKE

[Denys Dmytriyenko <denis@denix.org>]

On Tue, Mar 17, 2009 at 05:28:27PM +0100, Ulf Samuelsson wrote:
>> Yes, I know,
>> Maybe it is a good idea if the result of a build is published somewhere,
>> so that others do not have to go through this process.
>>
>> Right now, openembedded does not even build for me,
>> since the native perl build fails :-(
>>
>>> [1] http://en.wikipedia.org/wiki/Hyper-threading
>
> I now managed to build the Angstrom image for the AT91SAM9263 by applying
> a few fixes present in the ulf/linux branch.
> (Did not load on H/W yet though)
>
> perl-native was "fixed" by not building the library which crashed.
> libtheora was fixd, not to use docs/examples.
> busybox updated to 1.13.3

Any specific changes between 1.13.2 and 1.13.3 you need? Would be nice to 
cherry-pick into .dev branch...

> u-boot, gpsd was set to PARALLEL_MAKE = ""
> u-boot updated to 2009.01
> linux-2.6.28
> at91bootstrap-2.10
>
> Building with PARALLEL_MAKE & Multiple threads was not smooth.
> The build crashed several times, due to erros
> caused by parallellism. "gettext", "gpsd" and "u-boot" were culprits.

Would be nice to fix those. Tom Rini was working on fixing some of those 
issues.

> u-boot is NOT built cleanly, even after PARALLEL_MAKE was set to "".
> The actual compile completes and generates an u-boot.bin
> but this is not deployed to the result directory.
> There are logs for different stages, but after the compile, they exist, but 
> are empty.
>
> If I remove all the stamps and "bitbake -b <u-boot>" afterwards
> then the build completes correctly, and there is an 
> u-boot-at91sam9263ek.bin
> in the result directory.
>
> Can it be so, that a stage is started, before the previous stage has been 
> completed???

If there are no dependencies between packages, their stage tasks can run in 
parallel.

> ---------------------
> BTW: The total buildtime was around 2 hours. There is some overhead
> since the build was restarted 3-4 times, and local.conf was changed once.
> I started the build with PARALLEL_MAKE = "-j2" and BB_NUMBER_THREADS=8.
>
> I noticed that having a lot of threads helps most of the time,
> since the CPUs were executing at max frequency.
>
> When the cross compiler, was built,it was different,
> most of the cores were not used with this configuration.
> 2-3 max while compiling gcc.

Since gcc/cross is quite a big package and lots of other packages depend on 
it, it is being built for you only using 2 cores, as specified by 
PARALLEL_MAKE. Everything else is waiting for it to finish.

> Maybe the best approach is to build the Angstrom in several stages
> with different settings for parallelism.

-- 
Denys