Re: [PATCH 1/1] sstate.bbclass: fix parallel building issue

[Rongqing Li <rongqing.li@windriver.com>]

On 08/16/2013 04:25 PM, Rongqing Li wrote:
>>> Do we need some kind of a read/write lock on accessing those files.
>>> (Is this
>>> even something that we can do easily though the existing mechanisms?)
>>
>> It would kill performance for no good reason, been there, looked at
>> it...
>>
>> Cheers,
>>
>> Richard
>>
>
> I think reverting the below optimization maybe better than using lock
>
> commit 8c5544c2311b080bb212efb7f6b804db63e125f5
> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date:   Thu Oct 11 13:36:53 2012 +0100
>
>      scripts/cp-noerror: Try and use hardlinks if possible
>
>      Since we generally have lots of copies of the directories created
> using this tool, use
>      hardlinks where possible. This should save a little disk space and
> improve performance
>      slightly.
>
>      (From OE-Core rev: bfa11c028c2da093f7b4e6b7b1d611da90ae052f)
>
>      Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>
>
> -Roy
>
>

I think the upper commit saves lots of space, but the saved time maybe 
be ignored.

On my general building image.

1. aclocal size and numbers of files
         bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ du -sh aclocal
         768K    aclocal

         bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ ls aclocal|wc
              54      54     621
2. do hardlink copy 1000 times
         bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ cat ./aa
         #! /bin/bash

         n=0

         while [ $n -le 1000 ] ; do
             n=`expr "$n" + 1`
             cp -alf aclocal ./tmp/
             rm -rf ./tmp/aclocal
         done
         /bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ time ./aa

         real    0m4.416s
         user    0m0.084s
         sys     0m0.256s

2. do copy 1000 times
         bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ cat ./aa
         #! /bin/bash

         n=0

         while [ $n -le 1000 ] ; do
             n=`expr "$n" + 1`
             cp -rf aclocal ./tmp/
             rm -rf ./tmp/aclocal
         done

         bitbake_build/tmp/sysroots/x86_64-linux/usr/share$ time ./aa

         real    0m8.707s
         user    0m0.104s
         sys     0m0.324s


Since we need several hours to compile a image, several seconds 
improvement...


-- 
Best Reagrds,
Roy | RongQing Li