Re: [PATCH 1/1] libc-locale: split locale handling from libc recipe.

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2011-06-09 at 12:14 +0100, Phil Blundell wrote:
> On Wed, 2011-06-08 at 16:35 +0100, Richard Purdie wrote:
> > I'm not sure how this would reduce performance of builds of a few
> > threads, it should just make better use of any available "spare"
> > processing capacity throughout the build.
> 
> If I'm reading the patch right, it does involve a certain amount of
> extra copying around of the locale-related files since they need to be
> stashed away in some place where libc-locale.bb can find them later.  I
> don't think these files are especially big, but there are quite a few of
> them (which is the whole reason that libc's do_package was slow in the
> first place). 
> 
> I must admit that I'm slightly surprised that libc's packaging stage is
> becoming a bottleneck for anything other than the very smallest builds.
> If there's any substantial amount of other material being compiled then
> I would expect that there would be enough do_compile() work to keep the
> other threads busy until libc was done packaging.  That's not to say
> that I think this change is a bad idea, but I would be interested to
> know what the actual impact is for real-world workloads.
> 
> And, just on a point of principle, any time we're making a chance for
> performance-related reasons I think we should always have measurements
> to back it up. 

Totally agreed and this decision is being made on real world data even
if its perhaps not clearly being presented here.

$ cat buildstats/core-image-sato-qemux86/201106030912/eglibc-2.13-r1+svnr13356/do_package | grep time
eglibc-2.13-r1+svnr13356: do_package: Elapsed time: 828.01 seconds

As you can see, eglibc do_package takes about 14 minutes which is about
14% of our build time. That is a long time to block pretty much all
packaging activity, particularly if you have access to something with
several cores. When it does complete, even on my 4 core system you see a
"stampeding herd" of packaging happening on the build charts suggesting
a backlog does build up.

For those reasons I do think its a reasonable change.

Cheers,

Richard