Re: [PATCH] lib/oe/path: Fix performance issue got copyhardlinktree()

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Sat, 2013-11-09 at 21:43 +0100, Andrea Adami wrote:
> On Fri, Nov 8, 2013 at 4:42 PM, Bruce Ashfield
> <bruce.ashfield@windriver.com> wrote:
> > On 13-11-08 10:19 AM, Richard Purdie wrote:
> >>
> >> With the directory copy was added to avoid race issues, it wasn't noticed
> >> that
> >> tar was recursing the directories and copying files too. This is
> >> completely
> >> crazy when we hardlink those files in the next command.
> >>
> >> Resolve the issue by telling tar not to recurse. This gives a significant
> >> performance boost to various parts of the system (do_package for
> >> linux-yocto
> >> 256s -> 178s for example).
> >
> >
> > Every second makes a difference in this beast .. when adding up a
> > zillion package runs a week. :)
> >
> > Cheers,
> >
> > Bruce
> >
> >
> >>
> >> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> >> ---
> >> diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
> >> index 1310e38..d0588ba 100644
> >> --- a/meta/lib/oe/path.py
> >> +++ b/meta/lib/oe/path.py
> >> @@ -93,7 +93,7 @@ def copyhardlinktree(src, dst):
> >>       if (os.stat(src).st_dev ==  os.stat(dst).st_dev):
> >>           # Need to copy directories only with tar first since cp will
> >> error if two
> >>           # writers try and create a directory at the same time
> >> -        cmd = 'cd %s; find . -type d -print | tar -cf - -C %s -p
> >> --files-from - | tar -xf - -C %s' % (src, src, dst)
> >> +        cmd = 'cd %s; find . -type d -print | tar -cf - -C %s -p
> >> --files-from - --no-recursion | tar -xf - -C %s' % (src, src, dst)
> >>           check_output(cmd, shell=True, stderr=subprocess.STDOUT)
> >>           if os.path.isdir(src):
> >>               src = src + "/*"
> >>
> >>
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> After this patch do_populate_sysroot fails:

This should fix it:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=5237c8e2c1780983c000247ffa0c890b4bb5980a

Cheers,

Richard