From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: openembedded-core <openembedded-core@lists.openembedded.org>
Cc: "Hart, Darren" <darren.hart@intel.com>
Subject: Fetch/Unpack performance dependent on git version
Date: Sat, 19 Apr 2014 23:40:24 +0100 [thread overview]
Message-ID: <1397947224.16672.15.camel@ted> (raw)
I thought it might be worth highlighting to people that the performance
of the git fetcher *and* the unpack process is highly dependant on the
version of git.
Firstly, all versions of git 1.6+ appear to default to injecting an
fsync() after fetching objects. fsync() usage significantly hurts our
builds. It is also a bit pointless in our usecase, we can however
disable it with a simple tweak to the fetcher:
- ud.basecmd = data.getVar("FETCHCMD_git", d, True) or "git"
+ ud.basecmd = data.getVar("FETCHCMD_git", d, True) or "git -c core.fsyncobjectfiles=0"
Secondly, for older versions of git, git clone is running repository
coverage checks even for local cloning. E.g.:
git version 1.8.3.2 on my system:
$ bitbake linux-yocto -c fetch
$ time bitbake linux-yocto -c unpack
51.33user 1.93system 1:09.27elapsed 76%CPU (0avgtext+0avgdata 779380maxresident)k
after updating to git 1.9.2:
$ bitbake linux-yocto -c fetch
$ time bitbake linux-yocto -c unpack
4.28user 0.63system 0:04.36elapsed 112%CPU (0avgtext+0avgdata 260692maxresident)k
so 16 times faster!
The commit in question for anyone interested:
https://github.com/git/git/commit/125a05fd0b45416558923b753f6418c24208d443
Michael/Beth: Can we please upgrade the autobuilder to a daisy release
buildtools tarball which has git 1.9.x in it? This should massively
speed up certain parts of the autobuilder.
Stefan: Could you see which version of git is on the performance
benchmark machine? A run with the daisy build tools tarball would be
extremely interesting if its before 1.9.
Cheers,
Richard
next reply other threads:[~2014-04-19 22:40 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-04-19 22:40 Richard Purdie [this message]
2014-04-21 20:52 ` Fetch/Unpack performance dependent on git version Michael Halstead
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1397947224.16672.15.camel@ted \
--to=richard.purdie@linuxfoundation.org \
--cc=darren.hart@intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.