From: Jeff King <peff@peff.net>
To: steve.norman@thomsonreuters.com
Cc: Junio C Hamano <gitster@pobox.com>,
pclouds@gmail.com, git@vger.kernel.org
Subject: Re: [PATCH] index-pack: avoid excessive re-reading of pack directory
Date: Tue, 9 Jun 2015 13:41:06 -0400 [thread overview]
Message-ID: <20150609174106.GB32034@peff.net> (raw)
In-Reply-To: <20150609172436.GA32034@peff.net>
On Tue, Jun 09, 2015 at 01:24:36PM -0400, Jeff King wrote:
> I tested this on my system, and confirmed that for a "git clone
> --no-local --bare git.git":
>
> 1. It cuts the number of openat/getdents/close syscalls by several
> orders of magnitude.
>
> 2. The overall time drops from ~11.4s to ~10.5s. I suppose if I timed
> only the `index-pack` process, it would be even higher (as a
> percentage improvement).
Just for fun, I did a "git pack-objects --all --stdout" from linux.git,
and then timed "git index-pack --stdin" on it in an empty repo. With a
configured alternate pointing to another empty repo, just to make it
more unfair. And then I stored it all on a ramdisk to emphasize the cost
of the syscalls versus hitting the disk. The numbers I got were:
[before]
real 2m13.093s
user 3m31.884s
sys 0m55.208s
[after]
real 1m40.389s
user 3m10.776s
sys 0m26.012s
That's sort of a ridiculous test, but it does show that this was having
some impact even on "normal" systems without insane syscall latencies.
-Peff
next prev parent reply other threads:[~2015-06-09 17:41 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-21 13:13 Troubleshoot clone issue to NFS steve.norman
2015-05-21 14:00 ` Christian Couder
2015-05-21 14:31 ` Duy Nguyen
2015-05-21 14:38 ` Duy Nguyen
2015-05-21 15:53 ` steve.norman
2015-05-22 0:16 ` Duy Nguyen
2015-05-22 7:12 ` Jeff King
2015-05-22 8:35 ` steve.norman
2015-05-22 10:05 ` Duy Nguyen
2015-05-22 14:37 ` Junio C Hamano
2015-05-22 15:02 ` steve.norman
2015-05-22 23:51 ` [PATCH/RFC 0/3] using stat() to avoid re-scanning pack dir Jeff King
2015-05-22 23:51 ` [PATCH 1/3] stat_validity: handle non-regular files Jeff King
2015-05-23 11:00 ` Michael Haggerty
2015-05-24 8:29 ` Jeff King
2015-05-22 23:52 ` [PATCH 2/3] cache.h: move stat_validity definition up Jeff King
2015-05-22 23:54 ` [PATCH 3/3] prepare_packed_git: use stat_validity to avoid re-reading packs Jeff King
2015-05-23 1:19 ` [PATCH/RFC 0/3] using stat() to avoid re-scanning pack dir Duy Nguyen
2015-05-23 1:21 ` Duy Nguyen
2015-05-24 8:20 ` Jeff King
2015-05-24 9:00 ` Troubleshoot clone issue to NFS Duy Nguyen
2015-06-05 12:01 ` steve.norman
2015-06-05 12:18 ` Jeff King
2015-06-05 12:29 ` [PATCH] index-pack: avoid excessive re-reading of pack directory Jeff King
2015-06-09 17:24 ` Jeff King
2015-06-09 17:41 ` Jeff King [this message]
2015-06-10 3:46 ` Shawn Pearce
2015-06-10 14:00 ` Jeff King
2015-06-10 14:36 ` Duy Nguyen
2015-06-10 21:34 ` Shawn Pearce
2015-06-05 14:20 ` Troubleshoot clone issue to NFS steve.norman
2015-06-16 20:50 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150609174106.GB32034@peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=pclouds@gmail.com \
--cc=steve.norman@thomsonreuters.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).