From: git@jeffhostetler.com
To: git@vger.kernel.org
Cc: gitster@pobox.com, peff@peff.net,
Jeff Hostetler <jeffhost@microsoft.com>
Subject: [PATCH v2 0/2] read-cache: call verify_hdr() in a background thread
Date: Mon, 27 Mar 2017 21:09:37 +0000 [thread overview]
Message-ID: <1490648979-49580-1-git-send-email-git@jeffhostetler.com> (raw)
From: Jeff Hostetler <jeffhost@microsoft.com>
Version 2 of this patch series simplifies this to just
turn off the hash verification. Independent comments
from Linus and Peff suggested that we could just turn
this off and not worry about it. So I've updated this
patch to do that. I added a global variable to allow
the original code path to be used. I also added a
t/helper command to demonstrate the differences.
On the Linux repo, the effect is rather trivial:
$ ~/work/gfw/t/helper/test-skip-verify-index -c 3
0.029884 0 [cache_nr 57994]
0.031035 0 [cache_nr 57994]
0.024308 0 [cache_nr 57994]
0.028409 0 avg
0.018359 1 [cache_nr 57994]
0.017025 1 [cache_nr 57994]
0.011087 1 [cache_nr 57994]
0.015490 1 avg
On my Windows source tree (450MB index), I'm seeing a
savings of 0.6 seconds -- read_index() went from 1.2 to 0.6
seconds.
=====================
This patch contains a performance optimization to run
verify_hdr() in a background thread while the foreground
thread parses the index. This allows do_read_index() to
complete faster.
This idea was recently discussed on the mailing list in:
https://public-inbox.org/git/85221b97-759f-b7a9-1256-21515d163cbf@jeffhostetler.com/
during a discussion on sha1dc.
Our testing on Windows showed that verifying the SHA1 hash
on the index file takes approximately the same amount of time
as parsing the file and building the array of cache_entries.
(It could also be that having 2 threads better ammortizes the
page faults of reading from the mmap'd file.)
An earlier version of this change has been in use in GfW since December:
https://github.com/git-for-windows/git/pull/978
Jeff Hostetler (2):
read-cache: skip index SHA verification
skip_verify_index: helper test
Makefile | 1 +
cache.h | 5 +++
read-cache.c | 6 ++++
t/helper/.gitignore | 1 +
t/helper/test-skip-verify-index.c | 73 +++++++++++++++++++++++++++++++++++++++
5 files changed, 86 insertions(+)
create mode 100644 t/helper/test-skip-verify-index.c
--
2.7.4
next reply other threads:[~2017-03-27 21:09 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-27 21:09 git [this message]
2017-03-27 21:09 ` [PATCH v2 1/2] read-cache: skip index SHA verification git
2017-03-27 22:44 ` Jeff King
2017-03-27 23:32 ` Jonathan Nieder
2017-03-27 23:39 ` Jeff King
2017-03-28 15:27 ` Jeff Hostetler
2017-03-28 15:37 ` Jeff King
2017-03-27 21:09 ` [PATCH v2 2/2] skip_verify_index: helper test git
2017-03-27 22:45 ` [PATCH v2 0/2] read-cache: call verify_hdr() in a background thread Jeff King
2017-03-28 15:30 ` Jeff Hostetler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1490648979-49580-1-git-send-email-git@jeffhostetler.com \
--to=git@jeffhostetler.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jeffhost@microsoft.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).