From: "Jon Smirl" <jonsmirl@gmail.com>
To: "Git Mailing List" <git@vger.kernel.org>, "Nicolas Pitre" <nico@cam.org>
Subject: Better value for chunk_size when threaded
Date: Thu, 6 Dec 2007 18:58:34 -0500 [thread overview]
Message-ID: <9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com> (raw)
I tried some various ideas out for chunk_size and the best strategy I
found was to simply set it to a constant. How does 20,000 work on
other CPUs?
I'd turn on default threaded support with this change. With threads=1
versus non-threaded there is no appreciable difference in the time.
Is there an API to ask how many CPUs are in the system? It would be
nice to default the number of threads equal to the number of CPUs and
only use pack.threads=X to override.
Making all of this work by default should help when outside people
decide to do a massive import.
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 4f44658..4d73be8 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1645,7 +1645,7 @@ static void ll_find_deltas(struct object_entry
**list, unsigned list_size,
}
/* this should be auto-tuned somehow */
- chunk_size = window * 1000;
+ chunk_size = 20000;
do {
unsigned sublist_size = chunk_size;
with chunk_size = 20000, everything is on a q6600 4GB
threads = 5
time git repack -a -d -f --depth=250 --window=250
real 6m20.123s
user 20m25.841s
sys 0m5.520s
threads = 4
time git repack -a -d -f --depth=250 --window=250
real 6m15.525s
user 20m20.852s
sys 0m5.356s
threads = 4
time git repack -a -d -f
real 1m31.537s
user 3m2.063s
sys 0m3.064s
threads = 1
time git repack -a -d -f --depth=250 --window=250
real 18m46.005s
user 18m43.122s
sys 0m1.228s
threads = 1
time git repack -a -d -f
real 2m57.774s
user 2m54.211s
sys 0m1.228s
Non-threaded
time git repack -a -d -f --depth=250 --window=250
real 18m51.183s
user 18m46.538s
sys 0m1.604s
Non-threaded
time git repack -a -d -f
real 2m54.849s
user 2m51.267s
sys 0m1.412s
--
Jon Smirl
jonsmirl@gmail.com
next reply other threads:[~2007-12-06 23:58 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-12-06 23:58 Jon Smirl [this message]
2007-12-07 1:27 ` Better value for chunk_size when threaded Nicolas Pitre
2007-12-07 1:37 ` Jon Smirl
2007-12-07 3:25 ` Nicolas Pitre
2007-12-10 10:26 ` Andreas Ericsson
2007-12-10 13:46 ` Nicolas Pitre
2007-12-07 8:57 ` Andreas Ericsson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com \
--to=jonsmirl@gmail.com \
--cc=git@vger.kernel.org \
--cc=nico@cam.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).