git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Jon Smirl" <jonsmirl@gmail.com>
To: "Git Mailing List" <git@vger.kernel.org>, "Nicolas Pitre" <nico@cam.org>
Subject: Better value for chunk_size when threaded
Date: Thu, 6 Dec 2007 18:58:34 -0500	[thread overview]
Message-ID: <9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com> (raw)

I tried some various ideas out for chunk_size and the best strategy I
found was to simply set it to a constant. How does 20,000 work on
other CPUs?

I'd turn on default threaded support with this change. With threads=1
versus non-threaded there is no appreciable difference in the time.

Is there an API to ask how many CPUs are in the system? It would be
nice to default the number of threads equal to the number of CPUs and
only use pack.threads=X to override.

Making all of this work by default should help when outside people
decide to do a massive import.

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 4f44658..4d73be8 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1645,7 +1645,7 @@ static void ll_find_deltas(struct object_entry
**list, unsigned list_size,
        }

        /* this should be auto-tuned somehow */
-       chunk_size = window * 1000;
+       chunk_size = 20000;

        do {
                unsigned sublist_size = chunk_size;


with chunk_size = 20000, everything is on a q6600 4GB

threads = 5
time git repack -a -d -f --depth=250 --window=250
real    6m20.123s
user    20m25.841s
sys     0m5.520s


threads = 4
time git repack -a -d -f --depth=250 --window=250
real    6m15.525s
user    20m20.852s
sys     0m5.356s

threads = 4
time git repack -a -d -f
real    1m31.537s
user    3m2.063s
sys     0m3.064s

threads = 1
time git repack -a -d -f --depth=250 --window=250
real    18m46.005s
user    18m43.122s
sys     0m1.228s

threads = 1
time git repack -a -d -f
real    2m57.774s
user    2m54.211s
sys     0m1.228s

Non-threaded
time git repack -a -d -f --depth=250 --window=250
real    18m51.183s
user    18m46.538s
sys     0m1.604s

Non-threaded
time git repack -a -d -f
real    2m54.849s
user    2m51.267s
sys     0m1.412s

-- 
Jon Smirl
jonsmirl@gmail.com

             reply	other threads:[~2007-12-06 23:58 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-12-06 23:58 Jon Smirl [this message]
2007-12-07  1:27 ` Better value for chunk_size when threaded Nicolas Pitre
2007-12-07  1:37   ` Jon Smirl
2007-12-07  3:25     ` Nicolas Pitre
2007-12-10 10:26   ` Andreas Ericsson
2007-12-10 13:46     ` Nicolas Pitre
2007-12-07  8:57 ` Andreas Ericsson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com \
    --to=jonsmirl@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=nico@cam.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).