From: Jeff King <peff@peff.net>
To: Joachim B Haga <cjhaga@fys.uio.no>
Cc: git@vger.kernel.org
Subject: Re: Compression speed for large files
Date: Mon, 3 Jul 2006 17:45:03 -0400 [thread overview]
Message-ID: <20060703214503.GA3897@coredump.intra.peff.net> (raw)
In-Reply-To: <loom.20060703T124601-969@post.gmane.org>
On Mon, Jul 03, 2006 at 11:13:34AM +0000, Joachim B Haga wrote:
> often binary. In git, committing of large files is very slow; I have
> tested with a 45MB file, which takes about 1 minute to check in (on an
> intel core-duo 2GHz).
I know this has already been somewhat solved, but I found your numbers
curiously high. I work quite a bit with git and large files and I
haven't noticed this slowdown. Can you be more specific about your load?
Are you sure it is zlib?
On my 1.8Ghz Athlon, compressing 45MB of zeros into 20K takes about 2s.
Compressing 45MB of random data into a 45MB object takes 6.3s. In either
case, the commit takes only about 0.5s (since cogito stores the object
during the cg-add).
Is there some specific file pattern which is slow to compress?
-Peff
next prev parent reply other threads:[~2006-07-03 21:45 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-07-03 11:13 Compression speed for large files Joachim B Haga
2006-07-03 12:03 ` Alex Riesen
2006-07-03 12:42 ` Elrond
2006-07-03 13:44 ` Joachim B Haga
2006-07-03 13:32 ` Joachim Berdal Haga
[not found] ` <Pine.LN X.4.64.0607031030150.1213@localhost.localdomain>
2006-07-03 14:33 ` Nicolas Pitre
2006-07-03 14:54 ` Yakov Lerner
2006-07-03 15:17 ` Johannes Schindelin
2006-07-03 16:31 ` Linus Torvalds
2006-07-03 18:59 ` [PATCH] Make zlib compression level configurable, and change default Joachim B Haga
2006-07-03 19:33 ` Linus Torvalds
2006-07-03 19:50 ` Linus Torvalds
2006-07-03 20:11 ` Joachim B Haga
2006-07-03 19:02 ` [PATCH] Use configurable zlib compression level everywhere Joachim B Haga
2006-07-03 19:43 ` Junio C Hamano
2006-07-07 21:53 ` David Lang
2006-07-08 2:10 ` Johannes Schindelin
2006-07-03 21:45 ` Jeff King [this message]
2006-07-03 22:25 ` Compression speed for large files Joachim Berdal Haga
2006-07-03 23:02 ` Linus Torvalds
2006-07-04 5:42 ` Joachim Berdal Haga
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060703214503.GA3897@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=cjhaga@fys.uio.no \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).