From: Linus Torvalds <torvalds@osdl.org>
To: Junio C Hamano <junkio@cox.net>
Cc: Chris Lee <chris133@gmail.com>,
"Shawn O. Pearce" <spearce@spearce.org>,
git@vger.kernel.org
Subject: Re: git-svnimport failed and now git-repack hates me
Date: Fri, 5 Jan 2007 15:17:30 -0800 (PST) [thread overview]
Message-ID: <Pine.LNX.4.64.0701051515000.3661@woody.osdl.org> (raw)
In-Reply-To: <7v64bldqas.fsf@assigned-by-dhcp.cox.net>
On Fri, 5 Jan 2007, Junio C Hamano wrote:
>
> The problem Linus pointed out was that your SHA1_Update()
> implementations may not be prepared to hash the whole 2.3GB in
> one go. The one in "master" (and "maint", although I haven't
> done a v1.4.4.4 maintenance release yet) calls SHA1_Update()
> in chunks to work around that potential issue.
Well, I think Chris is worried about having it all mapped at the same
time.
It does actually end up forcing the kernel to do more work (it's harder to
re-use a mapped page than it is to reuse one that isn't), and in that
sense, if you have less than <n> GB of RAM and can't just keep it all in
memory at the same time, doing one large mmap is possibly more expensive
than chunking things up.
That said, I doubt it's a huge problem. If you can't fit the whole file in
memory, your real performance issue is going to be the IO, not the fact
that the kernel has to work a bit harder at unmapping pages ;)
Linus
next prev parent reply other threads:[~2007-01-05 23:17 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-01-03 23:52 git-svnimport failed and now git-repack hates me Chris Lee
2007-01-04 1:59 ` Linus Torvalds
2007-01-04 2:06 ` Shawn O. Pearce
2007-01-04 2:35 ` Shawn O. Pearce
2007-01-04 2:36 ` Chris Lee
2007-01-04 2:45 ` Shawn O. Pearce
2007-01-04 2:53 ` Chris Lee
2007-01-04 2:57 ` Shawn O. Pearce
2007-01-04 2:58 ` Chris Lee
2007-01-04 3:05 ` Shawn O. Pearce
2007-01-04 3:06 ` Chris Lee
2007-01-04 2:16 ` Chris Lee
2007-01-04 17:56 ` Chris Lee
2007-01-04 18:30 ` Linus Torvalds
2007-01-04 18:54 ` Chris Lee
2007-01-04 2:33 ` Eric Wong
2007-01-04 2:40 ` Randal L. Schwartz
2007-01-04 3:13 ` Eric Wong
2007-01-05 2:09 ` [PATCH] git-svn: make --repack work consistently between fetch and multi-fetch Eric Wong
2007-01-04 6:25 ` git-svnimport failed and now git-repack hates me Junio C Hamano
2007-01-04 7:26 ` [PATCH] pack-check.c::verify_packfile(): don't run SHA-1 update on huge data Junio C Hamano
2007-01-04 17:58 ` git-svnimport failed and now git-repack hates me Chris Lee
2007-01-04 20:22 ` Junio C Hamano
2007-01-05 17:19 ` Chris Lee
2007-01-05 19:05 ` Junio C Hamano
2007-01-05 19:33 ` Chris Lee
2007-01-05 19:39 ` Shawn O. Pearce
2007-01-05 20:48 ` Chris Lee
2007-01-05 21:37 ` Junio C Hamano
2007-01-05 21:57 ` Linus Torvalds
2007-01-05 22:18 ` alan
2007-01-07 0:36 ` Eric Wong
2007-01-05 22:39 ` Linus Torvalds
2007-01-05 22:48 ` Junio C Hamano
2007-01-05 23:00 ` Linus Torvalds
2007-01-05 23:02 ` Linus Torvalds
2007-01-05 23:44 ` Junio C Hamano
2007-01-05 23:59 ` Linus Torvalds
2007-01-06 0:06 ` Johannes Schindelin
2007-01-05 23:03 ` Chris Lee
2007-01-05 23:09 ` Junio C Hamano
2007-01-05 23:17 ` Linus Torvalds [this message]
2007-01-05 23:58 ` Junio C Hamano
2007-01-06 0:11 ` Linus Torvalds
2007-01-06 0:15 ` Linus Torvalds
2007-01-06 0:23 ` Junio C Hamano
2007-01-06 1:22 ` Linus Torvalds
2007-01-04 19:24 ` Chris Lee
2007-01-04 21:12 ` Linus Torvalds
2007-01-04 21:31 ` Sasha Khapyorsky
2007-01-04 22:04 ` Chris Lee
2007-01-07 0:17 ` [PATCH] git-svnimport: support for incremental import Sasha Khapyorsky
2007-01-07 18:12 ` Chris Lee
2007-01-07 18:59 ` Sasha Khapyorsky
2007-01-08 2:22 ` [PATCH] git-svnimport: fix edge revisions double importing Sasha Khapyorsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.64.0701051515000.3661@woody.osdl.org \
--to=torvalds@osdl.org \
--cc=chris133@gmail.com \
--cc=git@vger.kernel.org \
--cc=junkio@cox.net \
--cc=spearce@spearce.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).