Re: git-svnimport failed and now git-repack hates me

[Junio C Hamano <junkio@cox.net>]

"Chris Lee" <chris133@gmail.com> writes:

> On 1/5/07, Junio C Hamano <junkio@cox.net> wrote:
>> Subject: [PATCH] builtin-prune: memory diet.
>>
>> Somehow we forgot to turn save_commit_buffer off while walking
>> the reachable objects.  Releasing the memory for commit object
>> data that we do not use matters for large projects (for example,
>> about 90MB is saved while traversing linux-2.6 history).
>
> Is git-verify-pack supposed to mmap the entire packfile? Because the
> version I have maps 2.3GB into RAM and keeps it there until it's done.

Yes -- we need to hash the whole thing as well as doing other
checks on it.  Sliding mmap() in "next" will mmap that in chunks
of 32MB or 1GB, but its needing to read every byte of it does
not change.

The problem Linus pointed out was that your SHA1_Update()
implementations may not be prepared to hash the whole 2.3GB in
one go.  The one in "master" (and "maint", although I haven't
done a v1.4.4.4 maintenance release yet) calls SHA1_Update()
in chunks to work around that potential issue.