Re: Git chokes on large file

[Junio C Hamano <gitster@pobox.com>]

Duy Nguyen <pclouds@gmail.com> writes:

>>     $ git fsck --full --strict
>>     notice: HEAD points to an unborn branch (master)
>>     Checking object directories: 100% (256/256), done.
>>     fatal: Out of memory, malloc failed (tried to allocate 21474836481 bytes)
>
> Back trace for this one
> ...
> Not easy to fix. I started working on converting fsck to use
> index-pack code for pack verification. index-pack supports large files
> well, so in the end it might fix this (as well as speeding up fsck).
> But that work has stalled for a long time.

You need to have enough memory (virtual is fine if you have enough
time) to do fsck.  Some part of index-pack could be refactored into
a common helper function that could be called from fsck, but I think
it would be a lot of work.

>>     $ git commit -m Test.
>>     [master (root-commit) 3df3655] Test.
>>     fatal: Out of memory, malloc failed (tried to allocate 21474836481 bytes)

I suspect that this one is only because you are letting the status
which involves diff to kick in (and for that you would need to have
enough memory).  As you suggested, it might be a good idea to take
bigfilethreshold account when deciding if we would want to run diff.