Re: Git chokes on large file

[worley@alum.mit.edu (Dale R. Worley)]

> From: David Lang <david@lang.hm>

> well, as others noted, the problem is actually caused by doing the diffs, and 
> that is something that is a very common thing to do with source code.

To some degree, my attitude comes from When I Was A Boy, when you got
16k for both your bytecode and your data, so you never kept more than
one line of a file in memory unless you had to.

Regardless of that, the problem with "git fsck" is not due to doing
diffs, and "git commit" by default does diffs even if you don't ask
for them, so the observed problems cannot be subsumed under the label
"you asked for a diff of a file that can't be diffed".

> And I would assume that files of several MB would be able to fit in
> memory (again, this was assumed to be for development, and compilers
> take a lot of ram to run, so having enough ram to hold any
> individual source file while the compiler is _not_ using ram doesn't
> seem likely to be a problem)

At least the first versions of GCC only kept one function (and the
global symbol table) in memory at once, so you could compile a source
file that was larger than the available memory.

In any case, if Git is no longer limited to handling source files, it
needs to be updated so it can handle large files.

Dale