Re: [PATCH] pack-objects: only throw away data during memory pressure

[Nicolas Pitre <nico@cam.org>]

On Mon, 11 Feb 2008, Johannes Schindelin wrote:

> Hi,
> 
> On Mon, 11 Feb 2008, Martin Koegler wrote:
> 
> > What about this not really tested patch for dealing with memory pressure 
> > in git-pack-objects?
> > 
> > It will slow down the repack in the case of memory pressure, but missing 
> > memory will not affect the results.
> 
> It almost helped:
> 
> $ /usr/bin/time git repack -a -d -f --window=250 --depth=250
> Counting objects: 2477715, done.
> fatal: Out of memory, malloc failed411764)
> Command exited with non-zero status 1
> 10050.12user 240.63system 2:53:37elapsed 98%CPU (0avgtext+0avgdata 
> 0maxresident)k
> 0inputs+0outputs (29555major+94032945minor)pagefaults 0swaps
> 
> So, it ran longer until it ran out of memory.

What it can do for you is to limit the window memory usage much more 
without affecting the end result, say to 128MB.  Of course the repack is 
then going to progress much slower if active purging of the window 
memory is involved.

If you still run out of memory at that point then there is not much more 
to do besides using a new memory allocator that doesn't suffer as much 
from memory fragmentation, or find the possible memory leak that no one 
else found so far.


Nicolas