Re: git pull is slow

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

Hi,

On Sun, 13 Jul 2008, Shawn O. Pearce wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Sat, 12 Jul 2008, Stephan Hennig wrote:
> > > 
> > > Thanks for having a look at this!  What does "problem with the pack" 
> > > mean?  Do you think it is a Git problem (client or server side?) or just 
> > > a misconfiguration?
> > 
> > I thought that the blobs in the pack are just too similar.  That makes for 
> > a good compression, since you get many relatively small deltas.  But it 
> > also makes for a lot of work to reconstruct the blobs.
> > 
> > I suspected that you run out of space for the cache holding some 
> > reconstructed blobs (to prevent reconstructing all of them from scratch).
> ...
> > Whoa. As you can see, your puny little 3.3 megabyte pack is blown to a 
> > full 555 megabyte in RAM.
> ...
> > I expect this to touch the resolve_delta() function of index-pack.c in a 
> > major way, though.
> 
> Yea, that's going to be ugly.  The "cache" you speak of above is held on 
> the call stack as resolv_delta() recurses through the delta chain to 
> reconstruct objects and generate their SHA-1s.  There isn't a way to 
> release these objects when memory gets low so your worst case scenario 
> is a 100M+ blob with a delta chain of 50 or more - that will take you 5G 
> of memory to pass through index-pack.

Actually, there is...

You would only need to tap into the "release_pack_memory()" mechanism, 
adding a sort of a "smart pointer" that knows how to reconstruct its 
contents.

Ciao,
Dscho