Re: pack operation is thrashing my server

[Nicolas Pitre <nico@cam.org>]

On Sun, 7 Sep 2008, Jon Smirl wrote:

> On 9/7/08, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> >
> >
> >  On Sat, 6 Sep 2008, Jon Smirl wrote:
> >  >
> >
> > > When I was playing with those giant Mozilla packs speed of zlib wasn't
> >  > a big problem. Number one problem was the repack process exceeding 3GB
> >  > which forced me to get 64b hardware and 8GB of memory. If you start
> >  > swapping in a repack, kill it, it will probably take a month to
> >  > finish.
> >
> >
> > .. and you'd make things much much WORSE?
> 
> My observations on the Mozilla packs indicated that the problems were
> elsewhere in git, not in the decompression algorithms. Why does a
> single 2000 delta chain take 15% of the entire pack time? Something
> isn't right when long chains are processed which triggers far more
> decompressions than needed.

Please have a look at commit eac12e2d4d7f.  This fix improved things for 
my gcc repack tests.


Nicolas