Re: pack operation is thrashing my server

[Junio C Hamano <gitster@pobox.com>]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> Interestingly, it turns out that git also hits a sad performance downside 
> of using zlib.
>
> We always tend to set "stream.avail_out" to the exact size of the expected 
> output. And it turns out that that means that the fast-path case of 
> inffast.c doesn't trigger as often as it could. This (idiotic) patch 
> actually seems to help performance on git rev-list by about 5%.
>
> But maybe it's just me seeing things. But I did this because of the entry 
> assumptions in inflate_fast(), that code only triggers for the case of 
> strm->avail_out >= 258.
>
> Sad, if true.

This is reproducible  "rev-list --objects --all" in my copy of the kernel
repo takes around 47-48 seconds user time, and with the (idiotic) patch it
is cut down to 41-42 seconds.

(with patch)
41.41user 0.51system 0:41.93elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+134411minor)pagefaults 0swaps

(without patch)
47.21user 0.64system 0:47.85elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+134935minor)pagefaults 0swaps

One funny thing about your patch is that it also reduces the number of
minor faults; I would have expected that the additional memory wastage
(even though most of the allocated object buffer memory would be freed
immediately as soon as the caller is done with it) would result in larger
number of faults, not smaller, which is puzzling.