Re: git gc --aggressive led to about 40 times slower "git log --raw"

[David Kastrup <dak@gnu.org>]

David Kastrup <dak@gnu.org> writes:

> Duy Nguyen <pclouds@gmail.com> writes:
>
>> I can think of two improvements we could make, either increase cache
>> size dynamically (within limits) or make it configurable. If we have N
>> entries in worktree (both trees and blobs) and depth M, then we might
>> need to cache N*M objects for it to be effective. Christian, if you
>> want to experiment this, update MAX_DELTA_CACHE in sha1_file.c and
>> rebuild.
>
> Well, my optimized "git-blame" code is considerably hit by an
> aggressively packed Emacs repository so I took a look at it with the
> MAX_DELTA_CACHE value set to the default 256, and then 512, 1024, 2048.

[...]

> Trying with 16384:
> dak@lola:/usr/local/tmp/emacs$ time ../git/git blame src/xdisp.c >/dev/null
>
> real	2m8.000s
> user	0m54.968s
> sys	1m12.624s
>
> And memory consumption did not exceed about 200m all the while, so is
> far lower than what would have been available.

Of course, this has to do with delta_base_cache_limit defaulting to 16m.

> Something's _really_ fishy about that cache behavior.  Note that the
> _system_ time goes up considerably, not just user time.  Since the
> packs are zlib-packed, it's reasonable that more I/O time is also
> associated with more user time and it is well possible that the user
> time increase is entirely explainable by the larger amount of
> compressed data to access.
>
> But this stinks.

And an obvious contender for the stinking is that the "LRU" scheme used
here is _strictly_ freeing memory based on which cache entry has been
_created_ the longest time ago, not which cache entry has been
_accessed_ the longest time ago.  Which means a pure round-robin
strategy for freeing memory rather than LRU.

Let's see what happens when changing this.

-- 
David Kastrup