Re: git gc --aggressive led to about 40 times slower "git log --raw"

[David Kastrup <dak@gnu.org>]

Christian Jaeger <chrjae@gmail.com> writes:

> I've got a repository where "git log --raw > _somefile" took a few
> seconds in the past, but after an attempt at merging some commits that
> were collected in a clone of the same repo that was created about a
> year ago, I noticed that this command was now taking 3 minutes 7
> seconds. "git gc", "git fsck", "git clone file:///the/repo/.git" also
> now each took between ~4-10 minutes, also "git log --raw somefile" got
> equally unusably slow. With the help of the people on the IRC, I
> tracked it down to my recent use of "git gc --aggressive" in this
> repo. Running "git repack -a -d -f" solved it, now it's again taking
> 4-5 seconds. After running "git gc --aggressive" again for
> confirmation, "git log --raw > _somefile" was again slowed down,
> although now 'only' to 1 minute 34 seconds;

[...]

> I've now learned to avoid "git gc --aggressive". Perhaps there are
> some other conclusions to be drawn, I don't know.

I've seen the same with my ongoing work on git-blame with the current
Emacs Git mirror.  Aggressive packing reduces the repository size to
about a quarter, but it blows up the system time (mainly I/O)
significantly, quite reducing the total benefits of my algorithmic
improvements there.

There is also some quite visible additional time spent in zlib, so a
wild guess would be that zlib is not really suited to the massive amount
of directory entries of a Git object store.  Since the system time still
dominates, this guess would only make sense if Git over zlib kept
rereading the directory section of whatever compressed file we are
talking about.  But that's really a rather handwavy wild guess without
anything better than a hunch to back it up.  I don't even know what kind
of compression and/or packs are used: I've only ever messed myself with
the delta coding of the normal "unpacked" operation (there are a few
older commits from me on that).

-- 
David Kastrup