Re: Partitioned packs

[Linus Torvalds <torvalds@linux-foundation.org>]

On Tue, 3 Apr 2007, Chris Lee wrote:
> 
> These experiments include trying to see if there's a noticeable
> performance improvement by splitting out objects of different types
> into different packs. So far, it definitely seems to make a
> difference, though not the one I was initially expecting. For all of
> these tests, I did 'sysctl -w vm.drop_caches=3' before running, to
> effectively simulate a cold-cache run.

Ok, the wordwrap makes it a bit hard to read, but it looks like the 
single-pack always wins. Sometimes by a huge amount.

The reason is simple: not only are single packs well sorted anyway (so if 
you only look at commits, it will only look at the head of the pack 
anyway), but a single pack is much faster to look things up in: you can do 
a single binary lookup.

If you have multiple packs, you *may* be able to do a single binary 
lookup, but quite often you'll do one *failing* binary lookup, and then go 
on to the next pack - in other words, you'll do a linear search over a set 
of binary lookups.

So trying to partition things doesn't help (because the objects are 
already well sorted), and it does hurt.

That said, for most operations it's probably in the noise. Something bad 
happened for your "git-blame" thing for the "commits" and "everything 
else" case. Perhaps just unlucky ordering of packs.

		Linus