Re: [RFC] super indexes to span multiple packfiles

["Jon Smirl" <jonsmirl@gmail.com>]

Object's are not accessed in random order with git. Once an object
reference hits a pack file it is very likely that following references
will hit the same pack file. That's because you always find object
SHA's by following the chains.

So first place to look for an object is the same place the previous
object was found. If it isn't there order the search of the pack files
by creation data (just a heuristic). Make this list a circle and start
the search in the pack where the previous object was found. This can
all be done with the existing indexes.

I haven't been reading all of the messages on this subject, but is
this strategy enough to eliminate the need for a super index?

If you still need a super index, note that it may be good enough for
it to only contain the SHA's for objects that are externally
referenced. This index would be small and simply point to the correct
pack file index to find the object in. You could add a list of
dangling links to each packfile index to assist with building this
super index.

My work with databases leads me to believe that figuring out how to
pack everything into a smaller space always beats efforts put into
incrementally improving the indexing scheme. Packing into a smaller
space reduces the total IO needs and that's always a winner.

-- 
Jon Smirl
jonsmirl@gmail.com