Re: [RFC] super indexes to span multiple packfiles

[Nicolas Pitre <nico@cam.org>]

On Tue, 29 May 2007, Avi Kivity wrote:

> Jon Smirl wrote:
> > 
> > My work with databases leads me to believe that figuring out how to
> > pack everything into a smaller space always beats efforts put into
> > incrementally improving the indexing scheme. Packing into a smaller
> > space reduces the total IO needs and that's always a winner.
> > 
> 
> Another way to achieve that is to place objects that are accessed together
> nearby, and issue a larger read so as to bring them into cache.  I imagine
> that placing commit objects and associated tree and blobs in history order
> should help here (but maybe git already does that, I'm not familiar with the
> internals).

GIT already does that indeed, except for commit objects which are all 
together for better performances on history traversal operations.

After a fresh repack, the checkout of the latest revision should produce 
a nearly perfect linear and contigous access into the early portion of 
the same pack.  Things will get more random with access to objects 
further back in history of course, but those objects are less likely to 
be accessed as often.


Nicolas