Re: [PATCH 1/3] Lazily open pack index files on demand

[Nicolas Pitre <nico@cam.org>]

On Sun, 27 May 2007, Shawn O. Pearce wrote:

> Nicolas Pitre <nico@cam.org> wrote:
> > On Sat, 26 May 2007, Dana How wrote:
> > 
> > > On 5/26/07, Shawn O. Pearce <spearce@spearce.org> wrote:
> > > > In pack v4 we're likely to move the SHA-1 table from the .idx file
> > > > into the front of the .pack file.  This makes the .idx file hold
> > > > only the offsets and the CRC checkums of each object.  If we start
> > > > making a super index, we have to duplicate the SHA-1 table twice
> > > > (once in the .pack, again in the super index).
> > > 
> > > Hmm, hopefully the SHA-1 table can go at the _end_
> > > since with split packs that's the only time we know the number
> > > of objects in the pack... ;-)
> > 
> > Hmmm good point to consider.
> 
> The problem with putting the SHA-1 table at the end of the pack is
> it ruins the streaming for both unpack-objects and index-pack if
> we were to ever use pack v4 as a transport format.  Or just try
> to run a pack v4 packfile through unpack-objects, just locally,
> say to extract megablobs.  ;-)

Right.  In fact I think the SHA1 table could still remain at the 
beginning even if we don't know yet that the pack will be split. It 
would just happen to contain redundent entries.

In fact it would be impossible to store it at the end in the hope of 
trimming it down according to the written objects because the idea is to 
have commit and tree objects index into that table.  So if you cull 
entries from the table then indices in already written out objects won't 
match.


Nicolas