Re: Mozilla .git tree

[Shawn Pearce <spearce@spearce.org>]

Junio C Hamano <junkio@cox.net> wrote:
> > I think we probably should have done this when the binary headers
> > were introduced into loose objects.
> 
> No.  That was purely .pack format and did not affect .idx
> format.  Honestly .idx is purely a local matter and not as
> important to keep stable as the .pack format.

I think you missed the point of my comment.  I apologize for it
not being clear the first time.

The loose objects have format #1 (legacy text format compressed as
one big zlib stream) and now format #2 (pack object header followed
by data-only zlib stream).

In looking at this dictionary packing code that I'm working on right
now I may be introducing format #3 (dictionary based compression
rather than zlib compression).

We have no easy way to mark the version number of the loose objects.
I'm sure we can shoehorn in something though.
 
> > Sure the scheme you outlined allows a 64 bit difference but
> > uncompressed objects already can't be larger than 2**32-1...
> 
> Where do we have that limitation?

sha1_file.c uses "unsigned long" a lot to deal with size of an
object, deflated.  iirc unsigned long is only 32 bits even in many
64 bit architectures.  Or am I wrong?

> I think your "allow zlib to eat the remainder of the current
> window and slide window when it exhausts the current window"
> logic is a very good on and makes it unnecessary to know the
> tail of each object.

I'll send patches to sha1_file.c probably later tonight.  My last
round of offset changes was the start of that, this next round
should finish it.  Its mostly a copy-n-paste from fast-import.c
so it should go quickly.

-- 
Shawn.

-- 
VGER BF report: U 0.960524