Re: [PATCH] Support 64-bit indexes for pack files.

[Troy Telford <ttelford.groups@gmail.com>]

On Monday 26 February 2007, Shawn O. Pearce wrote:

> Nico and I are neck deep in our pack version 4 topic.  That topic
> hits all of the same code you touched with your patch.

It looked like that may have been the case; but I figured it wouldn't hurt.  
I've been coping with the 'too-small' index issue for about a year now, and 
have been on IRC and kept at least a lazy eye on it hoping that it would be 
attended to.

I've decided I need to start doing some C coding again, and it seemed to be as 
good an itch as any to scratch...

> Our topic also requires us to change the index file format, and
> in doing so we have decided to extend the index records to look
> something like the following[*1*]:
>
> 	object SHA-1
> 	64-bit offset within packfile
> 	32-bit index of next object in packfile
>
> The latter field is to help pack-objects reuse existing packfile
> data, as today it needs to sort everything on its own on the fly.
> Having that last field of data will help avoid that, and will keep
> the index nicely aligned for 64-bit accesses to the offset.

Exactly why I left the packfile itself alone.  Well, that and laziness.  I may 
have a huge repository, but I don't have single files so large that it needs 
a 32-bit index for the next object in the packfile.

Most of the things that large I've seen are binary anyway.

> I want to say your patch shouldn't be merged without even bothering
> to review it.  

No hard feelings on my part, it was as much a learning thing for me as 
anything else.

> The last time I was in this part of the git code 
> (implementing sliding mmap window) Nico and Junio also both went in
> here and rewrote huge chunks.  Their changes prevented sliding mmap
> window from applying.  It was 6 months before I got back around to
> rewriting the patch.

Ouch.

> Right now I'm neck deep in pack v4.  I hope to have the topic in
> pu-ready state by some time mid-next week, hopefully in time for
> Junio's git day.  I'm very unlikely to have the time to rewrite the
> topic again until late June/July if something like your patch goes
> in now.
>
> So would you mind waiting a couple of weeks for 64 bit indexes?

All I'm concerned with is being able to handle my (huge) source repository 
with git.  I wrote the patch in the hopes that it would accelerate the 
process, and that I'd learn something.  If all I have to do is wait that's 
fine.  I figured I'd at least be able to bring an idea or two to the table.  
If the code doesn't get accepted, but still get the desired functionality, I 
still met 1/2 of my goals in doing it.
-- 
Troy Telford