Re: [PATCH] Fix random fast-import errors when compiled with NO_MMAP

["Shawn O. Pearce" <spearce@spearce.org>]

Junio C Hamano <gitster@pobox.com> wrote:
> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
> > Btw, even with Shawn's patch, I wonder if the index_data usage is correct.
...
> So I do not think "index_data usage" can be incorrect, as there
> won't be any index_data usage with unfinalized pack, and the
> core part of git would not even have any mmap(2) (nor open fd)
> into its idx file before it is finalized.

Correct.
 
> By the way, I was quite puzzled how the gfi_unpack_entry()
> function manages to work correctly when it has to read an object
> it deltified based on another object it wrote into the same
> unfinalized pack earlier.  It knows where in the unfinalized
> pack it wrote the object, so it can find from its own "struct
> object_entry" the offset for the object, and calls
> unpack_entry() defined in the core to do the rest.
> 
> However, most of the core does not really know about the other
> objects in this half-built pack.  If the object is a delta,
> unpack_delta_entry() needs to find the delta base.  And it needs
> to do that without having the idx.
> 
> The trick (the code really needs a bit more documentation) is
> that gfi never writes anything but OFS_DELTA.  So the core, even
> though it does not have the corresponding idx file, does not
> have to look up the object (in fact it does not even know what
> object to look up for the base, it only knows the offset).

Yup.

Older fast-imports (pre OFS_DELTA) had to replicate a good chunk of
the unpack_entry() logic directly inside of fast-import.  But it
later occurred to me that OFS_DELTA simplifies the code and lets
me reuse more of the existing sha1_file.c implementation.

So yea, fast-import only emits OFS_DELTA, and does so only so that it
can pull this delta-base-unpack trick on the core, without actually
giving the core the corresponding idx file first.

-- 
Shawn.