Re: Questions about git-push for huge repositories

[Jeff King <peff@peff.net>]

On Tue, Sep 08, 2015 at 11:24:06AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > If you turn on reachability bitmaps, git _will_ do the thorough set
> > difference, because it becomes much cheaper to do so. E.g., try:
> >
> >     git repack -adb
> >
> > in repo A to build a single pack with bitmaps enabled. Then a subsequent
> > push should send only a single object (the new commit).
> 
> Hmph, A has the tip of B, and has a new commit B hasn't seen but A
> knows that new commit's tree matches the tree of the tip of B.
> 
> Wouldn't --thin transfer from A to B know to send only that new
> commit object without sending anything below the tree in such a
> case, even without the bitmap?

I started to write about that in my analysis, but it gets confusing
quickly. There are actually many tip trees, because A and B also share
all of their tags. We do not mark every blob of every tip tree as a
preferred base, because it is expensive to do so (and it just clogs our
object array).  Plus this only helps in the narrow circumstance that we
have the exact same tree as the tip (and not, say, the same tree as
master^, which I think it would be unreasonable to expect git to find).

But if we do:

  (cd ../B && git tag | git tag -d)

to delete all of the other tips besides master, leaving only the one
that we know has the same tree, I'd expect git to figure it out.

Certainly I would not expect it to save all of the delta compression,
in the sense that we may throw away on-disk delta bases to older objects
(because we don't realize the other side has those older objects). But I
would have thought before we even hit that phase, adding those objects
as "preferred bases" would have marked them as "do not send" in the
first place.

There is code in have_duplicate_entry() to handle this. I wonder why it
doesn't kick in.

-Peff