Re: Change set based shallow clone

[Junio C Hamano <junkio@cox.net>]

"Martin Langhoff" <martin.langhoff@gmail.com> writes:

> As far as I can see, this would not need any change to the upload-pack
> protocol.

That I have to disagree.  Take your earlier example of "I have X
and I learned you have A B C D".  Now the fetch that got X was a
commit-only one (but you have full tree for X), but you got X
from somebody else, not the uploader you are talking with right
now.  There is a common ancestor Y somewhere behind X, but
between Y and X you lack trees and blobs.

How would the current protocol work (I am explaining that it
won't be "not need any change")?  After you express interest for
A, B, C, D with "want" messages, you start telling "have X",
"have X~1", have "X~2",... to the uploader (X, X~1 etc. are
object names not symbolic).  Eventually the uploader would
recognize Y that is an ancestor of X that it has and will Ack
it, you stop traversing the ancestor of an acked one and say
"done".  So now we have a common ancestor and the upload side
knows it can omit commits behind  that common commit Y, trees
and blobs contained in Y.

See an Oops here?  You do NOT have trees and blobs associated
with commit Y.

I am not saying we should not change the protocol.  I am just
trying to explain that the problem is not something you can
fudge without changing the protocol.

As a first level approximation, we could in addition to the
commit object name have a bit that says "do I have trees and
blobs associated with the commit" bit on each "have" message (by
the way, this is _expensive_.  You have to traverse down the
tree contained in each commit using has_sha1_file() recursively
to see if you have anything missing from _each_ commit you send
to the other).  Alternatively, you can say "I have this commit,
but I do not have this tree and I do not even know if I have
blobs needed to complete that tree because I do not know what
that tree I am missing contains -- I may have them, I may not. I
truly do not know" to convey the same information.

Think about it a bit -- saying "I know I am missing this tree"
is one thing, but if we end up saying "I do not even know what I
am missing", it is like saying "don't care what I say, just send
everything to me".  Are we gaining much by having only commit
objects on our end?

Once you end up sending a full tree, it is like doing an initial
checkout over CVS when you think you are incrementally fetching,
and you are already lost.  For example, a recent kernel tarball
compressed with gzip is around 50MB; the history for 34k commits
since 2.6.12-rc2 fully packed is slightly less than 150MB.