Re: [BUG?] fetch into shallow sends a large number of objects

[Duy Nguyen <pclouds@gmail.com>]

On Tue, Mar 8, 2016 at 6:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jeff King <peff@peff.net> writes:
>
>> I don't know how the client invoked git, but we can guess what happened
>> and simulate with:
>>
>>   git tag shallow ecd7ea6033fe8a05d5c21f3a54355fded6942659
>>   git tag old 067f265bb512c95b22b83ccd121b9facbddcf6b1
>>   git tag new d7a6d9295d718c6015be496880f1a293bdd89185
>>
>>   git clone --no-local --bare --branch=shallow --depth=1 . clone.git
>>   cd clone.git
>>   git fetch origin old:refs/tags/old
>>   git fetch origin new:refs/tags/new
>>
>> Of the two follow-up fetches in the clone, the first is reasonably fast
>> (it just grabs a few new commits on top of the shallow base), but the
>> second is expensive (it grabs the merge which pulls in the whole
>> history). If we add "--depth=1" to each of those fetches, everything
>> remains fast.
>>
>> Is this user error to call "git fetch" without "--depth" in the
>> subsequent cases? Or should git remember that we are in a shallow repo,
>> and presume that the user by default wants to keep things shallow?
>
> Hmph, you shouldn't, and I somehow thought that you do not, have to
> explicitly say things like "--deepen" to break the original
> shallowness, but your example illustrates that the logic to do so is
> not well thought out.  A new side branch will prevent you from
> hitting an already-known shallow cut-off and traverse down to the
> root.

Yep. It "works" by design.

> Giving a random "depth" in subsequent fetch would however not work
> very well, I suspect, as that is very prone to make the part of the
> history the user originally obtained, and presumably used to build
> her own history, into an island that is unconnected to the updated
> tip of the history.

The new --deepen, --shallow-since and --shallow-exclude should be
better in this aspect and we can send them all the time without
affecting original cut points. Well, deepen can't be used here because
it needs shallow cut points as anchor in the first place.

> I also do not offhand think of a good way to use the topology or
> timestamp to figure out the best "depth" to truncate the side branch
> at.  The server side may be able to figure out that things before 'F'
> in your picture is not relevant for a client that has the shallow
> cut-off at 067f265, but the side branch can be forked arbitrarily
> long in the past, or it may not even share the ancient part of the
> history and has its own root commit.

If a shallow point can reach root without seeing another shallow
point, we can mark all reachable commits from it shallow. If it sees
another shallow point, maybe we can mark at the merge point of them..
We can also send "here is --depth=10, but only apply it on new refs".
That should mitigate the problem a bit. But I'm not sure if I can
solve it completely.
-- 
Duy