Re: Some issues when trying to set up a shallow git mirror server

[Richard Maw <richard.maw@codethink.co.uk>]

On Fri, Jan 08, 2016 at 01:37:02PM -0800, Junio C Hamano wrote:

Thanks for the write-up.
I knew most of how the protocol worked, but not all,
and I appreciate the write-up of how the extension could work.

If I have time I can see if I can implement this protocol extension myself.

<snip>
> While that consistency principle must hold everywhere in our system,
> it does not mean a client cannot ask a server to do something whose
> result it has to trust, at least to some degree, because there is
> fundamentally no way to independenty verify the result.

So there'll be a greater level of trust required in the transport protocol,
so you wouldn't necessarily want to allow this protocol over git:// or http://.

<snip>
> So if you want to do this, a new protocol extension needs to allow
> your updated sender (upload-pack) and receiver (fetch-pack) to work
> more like this:
> 
>  * The sender would advertise "I support that extension", while
>    giving the usual "here are my refs and its current values".
> 
>  * The receiver would say "I want to use that extension", and to
>    each of its "want" (which usually consists of "want" followed by
>    an object name and nothing else), it optionally adds names of the
>    objects it wants to verify ancestry relationship with.
> 
>    E.g. if you have O at the tip of the master branch and P at the
>    tip of the maint branch, the sender has N at both of these two
>    branches, and if you are updating your master and maint with
>    their master and maint, you would say something like "want N O P"
>    to tell the sender that you want history leading to N, and you
>    want to see if N is a descendant of O and if N is a descendant of
>    P.
> 
>  * The receiver and the sender then does the usual "have"/"ack"
>    exchange, which does not have to change any behaviour with this
>    extension.
> 
>  * Finally, when the sender sends out the resulting packfile, it
>    also has to tell the receiver which of the object pairs the
>    receiver asked it to check the ancestry relationship violate the
>    fast-forward rule.  In the earlier example of fast-forwarding O
>    and P with N, where the receiver asked "want N O P", the receiver
>    asked to check object pairs <N, O> and <N, P>.  If P fast-forwards
>    to N but O does not, then the sender would tell the receiver the
>    fact that "O does not fast forward to N".

So this would be another step after the receiver communicates "done",
but before the pack file itself gets sent,
so the sender can determine that it doesn't need to tell the reciever
the relationship between two commits,
since it knows you can work it out yourself?

Otherwise if it can probably communicate the relationships before the have list
is sent, and extend the shallow-update part of the protocol instead, which
might be simpler.

> With such an extension, your updated receiver can receive the
> necessary objects to update your history to "N", but notice that it
> would result in non-ff update to update master (that used to be O)
> with the new commit N.

Code-wise, does the following make sense?

1.  Add a field to `struct ref` to flag a "trusted fast-forward".
2.  Change `find_common()` in `fetch-pack.c`
    and `receive_needs()` in `upload-pack.c`
    to communicate the relationships we're interested in as above,
    and set the "trusted fast-forward" flag.
3.  Change `update_local_ref()` in `builtin/fetch.c`
    to check `ref->trusted_fast_forward || in_merge_bases(current, updated)`.