Weird interaction between Git protocol versions and git:// and https:// URLs (was: Re: Bug with Git shallow clones and submodules)

[Philippe Blain <levraiphilippeblain@gmail.com>]

Hi everyone,

Le 2021-08-15 à 10:31, Philippe Blain a écrit :

> 5. It falls back to fetching the needed commit by its hash:
> 
>      start git fetch origin 78bc27846101e481438a98c68cac47e4476085c0
> 
> 6. The server at https://dpdk.org/git/dpdk-stable refuses this request and you get
> 
>      error: Server does not allow request for unadvertised object 78bc27846101e481438a98c68cac47e4476085c0
> 
> OK so why does the server refuses the request ? If you take a look at [4], you will
> understand that it's probably because the configs 'uploadpack.allowReachableSHA1InWant'
> and 'uploadpack.allowAnySHA1InWant' are both false (i.e. their default value) on the
> https://dpdk.org/git/dpdk-stable server. So the behaviour here is not a bug, it's
> working as it should (but leads to a bad experience for you).
> 
> What is not unfortunately not mentioned in the documentation at [4]
> is that the two configs that I mention above only apply if version 0 of the Git transfer
> protocol is in effect. So this would mean that even though we specify '-c protocol.version=2'
> in the 'git clone' command in my script, the server uses protocol v0. Protocol v2 works
> starting in Git 2.18, so let's check what version is running at dpdk.org:
> 
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote https://dpdk.org/git/dpdk-stable 2>&1 | grep agent
> packet:          git< 6ee0521feb765d9105241a3f6693762c471655cf HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow deepen-since deepen-not deepen-relative no-progress include-tag multi_ack_detailed no-done symref=HEAD:refs/heads/default agent=git/2.20.1
> 
> OK, it's running Git 2.20.1. And the list of capabilities above implies that it's responding
> with protocol v0, even if my local Git version if 2.29, which defaults to protocol v2. If
> it were responding with protocol v2, we would see:
> 
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote https://github.com/git/git 2>&1 | grep agent
> packet:          git< agent=git/github-g78b452413e8b  # server version
> packet:          git< agent=git/github-g78b452413e8b
> packet:          git> agent=git/2.29.2  # client version
> packet:          git< agent=git/2.29.2
> 
> And grepping for "version" would confirm it:
> 
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote https://github.com/git/git 2>&1 | grep version
> packet:          git< version 2
> packet:          git< version 2
> 
> I can't explain why the server refuses to talk with protocol v2. Maybe Jonathan (CC-ed),
> who worked on the implementation of protocol v2, would be able to explain that.

I'm sorry to bump this, but I would really like to understand what is going on here.
I'm not familiar at all with the protocol code, but my understanding was that after
Git 2.18, if a client requested protocol v2, the server would answer with v2 (at least
for git:// and http[s]://, and also for ssh:// if AcceptEnv is correctly configured).
Here, Git 2.20.1 at https://dpdk.org/git/dpdk-stable answers with v0 for https://,
but with v2 for git:// !

> 
> And it gets weirder. If you instead use the Git URL (instead of the HTTPS URL)
> for the dpdk-stable submodule [2], then the server responds with protocol v2!
> 
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote git://dpdk.org/dpdk-stable 2>&1 | grep agent
> packet:          git< agent=git/2.20.1
> packet:          git> agent=git/2.29.2
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote git://dpdk.org/dpdk-stable 2>&1 | grep version
> packet:          git> git-upload-pack /dpdk-stable\0host=dpdk.org\0\0version=2\0
> packet:          git< version 2
> 
> And indeed, if you use the 'git://' URL in the 'git submodule add' command in my reproducer
> script, it succeeds !
> 
> It don't understand this behaviour either.
> 
--- snip ---
> 
> P.S. for Jonathan I noticed some additional weirdness while debugging this:
> Some servers respond with a different Git version depending on the protocol:
> 
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote  git://gcc.gnu.org/git/gcc.git 2>&1 | grep -E 'agent|version '
> packet:          git< version 2
> packet:          git< agent=git/2.26.2
> packet:          git> agent=git/2.29.2
> $ GIT_TRACE_PACKET=1 GIT_TRACE_BARE=1 git ls-remote  https://gcc.gnu.org/git/gcc.git 2>&1 | grep agent -E 'agent|version '
> packet:          git< 882f1d58bfa56737ff2de84c3cd1e0acfc318b86 HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow deepen-since deepen-not deepen-relative no-progress include-tag multi_ack_detailed no-done symref=HEAD:refs/heads/master agent=git/2.27.0
> 
> Here again, with HTTPS we fall back to protocol v0. The same behaviour happens
> with git://sourceware.org/git/binutils-gdb.git...

This is also puzzling to me...

Thanks,

Philippe.