Problem with git fetch over http://

Problem with git fetch over http://

[John Wiegley]

Hello, I'm using the Builtbot to pull sources from a Git repository,  
and have found that the Buildbot's Git step always uses git-fetch -- a  
detail of Buildbot which I currently have little control over.

This is no problem if the git:// protocol is used, but using fetch  
with http:// doesn't work.  This has been mentioned elsewhere:

   http://lists.berlios.de/pipermail/bitbake-dev/2007-November/000765.html

Here's a quick example of what I mean:

   cd /tmp ; mkdir foo ; cd foo ; git init

   git fetch git://newartisans.com/ledger.git	# works fine

   cd /tmp ; rm -fr foo ; mkdir foo ; cd foo ; git init

   git fetch http://newartisans.com/git/ledger.git  # fails

This is happening with git-1.5.5 on the server, Apache 2.2 and  
git-1.5.5.1 on the client.

Thanks,
   John

Re: Problem with git fetch over http://

[Junio C Hamano]

John Wiegley <johnw@newartisans.com> writes:

> Hello, I'm using the Builtbot to pull sources from a Git repository,
> and have found that the Buildbot's Git step always uses git-fetch -- a
> detail of Buildbot which I currently have little control over.
>
> This is no problem if the git:// protocol is used, but using fetch
> with http:// doesn't work.  This has been mentioned elsewhere:
>
>   http://lists.berlios.de/pipermail/bitbake-dev/2007-November/000765.html
>
> Here's a quick example of what I mean:
>
>   cd /tmp ; mkdir foo ; cd foo ; git init
>
>   git fetch git://newartisans.com/ledger.git	# works fine
>
>   cd /tmp ; rm -fr foo ; mkdir foo ; cd foo ; git init
>
>   git fetch http://newartisans.com/git/ledger.git  # fails

Is this related to this independent observation?

$ wget http://newartisans.com/git/ledger.git
--14:23:54--  http://newartisans.com/git/ledger.git
           => `ledger.git'
Resolving newartisans.com... 208.70.150.153
Connecting to newartisans.com|208.70.150.153|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://git.newartisans.com//ledger.git [following]
--14:23:55--  http://git.newartisans.com//ledger.git
           => `ledger.git'
Resolving git.newartisans.com... 208.70.150.153
Connecting to git.newartisans.com|208.70.150.153|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://git.newartisans.com/ledger.git/ [following]
--14:23:55--  http://git.newartisans.com/ledger.git/
           => `index.html'
Connecting to git.newartisans.com|208.70.150.153|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
14:23:55 ERROR 403: Forbidden.

Re: Problem with git fetch over http://

[Mike Hommey]

On Fri, Apr 25, 2008 at 02:24:24PM -0700, Junio C Hamano wrote:
> John Wiegley <johnw@newartisans.com> writes:
> 
> > Hello, I'm using the Builtbot to pull sources from a Git repository,
> > and have found that the Buildbot's Git step always uses git-fetch -- a
> > detail of Buildbot which I currently have little control over.
> >
> > This is no problem if the git:// protocol is used, but using fetch
> > with http:// doesn't work.  This has been mentioned elsewhere:
> >
> >   http://lists.berlios.de/pipermail/bitbake-dev/2007-November/000765.html
> >
> > Here's a quick example of what I mean:
> >
> >   cd /tmp ; mkdir foo ; cd foo ; git init
> >
> >   git fetch git://newartisans.com/ledger.git	# works fine
> >
> >   cd /tmp ; rm -fr foo ; mkdir foo ; cd foo ; git init
> >
> >   git fetch http://newartisans.com/git/ledger.git  # fails
> 
> Is this related to this independent observation?
> 
> $ wget http://newartisans.com/git/ledger.git
> --14:23:54--  http://newartisans.com/git/ledger.git
>            => `ledger.git'
> Resolving newartisans.com... 208.70.150.153
> Connecting to newartisans.com|208.70.150.153|:80... connected.
> HTTP request sent, awaiting response... 301 Moved Permanently
> Location: http://git.newartisans.com//ledger.git [following]
> --14:23:55--  http://git.newartisans.com//ledger.git
>            => `ledger.git'
> Resolving git.newartisans.com... 208.70.150.153
> Connecting to git.newartisans.com|208.70.150.153|:80... connected.
> HTTP request sent, awaiting response... 301 Moved Permanently
> Location: http://git.newartisans.com/ledger.git/ [following]
> --14:23:55--  http://git.newartisans.com/ledger.git/
>            => `index.html'
> Connecting to git.newartisans.com|208.70.150.153|:80... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 14:23:55 ERROR 403: Forbidden.

Probably, but even then, it also doesn't have a HEAD.

Mike

Re: Problem with git fetch over http://

[Daniel Barkalow]

On Fri, 25 Apr 2008, John Wiegley wrote:

> Hello, I'm using the Builtbot to pull sources from a Git repository, and have
> found that the Buildbot's Git step always uses git-fetch -- a detail of
> Buildbot which I currently have little control over.
> 
> This is no problem if the git:// protocol is used, but using fetch with
> http:// doesn't work.  This has been mentioned elsewhere:
> 
>  http://lists.berlios.de/pipermail/bitbake-dev/2007-November/000765.html
> 
> Here's a quick example of what I mean:
> 
>  cd /tmp ; mkdir foo ; cd foo ; git init
> 
>  git fetch git://newartisans.com/ledger.git	# works fine
> 
>  cd /tmp ; rm -fr foo ; mkdir foo ; cd foo ; git init
> 
>  git fetch http://newartisans.com/git/ledger.git  # fails
> 
> This is happening with git-1.5.5 on the server, Apache 2.2 and git-1.5.5.1 on
> the client.

John mentioned on IRC that clone works.

$ git-ls-remote http://newartisans.com/git/ledger.git
96751442d9eda537480867e599e4c604663f4530        refs/heads/master
365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
...

$ git-ls-remote git://newartisans.com/ledger.git
96751442d9eda537480867e599e4c604663f4530        HEAD
96751442d9eda537480867e599e4c604663f4530        refs/heads/master
365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
...

Looks like either server-info.c:update_info_refs() needs to also list 
HEAD, or transport.c:get_refs_via_curl() needs to guess that $url/HEAD 
exists and fetch it to add to the list, or both.

	-Daniel
*This .sig left intentionally blank*

Re: Problem with git fetch over http://

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> John mentioned on IRC that clone works.
>
> $ git-ls-remote http://newartisans.com/git/ledger.git
> 96751442d9eda537480867e599e4c604663f4530        refs/heads/master
> 365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
> f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
> 6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
> fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
> ...
>
> $ git-ls-remote git://newartisans.com/ledger.git
> 96751442d9eda537480867e599e4c604663f4530        HEAD
> 96751442d9eda537480867e599e4c604663f4530        refs/heads/master
> 365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
> f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
> 6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
> fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
> ...
>
> Looks like either server-info.c:update_info_refs() needs to also list 
> HEAD, or transport.c:get_refs_via_curl() needs to guess that $url/HEAD 
> exists and fetch it to add to the list, or both.

Wait a minute.  How did "git fetch http://host/repo.git" work before C
rewrite?

The reason I ask is because info/refs traditionally did not list HEAD and
fetch either would have failed to start walking from HEAD (iow, the issue
was very old), or it didn't care (iow, the issue is a regression after C
rewrite).

Re: Problem with git fetch over http://

[Daniel Barkalow]

On Fri, 25 Apr 2008, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > John mentioned on IRC that clone works.
> >
> > $ git-ls-remote http://newartisans.com/git/ledger.git
> > 96751442d9eda537480867e599e4c604663f4530        refs/heads/master
> > 365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
> > f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
> > 6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
> > fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
> > ...
> >
> > $ git-ls-remote git://newartisans.com/ledger.git
> > 96751442d9eda537480867e599e4c604663f4530        HEAD
> > 96751442d9eda537480867e599e4c604663f4530        refs/heads/master
> > 365baca5174232cc35e4ed190fd8ce93c3611c84        refs/heads/v2.4.1
> > f972a89073843fdbb2a1bef969d9ca70509d4bfe        refs/heads/v2.5
> > 6a81d760dc4ab9028b27430124702c28c0b894ed        refs/tags/cvs-20040410
> > fed3956a310a2ea8c8595b7fde7aaa0623492ca6        refs/tags/v0.1
> > ...
> >
> > Looks like either server-info.c:update_info_refs() needs to also list 
> > HEAD, or transport.c:get_refs_via_curl() needs to guess that $url/HEAD 
> > exists and fetch it to add to the list, or both.
> 
> Wait a minute.  How did "git fetch http://host/repo.git" work before C
> rewrite?
> 
> The reason I ask is because info/refs traditionally did not list HEAD and
> fetch either would have failed to start walking from HEAD (iow, the issue
> was very old), or it didn't care (iow, the issue is a regression after C
> rewrite).

I never managed to trace through the shell fetch code, but I suspect that 
it broke quite a long time ago (git-ls-remote.sh didn't list HEAD for 
http:// URLs, anyway). clone works because it uses entirely different 
code, and "git fetch origin" (when origin is an http URL) works, and "git 
fetch http://... branch" works (and is a workaround), and that's probably 
all that people use regularly. This only comes up because there's an 
automated fetch-and-build system that uses "<vcs> command <location>" to 
update, and it needed an actual quirk, rather than people just training 
their fingers to do things that work.

	-Daniel
*This .sig left intentionally blank*