Re: git clone stable-2.6.25.y fails over HTTP

Re: git clone stable-2.6.25.y fails over HTTP

[Greg KH]

On Wed, Jun 04, 2008 at 08:11:24AM -0700, Andrew Klossner wrote:
> Greg wrote:
> 
> > I don't think I remembered to run 'git-update-server-info' on the
> > repository.  I just did it, let me know if that works better or not.
> 
> I get the same error.
> 
>   -=- Andrew
> 
> % git clone http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2 .6.25.y.git
> Initialized empty Git repository in /work/andrew/http-git/linux-2.6.25.y/.git/
> Getting alternates list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Also look at http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Getting index for pack 6506d57e29d10a0ed9cca269f97296606aeb0b5b
> Getting index for pack 067caec25377d65e503d626a6c4f3f7e392406a3
> Getting index for pack 5434817c4def8a894fe4d61f55e8e4117485cbe7
> Getting pack 067caec25377d65e503d626a6c4f3f7e392406a3
>  which contains a82a9bfebc74862798a3f6949fe6f6a959c1fd4e
> walk a82a9bfebc74862798a3f6949fe6f6a959c1fd4e
> walk 15c2419876173c417a119b5665ecc62520e50a6c
> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Cannot obtain needed blob fdcce40226d7d4273a08cc4ef84bb25755a710a4
> while processing commit 15c2419876173c417a119b5665ecc62520e50a6c.


Hm, I really do not know, and I can duplicate this myself as well.

<adding the git mailing list>

Does anyone know what's going on here?  The stable git trees are created
by doing:
	git clone -s --bare
from Linus's main tree to the stable tree repo location.

It looks like git over http isn't following the link somehow to the main
repo?

Or is what is necessary to do is to clone Linus's tree and then pull
from the stable tree in this kind of situation (where the user can only
use git over http?)

thanks,

greg k-h

Re: git clone stable-2.6.25.y fails over HTTP

[Junio C Hamano]

Greg KH <greg@kroah.com> writes:

>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> ...
> It looks like git over http isn't following the link somehow to the main
> repo?

Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
path; it wants to serve it as /pub/scm/linux instead.

Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
but in 2.6.25.y it has extra /home/ftp in front of it.

A quick fix is obviously to remove the extra prefix but it makes me wonder
how these two "stable" repositories were prepared differently.  Could this
be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
repository slightly differently between them?

Re: git clone stable-2.6.25.y fails over HTTP

[Jakub Narebski]

Junio C Hamano <gitster@pobox.com> writes:

> Greg KH <greg@kroah.com> writes:
> 
>>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
>> ...
>> It looks like git over http isn't following the link somehow to the main
>> repo?
> 
> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
> path; it wants to serve it as /pub/scm/linux instead.
> 
> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
> but in 2.6.25.y it has extra /home/ftp in front of it.
> 
> A quick fix is obviously to remove the extra prefix but it makes me wonder
> how these two "stable" repositories were prepared differently.  Could this
> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> repository slightly differently between them?

Errr... isn't it the case for objects/info/http-alternates ?

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: git clone stable-2.6.25.y fails over HTTP

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Greg KH <greg@kroah.com> writes:
>> 
>>>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>>>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
>>> ...
>>> It looks like git over http isn't following the link somehow to the main
>>> repo?
>> 
>> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
>> path; it wants to serve it as /pub/scm/linux instead.
>> 
>> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
>> but in 2.6.25.y it has extra /home/ftp in front of it.
>> 
>> A quick fix is obviously to remove the extra prefix but it makes me wonder
>> how these two "stable" repositories were prepared differently.  Could this
>> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
>> repository slightly differently between them?
>
> Errr... isn't it the case for objects/info/http-alternates ?

You do not have to use that http specific hack.  On these machines, /pub
actually points at /home/ftp/pub with a symlink.

Re: git clone stable-2.6.25.y fails over HTTP

[Greg KH]

On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
> Greg KH <greg@kroah.com> writes:
> 
> >> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> >> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> > ...
> > It looks like git over http isn't following the link somehow to the main
> > repo?
> 
> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
> path; it wants to serve it as /pub/scm/linux instead.
> 
> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
> but in 2.6.25.y it has extra /home/ftp in front of it.
> 
> A quick fix is obviously to remove the extra prefix but it makes me wonder
> how these two "stable" repositories were prepared differently.  Could this
> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> repository slightly differently between them?

That's odd.

Chris was the one that created the .24 tree and I created the .25 tree,
so we must have done so differently, sorry.

I've edited the alternates file, so it should work now, sorry about
that.

thanks,

greg k-h

Re: git clone stable-2.6.25.y fails over HTTP

[Junio C Hamano]

Greg KH <greg@kroah.com> writes:

> On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
>
>> A quick fix is obviously to remove the extra prefix but it makes me wonder
>> how these two "stable" repositories were prepared differently.  Could this
>> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
>> repository slightly differently between them?
>
> That's odd.
>
> Chris was the one that created the .24 tree and I created the .25 tree,
> so we must have done so differently, sorry.

No need to be sorry.  Thanks for raising the issue up.

I think I have one possible explanation.  When "git clone" was rewritten
in C some time ago, it started to dereference symlinks, and newer git
would dereference /pub -> /home/ftp/pub when you do:

	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y

while the scripted version used to keep /pub/scm path.

A bit further digging is necessary to make sure if we can fix this easily,
but I suspect this is worth fixing.

Re: git clone stable-2.6.25.y fails over HTTP

[Greg KH]

On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote:
> Greg KH <greg@kroah.com> writes:
> 
> > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
> >
> >> A quick fix is obviously to remove the extra prefix but it makes me wonder
> >> how these two "stable" repositories were prepared differently.  Could this
> >> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> >> repository slightly differently between them?
> >
> > That's odd.
> >
> > Chris was the one that created the .24 tree and I created the .25 tree,
> > so we must have done so differently, sorry.
> 
> No need to be sorry.  Thanks for raising the issue up.
> 
> I think I have one possible explanation.  When "git clone" was rewritten
> in C some time ago, it started to dereference symlinks, and newer git
> would dereference /pub -> /home/ftp/pub when you do:
> 
> 	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y
> 
> while the scripted version used to keep /pub/scm path.
> 
> A bit further digging is necessary to make sure if we can fix this easily,
> but I suspect this is worth fixing.

I can duplicate what I did, and it did it again:
	on master.kernel.org:
	$ cd /pub/scm/linux/kernel/git/stable/
	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
	$ cat test/objects/info/alternates
	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects

I'm guessing this is because /pub/ is a symlink:
	$ ls -l /pub
	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub

	$ git --version
	git version 1.5.5.3

thanks,

greg k-h

Re: git clone stable-2.6.25.y fails over HTTP

[Daniel Barkalow]

On Wed, 4 Jun 2008, Greg KH wrote:

> On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote:
> > Greg KH <greg@kroah.com> writes:
> > 
> > > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
> > >
> > >> A quick fix is obviously to remove the extra prefix but it makes me wonder
> > >> how these two "stable" repositories were prepared differently.  Could this
> > >> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> > >> repository slightly differently between them?
> > >
> > > That's odd.
> > >
> > > Chris was the one that created the .24 tree and I created the .25 tree,
> > > so we must have done so differently, sorry.
> > 
> > No need to be sorry.  Thanks for raising the issue up.
> > 
> > I think I have one possible explanation.  When "git clone" was rewritten
> > in C some time ago, it started to dereference symlinks, and newer git
> > would dereference /pub -> /home/ftp/pub when you do:
> > 
> > 	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y
> > 
> > while the scripted version used to keep /pub/scm path.
> > 
> > A bit further digging is necessary to make sure if we can fix this easily,
> > but I suspect this is worth fixing.
> 
> I can duplicate what I did, and it did it again:
> 	on master.kernel.org:
> 	$ cd /pub/scm/linux/kernel/git/stable/
> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
> 	$ cat test/objects/info/alternates
> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> 
> I'm guessing this is because /pub/ is a symlink:
> 	$ ls -l /pub
> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub

We're using getcwd() to make the absolute path, but it looks like we 
should check whether $PWD is set and names the same directory, and, if so, 
base on that instead.

Actually, make_absolute_path() seems to go to an awful lot of trouble to 
make sure that there aren't symlinks in the resulting path. Is this 
actually necessary or even desireable for some reason? Should clone be 
just pasting $PWD (or, failing that, getcwd()) and the given path, and 
using that to identify the repo to clone?

	-Daniel
*This .sig left intentionally blank*

Re: git clone stable-2.6.25.y fails over HTTP

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> On Wed, 4 Jun 2008, Greg KH wrote:
> ...
>> I can duplicate what I did, and it did it again:
>> 	on master.kernel.org:
>> 	$ cd /pub/scm/linux/kernel/git/stable/
>> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
>> 	$ cat test/objects/info/alternates
>> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
>> 
>> I'm guessing this is because /pub/ is a symlink:
>> 	$ ls -l /pub
>> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub
>
> We're using getcwd() to make the absolute path, but it looks like we 
> should check whether $PWD is set and names the same directory, and, if so, 
> base on that instead.
>
> Actually, make_absolute_path() seems to go to an awful lot of trouble to 
> make sure that there aren't symlinks in the resulting path. Is this 
> actually necessary or even desireable for some reason? Should clone be 
> just pasting $PWD (or, failing that, getcwd()) and the given path, and 
> using that to identify the repo to clone?

I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused
this.  Perhaps 30% or so ;-).

However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
result in the same breakage, and that is something I consider gravely
broken.

If both are fixed by rethinking the make_absolute_path(), that would be
good.

Re: git clone stable-2.6.25.y fails over HTTP

[Greg KH]

On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > On Wed, 4 Jun 2008, Greg KH wrote:
> > ...
> >> I can duplicate what I did, and it did it again:
> >> 	on master.kernel.org:
> >> 	$ cd /pub/scm/linux/kernel/git/stable/
> >> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
> >> 	$ cat test/objects/info/alternates
> >> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> >> 
> >> I'm guessing this is because /pub/ is a symlink:
> >> 	$ ls -l /pub
> >> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub
> >
> > We're using getcwd() to make the absolute path, but it looks like we 
> > should check whether $PWD is set and names the same directory, and, if so, 
> > base on that instead.
> >
> > Actually, make_absolute_path() seems to go to an awful lot of trouble to 
> > make sure that there aren't symlinks in the resulting path. Is this 
> > actually necessary or even desireable for some reason? Should clone be 
> > just pasting $PWD (or, failing that, getcwd()) and the given path, and 
> > using that to identify the repo to clone?
> 
> I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused
> this.  Perhaps 30% or so ;-).

I agree, if I did something stupid like this, that's my fault.  I didn't
realize that it was stupid, so I guess a simple warning, "don't use
relative paths when cloning with alternates" would have told me so.

> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
> result in the same breakage, and that is something I consider gravely
> broken.

Yeah, especially as I never realized that /pub/ was a symlink in the
first place :)

thanks,

greg k-h

Re: git clone stable-2.6.25.y fails over HTTP

[Junio C Hamano]

Greg KH <greg@kroah.com> writes:

> On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
>
>> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
>> result in the same breakage, and that is something I consider gravely
>> broken.
>
> Yeah, especially as I never realized that /pub/ was a symlink in the
> first place :)

And it was even worse than that.  Even if you knew /pub was a symlink, you
could not have done anything about it, short of editing the alternates
file in the resulting repository by hand.

Up until now, that is.

Daniel's patch seems to fix the issue for me and it will be included in
tonight's pushout among other fixes.

Thanks.

Re: git clone stable-2.6.25.y fails over HTTP

[Greg KH]

On Fri, Jun 06, 2008 at 02:00:09PM -0700, Junio C Hamano wrote:
> Greg KH <greg@kroah.com> writes:
> 
> > On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
> >
> >> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
> >> result in the same breakage, and that is something I consider gravely
> >> broken.
> >
> > Yeah, especially as I never realized that /pub/ was a symlink in the
> > first place :)
> 
> And it was even worse than that.  Even if you knew /pub was a symlink, you
> could not have done anything about it, short of editing the alternates
> file in the resulting repository by hand.

Heh, true :)

> Up until now, that is.
> 
> Daniel's patch seems to fix the issue for me and it will be included in
> tonight's pushout among other fixes.

Great, thanks a lot for the fix everyone.

greg k-h

Re: git clone stable-2.6.25.y fails over HTTP

[Andrew Klossner]

Following up last week's problem with the stable-2.6.25.y tree:

>From the HTTP perspective, the tree is stuck in time.  The two new
tags v2.6.25.5 and v2.6.25.6 do not appear when I do this:

% git pull -v
From http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y
 = [up to date]      master     -> origin/master
Already up-to-date.

On my home machine, I verified that these tags appear as expected when
I pull with the git protocol.  But at work I'm behind a firewall and
am stuck with HTTP.

  -=- Andrew Klossner