1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

[Tim Shepard]

I have git 1.5.6.5 installed from the Debian/lenny package.

Poking around in http://git.kernel.org/ looking for a git repository
that might have the latest -rt development happening, I found

  http://git.kernel.org/?p=linux/kernel/git/rostedt/linux-2.6-rt.git

which looked promising.

But when I tried cloning it using:

    git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-rt.git linux-2.6-rt

it pulled several hundred megabytes through my (rather slow) DSL line
and then printed out

  error: Trying to write ref refs/tags/v2.6.11 with nonexistant object 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c
  fatal: Cannot update the ref 'refs/tags/v2.6.11'.

and then removed everything it had just pulled.


Looking at http://git.kernel.org/?p=linux/kernel/git/rostedt/linux-2.6-rt.git;a=tags
I see that apparently v2.6.11 and v.2.6.11-tree both point at a tree
object and not a commit.

Is this a bug in git?    (Even if there is something wrong with the
git repository I was trying to clone, the fact that it removed all the
work that took a long time to pull is annoying.)

Is this problem already fixed in some newer version of git?


			-Tim Shepard
			 shep@alum.mit.edu

Re: 1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

[Junio C Hamano]

Tim Shepard <shep@alum.mit.edu> writes:

> I have git 1.5.6.5 installed from the Debian/lenny package.
>
> Poking around in http://git.kernel.org/ looking for a git repository
> that might have the latest -rt development happening, I found
>
>   http://git.kernel.org/?p=linux/kernel/git/rostedt/linux-2.6-rt.git
>
> which looked promising.
>
> But when I tried cloning it using:
>
>     git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-rt.git linux-2.6-rt
>
> it pulled several hundred megabytes through my (rather slow) DSL line
> and then printed out
>
>   error: Trying to write ref refs/tags/v2.6.11 with nonexistant object 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c
>   fatal: Cannot update the ref 'refs/tags/v2.6.11'.
>
> and then removed everything it had just pulled.
>
> Looking at http://git.kernel.org/?p=linux/kernel/git/rostedt/linux-2.6-rt.git;a=tags
> I see that apparently v2.6.11 and v.2.6.11-tree both point at a tree
> object and not a commit.
>
> Is this a bug in git?

It is not a problem for the tag pointing to a tree, but linux-2.6-rt.git
tree uses (like everybody else) objects/alternates to borrow objects from
the repository of Linus.

I think we lost the alternate object store support when git-fetch was
rewritten from the original shell script (that did support fetching from
such a repository over rsync:// transport) to a reimplementation in C,
with commit b888d61 (Make fetch a builtin, 2007-09-10).  

Later, cd547b4 (fetch/push: readd rsync support, 2007-10-01) attempted to
resurrect some rsync support (b888d61 lost rsync support completely for
git-fetch), but introduced these lines in transport.c:

	/* NEEDSWORK: handle one level of alternates */
	result = run_command(&rsync);

These have not been touched ever since, and then finally commit 8434c2f
(Build in clone, 2008-04-27) killed support of cloning from a repository
that uses alternates for rsync transport for git-clone (before that, the
shell script version had a special case to still allow such operation over
rsync).

It appears practically nobody uses rsync:// transport anymore these days;
you are unfortunately the first one who noticed it.

Re: 1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

On Fri, Jan 09, 2009 at 01:24:19AM -0500, Tim Shepard <shep@alum.mit.edu> wrote:
> 
> 
> I have git 1.5.6.5 installed from the Debian/lenny package.
> 
> Poking around in http://git.kernel.org/ looking for a git repository
> that might have the latest -rt development happening, I found
> 
>   http://git.kernel.org/?p=linux/kernel/git/rostedt/linux-2.6-rt.git
> 
> which looked promising.
> 
> But when I tried cloning it using:
> 
>     git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-rt.git linux-2.6-rt

I would use the git:// link from gitweb.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: 1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

[Tim Shepard]

Junio,

Thank you for your good explanation.

Also thanks to Miklos Vajna who also replied to suggest using git:// transport.


(Over 3 years ago I used git glone to get a copy of torvalds/linux-2.6.git
 using rsync transport and I copied the recipe I used then.  It has been
 working for "git pull" updates, and for one from-scratch re-clone
 I did sometime last year. I had no reason to suspect it was broken.)

This morning I re-started the clone using git:// transport and it worked OK.


			-Tim Shepard
			 shep@alum.mit.edu

Re: 1.5.6.5 fails to clone git.kernel.org/[...]/rostedt/linux-2.6-rt

[Johannes Schindelin]

Hi,

On Fri, 9 Jan 2009, Junio C Hamano wrote:

> I think we lost the alternate object store support when git-fetch was 
> rewritten from the original shell script (that did support fetching from 
> such a repository over rsync:// transport) to a reimplementation in C, 
> with commit b888d61 (Make fetch a builtin, 2007-09-10).
> 
> Later, cd547b4 (fetch/push: readd rsync support, 2007-10-01) attempted to
> resurrect some rsync support (b888d61 lost rsync support completely for
> git-fetch), but introduced these lines in transport.c:
> 
> 	/* NEEDSWORK: handle one level of alternates */
> 	result = run_command(&rsync);

Indeed... And I know who's responsible for those lines.

However, I am swamped with work these days, and my Git time budget was 
_way_ overspent what with the recent patches.

So whoever would like to give it a go, go wild.

This is actually a very fine opportunity for people to get involved who 
always wanted to; it is a relatively low-hanging fruit.

It should just be a matter of getting objects/info/alternates (one can 
easily reuse a large part of the args[] array filled before the quoted 
code) into a temporary file.

If that does not succeed, return 0, otherwise fetch those objects, too 
(again reusing most of the args[] array).

It is that easy because objects as well as packs are immutable, so we can 
just build the union of objects/packs from the remote and its alternate.

Then all which is left to do is to add a test case to t/t5510-fetch.sh, 
and you're set.

As there are already test cases for rsync:// in it, it should be as simple 
as putting an empty file into a newly created directory, create an 
alternate for the "remote" pointing to the directory, fetching, and 
testing that the empty file was copied.

That's possible because rsync:// is dumb and does not verify the files it 
copied.

Oh, and don't forget to remove the NEEDSWORK comment :-)

And now I'm curious who's up for it...

Ciao,
Dscho