git-clone --reference problem?

git-clone --reference problem?

[David Woodhouse]

Should I expect cloning with alternates using '--reference' to be
transitive?

Using '--reference clone1 --reference linux-2.6' for the second clone
works OK, but surely it ought to work with just '--reference clone1'? 

It doesn't work -- it can't find the objects which are in the original
linux-2.6 tree....

 (linux-2.6 contains a current copy of linus' tree)

shinybook /shiny/git $ git-clone --reference linux-2.6 git://git.infradead.org/mtd-2.6.git clone1
Checking files out...
 100% (19552/19552) done
shinybook /shiny/git $ git-clone --reference clone1 git://git.infradead.org/mtd-2.6.git clone2
error: refs/reference-tmp/refs/tags/v2.6.13-rc7 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc2 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.14 does not point to a valid commit object!
   < ..... >
error: refs/reference-tmp/refs/tags/v2.6.14-rc1 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.15-rc2 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc6 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc4 does not point to a valid commit object!
error: Could not read 7e6684741b15e98dd52bd0dbcb08a4eb69857c23
error: Could not read 7e6684741b15e98dd52bd0dbcb08a4eb69857c23
error: Could not read 7260448207915a170bb812f8639a90a30329adce
error: Could not read 7260448207915a170bb812f8639a90a30329adce
error: Could not read acf8d9bd83be879328c558400d94ee61fc229672
error: Could not read acf8d9bd83be879328c558400d94ee61fc229672
error: Could not read c7270e76718f635bed33afe6c4751a270b2d031b
error: Could not read c7270e76718f635bed33afe6c4751a270b2d031b
error: Could not read 0a8763981774041f3fee0a71e016dcaa096fa3f8
error: Could not read 0a8763981774041f3fee0a71e016dcaa096fa3f8
error: Could not read 4a47136ddde07488a5741e6be267b8f5bddc407b
error: Could not read 4a47136ddde07488a5741e6be267b8f5bddc407b
error: Could not read 6cad1d2664ec091ba5a6f3e3e552cf550ec8c2e0
error: Could not read 6cad1d2664ec091ba5a6f3e3e552cf550ec8c2e0
   < ..... >
error: Could not read a0d4c65cd5642e7f7bbd2f806a69d20b2e43edc4
error: Could not read 611dd7a56906343db81c9fe340be0eb78c4ec260
error: Could not read 611dd7a56906343db81c9fe340be0eb78c4ec260
error: Could not read 227b4665ae0105348868d7a0a577209c8d16c6e3
error: Could not read 227b4665ae0105348868d7a0a577209c8d16c6e3
error: Could not read 9f19a4bfe0d0f0d203113d34d7455ceb82ff8341
error: Could not read 9f19a4bfe0d0f0d203113d34d7455ceb82ff8341
Checking files out...
error: git-checkout-index: unable to read sha1 file of .gitignore (27fd37621255799602d74e94d670ff7a1658d40a)
error: git-checkout-index: unable to read sha1 file of COPYING (ca442d313d86dc67e0a2e5d584b465bd382cbf5c)
error: git-checkout-index: unable to read sha1 file of CREDITS (787564bc248b1d6125fc42f3932966b60aa2f7f4)
error: git-checkout-index: unable to read sha1 file of Kbuild (2d4f95e4b89f7f81da6cb94b07e8449b3689ba37)
error: git-checkout-index: unable to read sha1 file of Makefile (fc8e08c419f09e81252f3aa41cb7f47524c0af60)
error: git-checkout-index: unable to read sha1 file of README (05e055530bbb687599dd732d6753c77ffa281ae5)
   < .....> 

-- 
dwmw2

Re: git-clone --reference problem?

[Junio C Hamano]

David Woodhouse <dwmw2@infradead.org> writes:

> Should I expect cloning with alternates using '--reference' to be
> transitive?

Not with the current code and design, but a patch that cleanly
does that can be considered for inclusion.  Adjusting relative
paths obtained from other repositories correctly, and avoiding
circular alternates by mistake would be tricky, though.

Re: git-clone --reference problem?

[Linus Torvalds]

On Sun, 23 Apr 2006, David Woodhouse wrote:
>
> Should I expect cloning with alternates using '--reference' to be
> transitive?

Well, certainly not now.

> Using '--reference clone1 --reference linux-2.6' for the second clone
> works OK, but surely it ought to work with just '--reference clone1'? 

Actually, I don't think using "--reference clone1 --reference linux-2.6" 
works OK at all - in the sense that it doesn't do what you _think_ it 
does.

It doesn't use two reference repos at all: it uses just one, which is the 
last one (ie linux-2.6).

Maybe something like this could work.

Untested, of course. It probably isn't even syntactically correct shell. 
Whatever. You get the idea.

(That while-loop could probably be simplified to just a 

  cat ""$reference/objects/info/alternates" >> "$GIT_DIR/objects/info/alternates"

because all alternates _should_ already be absolute paths, but hey, 
whatever. I also forget whether we decided that non-absolute paths were 
relative to the $reference directory, or to the $reference/objects/ 
directory. The while-loop should be fixed to match whatever that decision 
was (or just not accept anything but absolute paths: make it use a

	grep '^/' ..

instead of "cat"?)

		Linus

---
diff --git a/git-clone.sh b/git-clone.sh
index 0805168..df4c135 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -221,6 +221,13 @@ then
 		fi
 		reference=$(cd "$reference" && pwd)
 		echo "$reference/objects" >"$GIT_DIR/objects/info/alternates"
+		if test -s "$reference/objects/info/alternates"
+		then
+			while read alt
+			do
+				echo $(cd "$reference" && cd "$alt" && pwd) >> "$GIT_DIR/objects/info/alternates"
+			done < "$reference/objects/info/alternates"
+		fi
 		(cd "$reference" && tar cf - refs) |
 		(cd "$GIT_DIR/refs" &&
 		 mkdir reference-tmp &&

Re: git-clone --reference problem?

[Linus Torvalds]

On Sun, 23 Apr 2006, Linus Torvalds wrote:
>
> I also forget whether we decided that non-absolute paths were 
> relative to the $reference directory, or to the $reference/objects/ 
> directory.

Just checked. It should be relative to $reference/objects, and we should 
strip out lines that start with '#', since we allow comments.

Blah. That whole shell-thing is cerainly convenient for prototyping, but 
it would be wonderful if we had some enterprising young soul that rewrote 
these things in C and made them built-ins. It may be wasteful to use C for 
this, but it would make portability easier, and we have all those nice 
routines for validating and building alternate db lists already that do it 
right.

Oh, well.

		Linus

Re: git-clone --reference problem?

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Maybe something like this could work.
>
> Untested, of course. It probably isn't even syntactically correct shell. 
> Whatever. You get the idea.
>
> (That while-loop could probably be simplified to just a 
>
>   cat ""$reference/objects/info/alternates" >> "$GIT_DIR/objects/info/alternates"
>
> because all alternates _should_ already be absolute paths, but hey, 
> whatever.

I was hoping that the solution would be to do the recursive
alternate resolution on the runtime side (IOW, prepare_alt_odb()
in sha1_file.c).  Then the repository you borrow from _could_
change its own alternates after you set up your repository to
borrow from it.

Currently, if the repository you borrowed from suddenly starts
borrowing from somewhere else and pruned itself by running
"repack -a -d -l", the borrower would be in trouble, even with
your patch.