Re: efficient cloning

[Junio C Hamano <junkio@cox.net>]

James Cloos <cloos@jhcloos.com> writes:

> I presume I need to clone -s -l the local alternate, re-parent it to
> the new URL and grab anything missing, but how can I assure that it
> results in exactly the same repo as this script?

"The same repo as this script" is a very poor way to define what
you really want.  What is "git-repack -a -d -s"?

Guessing what you perhaps are trying to do:

 - You have /gits/linux-2.6/.git on your local disk that is a
   reasonably recent copy of the upstream Linux 2.6 repository.

 - You want to clone from whatever $1 is (maybe a subsystem
   tree, but we cannot tell from your question) to a new
   directory $2.

 - Presumably you know whatever $1 is is related to Linus
   repository and would want to take advantage of the fact that
   it shares many objects with /gits/linux-2.6/.git

It might be worth adding a --reference flag to git-clone like
this patch does.

However, this patch alone does not reduce the transferred data
during cloning any smaller if you are using the "$1" repository
over git native transport (including a local repository),
because the current clone-pack does not look at existing refs
(it was written assuming that there is _nothing_ in the cloned
repository at the beginning).  That needs a separate
enhancements.  Maybe it would be a good idea to deprecate
clone-pack altogether, use fetch-pack -k, and implement the
"copy upstream refs to our refs" logic in git-clone.sh.  We need
to do something like that if/when we are switching to use
$GIT_DIR/refs/remotes/ to store tracking branches outside
refs/heads anyway.

The rsync transport has been deprecated for some time, and it
does not handle alternates correctly anyway, so this patch does
not have any impact on that.

But if you are going to "$1" over http transport, this patch
would help because we stash away the existing refs obtained from
the reference repository under $GIT_DIR/refs/reference-tmp while
we run the fetch.

---
diff --git a/git-clone.sh b/git-clone.sh
index 4ed861d..73fb03c 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -9,7 +9,7 @@
 unset CDPATH
 
 usage() {
-	echo >&2 "Usage: $0 [--bare] [-l [-s]] [-q] [-u <upload-pack>] [-o <name>] [-n] <repo> [<dir>]"
+	echo >&2 "Usage: $0 [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [-o <name>] [-n] <repo> [<dir>]"
 	exit 1
 }
 
@@ -56,6 +56,7 @@ upload_pack=
 bare=
 origin=origin
 origin_override=
+reference=
 while
 	case "$#,$1" in
 	0,*) break ;;
@@ -68,6 +69,11 @@ while
         *,-s|*,--s|*,--sh|*,--sha|*,--shar|*,--share|*,--shared) 
           local_shared=yes; use_local=yes ;;
 	*,-q|*,--quiet) quiet=-q ;;
+	*,--reference=*)
+	  reference=`expr "$1" : '-[^=]*=\(.*\)'` ;;
+	*,--reference)
+	  case "$#" in 1) usage ;; esac
+	  reference="$1" ;;
 	1,-o) usage;;
 	*,-o)
 		git-check-ref-format "$2" || {
@@ -130,6 +136,23 @@ yes)
 	GIT_DIR="$D/.git" ;;
 esac
 
+# If given a reference we would first add that one; it has to name a
+# local repository that resembles the one being cloned.
+if test -d "$reference"
+then
+	reference=$(cd "$reference" && pwd)
+	if test -d "$reference/.git/objects"
+	then
+		reference="$reference/.git"
+	fi
+	echo "$reference/objects" >"$GIT_DIR/objects/info/alternates"
+	# Pretend we know about these heads - clone-pack does not
+	# honor them currently, but that can be rectified later.
+	mkdir "$GIT_DIR/refs/reference-tmp" 
+	(cd "$reference" && tar cf - refs) |
+	(cd "$GIT_DIR/refs/reference-tmp" && tar xf -)
+fi
+
 # We do local magic only when the user tells us to.
 case "$local,$use_local" in
 yes,yes)
@@ -229,6 +252,7 @@ yes,yes)
 esac
 
 cd "$D" || exit
+test -d "$GIT_DIR/refs/reference-tmp" && rm -fr "$GIT_DIR/refs/reference-tmp"
 
 if test -f "$GIT_DIR/HEAD" && test -z "$bare"
 then