Re: Git submodule enhancements

Re: Git submodule enhancements

[Lars Hjemli]

On Wed, Sep 24, 2008 at 10:00 AM, P. Christeas <p_christ@hol.gr> wrote:
> In my attempt to take some source off a submoduled repo, I tried writting a
> few lines of code into git-submodule.

Thanks, but please also send this to git@vger.kernel.org. And please
try do inline the patches, it makes it much easier to review; I've
pasted the patches below and inserted a few comments prefixed with
'lh>'.


>From 9edad86468933a21264ab7bed4608ea8a6e992e4 Mon Sep 17 00:00:00 2001
From: P. Christeas <p_christ@hol.gr>
Date: Wed, 24 Sep 2008 10:05:45 +0300
Subject: [PATCH] Git submodule archive: create series of archives, for
each module

This is a temporary solution to creating archives from repos
containing submodules. It will just create a series of archives,
each named after the name of the submodule.
---
 git-submodule.sh |   87 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 86 insertions(+), 1 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 1c39b59..2bce7f9 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -602,6 +602,91 @@ cmd_status()
 		fi
 	done
 }
+
+cmd_archive() {
+	# parse $args after "submodule ... archive".
+	smodules=""
+	verbose=""
+	format=tar
+	do_exec=
+	base_prefix=""
+	outname=
+	while test $# -ne 0
+	do
+		case "$1" in
+		-q|--quiet)
+			quiet=1
+			;;
+		-m|--module)
+			shift
+			smodules+="$1 "
+			;;

lh> Other submodule commands take optional module paths as final
lh> arguments. Why not use the same for `archive`?


+		--format)
+			shift
+			format="$1"
+			;;
+		--exec)
+			shift
+			do_exec="--exec=$1"
+			;;
+		--prefix)
+			shift
+			base_prefix="$1"
+			;;
+		-v|--verbose)
+			verbose="-v"
+			;;
+		-o|--output)
+			shift
+			outname="$1"
+			;;
+		--)
+			shift
+			break
+			;;
+		-*)
+			usage
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+	if [ -z $outname ] ; then
+		outname=$(basename $(pwd))-%m.$format
+	fi
+
+	if ! (echo $outname | grep '%m') > /dev/null  ; then
+		outname+="-%m"
+	fi
+	if ! (echo $outname | grep '^/') > /dev/null  ; then
+		outname=$(pwd)/"$outname"
+	fi
+	
+	#echo "Modules: $smodules"
+	#echo "Params: $@"
+	#echo "Outname: $outname"

lh> Please remove debug comments.


+	
+	module_list "$smodules" |
+	while read mode sha1 stage path
+	do
+		name=$(module_name "$path") || exit
+# 		url=$(git config submodule."$name".url)
+# 		if test -z "$url" || ! test -d "$path"/.git -o -f "$path"/.git
+# 		then
+# 			say "-$sha1 $path"
+# 			continue;
+# 		fi

lh> This should probably work similar to `update`, i.e. exit
lh> with an error if a submodule path is explicitly specified
lh> and the submodule isn't registered in .git/config, and
lh> silently skip any non-registered submodules if no
lh> paths were specified. And if the submodule is not
lh> skipped it probably should be an error if the submodule
lh>  isn't checked out.


+		
+		fname=$(echo $outname |sed 's/%m/'$name'/')

lh> This should probably be documented ;-)


+		say $fname
+		pushd "$path" > /dev/null
+		git archive --format=$format $do_exec $verbose
--prefix="$base_prefix$path/" $sha1 > \
+			"$fname"
+		popd > /dev/null

lh> What does `git archive` do when --exec is specified without
lh> --remote? What about error checking?


+	done
+}
 #
 # Sync remote urls for submodules
 # This makes the value for remote.$remote.url match the value
@@ -656,7 +741,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | update | status | summary | sync)
+	add | foreach | init | update | status | summary | sync | archive)
 		command=$1
 		;;
 	-q|--quiet)
--
1.6.0.1



>From 87b75f003d31994d0de6502342e4d0ad68665e80 Mon Sep 17 00:00:00 2001
From: P. Christeas <p_christ@hol.gr>
Date: Wed, 24 Sep 2008 10:48:04 +0300
Subject: [PATCH] Submodule init: cloned mode

If we try to clone a repo with git submodules, the 'submodule init'
command should bind the submodules to the source repo, rather than
the source's origin.

Signed-off-by: P. Christeas <p_christ@hol.gr>
---
 git-submodule.sh |   37 +++++++++++++++++++++++++------------
 1 files changed, 25 insertions(+), 12 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 2bce7f9..df5fcac 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -241,6 +241,12 @@ cmd_init()
 		-q|--quiet)
 			quiet=1
 			;;
+		-f|--force)
+			force=1
+			;;
+		--cloned)
+			cloned=1
+			;;
 		--)
 			shift
 			break
@@ -261,19 +267,26 @@ cmd_init()
 		# Skip already registered paths
 		name=$(module_name "$path") || exit
 		url=$(git config submodule."$name".url)
-		test -z "$url" || continue
-
-		url=$(git config -f .gitmodules submodule."$name".url)
-		test -z "$url" &&
-		die "No url found for submodule path '$path' in .gitmodules"
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			url=$(resolve_relative_url "$url") || exit
-			;;
-		esac
+		test -z "$url" || test -n "$force" || continue

+		if [ "x$cloned" != "x1" ] ; then
+			url=$(git config -f .gitmodules submodule."$name".url)
+			test -z "$url" &&
+			die "No url found for submodule path '$path' in .gitmodules"
+	
+			# Possibly a url relative to parent
+			case "$url" in
+			./*|../*)
+				url=$(resolve_relative_url "$url") || exit
+				;;
+			esac
+		else
+			# Cloned mode: we try to figure out the submodule
+			# path in the remote origin.
+			# FIXME: we do use "../" because the remote is the .git/
+			url=$(resolve_relative_url "../$path/.git")
+		fi
+		
 		git config submodule."$name".url "$url" ||
 		die "Failed to register url for submodule path '$path'"
--
1.6.0.1


It looks like --cloned requires the downstream to know which
submodules are available from the same remote as the supermodule (and
with the same path as registered in the supermodule), i.e. quite a
specific configuration. Is this really common enough to justify a
special option to `submodule init`?

Maybe the .gitmodules file could be extended to contain multiple urls
instead (i.e. both absolute and relative ones)? Then `submodule init`
could get options like --prefer-relative-url, --prefer-absolute-url
and --interactive. What do you think?

--
larsh

Re: Git submodule enhancements

[Lars Hjemli]

On Wed, Sep 24, 2008 at 11:13 AM, Lars Hjemli <hjemli@gmail.com> wrote:
> On Wed, Sep 24, 2008 at 10:00 AM, P. Christeas <p_christ@hol.gr> wrote:
> +                       # Cloned mode: we try to figure out the submodule
> +                       # path in the remote origin.
> +                       # FIXME: we do use "../" because the remote is the .git/
> +                       url=$(resolve_relative_url "../$path/.git")
> +               fi
> +
>                git config submodule."$name".url "$url" ||
>                die "Failed to register url for submodule path '$path'"
> --
> 1.6.0.1
>
>
> It looks like --cloned requires the downstream to know which
> submodules are available from the same remote as the supermodule (and
> with the same path as registered in the supermodule)

Hmm, after reading through this once more I guess you're trying to
clone a non-bare repository, possibly also lacking a .gitmodules file
(hence the --force)?

--
larsh

Re: Git submodule enhancements

[Lars Hjemli]

On Wed, Sep 24, 2008 at 11:13 AM, Lars Hjemli <hjemli@gmail.com> wrote:
> On Wed, Sep 24, 2008 at 10:00 AM, P. Christeas <p_christ@hol.gr> wrote:
> Subject: [PATCH] Git submodule archive: create series of archives, for
> each module

Btw: why doesn't

  $ git submodule foreach 'git archive HEAD > somewhere/$path.tar'

work for you?


--
larsh

Re: Git submodule enhancements

[P. Christeas]

On Wednesday 24 September 2008, Lars Hjemli wrote:
> It looks like --cloned requires the downstream to know which
> submodules are available from the same remote as the supermodule (and
> with the same path as registered in the supermodule), i.e. quite a
> specific configuration. Is this really common enough to justify a
> special option to `submodule init`?
I agree. My patch is only a temporary fix for the situation and may not cover 
all needs.

> Maybe the .gitmodules file could be extended to contain multiple urls
> instead (i.e. both absolute and relative ones)? Then `submodule init`
> could get options like --prefer-relative-url, --prefer-absolute-url
> and --interactive. What do you think?
I guess there is much room for improvement wrt. with the .gitmodules . For 
one, they should try to mimic the 'git clone' refspec logic, instead of being 
a static file. (in fact, the hooks could let us do that using scripts alone)

> Hmm, after reading through this once more I guess you're trying to
> clone a non-bare repository, possibly also lacking a .gitmodules file
> (hence the --force)?
Not really. I was trying to have the git-submodule script re-write the conf 
with a different url. Merely like running a 'submodule sync' at the same 
time.

> Btw: why doesn't
>   $ git submodule foreach 'git archive HEAD > somewhere/$path.tar'
> work for you?
In fact, it could. You could also replace HEAD with the $sha1 ..

In general, my comment on submodules is that it is a very powerful concept, 
but still lacks many real-life features. Reading the C code, I realized that 
we would have to rework all the core plumbing of git in order to have it work 
accross different git repos inside the same C process. That is a stopper. 
Otherwise, if we could only have read_tree_recursive() span to different gits, 
we could do really beautiful things.

Sorry for posting many ideas and only few patches!

Re: Git submodule enhancements

[René Scharfe]

P. Christeas schrieb:
> On Wednesday 24 September 2008, Lars Hjemli wrote:
>> Btw: why doesn't
>>   $ git submodule foreach 'git archive HEAD > somewhere/$path.tar'
>> work for you?
> In fact, it could. You could also replace HEAD with the $sha1 ..

By the way, have you tried git-archive-all.sh (announced here a month
ago and hosted here: http://github.com/meitar/git-archive-all.sh/wikis)?

René