[PATCH] Add git-submodule command

[PATCH] Add git-submodule command

[Lars Hjemli]

This command can be used to initialize, update and inspect submodules. It
uses a .gitmodules file, readable by git-config, in the top level directory
of the 'superproject' to specify a mapping between submodule paths and
repository url. There is currently no way to override the mappings in the
.gitmodules file, except by manually creating the subproject repository.

Example .gitmodules layout:

[module "git"]
	url = git://git.kernel.org/pub/scm/git/git.git

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

On 5/24/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Thu, 24 May 2007, Lars Hjemli wrote:
> > What I think would be nice is some porcelain support to manually init,
> > update and see the checked out version of selected subprojects, but as
> > standalone commands.
>
> Yes, a la git-remote. I'd be much happier with that, too, especially since
> I think that this can be a relatively small and easy-to-review script.

So, here it is. Please be kind :)

Btw: I've never managed to get asciidoc working on my machine, so the doc
isn't checked in any other format than plain text.


 Documentation/git-submodule.txt |   49 ++++++++++++
 Makefile                        |    2 +-
 git-submodule.sh                |  163 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 213 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/git-submodule.txt
 create mode 100755 git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
new file mode 100644
index 0000000..6ec917c
--- /dev/null
+++ b/Documentation/git-submodule.txt
@@ -0,0 +1,49 @@
+git-submodule(1)
+================
+
+NAME
+----
+git-submodule - Initialize, update or inspect submodules
+
+
+SYNOPSIS
+--------
+'git-submodule' [--init | --update | --cached] [--quiet] <path>...
+
+DESCRIPTION
+-----------
+The command shows the status of each specified submodule path, or all
+submodules if none is specified. Each submodule sha1 is prefixed with '-'
+if the submodule is uninitialized and '+' if the checked out version of
+the submodule is different from the commit sha1 stored in the index.
+
+
+OPTIONS
+-------
+<path>::
+	Path to submodule
+
+-i, --init::
+	Initialize the specified submodules, i.e. clone the git repository
+	specified in .gitmodules and checkout the sha1 specified in the
+	index.
+
+-u, --update::
+	Update the specified submodules, i.e. checkout the sha1 specified
+	in the index
+
+--cached::
+	Display the sha1 stored in the index, not the currently checked
+	out revsion.
+
+-q, --quiet::
+	Be quiet
+
+
+Author
+------
+Written by Lars Hjemli <hjemli@gmail.com>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..5cf2169 100644
--- a/Makefile
+++ b/Makefile
@@ -209,7 +209,7 @@ SCRIPT_SH = \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh \
-	git-lost-found.sh git-quiltimport.sh
+	git-lost-found.sh git-quiltimport.sh git-submodule.sh
 
 SCRIPT_PERL = \
 	git-add--interactive.perl \
diff --git a/git-submodule.sh b/git-submodule.sh
new file mode 100755
index 0000000..c4a1cc3
--- /dev/null
+++ b/git-submodule.sh
@@ -0,0 +1,163 @@
+#!/bin/sh
+#
+# git-submodule.sh: init, update or list git submodules
+#
+# Copyright (c) 2007 Lars Hjemli
+
+USAGE='[-i | --init | -u | --update] [-q | --quiet] [--cached] <path>...'
+. git-sh-setup
+require_work_tree
+
+init=
+update=
+quiet=
+cached=
+
+#
+# print stuff on stdout unless -q was specified
+#
+say()
+{
+	if test -z "$quiet"
+	then
+		echo -e "$@"
+	fi
+}
+
+#
+# Find all (requested) submodules, run clone + checkout on missing paths
+#
+# $@ = requested paths (default to all)
+#
+modules_init()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		test -d "$path/.git" && continue
+
+		if test -d "$path"
+		then
+			rmdir "$path" 2>/dev/null ||
+			die "Directory '$path' exist, but not as a submodule"
+		fi
+
+		test -e "$path" &&
+		die "A file already exist at path '$path'"
+
+		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$path' in .gitmodules"
+
+		git-clone "$url" "$path" ||
+		die "Clone of submodule '$path' failed"
+
+		$(cd "$path" && git-checkout -q "$sha1") ||
+		die "Checkout of submodule '$path' failed"
+
+		say "Submodule '$path' initialized"
+	done
+}
+
+#
+# Checkout correct revision of each initialized submodule
+#
+# $@ = requested paths (default to all)
+#
+modules_update()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "Submodule '$path' not initialized"
+			continue;
+		fi
+		subsha1=$(cd "$path" && git-rev-parse --verify HEAD) ||
+		die "Unable to find current revision of submodule '$path'"
+
+		if test "$subsha1" != "$sha1"
+		then
+			$(cd "$path" && git-fetch && git-checkout -q "$sha1") ||
+			die "Unable to checkout revision $sha1 of submodule '$path'"
+
+			say "Submodule '$path' reset to revision $sha1"
+		fi
+	done
+}
+
+#
+# List all registered submodules, prefixed with:
+#  - submodule not initialized
+#  + different version checked out
+#
+# If --cached was specified the revision in the index will be printed
+# instead of the currently checked out revision.
+#
+# $@ = requested paths (default to all)
+#
+modules_list()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "-$sha1 $path"
+			continue;
+		fi
+		revname=$(cd "$path" && git-describe $sha1)
+		if git diff-files --quiet -- "$path"
+		then
+			say " $sha1 $path\t($revname)"
+		else
+			if test -z "$cached"
+			then
+				sha1=$(cd "$path" && git-rev-parse --verify HEAD)
+				revname=$(cd "$path" && git-describe $sha1)
+			fi
+			say "+$sha1 $path\t($revname)"
+		fi
+	done
+}
+
+
+while case "$#" in 0) break ;; esac
+do
+	case "$1" in
+	-i|--init)
+		init=1
+		;;
+	-u|--update)
+		update=1
+		;;
+	-q|--quiet)
+		quiet=1
+		;;
+	--cached)
+		cached=1
+		;;
+	--)
+		break
+		;;
+	-*)
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+	shift
+done
+
+
+if test "$init" = "1"
+then
+	modules_init $@
+elif test "$update" = "1"
+then
+	modules_update $@
+else
+	modules_list $@
+fi
-- 
1.5.2.73.g18bece-dirty

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Lars Hjemli wrote:

> On 5/24/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Thu, 24 May 2007, Lars Hjemli wrote:
> > > What I think would be nice is some porcelain support to manually init,
> > > update and see the checked out version of selected subprojects, but as
> > > standalone commands.
> >
> > Yes, a la git-remote. I'd be much happier with that, too, especially since
> > I think that this can be a relatively small and easy-to-review script.
> 
> So, here it is. Please be kind :)

Thank you very much! And it looks small enough that I will review it right 
away.

> Btw: I've never managed to get asciidoc working on my machine, so the doc
> isn't checked in any other format than plain text.

No problem, I will check that.

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Lars Hjemli wrote:

> There is currently no way to override the mappings in the .gitmodules 
> file, except by manually creating the subproject repository.

I think that is okay. We can add that easily at a later stage, and the 
script is much easier without that logic.

> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt

Looks good here, I checked with asciidoc.

> diff --git a/git-submodule.sh b/git-submodule.sh
> new file mode 100755
> index 0000000..c4a1cc3
> --- /dev/null
> +++ b/git-submodule.sh
> @@ -0,0 +1,163 @@
> +#!/bin/sh
> +#
> +# git-submodule.sh: init, update or list git submodules
> +#
> +# Copyright (c) 2007 Lars Hjemli
> +
> +USAGE='[-i | --init | -u | --update] [-q | --quiet] [--cached] <path>...'
> +. git-sh-setup
> +require_work_tree
> +
> +init=
> +update=
> +quiet=
> +cached=
> +
> +#
> +# print stuff on stdout unless -q was specified
> +#
> +say()
> +{
> +	if test -z "$quiet"
> +	then
> +		echo -e "$@"
> +	fi
> +}
> +
> +#
> +# Find all (requested) submodules, run clone + checkout on missing paths
> +#
> +# $@ = requested paths (default to all)
> +#
> +modules_init()
> +{
> +	git ls-files --stage -- $@ | grep -e '^160000 ' |

Any reason you read in the stage? It does not seem that you use it.

> +	while read mode sha1 stage path
> +	do
> +		test -d "$path/.git" && continue
> +
> +		if test -d "$path"
> +		then
> +			rmdir "$path" 2>/dev/null ||
> +			die "Directory '$path' exist, but not as a submodule"
> +		fi
> +
> +		test -e "$path" &&
> +		die "A file already exist at path '$path'"
> +
> +		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)

I like that command ;-)

> +		test -z "$url" &&
> +		die "No url found for submodule '$path' in .gitmodules"
> +
> +		git-clone "$url" "$path" ||
> +		die "Clone of submodule '$path' failed"
> +
> +		$(cd "$path" && git-checkout -q "$sha1") ||
> +		die "Checkout of submodule '$path' failed"
> +
> +		say "Submodule '$path' initialized"
> +	done
> +}
> +
> +#
> +# Checkout correct revision of each initialized submodule
> +#
> +# $@ = requested paths (default to all)
> +#
> +modules_update()
> +{
> +	git ls-files --stage -- $@ | grep -e '^160000 ' |

Same here.

> +	while read mode sha1 stage path
> +	do
> +		if ! test -d "$path/.git"
> +		then
> +			say "Submodule '$path' not initialized"
> +			continue;
> +		fi
> +		subsha1=$(cd "$path" && git-rev-parse --verify HEAD) ||

Maybe it would be a better idea to use "git --git-dir="$path" rev-parse 
..."? Just in case somebody calls this with GIT_DIR overridden...

Or, unset GIT_DIR explicitely.

> +		die "Unable to find current revision of submodule '$path'"
> +
> +		if test "$subsha1" != "$sha1"
> +		then
> +			$(cd "$path" && git-fetch && git-checkout -q "$sha1") ||

This will make a detached HEAD, right? Do you want that? (I am not really 
interested in submodules myself, so I haven't thought about it, and I 
haven't followed that monster discussion.)

> +			die "Unable to checkout revision $sha1 of submodule '$path'"
> +
> +			say "Submodule '$path' reset to revision $sha1"

I'd rather not say "reset", since this has a different meaning in Git, but 
rather "set to revision $sha1".

> +		fi
> +	done
> +}
> +
> +#
> +# List all registered submodules, prefixed with:
> +#  - submodule not initialized
> +#  + different version checked out
> +#
> +# If --cached was specified the revision in the index will be printed
> +# instead of the currently checked out revision.
> +#
> +# $@ = requested paths (default to all)
> +#
> +modules_list()
> +{
> +	git ls-files --stage -- $@ | grep -e '^160000 ' |
> +	while read mode sha1 stage path
> +	do
> +		if ! test -d "$path/.git"
> +		then
> +			say "-$sha1 $path"
> +			continue;
> +		fi
> +		revname=$(cd "$path" && git-describe $sha1)
> +		if git diff-files --quiet -- "$path"
> +		then
> +			say " $sha1 $path\t($revname)"
> +		else
> +			if test -z "$cached"
> +			then
> +				sha1=$(cd "$path" && git-rev-parse --verify HEAD)
> +				revname=$(cd "$path" && git-describe $sha1)
> +			fi
> +			say "+$sha1 $path\t($revname)"
> +		fi
> +	done
> +}
> +
> +
> +while case "$#" in 0) break ;; esac
> +do
> +	case "$1" in
> +	-i|--init)
> +		init=1
> +		;;
> +	-u|--update)
> +		update=1
> +		;;
> +	-q|--quiet)
> +		quiet=1
> +		;;
> +	--cached)
> +		cached=1
> +		;;
> +	--)
> +		break
> +		;;
> +	-*)
> +		usage
> +		;;
> +	*)
> +		break
> +		;;
> +	esac
> +	shift
> +done
> +
> +
> +if test "$init" = "1"
> +then
> +	modules_init $@
> +elif test "$update" = "1"
> +then
> +	modules_update $@
> +else
> +	modules_list $@
> +fi

I'll let Junio comment on that command line parsing...

All in all, I like it: it is short, to the point, and it should do the job 
(maybe with a few enhancements like "--update" without arguments means 
_all_ submodules).

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 25 May 2007, Lars Hjemli wrote:
>
> > There is currently no way to override the mappings in the .gitmodules
> > file, except by manually creating the subproject repository.
>
> I think that is okay. We can add that easily at a later stage, and the
> script is much easier without that logic.

Yes. And it is sort of a feature: if you've cloned the submodule using
a different (aka local) url, 'git submodule --init' will leave it
alone.

>
> > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
>
> Looks good here, I checked with asciidoc.

Thanks.

>
> > diff --git a/git-submodule.sh b/git-submodule.sh
> > new file mode 100755
> > index 0000000..c4a1cc3
> > --- /dev/null
> > +++ b/git-submodule.sh
> > @@ -0,0 +1,163 @@
> > +#!/bin/sh
> > +#
> > +# git-submodule.sh: init, update or list git submodules
> > +#
> > +# Copyright (c) 2007 Lars Hjemli
> > +
> > +USAGE='[-i | --init | -u | --update] [-q | --quiet] [--cached] <path>...'
> > +. git-sh-setup
> > +require_work_tree
> > +
> > +init=
> > +update=
> > +quiet=
> > +cached=
> > +
> > +#
> > +# print stuff on stdout unless -q was specified
> > +#
> > +say()
> > +{
> > +     if test -z "$quiet"
> > +     then
> > +             echo -e "$@"
> > +     fi
> > +}
> > +
> > +#
> > +# Find all (requested) submodules, run clone + checkout on missing paths
> > +#
> > +# $@ = requested paths (default to all)
> > +#
> > +modules_init()
> > +{
> > +     git ls-files --stage -- $@ | grep -e '^160000 ' |
>
> Any reason you read in the stage? It does not seem that you use it.

Are there any other way to get the mode info?


>
> > +     while read mode sha1 stage path
> > +     do
> > +             test -d "$path/.git" && continue
> > +
> > +             if test -d "$path"
> > +             then
> > +                     rmdir "$path" 2>/dev/null ||
> > +                     die "Directory '$path' exist, but not as a submodule"
> > +             fi
> > +
> > +             test -e "$path" &&
> > +             die "A file already exist at path '$path'"
> > +
> > +             url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
>
> I like that command ;-)
>
> > +             test -z "$url" &&
> > +             die "No url found for submodule '$path' in .gitmodules"
> > +
> > +             git-clone "$url" "$path" ||
> > +             die "Clone of submodule '$path' failed"
> > +
> > +             $(cd "$path" && git-checkout -q "$sha1") ||
> > +             die "Checkout of submodule '$path' failed"
> > +
> > +             say "Submodule '$path' initialized"
> > +     done
> > +}
> > +
> > +#
> > +# Checkout correct revision of each initialized submodule
> > +#
> > +# $@ = requested paths (default to all)
> > +#
> > +modules_update()
> > +{
> > +     git ls-files --stage -- $@ | grep -e '^160000 ' |
>
> Same here.
>
> > +     while read mode sha1 stage path
> > +     do
> > +             if ! test -d "$path/.git"
> > +             then
> > +                     say "Submodule '$path' not initialized"
> > +                     continue;
> > +             fi
> > +             subsha1=$(cd "$path" && git-rev-parse --verify HEAD) ||
>
> Maybe it would be a better idea to use "git --git-dir="$path" rev-parse
> ..."? Just in case somebody calls this with GIT_DIR overridden...
>
> Or, unset GIT_DIR explicitely.

Hmm, that's annoying (overridden GIT_DIR). I guess 'git --git-dir
$path/.git' would be the easiest solution.


>
> > +             die "Unable to find current revision of submodule '$path'"
> > +
> > +             if test "$subsha1" != "$sha1"
> > +             then
> > +                     $(cd "$path" && git-fetch && git-checkout -q "$sha1") ||
>
> This will make a detached HEAD, right? Do you want that? (I am not really
> interested in submodules myself, so I haven't thought about it, and I
> haven't followed that monster discussion.)

Well, we might want to be smarter about this, but on the other hand:
if the user cares, he can always do 'cd $path && git checkout
$branch', since 'git submodule -u' will skip submodules with the
correct commit checked out.

>
> > +                     die "Unable to checkout revision $sha1 of submodule '$path'"
> > +
> > +                     say "Submodule '$path' reset to revision $sha1"
>
> I'd rather not say "reset", since this has a different meaning in Git, but
> rather "set to revision $sha1".

Ok.

>
> > +             fi
> > +     done
> > +}
> > +
> > +#
> > +# List all registered submodules, prefixed with:
> > +#  - submodule not initialized
> > +#  + different version checked out
> > +#
> > +# If --cached was specified the revision in the index will be printed
> > +# instead of the currently checked out revision.
> > +#
> > +# $@ = requested paths (default to all)
> > +#
> > +modules_list()
> > +{
> > +     git ls-files --stage -- $@ | grep -e '^160000 ' |
> > +     while read mode sha1 stage path
> > +     do
> > +             if ! test -d "$path/.git"
> > +             then
> > +                     say "-$sha1 $path"
> > +                     continue;
> > +             fi
> > +             revname=$(cd "$path" && git-describe $sha1)
> > +             if git diff-files --quiet -- "$path"
> > +             then
> > +                     say " $sha1 $path\t($revname)"
> > +             else
> > +                     if test -z "$cached"
> > +                     then
> > +                             sha1=$(cd "$path" && git-rev-parse --verify HEAD)
> > +                             revname=$(cd "$path" && git-describe $sha1)
> > +                     fi
> > +                     say "+$sha1 $path\t($revname)"
> > +             fi
> > +     done
> > +}
> > +
> > +
> > +while case "$#" in 0) break ;; esac
> > +do
> > +     case "$1" in
> > +     -i|--init)
> > +             init=1
> > +             ;;
> > +     -u|--update)
> > +             update=1
> > +             ;;
> > +     -q|--quiet)
> > +             quiet=1
> > +             ;;
> > +     --cached)
> > +             cached=1
> > +             ;;
> > +     --)
> > +             break
> > +             ;;
> > +     -*)
> > +             usage
> > +             ;;
> > +     *)
> > +             break
> > +             ;;
> > +     esac
> > +     shift
> > +done
> > +
> > +
> > +if test "$init" = "1"
> > +then
> > +     modules_init $@
> > +elif test "$update" = "1"
> > +then
> > +     modules_update $@
> > +else
> > +     modules_list $@
> > +fi
>
> I'll let Junio comment on that command line parsing...

Heh, I'm a shell illiterate...

>
> All in all, I like it: it is short, to the point, and it should do the job
> (maybe with a few enhancements like "--update" without arguments means
> _all_ submodules).

Well, it does (or should) update all initialized submodules, but maybe
that's not what you meant?

Thanks for the review!

--
larsh

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Lars Hjemli wrote:

> On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > 
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> > 
> > > +modules_init()
> > > +{
> > > +     git ls-files --stage -- $@ | grep -e '^160000 ' |
> > 
> > Any reason you read in the stage? It does not seem that you use it.
> 
> Are there any other way to get the mode info?

Ah, I missed that. Right.

> > > +             subsha1=$(cd "$path" && git-rev-parse --verify HEAD) ||
> > 
> > Maybe it would be a better idea to use "git --git-dir="$path" 
> > rev-parse ..."? Just in case somebody calls this with GIT_DIR 
> > overridden...
> > 
> > Or, unset GIT_DIR explicitely.
> 
> Hmm, that's annoying (overridden GIT_DIR). I guess 'git --git-dir 
> $path/.git' would be the easiest solution.

But you have to repeat it on every subsequent Git command. OTOH if you 
unset GIT_DIR, you can no longer be sure that you have the correct git dir 
for Git calls in the superproject...

> > > +             die "Unable to find current revision of submodule '$path'"
> > > +
> > > +             if test "$subsha1" != "$sha1"
> > > +             then
> > > +                     $(cd "$path" && git-fetch && git-checkout -q
> > "$sha1") ||
> > 
> > This will make a detached HEAD, right? Do you want that? (I am not 
> > really interested in submodules myself, so I haven't thought about it, 
> > and I haven't followed that monster discussion.)
> 
> Well, we might want to be smarter about this, but on the other hand: if 
> the user cares, he can always do 'cd $path && git checkout $branch', 
> since 'git submodule -u' will skip submodules with the correct commit 
> checked out.

Fair enough, I guess...

> > I'll let Junio comment on that command line parsing...
> 
> Heh, I'm a shell illiterate...

;-)

I guess Junio would like one of his famous

	case ,"$init","$update",[...] in
	*1*1) usage
	esac

to prevent running with two actions...

> > All in all, I like it: it is short, to the point, and it should do the 
> > job (maybe with a few enhancements like "--update" without arguments 
> > means _all_ submodules).
> 
> Well, it does (or should) update all initialized submodules, but maybe 
> that's not what you meant?

Oops. I meant "init". I mean, most people who want to clone a superproject 
want the submodules being initialized without hassles, probably. But maybe 
that should be another option: "--clone-superproject" or something. Dunno. 
There's time for that after the initial git-submodule.

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Fri, 25 May 2007, Lars Hjemli wrote:
> > On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > On Fri, 25 May 2007, Lars Hjemli wrote:
> > > > +             die "Unable to find current revision of submodule '$path'"
> > > > +
> > > > +             if test "$subsha1" != "$sha1"
> > > > +             then
> > > > +                     $(cd "$path" && git-fetch && git-checkout -q
> > > "$sha1") ||
> > >
> > > This will make a detached HEAD, right? Do you want that? (I am not
> > > really interested in submodules myself, so I haven't thought about it,
> > > and I haven't followed that monster discussion.)
> >
> > Well, we might want to be smarter about this, but on the other hand: if
> > the user cares, he can always do 'cd $path && git checkout $branch',
> > since 'git submodule -u' will skip submodules with the correct commit
> > checked out.
>
> Fair enough, I guess...

The only alternative I can think of is to check if module.$path.branch
is specified in .gitmodules: if it is, and it's pointing at the wanted
sha1, the right thing would be to checkout that branch. Any other sort
of dwim is bound to get it wrong.


>
> > > I'll let Junio comment on that command line parsing...
> >
> > Heh, I'm a shell illiterate...
>
> ;-)
>
> I guess Junio would like one of his famous
>
>         case ,"$init","$update",[...] in
>         *1*1) usage
>         esac
>
> to prevent running with two actions...

Ahh, that's something I can handle


>
> > > All in all, I like it: it is short, to the point, and it should do the
> > > job (maybe with a few enhancements like "--update" without arguments
> > > means _all_ submodules).
> >
> > Well, it does (or should) update all initialized submodules, but maybe
> > that's not what you meant?
>
> Oops. I meant "init".

Hmm, it does (or should) clone all submodules if you run 'git
submodule --init' (no paths specified). Did it fail for you?


-- 
larsh

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Lars Hjemli wrote:

> On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> > > On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > > All in all, I like it: it is short, to the point, and it should do 
> > > > the job (maybe with a few enhancements like "--update" without 
> > > > arguments means _all_ submodules).
> > >
> > > Well, it does (or should) update all initialized submodules, but 
> > > maybe that's not what you meant?
> > 
> > Oops. I meant "init".
> 
> Hmm, it does (or should) clone all submodules if you run 'git submodule 
> --init' (no paths specified). Did it fail for you?

I don't have any superproject to try with ;-)

Actually, I missed the "$@" in modules_init. You might want to change the 
documentation, though, since it suggests (at least to yours truly) that 
you _need_ to pass a path with "--init".

Even happier,
Dscho

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 25 May 2007, Lars Hjemli wrote:
>
> > On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > On Fri, 25 May 2007, Lars Hjemli wrote:
> > > > On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > > All in all, I like it: it is short, to the point, and it should do
> > > > > the job (maybe with a few enhancements like "--update" without
> > > > > arguments means _all_ submodules).
> > > >
> > > > Well, it does (or should) update all initialized submodules, but
> > > > maybe that's not what you meant?
> > >
> > > Oops. I meant "init".
> >
> > Hmm, it does (or should) clone all submodules if you run 'git submodule
> > --init' (no paths specified). Did it fail for you?
>
> I don't have any superproject to try with ;-)

You could always try cgit ;-)

>
> Actually, I missed the "$@" in modules_init. You might want to change the
> documentation, though, since it suggests (at least to yours truly) that
> you _need_ to pass a path with "--init".

Ok, will do.

--
larsh

[PATCH] Add git-submodule command

[Lars Hjemli]

This command can be used to initialize, update and inspect submodules. It
uses a .gitmodules file, readable by git-config, in the top level directory
of the 'superproject' to specify a mapping between submodule paths and
repository url. There is currently no way to override the mappings in the
.gitmodules file, except by manually creating the subproject repository.

Example .gitmodules layout:

[module "git"]
	url = git://git.kernel.org/pub/scm/git/git.git
	branch = maint

The branch-key in .gitmodules is optional. When specified, 'git-submodule -u'
will checkout the named branch if its tip matches the commit-sha1 in the
superprojects index. Otherwise, 'git-submodule -u' will create a detached
HEAD in the submodule.

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

This is an updated patch that hopefully deals with all the previous issues.

I've also added support for specifying module.$path.branch in .gitmodules,
but not support for general url rewriting based on local config. If that
is a wanted feature, it should be trivial to add later on.

Btw: testing this quickly becomes tedious, so I'll try to make a proper
testscript later tonight.


 Documentation/git-submodule.txt |   63 ++++++++++++++
 Makefile                        |    2 +-
 git-submodule.sh                |  178 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 242 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/git-submodule.txt
 create mode 100755 git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
new file mode 100644
index 0000000..81236aa
--- /dev/null
+++ b/Documentation/git-submodule.txt
@@ -0,0 +1,63 @@
+git-submodule(1)
+================
+
+NAME
+----
+git-submodule - Initialize, update or inspect submodules
+
+
+SYNOPSIS
+--------
+'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
+
+
+DESCRIPTION
+-----------
+The command shows the status of each specified submodule path, or all
+submodules if none is specified. Each submodule sha1 is prefixed with '-'
+if the submodule is uninitialized and '+' if the checked out version of
+the submodule is different from the commit sha1 stored in the index.
+
+
+OPTIONS
+-------
+<path>::
+	Path to submodule(s)
+
+-i, --init::
+	Initialize the specified submodules, i.e. clone the git repository
+	specified in .gitmodules and checkout the sha1 specified in the
+	index.
+
+-u, --update::
+	Update the specified submodules, i.e. checkout the sha1 specified
+	in the index
+
+--cached::
+	Display the sha1 stored in the index, not the currently checked
+	out revsion.
+
+-q, --quiet::
+	Be quiet
+
+
+FILES
+-----
+When cloning submodules, a .gitmodules file in the top-level directory
+of the containing work-tree is examined for the url of each submodule.
+The url is the value of the key module.$path.url.
+
+When updating submodules, the same .gitmodules file is examined for a key
+named 'module.$path.branch'. If found, and if the named branch is currently
+at the same revision as the commit-id in the containing repositories index,
+the specified branch will be checked out in the submodule. If not found, or
+if the branch isn't currently positioned at the wanted revision, a checkout
+of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
+
+Author
+------
+Written by Lars Hjemli <hjemli@gmail.com>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..5cf2169 100644
--- a/Makefile
+++ b/Makefile
@@ -209,7 +209,7 @@ SCRIPT_SH = \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh \
-	git-lost-found.sh git-quiltimport.sh
+	git-lost-found.sh git-quiltimport.sh git-submodule.sh
 
 SCRIPT_PERL = \
 	git-add--interactive.perl \
diff --git a/git-submodule.sh b/git-submodule.sh
new file mode 100755
index 0000000..51a0f47
--- /dev/null
+++ b/git-submodule.sh
@@ -0,0 +1,178 @@
+#!/bin/sh
+#
+# git-submodules.sh: init, update or list git submodules
+#
+# Copyright (c) 2007 Lars Hjemli
+
+USAGE='[--init | --update | --cached] [--quiet] [--] [<path>...]'
+. git-sh-setup
+require_work_tree
+
+init=
+update=
+quiet=
+cached=
+
+#
+# print stuff on stdout unless -q was specified
+#
+say()
+{
+	if test -z "$quiet"
+	then
+		echo -e "$@"
+	fi
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_init()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		test -d "$path/.git" && continue
+
+		if test -d "$path"
+		then
+			rmdir "$path" 2>/dev/null ||
+			die "Directory '$path' exist, but not as a submodule"
+		fi
+
+		test -e "$path" &&
+		die "A file already exist at path '$path'"
+
+		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$path' in .gitmodules"
+
+		git-clone "$url" "$path" ||
+		die "Clone of submodule '$path' failed"
+
+		$(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
+		die "Checkout of submodule '$path' failed"
+
+		say "Submodule '$path' initialized"
+	done
+}
+
+#
+# Checkout correct revision of each initialized submodule
+#
+# $@ = requested paths (default to all)
+#
+modules_update()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode rev stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "Submodule '$path' not initialized"
+			continue;
+		fi
+		subsha1=$(unset GIT_DIR && cd "$path" && git-rev-parse --verify HEAD) ||
+		die "Unable to find current revision of submodule '$path'"
+
+		if test "$subsha1" != "$rev"
+		then
+			$(unset GIT_DIR && cd "$path" && git-fetch)
+			branch=$(GIT_CONFIG=.gitmodules git-config module."$path".branch)
+			if test "$branch" != ""
+			then
+				branch_sha1=$(unset GIT_DIR && cd "$path" &&
+					git-rev-parse --verify "$branch")
+				if test "$branch_sha1" = "$rev"
+				then
+					rev="$branch"
+				fi
+			fi
+			$(unset GIT_DIR && cd "$path" && git-checkout -q "$rev") ||
+			die "Unable to checkout '$rev' in submodule '$path'"
+
+			say "Submodule '$path': checked out '$rev'"
+		fi
+	done
+}
+
+#
+# List all registered submodules, prefixed with:
+#  - submodule not initialized
+#  + different version checked out
+#
+# If --cached was specified the revision in the index will be printed
+# instead of the currently checked out revision.
+#
+# $@ = requested paths (default to all)
+#
+modules_list()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "-$sha1 $path"
+			continue;
+		fi
+		revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+		if git diff-files --quiet -- "$path"
+		then
+			say " $sha1 $path\t($revname)"
+		else
+			if test -z "$cached"
+			then
+				sha1=$(unset GIT_DIR && cd "$path" && git-rev-parse --verify HEAD)
+				revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+			fi
+			say "+$sha1 $path\t($revname)"
+		fi
+	done
+}
+
+
+while case "$#" in 0) break ;; esac
+do
+	case "$1" in
+	-i|--init)
+		init=1
+		;;
+	-u|--update)
+		update=1
+		;;
+	-q|--quiet)
+		quiet=1
+		;;
+	--cached)
+		cached=1
+		;;
+	--)
+		break
+		;;
+	-*)
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+	shift
+done
+
+case "$init,$update,$cached" in
+1,,)
+	modules_init $@
+	;;
+,1,)
+	modules_update $@
+	;;
+,,*)
+	modules_list $@
+	;;
+*)
+	usage
+	;;
+esac
-- 
1.5.2.74.gea9f

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Lars Hjemli wrote:

> Btw: testing this quickly becomes tedious, so I'll try to make a proper 
> testscript later tonight.

Very good.

> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]

I did not realize this earlier, but we seem to have more and more programs 
where actions are specified without "--", i.e. "git-svn fetch", or 
"git-bundle create".

I actually like that, to separate actions from options. Hmm?

> +-i, --init::
> +	Initialize the specified submodules, i.e. clone the git repository
> +	specified in .gitmodules and checkout the sha1 specified in the
> +	index.

How about "Initialize the submodules...", and then another sentence "If 
you do not want to initialize all submodules, you can specify the subset 
to initialize"?

> +-u, --update::
> +	Update the specified submodules, i.e. checkout the sha1 specified
> +	in the index

The full stop is missing here. And again, I would add another sentence 
"Submodules which have not been initialized are not touched by this 
operation."

> +FILES
> +-----
> +When cloning submodules, a .gitmodules file in the top-level directory
> +of the containing work-tree is examined for the url of each submodule.
> +The url is the value of the key module.$path.url.

IIRC Junio talked about a name for overriding. But I think it would be 
even better to to override by mapping the URLs from .gitmodules to the 
locally-wanted URLs.

Junio?

> +When updating submodules, the same .gitmodules file is examined for a key
> +named 'module.$path.branch'. If found, and if the named branch is currently 
> +at the same revision as the commit-id in the containing repositories index, 
> +the specified branch will be checked out in the submodule. If not found, or 
> +if the branch isn't currently positioned at the wanted revision, a checkout
> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.

A very good description, and I think this is the only method to checkout 
the submodule which makes sense. (Just maybe default the value of 
module.<path>.branch to "master"?)

> +++ b/git-submodule.sh
> @@ -0,0 +1,178 @@
> +#!/bin/sh
> +#
> +# git-submodules.sh: init, update or list git submodules
> +#
> +# Copyright (c) 2007 Lars Hjemli
> +
> +USAGE='[--init | --update | --cached] [--quiet] [--] [<path>...]'
> +. git-sh-setup
> +require_work_tree

Maybe

	test -f "$GIT_DIR"/.gitmodules || die "Not a superproject"

Hmm?

> +			rmdir "$path" 2>/dev/null ||

Just out of curiousity: is rmdir portable? I always used "rm -r"...

> +case "$init,$update,$cached" in
> +1,,)
> +	modules_init $@
> +	;;

:-)

Now I run out of comments...

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Fri, 25 May 2007, Lars Hjemli wrote:
>
>> Btw: testing this quickly becomes tedious, so I'll try to make a proper 
>> testscript later tonight.
>
> Very good.
>
>> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
>
> I did not realize this earlier, but we seem to have more and more programs 
> where actions are specified without "--", i.e. "git-svn fetch", or 
> "git-bundle create".
>
> I actually like that, to separate actions from options. Hmm?

I think it is a sensible thing to do for this kind of "wrapper
of different functionalities related to one area".

>> +-i, --init::
>> +	Initialize the specified submodules, i.e. clone the git repository
>> +	specified in .gitmodules and checkout the sha1 specified in the
>> +	index.
>
> How about "Initialize the submodules...", and then another sentence "If 
> you do not want to initialize all submodules, you can specify the subset 
> to initialize"?


>> +FILES
>> +-----
>> +When cloning submodules, a .gitmodules file in the top-level directory
>> +of the containing work-tree is examined for the url of each submodule.
>> +The url is the value of the key module.$path.url.
>
> IIRC Junio talked about a name for overriding. But I think it would be 
> even better to to override by mapping the URLs from .gitmodules to the 
> locally-wanted URLs.
>
> Junio?

I really do not want that (mis)conception that .gitmodules
specify the default and .git/config the override.  I really
think we should use the .git/config as _the_ only authority to
get URL, but keyed with the three-level scheme, with URL in
.gitmodules used _solely_ as a hint when setting up the URL in
the .git/config file.

	cf. $gmane/47502, 47548, 47621

>> +When updating submodules, the same .gitmodules file is examined for a key
>> +named 'module.$path.branch'. If found, and if the named branch is currently 
>> +at the same revision as the commit-id in the containing repositories index, 
>> +the specified branch will be checked out in the submodule. If not found, or 
>> +if the branch isn't currently positioned at the wanted revision, a checkout
>> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
>
> A very good description, and I think this is the only method to checkout 
> the submodule which makes sense. (Just maybe default the value of 
> module.<path>.branch to "master"?)

I suspect leaving the HEAD always detached if the superproject
tree names a concrete commit object name would be less confusing
and consistent.  When the name of the commit object in the
superproject tree and/or index is 0{40}, it would be a good
extension to use "whatever commit that happens to be at the tip
of this branch" taken from the .gitmodules file.

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> >
> >> Btw: testing this quickly becomes tedious, so I'll try to make a proper
> >> testscript later tonight.
> >
> > Very good.
> >
> >> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
> >
> > I did not realize this earlier, but we seem to have more and more programs
> > where actions are specified without "--", i.e. "git-svn fetch", or
> > "git-bundle create".
> >
> > I actually like that, to separate actions from options. Hmm?
>
> I think it is a sensible thing to do for this kind of "wrapper
> of different functionalities related to one area".

Ok, there seems to be general agreement on this, so how about
something like this instead:

  git-submodule [--quiet] [--cached] [init | update] [--] [<path>...]

or would you rather have

  git-submodule [--quiet] [--cached] <cmd> [<path>...]

with cmd being one of init, update, status?

> >> +FILES
> >> +-----
> >> +When cloning submodules, a .gitmodules file in the top-level directory
> >> +of the containing work-tree is examined for the url of each submodule.
> >> +The url is the value of the key module.$path.url.
> >
> > IIRC Junio talked about a name for overriding. But I think it would be
> > even better to to override by mapping the URLs from .gitmodules to the
> > locally-wanted URLs.
> >
> > Junio?
>
> I really do not want that (mis)conception that .gitmodules
> specify the default and .git/config the override.  I really
> think we should use the .git/config as _the_ only authority to
> get URL, but keyed with the three-level scheme, with URL in
> .gitmodules used _solely_ as a hint when setting up the URL in
> the .git/config file.
>
>         cf. $gmane/47502, 47548, 47621
>

I've read these articles, but I think much of the concerns about
trusting the url supplied by upstream goes away when the submodule
clone/checkout isn't an integrated part of the superproject
clone/checkout. Besides, if you trust your upstream enough to clone
their repository (the superproject), why wouldn't you trust the data
(.gitmodules) in that very repository?

Still, a way to locally override the suggested url is probably needed.
 This can be easily achieved by either yours or Linus' suggestion
about 'url rewriting'. And if clone/checkout doesn't touch the
submodules at all, the downstream user can look at/override the
.gitmodules file in his local tree before deciding to do the submodule
 clone or checkout. I don't think there is any need for an interactive
tool here.

Another possibility is simply doing the submodule clone/checkout by
hand (i.e. do 'git clone preferred-url path', don't do 'git submodule
init path').


> >> +When updating submodules, the same .gitmodules file is examined for a key
> >> +named 'module.$path.branch'. If found, and if the named branch is currently
> >> +at the same revision as the commit-id in the containing repositories index,
> >> +the specified branch will be checked out in the submodule. If not found, or
> >> +if the branch isn't currently positioned at the wanted revision, a checkout
> >> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
> >
> > A very good description, and I think this is the only method to checkout
> > the submodule which makes sense. (Just maybe default the value of
> > module.<path>.branch to "master"?)
>
> I suspect leaving the HEAD always detached if the superproject
> tree names a concrete commit object name would be less confusing
> and consistent.

And easier to explain :)


>When the name of the commit object in the
> superproject tree and/or index is 0{40}, it would be a good
> extension to use "whatever commit that happens to be at the tip
> of this branch" taken from the .gitmodules file.
>

I really can't imagine what kind of superproject would have such a
setup. Why would this be needed?

-- 
larsh

Re: [PATCH] Add git-submodule command

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> ...
>> I really do not want that (mis)conception that .gitmodules
>> specify the default and .git/config the override.  I really
>> think we should use the .git/config as _the_ only authority to
>> get URL, but keyed with the three-level scheme, with URL in
>> .gitmodules used _solely_ as a hint when setting up the URL in
>> the .git/config file.
>>
>>         cf. $gmane/47502, 47548, 47621
>>
>
> I've read these articles, but I think much of the concerns about
> trusting the url supplied by upstream goes away when the submodule
> clone/checkout isn't an integrated part of the superproject
> clone/checkout. Besides, if you trust your upstream enough to clone
> their repository (the superproject), why wouldn't you trust the data
> (.gitmodules) in that very repository?

It's not about trusting.  You would need to support the mapping
for network connectivity reasons, and you would also need to
notice and reconfirm when the suggested URL in .gitmodules
changes (perhaps because the upstream relocated from sf.net to
repo.or.cz ;-), you would need something like what I described
in order to keep track of user preference for each submodule in
.git/config anyway.  If that "mapping" ends up to be ident
mapping for most people, that is fine.  At least by always doing
the three-level mapping we would not have any special case in
the code, and this is not the performance critical part of the
system.

I think the response to the case when upstream repository
relocates from the ".gitmodule for default, .git/config for
override" camp would be "you asked to override in .git/config,
so it is your job to notice the change in .gitmodules and adjust
your override URL".  That is a serious mistake in usability
point of view.  Repository relocation would (hopefully) seldom
happen, but when it does happen, things either would break
(which is easier to diagnose and manually fix up), or things
clone fine but we reach a wrong repository (which is harder to
notice, as "fetch" may succeed -- it just would not fetch the
right commit).  Being able to notice when upstream repository
relocates and to ask for confirmation when that happens would
eliminate a lot of confusion from that.

> Another possibility is simply doing the submodule clone/checkout by
> hand (i.e. do 'git clone preferred-url path', don't do 'git submodule
> init path').

But that is what this patch is trying to help the users, isn't
it?  It reduces the attractiveness of this new tool greatly if
you give up there.

>>When the name of the commit object in the
>> superproject tree and/or index is 0{40}, it would be a good
>> extension to use "whatever commit that happens to be at the tip
>> of this branch" taken from the .gitmodules file.
>
> I really can't imagine what kind of superproject would have such a
> setup. Why would this be needed?

"We would work with any working version of Linux 2.6 kernel"
would be a sensible thing to say, I would think.

It's purely optional, and as you seem to agree always detaching
HEAD is easier to explain, you do not need "module.$path.branch"
at all.  I just mentioned 0{40} as a possible use case for that
configuration variable.

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

[This thread is starting to get long, sorry for even more noise about
the submodule stuff]

On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> "Lars Hjemli" <hjemli@gmail.com> writes:
>
> > On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> > ...
> >> I really do not want that (mis)conception that .gitmodules
> >> specify the default and .git/config the override.  I really
> >> think we should use the .git/config as _the_ only authority to
> >> get URL, but keyed with the three-level scheme, with URL in
> >> .gitmodules used _solely_ as a hint when setting up the URL in
> >> the .git/config file.
> >>
> >>         cf. $gmane/47502, 47548, 47621
> >>
> >
> > I've read these articles, but I think much of the concerns about
> > trusting the url supplied by upstream goes away when the submodule
> > clone/checkout isn't an integrated part of the superproject
> > clone/checkout. Besides, if you trust your upstream enough to clone
> > their repository (the superproject), why wouldn't you trust the data
> > (.gitmodules) in that very repository?
>
> It's not about trusting.  You would need to support the mapping
> for network connectivity reasons, and you would also need to
> notice and reconfirm when the suggested URL in .gitmodules
> changes (perhaps because the upstream relocated from sf.net to
> repo.or.cz ;-), you would need something like what I described
> in order to keep track of user preference for each submodule in
> .git/config anyway.  If that "mapping" ends up to be ident
> mapping for most people, that is fine.  At least by always doing
> the three-level mapping we would not have any special case in
> the code, and this is not the performance critical part of the
> system.
>
> I think the response to the case when upstream repository
> relocates from the ".gitmodule for default, .git/config for
> override" camp would be "you asked to override in .git/config,
> so it is your job to notice the change in .gitmodules and adjust
> your override URL".  That is a serious mistake in usability
> point of view.  Repository relocation would (hopefully) seldom
> happen, but when it does happen, things either would break
> (which is easier to diagnose and manually fix up), or things
> clone fine but we reach a wrong repository (which is harder to
> notice, as "fetch" may succeed -- it just would not fetch the
> right commit).  Being able to notice when upstream repository
> relocates and to ask for confirmation when that happens would
> eliminate a lot of confusion from that.

Basically, I'd say that as long as the superproject names the sha1 of
the submodule commit, nothing else matters.

If you track a submodule, you would easily notice it if the submodule
has the 'wrong' commit checked out (git diff, git status, git
submodule status). And 'git submodule update' would synchronize the
submodules you have decided to track, or error out with a message like
"Unable to checkout '$sha1' in submodule '$path'".

This is when the ugly sides of submodules raises its head (the
'official' repo has moved, my local repo is out of date, whatever). I
just don't see the need for solving those problems now. The 'git
submodule' command would make the common cases easier. Hopefully(?)
that would encourage more people to test/use submodules, and the
problems that actually _needs_ solving will then show up in due time.

> > Another possibility is simply doing the submodule clone/checkout by
> > hand (i.e. do 'git clone preferred-url path', don't do 'git submodule
> > init path').
>
> But that is what this patch is trying to help the users, isn't
> it?  It reduces the attractiveness of this new tool greatly if
> you give up there.

Well, I happen to think that the average user of submodules wouldn't
care the slightest bit where the submodule was cloned from, as long as
the sha1 matches. So no, the patch was more about the lack of
submodule porcelain and less about completeness.


> >>When the name of the commit object in the
> >> superproject tree and/or index is 0{40}, it would be a good
> >> extension to use "whatever commit that happens to be at the tip
> >> of this branch" taken from the .gitmodules file.
> >
> > I really can't imagine what kind of superproject would have such a
> > setup. Why would this be needed?
>
> "We would work with any working version of Linux 2.6 kernel"
> would be a sensible thing to say, I would think.

Maybe. I wouldn't want to automatically track the tip of _any_ branch,
since I would have no way of knowing if what works today will also
work tomorrow (or even compile).

> It's purely optional, and as you seem to agree always detaching
> HEAD is easier to explain, you do not need "module.$path.branch"
> at all.  I just mentioned 0{40} as a possible use case for that
> configuration variable.

Ok.

I'll redo the patch, removing the branch-specific things, and try to shut up :)

-- 
larsh

[PATCH] Add git-submodule command

[Lars Hjemli]

This command can be used to initialize, update and inspect submodules. It
uses a .gitmodules file, readable by git-config, in the top level directory
of the 'superproject' to specify a mapping between submodule paths and
repository url. There is currently no way to override the mappings in the
.gitmodules file, except by manually creating the subproject repository.

Example .gitmodules layout:

[module "git"]
	url = git://git.kernel.org/pub/scm/git/git.git

With this entry in .gitmodules (and a commit reference in the index entry for
the path "git"), the command 'git submodule init' will clone the repository
at kernel.org into the directory "git".

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

On 5/26/07, Lars Hjemli <hjemli@gmail.com> wrote:
> I'll redo the patch, removing the branch-specific things, and try to
> shut up :)

This is my final uttering ;-)


 .gitignore                      |    1 +
 Documentation/git-submodule.txt |   65 +++++++++++++++
 Makefile                        |    2 +-
 git-submodule.sh                |  172 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 239 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/git-submodule.txt
 create mode 100755 git-submodule.sh

diff --git a/.gitignore b/.gitignore
index 4dc0c39..8fc4923 100644
--- a/.gitignore
+++ b/.gitignore
@@ -126,6 +126,7 @@ git-ssh-push
 git-ssh-upload
 git-status
 git-stripspace
+git-submodule
 git-svn
 git-svnimport
 git-symbolic-ref
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
new file mode 100644
index 0000000..2135331
--- /dev/null
+++ b/Documentation/git-submodule.txt
@@ -0,0 +1,65 @@
+git-submodule(1)
+================
+
+NAME
+----
+git-submodule - Initialize, update or inspect submodules
+
+
+SYNOPSIS
+--------
+'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
+
+
+COMMANDS
+--------
+status::
+	Show the status of the submodules. This will print the sha1 of the
+	currently checked out commit for each submodule, along with the
+	submodule path and the output of gitlink:git-describe[1] for the
+	sha1. Each sha1 will be prefixed with '-' if the submodule is not
+	initialized and '+' if the currently checked out submodule commit
+	does not match the sha1 found in the index of the containing
+	repository. This command is the default command for git-submodule.
+
+init::
+	Initialize the submodules, i.e. clone the git repositories specified
+	in the .gitmodules file and checkout the submodule commits specified
+	in the index of the containing repository. This will make the
+	submodules HEAD be detached.
+
+update::
+	Update the initialized submodules, i.e. checkout the submodule commits
+	specified in the index of the containing repository. This will make
+	the submodules HEAD be detached.
+
+
+OPTIONS
+-------
+-q, --quiet::
+	Only print error messages.
+
+--cached::
+	Display the sha1 stored in the index, not the sha1 of the currently
+	checked out submodule commit. This option is only valid for the
+	status command.
+
+<path>::
+	Path to submodule(s). When specified this will restrict the command
+	to only operate on the submodules found at the specified paths.
+
+FILES
+-----
+When cloning submodules, a .gitmodules file in the top-level directory
+of the containing repository is used to find the url of each submodule.
+This file should be formatted in the same way as $GIR_DIR/config. The key
+to each submodule url is "module.$path.url".
+
+
+AUTHOR
+------
+Written by Lars Hjemli <hjemli@gmail.com>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..5cf2169 100644
--- a/Makefile
+++ b/Makefile
@@ -209,7 +209,7 @@ SCRIPT_SH = \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh \
-	git-lost-found.sh git-quiltimport.sh
+	git-lost-found.sh git-quiltimport.sh git-submodule.sh
 
 SCRIPT_PERL = \
 	git-add--interactive.perl \
diff --git a/git-submodule.sh b/git-submodule.sh
new file mode 100755
index 0000000..247b1ee
--- /dev/null
+++ b/git-submodule.sh
@@ -0,0 +1,172 @@
+#!/bin/sh
+#
+# git-submodules.sh: init, update or list git submodules
+#
+# Copyright (c) 2007 Lars Hjemli
+
+USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+. git-sh-setup
+require_work_tree
+
+init=
+update=
+status=
+quiet=
+cached=
+
+#
+# print stuff on stdout unless -q was specified
+#
+say()
+{
+	if test -z "$quiet"
+	then
+		echo -e "$@"
+	fi
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_init()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		test -d "$path/.git" && continue
+
+		if test -d "$path"
+		then
+			rmdir "$path" 2>/dev/null ||
+			die "Directory '$path' exist, but not as a submodule"
+		fi
+
+		test -e "$path" &&
+		die "A file already exist at path '$path'"
+
+		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$path' in .gitmodules"
+
+		git-clone "$url" "$path" ||
+		die "Clone of submodule '$path' failed"
+
+		$(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
+		die "Checkout of submodule '$path' failed"
+
+		say "Submodule '$path' initialized"
+	done
+}
+
+#
+# Checkout correct revision of each initialized submodule
+#
+# $@ = requested paths (default to all)
+#
+modules_update()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "Submodule '$path' not initialized"
+			continue;
+		fi
+		subsha1=$(unset GIT_DIR && cd "$path" &&
+			git-rev-parse --verify HEAD) ||
+		die "Unable to find current revision of submodule '$path'"
+
+		if test "$subsha1" != "$sha1"
+		then
+			$(unset GIT_DIR && cd "$path" && git-fetch &&
+				git-checkout -q "$sha1") ||
+			die "Unable to checkout '$sha1' in submodule '$path'"
+
+			say "Submodule '$path': checked out '$sha1'"
+		fi
+	done
+}
+
+#
+# List all registered submodules, prefixed with:
+#  - submodule not initialized
+#  + different revision checked out
+#
+# If --cached was specified the revision in the index will be printed
+# instead of the currently checked out revision.
+#
+# $@ = requested paths (default to all)
+#
+modules_list()
+{
+	git ls-files --stage -- $@ | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path/.git"
+		then
+			say "-$sha1 $path"
+			continue;
+		fi
+		revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+		if git diff-files --quiet -- "$path"
+		then
+			say " $sha1 $path\t($revname)"
+		else
+			if test -z "$cached"
+			then
+				sha1=$(unset GIT_DIR && cd "$path" && git-rev-parse --verify HEAD)
+				revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+			fi
+			say "+$sha1 $path\t($revname)"
+		fi
+	done
+}
+
+while case "$#" in 0) break ;; esac
+do
+	case "$1" in
+	init)
+		init=1
+		;;
+	update)
+		update=1
+		;;
+	status)
+		status=1
+		;;
+	-q|--quiet)
+		quiet=1
+		;;
+	--cached)
+		cached=1
+		;;
+	--)
+		break
+		;;
+	-*)
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+	shift
+done
+
+case "$init,$update,$status,$cached" in
+1,,,)
+	modules_init $@
+	;;
+,1,,)
+	modules_update $@
+	;;
+,,*,*)
+	modules_list $@
+	;;
+*)
+	usage
+	;;
+esac
-- 
1.5.2.74.ga303

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Sat, 26 May 2007, Lars Hjemli wrote:

> On 5/26/07, Lars Hjemli <hjemli@gmail.com> wrote:
> > I'll redo the patch, removing the branch-specific things, and try to
> > shut up :)
> 
> This is my final uttering ;-)

Don't be so shy. We're really making progress here, methinks.

Your version looks good to me.

Ciao,
Dscho "awaiting with impatience the test cases"

Re: [PATCH] Add git-submodule command

[Junio C Hamano]

Lars Hjemli <hjemli@gmail.com> writes:

> With this entry in .gitmodules (and a commit reference in the index entry for
> the path "git"), the command 'git submodule init' will clone the repository
> at kernel.org into the directory "git".
>
> Signed-off-by: Lars Hjemli <hjemli@gmail.com>
> ---
>
> On 5/26/07, Lars Hjemli <hjemli@gmail.com> wrote:
>> I'll redo the patch, removing the branch-specific things, and try to
>> shut up :)

Hey, don't shut up.  Starting small and covering corner cases
incrementally is really the right approach.

> +status::
> +	Show the status of the submodules. This will print the sha1 of the
> +	currently checked out commit for each submodule, along with the
> +	submodule path and the output of gitlink:git-describe[1] for the
> +	sha1. Each sha1 will be prefixed with '-' if the submodule is not
> +	initialized and '+' if the currently checked out submodule commit
> +	does not match the sha1 found in the index of the containing
> +	repository. This command is the default command for git-submodule.

(markup) probably you would want `` there for typewriter face.

(wording) didn't we have "the name of the hash function is
SHA-1" patch earlier?  I'd personally prefer calling them
"object names", though...

Other than that, the command description is very nicely done,
which means the design of the command set hence the semantics is
cleanly done.  Good job.

> diff --git a/git-submodule.sh b/git-submodule.sh
> new file mode 100755
> index 0000000..247b1ee
> --- /dev/null
> +++ b/git-submodule.sh
> @@ -0,0 +1,172 @@
> ...
> +#
> +# print stuff on stdout unless -q was specified
> +#
> +say()
> +{
> +	if test -z "$quiet"
> +	then
> +		echo -e "$@"
> +	fi
> +}

We tend to avoid "echo -e" (not POSIX).  I do not see any string
you feed to this function that you would _want_ backslash
escaped sequences (actually I would suspect you would not want
them).

> +
> +#
> +# Run clone + checkout on missing submodules
> +#
> +# $@ = requested paths (default to all)
> +#
> +modules_init()
> +{
> +	git ls-files --stage -- $@ | grep -e '^160000 ' |

Did you mean "$@", i.e. inside double-quotes?

Because this pattern would appear a lot in superproject support,
it might be a good idea to give a new option, --subprojects, to
git-ls-files to limit its output to 160000 entries, but that is
a minor detail.

> +	while read mode sha1 stage path
> +	do

We would need to undo the shell-safety "quoted" output of paths
here.  I suspect it would be much easier to code this in Perl or
Python, do the "grep -e" part above in the script, when we start
caring about unwrapping c-quoting of path (or "ls-files -z").

But that is a minor detail we could fix up later.

> +		test -d "$path/.git" && continue
> +
> +		if test -d "$path"
> +		then
> +			rmdir "$path" 2>/dev/null ||
> +			die "Directory '$path' exist, but not as a submodule"
> +		fi

Could the currently checked-out $path be a symlink to another
directory, and what does the code do in such a case?

> +
> +		test -e "$path" &&
> +		die "A file already exist at path '$path'"

"test -e" is a relatively new invention and I tended to stay
away from it, but it should be safe to use these days...

> +		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
> +		test -z "$url" &&
> +		die "No url found for submodule '$path' in .gitmodules"
> +
> +		git-clone "$url" "$path" ||
> +		die "Clone of submodule '$path' failed"

"git-clone -n" please, as you will checkout something different
in the next step anyway.

> +
> +		$(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
> +		die "Checkout of submodule '$path' failed"

Lose $() around this, as it is not producing a string which is
the name of the command to run.  You do want a subshell here
because of chdir, so instead of losing $(), replace them with
().

> ...
> +modules_update()
> +{
>...
> +		if test "$subsha1" != "$sha1"
> +		then
> +			$(unset GIT_DIR && cd "$path" && git-fetch &&
> +				git-checkout -q "$sha1") ||
> +			die "Unable to checkout '$sha1' in submodule '$path'"

Likewise.

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/26/07, Junio C Hamano <junkio@cox.net> wrote:
> Lars Hjemli <hjemli@gmail.com> writes:
>
> > With this entry in .gitmodules (and a commit reference in the index entry for
> > the path "git"), the command 'git submodule init' will clone the repository
> > at kernel.org into the directory "git".
> >
> > Signed-off-by: Lars Hjemli <hjemli@gmail.com>
> > ---
> >
> > On 5/26/07, Lars Hjemli <hjemli@gmail.com> wrote:
> >> I'll redo the patch, removing the branch-specific things, and try to
> >> shut up :)
>
> Hey, don't shut up.  Starting small and covering corner cases
> incrementally is really the right approach.

Ok, my meds have kicked in and I'm ready for more ;-)

>
> > +status::
> > +     Show the status of the submodules. This will print the sha1 of the
> > +     currently checked out commit for each submodule, along with the
> > +     submodule path and the output of gitlink:git-describe[1] for the
> > +     sha1. Each sha1 will be prefixed with '-' if the submodule is not
> > +     initialized and '+' if the currently checked out submodule commit
> > +     does not match the sha1 found in the index of the containing
> > +     repository. This command is the default command for git-submodule.
>
> (markup) probably you would want `` there for typewriter face.

Ok, will fix

>
> (wording) didn't we have "the name of the hash function is
> SHA-1" patch earlier?  I'd personally prefer calling them
> "object names", though...

Some quick grepping wasn't helpfull:

~/src/git/Documentation$ git grep sha1 | wc -l
98
~/src/git/Documentation$ git grep SHA1 | wc -l
80
~/src/git/Documentation$ git grep SHA-1 | wc -l
22

Would you prefer SHA-1?

>
> Other than that, the command description is very nicely done,
> which means the design of the command set hence the semantics is
> cleanly done.  Good job.

Thanks!

>
> > diff --git a/git-submodule.sh b/git-submodule.sh
> > new file mode 100755
> > index 0000000..247b1ee
> > --- /dev/null
> > +++ b/git-submodule.sh
> > @@ -0,0 +1,172 @@
> > ...
> > +#
> > +# print stuff on stdout unless -q was specified
> > +#
> > +say()
> > +{
> > +     if test -z "$quiet"
> > +     then
> > +             echo -e "$@"
> > +     fi
> > +}
>
> We tend to avoid "echo -e" (not POSIX).  I do not see any string
> you feed to this function that you would _want_ backslash
> escaped sequences (actually I would suspect you would not want
> them).

I do use \t between submodule path and the result of git-describe, but
it's not really needed. I'll drop it.

>
> > +
> > +#
> > +# Run clone + checkout on missing submodules
> > +#
> > +# $@ = requested paths (default to all)
> > +#
> > +modules_init()
> > +{
> > +     git ls-files --stage -- $@ | grep -e '^160000 ' |
>
> Did you mean "$@", i.e. inside double-quotes?

That looks right, yes


> > +             test -d "$path/.git" && continue
> > +
> > +             if test -d "$path"
> > +             then
> > +                     rmdir "$path" 2>/dev/null ||
> > +                     die "Directory '$path' exist, but not as a submodule"
> > +             fi
>
> Could the currently checked-out $path be a symlink to another
> directory, and what does the code do in such a case?

This I will have to test, but I _suspect_ it would fail on "test -e
path" _unless_ the "test -d $path/.git" kicks in.

>
> > +
> > +             test -e "$path" &&
> > +             die "A file already exist at path '$path'"
>
> "test -e" is a relatively new invention and I tended to stay
> away from it, but it should be safe to use these days...

Would you prefer separate -f and -l instead? Hmm, that would match up
nicely with your concern about symlinks ;-)

>
> > +             url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
> > +             test -z "$url" &&
> > +             die "No url found for submodule '$path' in .gitmodules"
> > +
> > +             git-clone "$url" "$path" ||
> > +             die "Clone of submodule '$path' failed"
>
> "git-clone -n" please, as you will checkout something different
> in the next step anyway.

Nice, I hadn't noticed -n, will fix

>
> > +
> > +             $(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
> > +             die "Checkout of submodule '$path' failed"
>
> Lose $() around this, as it is not producing a string which is
> the name of the command to run.  You do want a subshell here
> because of chdir, so instead of losing $(), replace them with
> ().

Heh, there's always something new to learn on this list, thanks.

--
larsh

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Junio C Hamano wrote:

> Lars Hjemli <hjemli@gmail.com> writes:
> 
> > +
> > +#
> > +# Run clone + checkout on missing submodules
> > +#
> > +# $@ = requested paths (default to all)
> > +#
> > +modules_init()
> > +{
> > +	git ls-files --stage -- $@ | grep -e '^160000 ' |
> 
> Did you mean "$@", i.e. inside double-quotes?
> 
> Because this pattern would appear a lot in superproject support,
> it might be a good idea to give a new option, --subprojects, to
> git-ls-files to limit its output to 160000 entries, but that is
> a minor detail.

I think that makes sense. It would also help the next one:

> 
> > +	while read mode sha1 stage path
> > +	do
> 
> We would need to undo the shell-safety "quoted" output of paths
> here.  I suspect it would be much easier to code this in Perl or
> Python, do the "grep -e" part above in the script, when we start
> caring about unwrapping c-quoting of path (or "ls-files -z").

I would prefer this to be in shell... so to be easier make it a builtin 
later, when the script has evolved into a stable state. By introducing 
--subprojects to git-ls-files, you can roll your own quoting just for 
git-submodule, as needed.

> But that is a minor detail we could fix up later.

I agree. It makes for a nice incremental patch.

> > +		test -d "$path/.git" && continue
> > +
> > +		if test -d "$path"
> > +		then
> > +			rmdir "$path" 2>/dev/null ||
> > +			die "Directory '$path' exist, but not as a submodule"
> > +		fi
> 
> Could the currently checked-out $path be a symlink to another
> directory, and what does the code do in such a case?

I am not quite sure if you should allow that... So, IMHO this is stuff to 
discuss after the initial revision (and after we have a test case, so we 
can play around with symlinks safely).

Ciao,
Dscho

[PATCH] Add git-submodule command

[Lars Hjemli]

This command can be used to initialize, update and inspect submodules. It
uses a .gitmodules file, readable by git-config, in the top level directory
of the 'superproject' to specify a mapping between submodule paths and
repository url.

Example .gitmodules layout:

[module "git"]
	url = git://git.kernel.org/pub/scm/git/git.git

With this entry in .gitmodules (and a commit reference in the index entry for
the path "git"), the command 'git submodule init' will clone the repository
at kernel.org into the directory "git".

Known issues
============
There is currently no way to override the url found in the .gitmodules file,
except by manually creating the subproject repository. The place to fix this
in the script has a rather long comment about a possible plan.

Funny paths will be quoted in the output from git-ls-files, but git-submodule
does not attempt to unquote (or even detect the presence of) such paths.

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

So here is another attempt on initial submodule porcelain, still trying to
keep it simple. A proper test-script hasn't materialized yet, sorry about
that.

 .gitignore                      |    1 +
 Documentation/cmd-list.perl     |    1 +
 Documentation/git-submodule.txt |   65 +++++++++++++
 Makefile                        |    2 +-
 git-submodule.sh                |  195 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 263 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/git-submodule.txt
 create mode 100755 git-submodule.sh

diff --git a/.gitignore b/.gitignore
index 4dc0c39..8fc4923 100644
--- a/.gitignore
+++ b/.gitignore
@@ -126,6 +126,7 @@ git-ssh-push
 git-ssh-upload
 git-status
 git-stripspace
+git-submodule
 git-svn
 git-svnimport
 git-symbolic-ref
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index 443802a..51dc6b4 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -180,6 +180,7 @@ git-ssh-fetch                           synchingrepositories
 git-ssh-upload                          synchingrepositories
 git-status                              mainporcelain
 git-stripspace                          purehelpers
+git-submodule                           mainporcelain
 git-svn                                 foreignscminterface
 git-svnimport                           foreignscminterface
 git-symbolic-ref                        plumbingmanipulators
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
new file mode 100644
index 0000000..02932a7
--- /dev/null
+++ b/Documentation/git-submodule.txt
@@ -0,0 +1,65 @@
+git-submodule(1)
+================
+
+NAME
+----
+git-submodule - Initialize, update or inspect submodules
+
+
+SYNOPSIS
+--------
+'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
+
+
+COMMANDS
+--------
+status::
+	Show the status of the submodules. This will print the SHA-1 of the
+	currently checked out commit for each submodule, along with the
+	submodule path and the output of gitlink:git-describe[1] for the
+	SHA-1. Each SHA-1 will be prefixed with "-" if the submodule is not
+	initialized and "+" if the currently checked out submodule commit
+	does not match the SHA-1 found in the index of the containing
+	repository. This command is the default command for git-submodule.
+
+init::
+	Initialize the submodules, i.e. clone the git repositories specified
+	in the .gitmodules file and checkout the submodule commits specified
+	in the index of the containing repository. This will make the
+	submodules HEAD be detached.
+
+update::
+	Update the initialized submodules, i.e. checkout the submodule commits
+	specified in the index of the containing repository. This will make
+	the submodules HEAD be detached.
+
+
+OPTIONS
+-------
+-q, --quiet::
+	Only print error messages.
+
+--cached::
+	Display the SHA-1 stored in the index, not the SHA-1 of the currently
+	checked out submodule commit. This option is only valid for the
+	status command.
+
+<path>::
+	Path to submodule(s). When specified this will restrict the command
+	to only operate on the submodules found at the specified paths.
+
+FILES
+-----
+When cloning submodules, a .gitmodules file in the top-level directory
+of the containing repository is used to find the url of each submodule.
+This file should be formatted in the same way as $GIR_DIR/config. The key
+to each submodule url is "module.$path.url".
+
+
+AUTHOR
+------
+Written by Lars Hjemli <hjemli@gmail.com>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..5cf2169 100644
--- a/Makefile
+++ b/Makefile
@@ -209,7 +209,7 @@ SCRIPT_SH = \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh \
-	git-lost-found.sh git-quiltimport.sh
+	git-lost-found.sh git-quiltimport.sh git-submodule.sh
 
 SCRIPT_PERL = \
 	git-add--interactive.perl \
diff --git a/git-submodule.sh b/git-submodule.sh
new file mode 100755
index 0000000..1601bca
--- /dev/null
+++ b/git-submodule.sh
@@ -0,0 +1,195 @@
+#!/bin/sh
+#
+# git-submodules.sh: init, update or list git submodules
+#
+# Copyright (c) 2007 Lars Hjemli
+
+USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+. git-sh-setup
+require_work_tree
+
+init=
+update=
+status=
+quiet=
+cached=
+
+#
+# print stuff on stdout unless -q was specified
+#
+say()
+{
+	if test -z "$quiet"
+	then
+		echo "$@"
+	fi
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_init()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		# Skip submodule paths that already contain a .git directory.
+		# This will also trigger if $path is a symlink to a git
+		# repository
+		test -d "$path"/.git && continue
+
+		# If there already is a directory at the submodule path,
+		# expect it to be empty (since that is the default checkout
+		# action) and try to remove it.
+		# Note: if $path is a symlink to a directory the test will
+		# succeed but the rmdir will fail. We might want to fix this.
+		if test -d "$path"
+		then
+			rmdir "$path" 2>/dev/null ||
+			die "Directory '$path' exist, but is neither empty "
+			    "nor a git repository"
+		fi
+
+		test -e "$path"
+		die "A file already exist at path '$path'"
+
+		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$path' in .gitmodules"
+
+		# MAYBE FIXME: this would be the place to check GIT_CONFIG
+		# for a preferred url for this submodule, possibly like this:
+		#
+		# modname=$(GIT_CONFIG=.gitmodules git-config module."$path".name)
+		# alturl=$(git-config module."$modname".url)
+		#
+		# This would let the versioned .gitmodules file use the submodule
+		# path as key, while the unversioned GIT_CONFIG would use the
+		# logical modulename (if present) as key. But this would need
+		# another fallback mechanism if the module wasn't named.
+
+		git-clone -n "$url" "$path" ||
+		die "Clone of submodule '$path' failed"
+
+		(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
+		die "Checkout of submodule '$path' failed"
+
+		say "Submodule '$path' initialized"
+	done
+}
+
+#
+# Checkout correct revision of each initialized submodule
+#
+# $@ = requested paths (default to all)
+#
+modules_update()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path"/.git
+		then
+			# Only mention uninitialized submodules when its
+			# path have been specified
+			test "$#" != "0" &&
+			say "Submodule '$path' not initialized"
+			continue;
+		fi
+		subsha1=$(unset GIT_DIR && cd "$path" &&
+			git-rev-parse --verify HEAD) ||
+		die "Unable to find current revision of submodule '$path'"
+
+		if test "$subsha1" != "$sha1"
+		then
+			(unset GIT_DIR && cd "$path" && git-fetch &&
+				git-checkout -q "$sha1") ||
+			die "Unable to checkout '$sha1' in submodule '$path'"
+
+			say "Submodule '$path': checked out '$sha1'"
+		fi
+	done
+}
+
+#
+# List all registered submodules, prefixed with:
+#  - submodule not initialized
+#  + different revision checked out
+#
+# If --cached was specified the revision in the index will be printed
+# instead of the currently checked out revision.
+#
+# $@ = requested paths (default to all)
+#
+modules_list()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path"/.git
+		then
+			say "-$sha1 $path"
+			continue;
+		fi
+		revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+		if git diff-files --quiet -- "$path"
+		then
+			say " $sha1 $path ($revname)"
+		else
+			if test -z "$cached"
+			then
+				sha1=$(unset GIT_DIR && cd "$path" && git-rev-parse --verify HEAD)
+				revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+			fi
+			say "+$sha1 $path ($revname)"
+		fi
+	done
+}
+
+while case "$#" in 0) break ;; esac
+do
+	case "$1" in
+	init)
+		init=1
+		;;
+	update)
+		update=1
+		;;
+	status)
+		status=1
+		;;
+	-q|--quiet)
+		quiet=1
+		;;
+	--cached)
+		cached=1
+		;;
+	--)
+		break
+		;;
+	-*)
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+	shift
+done
+
+case "$init,$update,$status,$cached" in
+1,,,)
+	modules_init "$@"
+	;;
+,1,,)
+	modules_update "$@"
+	;;
+,,*,*)
+	modules_list "$@"
+	;;
+*)
+	usage
+	;;
+esac
-- 
1.5.2.74.ga303-dirty

Re: [PATCH] Add git-submodule command

[Simon Hausmann]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On Saturday 26 May 2007 15:56:40 Lars Hjemli wrote:
[...]
> +modules_init()
> +{
> +	git ls-files --stage -- "$@" | grep -e '^160000 ' |
> +	while read mode sha1 stage path
> +	do
> +		# Skip submodule paths that already contain a .git directory.
> +		# This will also trigger if $path is a symlink to a git
> +		# repository
> +		test -d "$path"/.git && continue
> +
> +		# If there already is a directory at the submodule path,
> +		# expect it to be empty (since that is the default checkout
> +		# action) and try to remove it.
> +		# Note: if $path is a symlink to a directory the test will
> +		# succeed but the rmdir will fail. We might want to fix this.
> +		if test -d "$path"
> +		then
> +			rmdir "$path" 2>/dev/null ||
> +			die "Directory '$path' exist, but is neither empty "
> +			    "nor a git repository"
> +		fi
> +
> +		test -e "$path"
> +		die "A file already exist at path '$path'"

Isn't there a && missing at the end of the test line? :)


Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/26/07, Simon Hausmann <simon@lst.de> wrote:
> On Saturday 26 May 2007 15:56:40 Lars Hjemli wrote:
> > +             test -e "$path"
> > +             die "A file already exist at path '$path'"
>
> Isn't there a && missing at the end of the test line? :)
>

Hmph. There's always something slipping through, thanks for noticing.
I will make the testscript now....

-- 
larsh

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> >
> >> +FILES
> >> +-----
> >> +When cloning submodules, a .gitmodules file in the top-level directory
> >> +of the containing work-tree is examined for the url of each submodule.
> >> +The url is the value of the key module.$path.url.
> >
> > IIRC Junio talked about a name for overriding. But I think it would be 
> > even better to to override by mapping the URLs from .gitmodules to the 
> > locally-wanted URLs.
> >
> > Junio?
> 
> I really do not want that (mis)conception that .gitmodules
> specify the default and .git/config the override.  I really
> think we should use the .git/config as _the_ only authority to
> get URL, but keyed with the three-level scheme, with URL in
> .gitmodules used _solely_ as a hint when setting up the URL in
> the .git/config file.
> 
> 	cf. $gmane/47502, 47548, 47621

Yes, I skipped over the first two, and concentrated on the third. If 
.gitmodules are used to initialize what is in .git/config, I agree.

However, could we have that as a separate commit, to keep things simple 
enough so that I can understand them?

> >> +When updating submodules, the same .gitmodules file is examined for a key
> >> +named 'module.$path.branch'. If found, and if the named branch is currently 
> >> +at the same revision as the commit-id in the containing repositories index, 
> >> +the specified branch will be checked out in the submodule. If not found, or 
> >> +if the branch isn't currently positioned at the wanted revision, a checkout
> >> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
> >
> > A very good description, and I think this is the only method to checkout 
> > the submodule which makes sense. (Just maybe default the value of 
> > module.<path>.branch to "master"?)
> 
> I suspect leaving the HEAD always detached if the superproject
> tree names a concrete commit object name would be less confusing
> and consistent.  When the name of the commit object in the
> superproject tree and/or index is 0{40}, it would be a good
> extension to use "whatever commit that happens to be at the tip
> of this branch" taken from the .gitmodules file.

Yes, that's a much better idea.

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Linus Torvalds]

On Fri, 25 May 2007, Johannes Schindelin wrote:
>
> I did not realize this earlier, but we seem to have more and more programs 
> where actions are specified without "--", i.e. "git-svn fetch", or 
> "git-bundle create".

Hey, don't forget "git bisect", the granddaddy of them all.

> I actually like that, to separate actions from options. Hmm?

I agree. If something effectively always takes a separate command, it's 
not an option, it's a subcommand.

		Linus

Re: [PATCH] Add git-submodule command

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Linus Torvalds wrote:

> On Fri, 25 May 2007, Johannes Schindelin wrote:
> >
> > I did not realize this earlier, but we seem to have more and more programs 
> > where actions are specified without "--", i.e. "git-svn fetch", or 
> > "git-bundle create".
> 
> Hey, don't forget "git bisect", the granddaddy of them all.

Oh, sorry! How could I? Respect the elders. ;-)

Ciao,
Dscho

Re: [PATCH] Add git-submodule command

[Lars Hjemli]

On 5/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> [* many usefull comments about the docs *]

Thanks, will fix

> > +USAGE='[--init | --update | --cached] [--quiet] [--] [<path>...]'
> > +. git-sh-setup
> > +require_work_tree
>
> Maybe
>
>         test -f "$GIT_DIR"/.gitmodules || die "Not a superproject"
>
> Hmm?

Yeah, maybe. But the command will only touch entries with mode 160000
anyway, so it might not be a big deal.

>
> > +                     rmdir "$path" 2>/dev/null ||
>
> Just out of curiousity: is rmdir portable? I always used "rm -r"...

I have no idea, really. But the reason for using rmdir was this
section of the man page:

NAME
       rmdir - remove empty directories

I do not want to delete an unempty directory. But there is probably
some better way to do this?

--
larsh