[PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Sven Verdoolaege]

To make a submodule effectively usable, the path and
a URL where the submodule can be cloned need to be stored
in .gitmodules.  This subcommand takes care of setting
this information after cloning the new submodule.
Only the index is updated, so, if needed, the user may still
change the URL or switch to a different branch of the submodule
before committing.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
I didn't get any reaction (negative or positive) to this patch.
The only difference with the previous version is that I know
check for the existence of the destination path before calling
module_clone.

skimo

 Documentation/git-submodule.txt |    8 ++++
 git-submodule.sh                |   68 +++++++++++++++++++++++++++++++++++---
 2 files changed, 70 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index f8fb80f..bf9cbf9 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,11 +8,19 @@ git-submodule - Initialize, update or inspect submodules
 
 SYNOPSIS
 --------
+'git-submodule' [--quiet] add <repository> [<path>]
 'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
 
 
 COMMANDS
 --------
+add::
+	Add the given repository as a submodule at the given path
+	to the changeset to be committed next.  In particular, the
+	repository is cloned at the specified path, added to the
+	changeset and registered in .gitmodules.   If no path is
+	specified, the path is deduced from the repository specification.
+
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
diff --git a/git-submodule.sh b/git-submodule.sh
index 89a3885..3df7121 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1,13 +1,14 @@
 #!/bin/sh
 #
-# git-submodules.sh: init, update or list git submodules
+# git-submodules.sh: add, init, update or list git submodules
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'
 . git-sh-setup
 require_work_tree
 
+add=
 init=
 update=
 status=
@@ -25,6 +26,17 @@ say()
 	fi
 }
 
+get_repo_base() {
+	(
+		cd "`/bin/pwd`" &&
+		cd "$1" || cd "$1.git" &&
+		{
+			cd .git
+			pwd
+		}
+	) 2>/dev/null
+}
+
 #
 # Map submodule path to submodule name
 #
@@ -66,6 +78,44 @@ module_clone()
 }
 
 #
+# Add a new submodule to the working tree, .gitmodules and the index
+#
+# $@ = repo [path]
+#
+module_add()
+{
+	repo=$1
+	path=$2
+
+	# Turn the source into an absolute path if
+	# it is local
+	if base=$(get_repo_base "$repo"); then
+		repo="$base"
+	fi
+
+	# Guess path from repo if not specified or strip trailing slashes
+	if test -z "$path"; then
+		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
+	else
+		path=$(echo "$path" | sed -e 's|/*$||')
+	fi
+
+	test -e "$path" &&
+	die "'$path' already exists"
+
+	module_clone "$path" "$repo" || exit
+	(unset GIT_DIR && cd "$path" && git checkout -q) ||
+	die "Unable to checkout submodule '$path'"
+	git add "$path" ||
+	die "Failed to add submodule '$path'"
+
+	GIT_CONFIG=.gitmodules git config submodule."$path".path "$path" &&
+	GIT_CONFIG=.gitmodules git config submodule."$path".url "$repo" &&
+	git add .gitmodules ||
+	die "Failed to register submodule '$path'"
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -173,6 +223,9 @@ modules_list()
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
+	add)
+		add=1
+		;;
 	init)
 		init=1
 		;;
@@ -201,14 +254,17 @@ do
 	shift
 done
 
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$add,$init,$update,$status,$cached" in
+1,,,,)
+	module_add "$@"
+	;;
+,1,,,)
 	modules_init "$@"
 	;;
-,1,,)
+,,1,,)
 	modules_update "$@"
 	;;
-,,*,*)
+,,,*,*)
 	modules_list "$@"
 	;;
 *)
-- 
1.5.2.2.250.g1cd90-dirty

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Lars Hjemli]

On 6/21/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> I didn't get any reaction (negative or positive) to this patch.

Sorry for the late reply, I've been buried in day-job stuff lately
(and still is).

The patch looks sane to me. I'll try to play around with it tomorrow
and give some more feedback.

--
larsh

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> On 6/21/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
>> I didn't get any reaction (negative or positive) to this patch.
>
> Sorry for the late reply, I've been buried in day-job stuff lately
> (and still is).
>
> The patch looks sane to me. I'll try to play around with it tomorrow
> and give some more feedback.

Thanks.

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

>  COMMANDS
>  --------
> +add::
> +	Add the given repository as a submodule at the given path
> +	to the changeset to be committed next.  In particular, the
> +	repository is cloned at the specified path, added to the
> +	changeset and registered in .gitmodules.   If no path is
> +	specified, the path is deduced from the repository specification.
> +

Somehow "git submodule add $URL $my_subdirectory" feels
unnatural, although it certainly is simpler to write the command
usage string.  Wouldn't a commit on the maintenance branch of
cgit.git want to say "Add the 'maint' branch of git.git as my
submodule", for example?

The alternatives I can come up with do not feel right either, though.

	git submodule $my_subdirectory $URL [$branch]
	git submodule $URL [--branch $branch] $my_subdirectory

Hmmm...

> diff --git a/git-submodule.sh b/git-submodule.sh
> index 89a3885..3df7121 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -1,13 +1,14 @@
>  #!/bin/sh
>  #
> -# git-submodules.sh: init, update or list git submodules
> +# git-submodules.sh: add, init, update or list git submodules
>  #
>  # Copyright (c) 2007 Lars Hjemli
>  
> -USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
> +USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'

Can a single repo added at more than one path with this syntax?
I do not see that in the code, but this implies such.

>  . git-sh-setup
>  require_work_tree
>  
> +add=
>  init=
>  update=
>  status=
> @@ -25,6 +26,17 @@ say()
>  	fi
>  }
>  
> +get_repo_base() {
> +	(
> +		cd "`/bin/pwd`" &&
> +		cd "$1" || cd "$1.git" &&
> +		{
> +			cd .git
> +			pwd
> +		}
> +	) 2>/dev/null
> +}
> +

I've seen this code before elsewhere.  We do not need to
refactor right now with this patch, but please mark this copy
with something like:

	# NEEDSWORK: identical function exists in get_repo_base
        # in clone.sh
	get_repo_base () {
        	...

as a reminder.

> @@ -66,6 +78,44 @@ module_clone()
>  }
>  
>  #
> +# Add a new submodule to the working tree, .gitmodules and the index
> +#
> +# $@ = repo [path]
> +#
> +module_add()
> +{
> +	repo=$1
> +	path=$2
> +
> +	# Turn the source into an absolute path if
> +	# it is local
> +	if base=$(get_repo_base "$repo"); then
> +		repo="$base"
> +	fi
> +
> +	# Guess path from repo if not specified or strip trailing slashes
> +	if test -z "$path"; then
> +		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
> +	else
> +		path=$(echo "$path" | sed -e 's|/*$||')
> +	fi
> +
> +	test -e "$path" &&
> +	die "'$path' already exists"
> +
> +	module_clone "$path" "$repo" || exit

 - module_clone catches the "$path already exists" case; but the
   test is done differently.  One particular case of "an empty
   directory exists" is allowed there, but you are dying early
   to forbid it.  Is that warranted?  My gut feeling is that
   they should share the same check, iow, don't check yourself
   but have module_clone take care of the error case.

 - If $path does not exist in the worktree (because it hasn't
   been checked out), but does exist in the index, what should
   happen?  Should it be flagged as an error (in module_clone,
   not here)?

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Sven Verdoolaege]

On Sat, Jun 23, 2007 at 12:58:03PM -0700, Junio C Hamano wrote:
> Somehow "git submodule add $URL $my_subdirectory" feels
> unnatural, although it certainly is simpler to write the command

The order of the arguments is the same as those of git-clone and
you can read it as "git: add submodule $URL (at) $my_subdirectory"

> usage string.  Wouldn't a commit on the maintenance branch of
> cgit.git want to say "Add the 'maint' branch of git.git as my
> submodule", for example?

Sounds plausible

> The alternatives I can come up with do not feel right either, though.
> 
> 	git submodule $my_subdirectory $URL [$branch]
> 	git submodule $URL [--branch $branch] $my_subdirectory

I'll add the latter, although it's not clear if you actually
want me to.

> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index 89a3885..3df7121 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -1,13 +1,14 @@
> >  #!/bin/sh
> >  #
> > -# git-submodules.sh: init, update or list git submodules
> > +# git-submodules.sh: add, init, update or list git submodules
> >  #
> >  # Copyright (c) 2007 Lars Hjemli
> >  
> > -USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
> > +USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'
> 
> Can a single repo added at more than one path with this syntax?

No.  I was trying to be brief.  The more correct syntax would be the one
in the documentation, but I thought that would be a bit lengthy for
USAGE.

> I've seen this code before elsewhere.  We do not need to
> refactor right now with this patch, but please mark this copy
> with something like:
> 
> 	# NEEDSWORK: identical function exists in get_repo_base
>         # in clone.sh
> 	get_repo_base () {
>         	...

OK

> > +	test -e "$path" &&
> > +	die "'$path' already exists"
> > +
> > +	module_clone "$path" "$repo" || exit
> 
>  - module_clone catches the "$path already exists" case; but the
>    test is done differently.  One particular case of "an empty
>    directory exists" is allowed there, but you are dying early
>    to forbid it.  Is that warranted?  My gut feeling is that
>    they should share the same check, iow, don't check yourself
>    but have module_clone take care of the error case.

They're different because submodule update (which also calls module_clone)
is performed on a module that already exists in the repo and
was therefore checked out by git as an empty directory.  If you
add a new submodule, then there is no reason for the subdirectory
to exist already.

>  - If $path does not exist in the worktree (because it hasn't
>    been checked out), but does exist in the index, what should
>    happen?  Should it be flagged as an error (in module_clone,
>    not here)?

Good question.  It should fail.  However, I think this new check
does belong here, because when module_clone is called from
modules_update, the path _should_ exist in the index.

skimo

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

> They're different because submodule update (which also calls module_clone)
> is performed on a module that already exists in the repo and
> was therefore checked out by git as an empty directory.  If you
> add a new submodule, then there is no reason for the subdirectory
> to exist already.

Ah, you are absolutely right.  Thanks for the clarification.

By the way, when I have to say "thanks for the clarification", I
think it is an indication that the relevant parts of the code
need a bit of commenting ;-).

[PATCH] git-submodule: provide easy way of adding new submodules

[Sven Verdoolaege]

To make a submodule effectively usable, the path and
a URL where the submodule can be cloned need to be stored
in .gitmodules.  This subcommand takes care of setting
this information after cloning the new submodule.
Only the index is updated, so, if needed, the user may still
change the URL or switch to a different branch of the submodule
before committing.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
On Sun, Jun 24, 2007 at 10:50:48AM -0700, Junio C Hamano wrote:
> Sven Verdoolaege <skimo@kotnet.org> writes:
> 
> > They're different because submodule update (which also calls module_clone)
> > is performed on a module that already exists in the repo and
> > was therefore checked out by git as an empty directory.  If you
> > add a new submodule, then there is no reason for the subdirectory
> > to exist already.
> 
> Ah, you are absolutely right.  Thanks for the clarification.
> 
> By the way, when I have to say "thanks for the clarification", I
> think it is an indication that the relevant parts of the code
> need a bit of commenting ;-).

How's this ?

skimo

 Documentation/git-submodule.txt |   11 ++++
 git-submodule.sh                |  102 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 107 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index f8fb80f..7f0904e 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,11 +8,19 @@ git-submodule - Initialize, update or inspect submodules
 
 SYNOPSIS
 --------
+'git-submodule' [--quiet] [-b branch] add <repository> [<path>]
 'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
 
 
 COMMANDS
 --------
+add::
+	Add the given repository as a submodule at the given path
+	to the changeset to be committed next.  In particular, the
+	repository is cloned at the specified path, added to the
+	changeset and registered in .gitmodules.   If no path is
+	specified, the path is deduced from the repository specification.
+
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
@@ -39,6 +47,9 @@ OPTIONS
 -q, --quiet::
 	Only print error messages.
 
+-b, --branch::
+	Branch of repository to add as submodule.
+
 --cached::
 	Display the SHA-1 stored in the index, not the SHA-1 of the currently
 	checked out submodule commit. This option is only valid for the
diff --git a/git-submodule.sh b/git-submodule.sh
index 89a3885..67c78f2 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1,13 +1,15 @@
 #!/bin/sh
 #
-# git-submodules.sh: init, update or list git submodules
+# git-submodules.sh: add, init, update or list git submodules
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [add <repo> [-b branch]|status|init|update] [--] [<path>...]'
 . git-sh-setup
 require_work_tree
 
+add=
+branch=
 init=
 update=
 status=
@@ -25,6 +27,18 @@ say()
 	fi
 }
 
+# NEEDSWORK: identical function exists in get_repo_base in clone.sh
+get_repo_base() {
+	(
+		cd "`/bin/pwd`" &&
+		cd "$1" || cd "$1.git" &&
+		{
+			cd .git
+			pwd
+		}
+	) 2>/dev/null
+}
+
 #
 # Map submodule path to submodule name
 #
@@ -42,6 +56,11 @@ module_name()
 #
 # Clone a submodule
 #
+# Prior to calling, modules_update checks that a possibly existing
+# path is not a git repository.
+# Likewise, module_add checks that path does not exist at all,
+# since it is the location of a new submodule.
+#
 module_clone()
 {
 	path=$1
@@ -66,6 +85,53 @@ module_clone()
 }
 
 #
+# Add a new submodule to the working tree, .gitmodules and the index
+#
+# $@ = repo [path]
+#
+# optional branch is stored in global branch variable
+#
+module_add()
+{
+	repo=$1
+	path=$2
+
+	if test -z "$repo"; then
+		usage
+	fi
+
+	# Turn the source into an absolute path if
+	# it is local
+	if base=$(get_repo_base "$repo"); then
+		repo="$base"
+	fi
+
+	# Guess path from repo if not specified or strip trailing slashes
+	if test -z "$path"; then
+		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
+	else
+		path=$(echo "$path" | sed -e 's|/*$||')
+	fi
+
+	test -e "$path" &&
+	die "'$path' already exists"
+
+	git-ls-files --error-unmatch "$path" > /dev/null 2>&1 &&
+	die "'$path' already exists in the index"
+
+	module_clone "$path" "$repo" || exit
+	(unset GIT_DIR && cd "$path" && git checkout -q ${branch:+-b "$branch" "origin/$branch"}) ||
+	die "Unable to checkout submodule '$path'"
+	git add "$path" ||
+	die "Failed to add submodule '$path'"
+
+	GIT_CONFIG=.gitmodules git config submodule."$path".path "$path" &&
+	GIT_CONFIG=.gitmodules git config submodule."$path".url "$repo" &&
+	git add .gitmodules ||
+	die "Failed to register submodule '$path'"
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -173,6 +239,9 @@ modules_list()
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
+	add)
+		add=1
+		;;
 	init)
 		init=1
 		;;
@@ -185,6 +254,14 @@ do
 	-q|--quiet)
 		quiet=1
 		;;
+	-b|--branch)
+		case "$2" in
+		'')
+			usage
+			;;
+		esac
+		branch="$2"; shift
+		;;
 	--cached)
 		cached=1
 		;;
@@ -201,14 +278,27 @@ do
 	shift
 done
 
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$add,$branch" in
+1,*)
+	;;
+,)
+	;;
+,*)
+	usage
+	;;
+esac
+
+case "$add,$init,$update,$status,$cached" in
+1,,,,)
+	module_add "$@"
+	;;
+,1,,,)
 	modules_init "$@"
 	;;
-,1,,)
+,,1,,)
 	modules_update "$@"
 	;;
-,,*,*)
+,,,1,*)
 	modules_list "$@"
 	;;
 *)
-- 
1.5.2.2.1064.g59b4

Re: [PATCH] git-submodule: provide easy way of adding new submodules

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

> To make a submodule effectively usable, the path and
> a URL where the submodule can be cloned need to be stored
> in .gitmodules.  This subcommand takes care of setting
> this information after cloning the new submodule.
> Only the index is updated, so, if needed, the user may still
> change the URL or switch to a different branch of the submodule
> before committing.
>
> Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
> ---
> On Sun, Jun 24, 2007 at 10:50:48AM -0700, Junio C Hamano wrote:
>> Sven Verdoolaege <skimo@kotnet.org> writes:
>> 
>> > They're different because submodule update (which also calls module_clone)
>> > is performed on a module that already exists in the repo and
>> > was therefore checked out by git as an empty directory.  If you
>> > add a new submodule, then there is no reason for the subdirectory
>> > to exist already.
>> 
>> Ah, you are absolutely right.  Thanks for the clarification.
>> 
>> By the way, when I have to say "thanks for the clarification", I
>> think it is an indication that the relevant parts of the code
>> need a bit of commenting ;-).
>
> How's this ?

I am not sure about the usefulness of "-b branch" thing myself,
but other than that, looks very sane to me.

Lars?

Re: [PATCH] git-submodule: provide easy way of adding new submodules

[Sven Verdoolaege]

On Mon, Jun 25, 2007 at 12:09:09AM -0700, Junio C Hamano wrote:
> I am not sure about the usefulness of "-b branch" thing myself,
> but other than that, looks very sane to me.

So, do you want me to send it in again, without the branch thing?

skimo

Re: [PATCH] git-submodule: provide easy way of adding new submodules

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

> On Mon, Jun 25, 2007 at 12:09:09AM -0700, Junio C Hamano wrote:
>> I am not sure about the usefulness of "-b branch" thing myself,
>> but other than that, looks very sane to me.
>
> So, do you want me to send it in again, without the branch thing?

It is already scheduled for 1.5.3 and is part of 'master'.