[PATCH v2 1/5] git-submodule.sh: Add Long Usage instead of simple usage

[PATCH v2 1/5] git-submodule.sh: Add Long Usage instead of simple usage

[imyousuf]

From: Imran M Yousuf <imyousuf@smartitengineering.com>

With the already available commands the synopsis is quite hard to
read; thus converted it to Long usage instead. In process also
updated the file comment.

Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
 git-submodule.sh |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index ce0f00c..a5ee2e5 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1,12 +1,17 @@
 #!/bin/sh
 #
-# git-submodules.sh: add, init, update or list git submodules
+# git-submodules.sh: add, init, update, summary or status git submodules
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE="[--quiet] [--cached] \
-[add <repo> [-b branch]|status|init|update|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]"
+USAGE="<command> <options>
+Use $0 -h for more details"
+# Did not use '\' at the end of the lines to ensure that each synopsis
+# are in a separate line
+LONG_USAGE="$0 add [-q|--quiet] [-b|--branch branch] <repository> [<path>]
+$0 [status] [-q|--quiet] [-c|--cached] [--] [<path>...]
+$0 init|update [-q|--quiet] [--] [<path>...]
+$0 summary [--cached] [-n|--summary-limit <n>] [<commit>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
-- 
1.5.4.2

[PATCH v2 2/5] git-submodule.sh: Add recurse subcommand with basic options

[imyousuf]

From: Imran M Yousuf <imyousuf@smartitengineering.com>

The purpose of the recurse command in the git submodule is to recurse
a command in its submodule. For example if one wants to do a diff on its
project with submodules at once, one can simply do
	git-submodule recurse diff HEAD
and would see the diff for all the modules it contains.

The recurse commands behavior can be customized with several arguments
that it accepts. The synopsis for the recurse command is:

	git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
	[-d|--depth <recursion depth>] [-b|--breadth-first]
	<git command> [<arguments> ...]

There are commands that can fail for a certain submodule but succeed for
others; if one wants to stop execution once the top level module's execution
fails, one can specify [-e|--exit-after-error]. It will ensure that once
execution of git <command> fails in the top level module it will not recurse
into its submodules.

If the project has submodule hierarchy upto n depth and we want to restrict
recursion to (n-p) depth; we can use the [-d|--depth <recursion depth>] option.
Value has to be greater than 0 and command will at least recurse into the first
depth. If depth is specified to p than all depths <= p will be recursed over.

While discussion on the recurse command one thing which was put forward
in several occassions is that there might be scenario where a command should be
executed over the child module before the parent module.

	For such scenario [-b|--breadth-first] option can be used; one use case
in particular presented as an example is git commit; where almost everybody
mentioned that they prefer to commit the child module before the parent and
default will enable just that.

	E.g. p -> a, b, c, e; a ->d is a module structure. If the following command is
used,

	git submodule recurse commit -a

it will execute git commit -a in the following sequence - d, a, b, c, e, p.

	Now if one want to instead go in a breadth first manner then one can
specify -b option. E.g. if the above command is -

	git submodule recurse -b commit -a

it will execute git commit -a in the following sequence - p, a, d, b, c, e.

Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
 git-submodule.sh |  132 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 130 insertions(+), 2 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index a5ee2e5..8161d51 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -11,7 +11,8 @@ Use $0 -h for more details"
 LONG_USAGE="$0 add [-q|--quiet] [-b|--branch branch] <repository> [<path>]
 $0 [status] [-q|--quiet] [-c|--cached] [--] [<path>...]
 $0 init|update [-q|--quiet] [--] [<path>...]
-$0 summary [--cached] [-n|--summary-limit <n>] [<commit>]"
+$0 summary [--cached] [-n|--summary-limit <n>] [<commit>]
+$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] <git command> [<args> ...]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -20,6 +21,10 @@ command=
 branch=
 quiet=
 cached=
+depth=0
+current_depth=0
+depth_first=1
+on_error=
 
 #
 # print stuff on stdout unless -q was specified
@@ -580,6 +585,129 @@ cmd_status()
 	done
 }
 
+# Check whether the submodule is initialized or not
+initialize_sub_module()
+{
+	if test ! -d "$1"/.git
+	then
+		say "Submodule $1 is not initialized and skipped"
+		return 1
+	# Returns true if submodule is already initialized
+	elif test -d "$1"/.git
+	then
+		return 0
+	fi
+}
+
+# This function simply checks whether the depth is traverseable in terms of
+# depth and if so then it sequentially traverses its submodules
+traverse_submodules()
+{
+	# If current depth is the range specified than it will continue
+	# else return with success
+	if test "$depth" -gt 0 &&
+		test "$current_depth" -ge "$depth"
+	then
+		return 0;
+	fi
+	# If submodules exists than it will traverse over them
+	if test -f .gitmodules
+	then
+		# Incrementing the depth for the next level of submodules
+		current_depth=$(($current_depth + 1))
+                for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
+                        traverse_module "$mod_path" "$@"
+                done
+		# Decremented the depth to bring it back to the depth of
+		# the current submodule
+		current_depth=$(($current_depth - 1))
+	fi
+}
+
+# This actually traverses a submodule; checks whether the its initialized
+# or not, does nothing if not initialized.
+traverse_module()
+{
+	# Will work in the submodule if and only if its initialized
+	initialize_sub_module "$1" &&
+	(
+		submod_path="$1"
+		shift
+		cd "$submod_path"
+		# If depth-first is specified in that case submodules are
+		# are traversed before executing the command on this submodule
+		test -n "$depth_first" && traverse_submodules "$@"
+		# pwd is mentioned in order to enable the ser to distinguish
+		# between same name modules, e.g. a/lib and b/lib.
+		say "git submodule recurse $submod_path $*"
+		git "$@"
+		# if exit on error is specifed than script will exit if any
+		# command fails. As there is no transaction there will be
+		# no rollback either
+		# TODO - If possible facilitate transaction
+		if test "$?" -ne 0 && test -n "$on_error"
+		then
+			die "FAILED: git submodule $submod_path $*"
+		fi
+		# If depth-first is not specified in that case submodules are
+		# are traversed after executing the command on this submodule
+		test -z "$depth_first" && traverse_submodules "$@"
+	)
+}
+
+# Propagates or recurses over all the submodules at any depth with any
+# git command, e.g. git-clone, git-status, git-commit etc., with the
+# arguments supplied exactly as it would have been supplied to the command
+# otherwise. This actually starts the recursive propagation.
+cmd_recurse() {
+	while :
+	do
+		case "$1" in
+		-q|--quiet)
+			quiet=1
+			;;
+		-d|--depth)
+			shift
+			if test -z "$1"
+			then
+				echo "No <recursion depth> specified"
+				usage
+			# Arithmatic operation will give an error if depth is not number
+			# thus chose to check intergerness with regular expression.
+			# $1 is underquoted becuase the expr is in quotation
+			elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"
+			then
+				depth="$1"
+			else
+				echo "<recursion depth> not an integer"
+				usage
+			fi
+			;;
+		-b|--breadth-first)
+			depth_first=
+			;;
+		-e|--exit-after-error)
+			on_error=1
+			;;
+		-*)
+			usage
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+	test "$#" -le 0 && die "No git command specified"
+	project_home="$(pwd)"
+	if test -d "$project_home"/.git/
+	then
+		traverse_module . "$@"
+	else
+		die "$project_home not a git repo thus exiting"
+	fi
+}
+
 # This loop parses the command line arguments to find the
 # subcommand name to dispatch.  Parsing of the subcommand specific
 # options are primarily done by the subcommand implementations.
@@ -589,7 +717,7 @@ cmd_status()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | init | update | status | summary)
+	add | init | update | status | summary |recurse)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.5.4.2

[PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[imyousuf]

From: Imran M Yousuf <imyousuf@smartitengineering.com>

There is a scenario which has been put forward several times in
discussion over the recurse subcommand and it is that commands chould have
different arguments for different modules.

For example, one module could want to checkout 'master', while another might want
to checkout 'work'. The [-a|--customized-argument] argument provides platform
just for that. Consider the following command and its followup for further info:

	git submodule recurse -a checkout

	Submodule b is not initialized and skipped
	git submodule recurse a checkout
	Please provide an argument: master
	Press y to provide another arg...
	git checkout master
	Already on branch "master"
	Submodule d is not initialized and skipped
	git submodule recurse . checkout
	Please provide an argument: master
	Press y to provide another arg...
	git checkout master
	Already on branch "master"

This command would also come in handy for diffs and other commands.

Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
 git-submodule.sh |   53 +++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 8161d51..314652d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -12,7 +12,7 @@ LONG_USAGE="$0 add [-q|--quiet] [-b|--branch branch] <repository> [<path>]
 $0 [status] [-q|--quiet] [-c|--cached] [--] [<path>...]
 $0 init|update [-q|--quiet] [--] [<path>...]
 $0 summary [--cached] [-n|--summary-limit <n>] [<commit>]
-$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] <git command> [<args> ...]"
+$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] [-a|--customized-argument] <git command> [<args> ...]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -25,6 +25,8 @@ depth=0
 current_depth=0
 depth_first=1
 on_error=
+use_custom_args=
+custom_args=
 
 #
 # print stuff on stdout unless -q was specified
@@ -585,6 +587,40 @@ cmd_status()
 	done
 }
 
+# Take arguments from user to pass as custom arguments and execute the command
+exec_with_custom_args()
+{
+	input=
+	arg_index=0
+	eval_str="set "
+	while test $# -gt 0
+	do
+		arg_index=$(($arg_index + 1))
+		var='$'"$arg_index"
+		input="$1"
+		eval_str="$eval_str $var \"$input\""
+		shift
+	done
+	while :
+	do
+		arg_index=$(($arg_index + 1))
+		printf "Please provide an argument: "
+		read input
+		var='$'"$arg_index"
+		eval_str="$eval_str $var \"$input\""
+		printf "Press y to provide another arg... "
+		read keypress
+		if test "$keypress" != "y" &&
+			test "$keypress" != "Y"
+		then
+			break
+		fi
+	done
+	eval $eval_str
+	say "$*"
+	"$@"
+}
+
 # Check whether the submodule is initialized or not
 initialize_sub_module()
 {
@@ -637,10 +673,16 @@ traverse_module()
 		# If depth-first is specified in that case submodules are
 		# are traversed before executing the command on this submodule
 		test -n "$depth_first" && traverse_submodules "$@"
-		# pwd is mentioned in order to enable the ser to distinguish
-		# between same name modules, e.g. a/lib and b/lib.
 		say "git submodule recurse $submod_path $*"
-		git "$@"
+		if test -n "$use_custom_args"
+		then
+			# Execute the commands after taking the arguments
+			# Please note that one input is for one argument
+			# only.
+			exec_with_custom_args git "$@"
+		else
+			git "$@"
+		fi
 		# if exit on error is specifed than script will exit if any
 		# command fails. As there is no transaction there will be
 		# no rollback either
@@ -689,6 +731,9 @@ cmd_recurse() {
 		-e|--exit-after-error)
 			on_error=1
 			;;
+		-a|--customized-argument)
+			use_custom_args=1
+			;;
 		-*)
 			usage
 			;;
-- 
1.5.4.2

[PATCH v2 4/5] git-submodule.sh: Add pre command argument to git submodule recurse subcommand

[imyousuf]

From: Imran M Yousuf <imyousuf@smartitengineering.com>

I usually feel that when typing a command, being able to see some options
come in handy. For example if I can see the available branches before checking
out a branch that would be useful, IOW, if I could do 'git branch' before git
checkout it would be helpful.

It is now possible using the [-p|--pre-command] option. Using this
subcommand command argument one can actually execute another command before
specifying the arguments or the original command getting executed.

	git submodule recurse -a -p checkout

it will prompt the user for the pre command until one is satisfied and later
the original command with the custom argument will get executed.

Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
 git-submodule.sh |   29 ++++++++++++++++++++++++++++-
 1 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 314652d..dd80850 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -12,7 +12,7 @@ LONG_USAGE="$0 add [-q|--quiet] [-b|--branch branch] <repository> [<path>]
 $0 [status] [-q|--quiet] [-c|--cached] [--] [<path>...]
 $0 init|update [-q|--quiet] [--] [<path>...]
 $0 summary [--cached] [-n|--summary-limit <n>] [<commit>]
-$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] [-a|--customized-argument] <git command> [<args> ...]"
+$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] [-a|--customized-argument] [-p|--pre-command] <git command> [<args> ...]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -27,6 +27,7 @@ depth_first=1
 on_error=
 use_custom_args=
 custom_args=
+pre_cmd=
 
 #
 # print stuff on stdout unless -q was specified
@@ -587,6 +588,28 @@ cmd_status()
 	done
 }
 
+# Take command from user and execute it until user wants to discontinue
+do_pre_command()
+{
+	say "Starting pre-comamnd execution!"
+	while :
+	do
+		(
+			printf "Please provide a command: "
+			read pre_command
+			test -z "$pre_command" ||
+			eval "$pre_command"
+		)
+		printf "Press y to continue with another shell command... "
+		read keypress
+		if test "$keypress" != "y" &&
+			test "$keypress" != "Y"
+		then
+			break
+		fi
+	done
+}
+
 # Take arguments from user to pass as custom arguments and execute the command
 exec_with_custom_args()
 {
@@ -673,6 +696,7 @@ traverse_module()
 		# If depth-first is specified in that case submodules are
 		# are traversed before executing the command on this submodule
 		test -n "$depth_first" && traverse_submodules "$@"
+		test -n "$pre_cmd" && do_pre_command
 		say "git submodule recurse $submod_path $*"
 		if test -n "$use_custom_args"
 		then
@@ -734,6 +758,9 @@ cmd_recurse() {
 		-a|--customized-argument)
 			use_custom_args=1
 			;;
+		-p|--pre-command)
+			pre_cmd=1
+			;;
 		-*)
 			usage
 			;;
-- 
1.5.4.2

[PATCH v2 5/5] Documentation/git-submodule.txt: Add documentation for the recurse subcommand

[imyousuf]

From: Imran M Yousuf <imyousuf@smartitengineering.com>

Documentation with brief description is added for the recurse
sucommand along with its arguments and their nature.

Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
 Documentation/git-submodule.txt |   35 +++++++++++++++++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 6ffd896..9a95522 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,6 +13,10 @@ SYNOPSIS
 'git-submodule' [--quiet] status [--cached] [--] [<path>...]
 'git-submodule' [--quiet] [init|update] [--] [<path>...]
 'git-submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
+'git-submodule' recurse [-q|--quiet] [-e|--exit-after-error]
+		[-d|--depth <recursion depth>] [-b|--breadth-first]
+		[-a|--customized-argument] [-p|--pre-command]
+		<git command> [<arg> ...]"
 
 
 COMMANDS
@@ -54,6 +58,10 @@ summary::
 	in the submodule between the given super project commit and the
 	index or working tree (switched by --cached) are shown.
 
+recurse::
+	Recurse, IOW propagate, command to its submodules if they are
+	initialized.
+
 OPTIONS
 -------
 -q, --quiet::
@@ -78,6 +86,33 @@ OPTIONS
 	Path to submodule(s). When specified this will restrict the command
 	to only operate on the submodules found at the specified paths.
 
+-e, --exit-after-error::
+	This option is only valid for the recurse command. If its provided then
+	then command will not be recursed into any other module once a command
+	has failed
+-d, --depth <recursion depth>::
+	This option is only valid for the recurse command. If its provided then
+	then the command will be recursed upto <recursion depth> only.
+-b, --breadth-first::
+	This option is only valid for the recurse command. If its provided then
+	the command will execute in the current node before traversing to its
+	child, else it will first traverse the children before executing in the
+	current node.
+-a, --customized-argument::
+	This option is only valid for the recurse command. If its provided then
+	user will be prompted for an argument for the <git command> specified.
+	Its particularly useful when one wants to supply different arguments
+	for the same <git command> for different submodules; for example,
+	checking out a branch, one might want branch to differ from submodule
+	to submodule
+-p, --pre-command::
+	This option is only valid for the recurse command. If its provided then
+	user will be prompted for a shell command, e.g. 'ls -al', 'pwd' etc.
+<git command> [<arg>...]::
+	Any git command and their argument. For example, to get the status use
+	'status' as <git command> and '-s' or '-o' or any other 'git status'
+	arguments as <arg>
+
 FILES
 -----
 When initializing submodules, a .gitmodules file in the top-level directory
-- 
1.5.4.2

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Junio C Hamano]

imyousuf@gmail.com writes:

> From: Imran M Yousuf <imyousuf@smartitengineering.com>
>
> There is a scenario which has been put forward several times in
> discussion over the recurse subcommand and it is that commands chould have
> different arguments for different modules.
>
> For example, one module could want to checkout 'master', while another might want
> to checkout 'work'. The [-a|--customized-argument] argument provides platform
> just for that. Consider the following command and its followup for further info:
>
> 	git submodule recurse -a checkout
>
> 	Submodule b is not initialized and skipped
> 	git submodule recurse a checkout
> 	Please provide an argument: master
> 	Press y to provide another arg...
> 	git checkout master
> 	Already on branch "master"
> 	Submodule d is not initialized and skipped
> 	git submodule recurse . checkout
> 	Please provide an argument: master
> 	Press y to provide another arg...
> 	git checkout master
> 	Already on branch "master"

Is it only me who finds this UI (and the one 4/5 further introduces)
somewhat clumsy and extremely ugly?

I am almost tempted to suggest going to the extreme and spawn interactive
shell in each repository directory, like this:

    toplevel$ git submodule recurse -a
    (info) Submodule b is not initialized and skipped
    (info) git submodule recurse 'a'
    (info) we now give you a shell in that directory.  Do whatever you
    (info) like and type cntl-D (or "exit") once you are done.
    toplevel/a$ git checkout
    toplevel/a$ exit
    (info) Submodule d is not initialized and skipped
    (info) git submodule recurse '.'
    (info) we now give you a shell in that directory.  Do whatever you
    (info) like and type cntl-D (or "exit") once you are done.
    toplevel/.$ git checkout
    toplevel/.$ exit
    (info) git submodule recurse recursion ended.
    toplevel$

so that the users can do whatever they want there.

If we want a useful and flexible "recurse", perhaps the only thing we need
to do is a command that lists a submodule directory path, one path at a
time, in optionally different traversal order and depth cutoff, so that
the user can feed it to xargs and do whatever they want to run in there.

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Johan Herland]

On Tuesday 13 May 2008, Junio C Hamano wrote:
> If we want a useful and flexible "recurse", perhaps the only thing we need
> to do is a command that lists a submodule directory path, one path at a
> time, in optionally different traversal order and depth cutoff, so that
> the user can feed it to xargs and do whatever they want to run in there.

Yes! As more and more porcelains on top of the git submodule plumbing is
created, this is definitely something that is (or will be) sorely needed.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Sverre Rabbelier]

On Tue, May 13, 2008 at 12:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
> If we want a useful and flexible "recurse", perhaps the only thing we need
> to do is a command that lists a submodule directory path, one path at a
> time, in optionally different traversal order and depth cutoff, so that
> the user can feed it to xargs and do whatever they want to run in there.

How about Windows? Do we want to depend on something like
http://gnuwin32.sourceforge.net/packages/findutils.htm or does msysgit
ship with xargs? (Or do we not intend to build internal commands upon
this system and leave using the output of "recurse" to the user?)

-- 
Cheers,

Sverre Rabbelier

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Johannes Schindelin]

Hi,

On Sun, 18 May 2008, Sverre Rabbelier wrote:

> On Tue, May 13, 2008 at 12:43 AM, Junio C Hamano <gitster@pobox.com> 
> wrote:
>
> > If we want a useful and flexible "recurse", perhaps the only thing we 
> > need to do is a command that lists a submodule directory path, one 
> > path at a time, in optionally different traversal order and depth 
> > cutoff, so that the user can feed it to xargs and do whatever they 
> > want to run in there.
> 
> How about Windows? Do we want to depend on something like 
> http://gnuwin32.sourceforge.net/packages/findutils.htm or does msysgit 
> ship with xargs? (Or do we not intend to build internal commands upon 
> this system and leave using the output of "recurse" to the user?)

git-repack already relies on "find", and if you have no silly naming 
scheme *1*, you do not need "xargs".  Besides, we do ship xargs.exe.

Ciao,
Dscho

*1* Oops, this is Windows, right?

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Sverre Rabbelier]

On Sun, May 18, 2008 at 5:32 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> git-repack already relies on "find", and if you have no silly naming
> scheme *1*, you do not need "xargs".  Besides, we do ship xargs.exe.

Ok, awesome, sounds like a good solution to me then.

-- 
Cheers,

Sverre Rabbelier

Re: [PATCH v2 3/5] git-submodule.sh: Add Custom argument input support to git submodule recurse subcommand

[Imran M Yousuf]

On Tue, May 13, 2008 at 4:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> For example, one module could want to checkout 'master', while another might want
>> to checkout 'work'. The [-a|--customized-argument] argument provides platform
>> just for that. Consider the following command and its followup for further info:
>>......
>.......
> If we want a useful and flexible "recurse", perhaps the only thing we need
> to do is a command that lists a submodule directory path, one path at a
> time, in optionally different traversal order and depth cutoff, so that
> the user can feed it to xargs and do whatever they want to run in there.
>
I am not sure whether I understand it correctly or not, we could
simply provide the shell window by invoking 'sh', right? If we do that
then is there need to use xargs? I am not particularly experienced
with xargs, I was thinking of something as follows:

#!/bin/sh
echo "Submodule a"
(
PS1="module/submodule$" sh
)
echo "" #In case of ctrl+D to ensure that there is a blank line
echo "Submodule a ends"

I do agree that it would be powerful enough to serve all the necessary
purpose for traversing git submodules. If my assumption is right in
what we want then I will resubmit the patch with required changes.

-- 
Imran M Yousuf

Re: [PATCH v2 2/5] git-submodule.sh: Add recurse subcommand with basic options

[Junio C Hamano]

imyousuf@gmail.com writes:

> The recurse commands behavior can be customized with several arguments
> that it accepts. The synopsis for the recurse command is:
>
> 	git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
> 	[-d|--depth <recursion depth>] [-b|--breadth-first]
> 	<git command> [<arguments> ...]

Is there a reason to limit the command that can be run per submodule to
only "git" commands?  To me, this "recurse" looks like a glorified "find"
command that can trigger its action only to submodule directories, but
limits what can be given to its -exec option to "git" commands.  While it
would not make sense to give certain git command to recurse (e.g. neither
"git show 65ea3b8" nor "git clone $there" would make any sense), it would
be handy if we can give certain non-git commands to it (e.g. "du -sh").

> @@ -580,6 +585,129 @@ cmd_status()
>  	done
>  }
>  
> +# Check whether the submodule is initialized or not
> +initialize_sub_module()

Everybody else seems to spell "<do-something>_submodule"; should this be
any different?

> +{
> +	if test ! -d "$1"/.git
> +	then
> +		say "Submodule $1 is not initialized and skipped"
> +		return 1
> +	# Returns true if submodule is already initialized

Micronit; s/Returns/Return/.  A sentence that begins with a capitalized
verb in comments is almost always in imperative mood, not third-person
singular present.

> +	elif test -d "$1"/.git
> +	then
> +		return 0
> +	fi
> +}

Otherwise, what does it return?  Do you need elif there, or just "else"?

> +# This function simply checks whether the depth is traverseable in terms of
> +# depth and if so then it sequentially traverses its submodules
> +traverse_submodules()
> +{
> +	# If current depth is the range specified than it will continue
> +	# else return with success
> +	if test "$depth" -gt 0 &&
> +		test "$current_depth" -ge "$depth"
> +	then
> +		return 0;
> +	fi
> +	# If submodules exists than it will traverse over them
> +	if test -f .gitmodules
> +	then
> +		# Incrementing the depth for the next level of submodules
> +		current_depth=$(($current_depth + 1))
> +                for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
> +                        traverse_module "$mod_path" "$@"
> +                done
> +		# Decremented the depth to bring it back to the depth of
> +		# the current submodule
> +		current_depth=$(($current_depth - 1))
> +	fi
> +}

This makes me wonder if you should be iterating over .gitmodules, or
perhaps you may want to iterate over output of git-ls-files (picking
entries of gitlink type).  How should a local change that adds a new
submodule or removes an existing submodule, or moves an existing submodule
interact with "submodule recurse"? 

Also the same micronits (s/Incrementing/Increment/; s/Decremented/Decrement/).

Even if iterating over .gitmodules entries is a good idea, I suspect that
sed script is too fragile.  Doesn't .gitmodules use the same format as git
configuration files, allowing spaces around values, value quoting and
trailing comments on the same line?

> +# This actually traverses a submodule; checks whether the its initialized
> +# or not, does nothing if not initialized.

s/the //;?

> +traverse_module()
> +{
> +	# Will work in the submodule if and only if its initialized
> +	initialize_sub_module "$1" &&

"initialize_sub_module" does not sound like a function that checks if it
is initialized, but more like a function to, eh, initialize the submodule.
Perhaps the function should be renamed to make it clearer that it is a
predicate?

> +	(
> +		submod_path="$1"
> +		shift
> +		cd "$submod_path"
> +		# If depth-first is specified in that case submodules are
> +		# are traversed before executing the command on this submodule
> +		test -n "$depth_first" && traverse_submodules "$@"
> +		# pwd is mentioned in order to enable the ser to distinguish
> +		# between same name modules, e.g. a/lib and b/lib.
> +		say "git submodule recurse $submod_path $*"
> +		git "$@"
> +		# if exit on error is specifed than script will exit if any
> +		# command fails. As there is no transaction there will be
> +		# no rollback either

s/than/then/;?

> +		# TODO - If possible facilitate transaction
> +		if test "$?" -ne 0 && test -n "$on_error"
> +		then
> +			die "FAILED: git submodule $submod_path $*"

Dying before doing further damage to the repository tree may be a good
idea, but I did not see the calling loop in traverse_submodules pay
attention to the exit code from here.

> +		fi
> +		# If depth-first is not specified in that case submodules are
> +		# are traversed after executing the command on this submodule
> +		test -z "$depth_first" && traverse_submodules "$@"
> +	)
> +}
> +
> +# Propagates or recurses over all the submodules at any depth with any
> +# git command, e.g. git-clone, git-status, git-commit etc., with the
> +# arguments supplied exactly as it would have been supplied to the command
> +# otherwise. This actually starts the recursive propagation.

Is "git-clone" a good example to give here?  What would that mean to
recurse into each submodule directories in a superproject to run "clone"?

> +cmd_recurse() {
> +	while :
> +	do
> +		case "$1" in
> +		-q|--quiet)
> +			quiet=1
> +			;;
> +		-d|--depth)
> +			shift
> +			if test -z "$1"
> +			then
> +				echo "No <recursion depth> specified"
> +				usage
> +			# Arithmatic operation will give an error if depth is not number
> +			# thus chose to check intergerness with regular expression.
> +			# $1 is underquoted becuase the expr is in quotation
> +			elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"

Huh?

	$ a='1 2 3'
        $ expr $a : '[1-9]'
	expr: syntax error
	$ expr "$a" : '[1-9]'
        1
        $ z=$(expr $a : '[1-9]')
	expr: syntax error
	$ z=$(expr "$a" : '[1-9]')
        $ echo $z
        1
        $ echo "$(expr $a : '[1-9]')"
	expr: syntax error
        $ echo "$(expr "$a" : '[1-9]')"
        1

If you want to make sure that $(( ... )) would not choke with given "$1",
you can check by attempting to do a simple $(( ... )) to see if it errors
out, which would be simpler.

	if test -z "$1"
        then
        	...
	elif ! echo $(( "$1" + 0 )) >/dev/null
        then
        	die "$1 is not an integer"
	...

Re: [PATCH v2 2/5] git-submodule.sh: Add recurse subcommand with basic options

[Imran M Yousuf]

On Mon, May 12, 2008 at 7:20 AM, Junio C Hamano <junio@pobox.com> wrote:
> imyousuf@gmail.com writes:
>
>  > The recurse commands behavior can be customized with several arguments
>  > that it accepts. The synopsis for the recurse command is:
>  >
>  >       git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
>  >       [-d|--depth <recursion depth>] [-b|--breadth-first]
>  >       <git command> [<arguments> ...]
>
>  Is there a reason to limit the command that can be run per submodule to
>  only "git" commands?  To me, this "recurse" looks like a glorified "find"
>  command that can trigger its action only to submodule directories, but
>  limits what can be given to its -exec option to "git" commands.  While it
>  would not make sense to give certain git command to recurse (e.g. neither
>  "git show 65ea3b8" nor "git clone $there" would make any sense), it would
>  be handy if we can give certain non-git commands to it (e.g. "du -sh").

I do agree how the recurse command looks, but considering that it is a
'git submodule' subcommand I thought having a general command might
have faced a greater criticism from the community. Similarly about not
allowing certain git commands is also in my list for the later version
as it would require a bigger discussion in the community.

>
>
>  > @@ -580,6 +585,129 @@ cmd_status()
>  >       done
>  >  }
>  >
>  > +# Check whether the submodule is initialized or not
>  > +initialize_sub_module()
>
>  Everybody else seems to spell "<do-something>_submodule"; should this be
>  any different?
>
>
>  > +{
>  > +     if test ! -d "$1"/.git
>  > +     then
>  > +             say "Submodule $1 is not initialized and skipped"
>  > +             return 1
>  > +     # Returns true if submodule is already initialized
>
>  Micronit; s/Returns/Return/.  A sentence that begins with a capitalized
>  verb in comments is almost always in imperative mood, not third-person
>  singular present.
>

Got it, thanks for the correction.

>
>  > +     elif test -d "$1"/.git
>  > +     then
>  > +             return 0
>  > +     fi
>  > +}
>
>  Otherwise, what does it return?  Do you need elif there, or just "else"?
>

Yup, else would be sufficient. Sorry for the mistake

>
>  > +# This function simply checks whether the depth is traverseable in terms of
>  > +# depth and if so then it sequentially traverses its submodules
>  > +traverse_submodules()
>  > +{
>  > +     # If current depth is the range specified than it will continue
>  > +     # else return with success
>  > +     if test "$depth" -gt 0 &&
>  > +             test "$current_depth" -ge "$depth"
>  > +     then
>  > +             return 0;
>  > +     fi
>  > +     # If submodules exists than it will traverse over them
>  > +     if test -f .gitmodules
>  > +     then
>  > +             # Incrementing the depth for the next level of submodules
>  > +             current_depth=$(($current_depth + 1))
>  > +                for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
>  > +                        traverse_module "$mod_path" "$@"
>  > +                done
>  > +             # Decremented the depth to bring it back to the depth of
>  > +             # the current submodule
>  > +             current_depth=$(($current_depth - 1))
>  > +     fi
>  > +}
>
>  This makes me wonder if you should be iterating over .gitmodules, or
>  perhaps you may want to iterate over output of git-ls-files (picking
>  entries of gitlink type).  How should a local change that adds a new
>  submodule or removes an existing submodule, or moves an existing submodule
>  interact with "submodule recurse"?

Actually once I am done with the recurse command I was planning to add
submodule mv and rm subcommands :). About the git-ls-files command yes
that is also an option, but in case of move it would require editing
.gitmodules and .git/config. AFAIK user need to currently manually
edit them for updating, hoping to write a shell script to get it done.

About it interacting with these changes, as long as the .gitmodules
file is updated correctly it should not be a problem, but if it
becomes inconsistent then it will chokes. In this regard, I checked
how 'git submodule update' works and it uses git-ls-fiiles --stage
with grep to find the gitlinks path and then search them through
.git/config, but it also faces the same problem if move is done
manually without changing the files. Also to be noted is the status
command also uses git-ls-files.

The reason why I did .gitsubmodule is I want to introduce auto-init
and update as an option, and plan to do it once the basic recurse
patches are accepted :). Then reading the .gitmodules would have been
necessary.

About the sed script another option would be to use -

git config -f ./.gitmodules --get-regexp '^submodule\..*\.path$' |
sed -n -e 's|^submodule\.\(.*\)\.path \(.*\)$|\2|p'

Will using this be preferable? I think so :).

>
>  Also the same micronits (s/Incrementing/Increment/; s/Decremented/Decrement/).
>
>  Even if iterating over .gitmodules entries is a good idea, I suspect that
>  sed script is too fragile.  Doesn't .gitmodules use the same format as git
>  configuration files, allowing spaces around values, value quoting and
>  trailing comments on the same line?

I agree on the fragile point and I think I will replace it with the
one I mentioned above.

>
>
>  > +# This actually traverses a submodule; checks whether the its initialized
>  > +# or not, does nothing if not initialized.
>
>  s/the //;?
>
>
>  > +traverse_module()
>  > +{
>  > +     # Will work in the submodule if and only if its initialized
>  > +     initialize_sub_module "$1" &&
>
>  "initialize_sub_module" does not sound like a function that checks if it
>  is initialized, but more like a function to, eh, initialize the submodule.
>  Perhaps the function should be renamed to make it clearer that it is a
>  predicate?

I thought of renaming it but I was a bit lazy as I am writing another
patch for auto initialize :).

>
>
>  > +     (
>  > +             submod_path="$1"
>  > +             shift
>  > +             cd "$submod_path"
>  > +             # If depth-first is specified in that case submodules are
>  > +             # are traversed before executing the command on this submodule
>  > +             test -n "$depth_first" && traverse_submodules "$@"
>  > +             # pwd is mentioned in order to enable the ser to distinguish
>  > +             # between same name modules, e.g. a/lib and b/lib.
>  > +             say "git submodule recurse $submod_path $*"
>  > +             git "$@"
>  > +             # if exit on error is specifed than script will exit if any
>  > +             # command fails. As there is no transaction there will be
>  > +             # no rollback either
>
>  s/than/then/;?
>
>
>  > +             # TODO - If possible facilitate transaction
>  > +             if test "$?" -ne 0 && test -n "$on_error"
>  > +             then
>  > +                     die "FAILED: git submodule $submod_path $*"
>
>  Dying before doing further damage to the repository tree may be a good
>  idea, but I did not see the calling loop in traverse_submodules pay
>  attention to the exit code from here.

Thanks for pointing out this bug, will fix it in the next version.

>
>
>  > +             fi
>  > +             # If depth-first is not specified in that case submodules are
>  > +             # are traversed after executing the command on this submodule
>  > +             test -z "$depth_first" && traverse_submodules "$@"
>  > +     )
>  > +}
>  > +
>  > +# Propagates or recurses over all the submodules at any depth with any
>  > +# git command, e.g. git-clone, git-status, git-commit etc., with the
>  > +# arguments supplied exactly as it would have been supplied to the command
>  > +# otherwise. This actually starts the recursive propagation.
>
>  Is "git-clone" a good example to give here?  What would that mean to
>  recurse into each submodule directories in a superproject to run "clone"?
>

I agree that git-clone is infact a bad example, will remove it :).

>
>  > +cmd_recurse() {
>  > +     while :
>  > +     do
>  > +             case "$1" in
>  > +             -q|--quiet)
>  > +                     quiet=1
>  > +                     ;;
>  > +             -d|--depth)
>  > +                     shift
>  > +                     if test -z "$1"
>  > +                     then
>  > +                             echo "No <recursion depth> specified"
>  > +                             usage
>  > +                     # Arithmatic operation will give an error if depth is not number
>  > +                     # thus chose to check intergerness with regular expression.
>  > +                     # $1 is underquoted becuase the expr is in quotation
>  > +                     elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"
>
>  Huh?
>
>         $ a='1 2 3'
>         $ expr $a : '[1-9]'
>         expr: syntax error
>         $ expr "$a" : '[1-9]'
>         1
>         $ z=$(expr $a : '[1-9]')
>         expr: syntax error
>         $ z=$(expr "$a" : '[1-9]')
>         $ echo $z
>         1
>         $ echo "$(expr $a : '[1-9]')"
>         expr: syntax error
>         $ echo "$(expr "$a" : '[1-9]')"
>         1
>
>  If you want to make sure that $(( ... )) would not choke with given "$1",
>  you can check by attempting to do a simple $(( ... )) to see if it errors
>  out, which would be simpler.
>
>
>         if test -z "$1"
>         then
>                 ...
>         elif ! echo $(( "$1" + 0 )) >/dev/null
>         then
>                 die "$1 is not an integer"
>         ...

This was what I was looking for a simpler and cleaner way :), thanks a
lot Junio.

BTW: its nice to see your emails once again :).

Best regards,

Imran

>
>



-- 
Imran M Yousuf
Email: imran@smartitengineering.com
Mobile: +880-1711402557