[PATCH] git-submodule - Add 'foreach' subcommand

[PATCH] git-submodule - Add 'foreach' subcommand

[Mark Levedahl]

submodule foreach <command-list> will execute the list of commands in
each currently checked out submodule directory. The list of commands
is arbitrary as long as it is acceptable to sh. The variables '$path'
and '$sha1' are availble to the command-list, defining the submodule
path relative to the superproject and the submodules's commitID as
recorded in the superproject (this may be different than HEAD in the
submodule).

This utility is inspired by a number of threads on the mailing list
looking for ways to better integrate submodules in a tree and work
with them as a unit. This could include fetching a new branch in each
from a given source, or possibly checking out a given named branch in
each. Currently, there is no consensus as to what additional commands
should be implemented in the porcelain, requiring all users whose needs
exceed that of git-submodule to do their own scripting. The foreach
command is intended to support such scripting, and in particular does
no error checking and produces no output, thus allowing end users
complete control over any information printed out and over what
constitutes an error.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   12 ++++++++++++
 git-submodule.sh                |   22 ++++++++++++++++++++--
 2 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index bf33b0c..5b52f35 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
+'git submodule' foreach <command-list>
 
 
 DESCRIPTION
@@ -123,6 +124,17 @@ summary::
 	in the submodule between the given super project commit and the
 	index or working tree (switched by --cached) are shown.
 
+foreach::
+	Executes an arbitrary list of commands in each checked out submodule.
+	$path is the name of the submodule directory relative to the
+	superproject, and $sha1 is the commit as recorded in the superproject.
+	And submodules defined in the superproject but not checked out are
+	ignored by this command, and an empty command-list provides no output.
++
+As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
+show the path and currently checked out commit for each submodule.
+
+
 OPTIONS
 -------
 -q::
diff --git a/git-submodule.sh b/git-submodule.sh
index b40f876..abbc7bb 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]"
+[--] [<path>...]|[foreach <command-list>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -199,6 +199,24 @@ cmd_add()
 }
 
 #
+# Execute an arbitrary command sequence in each checked out
+# submodule
+#
+# $@ = command to execute
+#
+cmd_foreach()
+{
+	git ls-files --stage | grep '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if test -e "$path"/.git
+		then
+			(cd "$path" && eval "$@")
+		fi
+	done
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -583,7 +601,7 @@ cmd_status()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | init | update | status | summary)
+	add | foreach | init | update | status | summary)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.6.0.rc2.44.g974eb

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Johan Herland]

On Sunday 10 August 2008, Mark Levedahl wrote:
> submodule foreach <command-list> will execute the list of commands in
> each currently checked out submodule directory. The list of commands
> is arbitrary as long as it is acceptable to sh. The variables '$path'
> and '$sha1' are availble to the command-list, defining the submodule
> path relative to the superproject and the submodules's commitID as
> recorded in the superproject (this may be different than HEAD in the
> submodule).
>
> This utility is inspired by a number of threads on the mailing list
> looking for ways to better integrate submodules in a tree and work
> with them as a unit. This could include fetching a new branch in each
> from a given source, or possibly checking out a given named branch in
> each. Currently, there is no consensus as to what additional commands
> should be implemented in the porcelain, requiring all users whose needs
> exceed that of git-submodule to do their own scripting. The foreach
> command is intended to support such scripting, and in particular does
> no error checking and produces no output, thus allowing end users
> complete control over any information printed out and over what
> constitutes an error.
>
> Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>

Hi,

I was about to send an equivalent patch, but you beat me to it. Yours is 
shorter and more elegant as well, so thanks for saving me the 
embarrasment ;)

So, FWIW:

Liked-by: Johan Herland <johan@herland.net>

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Junio C Hamano]

Mark Levedahl <mlevedahl@gmail.com> writes:

> This utility is inspired by a number of threads on the mailing list
> looking for ways to better integrate submodules in a tree and work
> with them as a unit.

Thanks for keeping the topic going

> +foreach::
> +	Executes an arbitrary list of commands in each checked out submodule.
> +	$path is the name of the submodule directory relative to the
> +	superproject, and $sha1 is the commit as recorded in the superproject.
> +	And submodules defined in the superproject but not checked out are
> +	ignored by this command, and an empty command-list provides no output.
> ++
> +As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
> +show the path and currently checked out commit for each submodule.

I think this is a sensible building block to have.

> +cmd_foreach()
> +{
> +	git ls-files --stage | grep '^160000 ' |
> +	while read mode sha1 stage path
> +	do
> +		if test -e "$path"/.git
> +		then
> +			(cd "$path" && eval "$@")
> +		fi
> +	done
> +}

I suspect we may want to stop the loop upon failure from eval (any user
that wants to run command in all submodules no matter what could add '||:'
to the cmd list), but other than that this is a simple enough and good
addition.

[PATCH] git-submodule - Add 'foreach' subcommand

[Mark Levedahl]

submodule foreach <command-list> will execute the list of commands in
each currently checked out submodule directory. The list of commands
is arbitrary as long as it is acceptable to sh. The variables '$path'
and '$sha1' are availble to the command-list, defining the submodule
path relative to the superproject and the submodules's commitID as
recorded in the superproject (this may be different than HEAD in the
submodule).

This utility is inspired by a number of threads on the mailing list
looking for ways to better integrate submodules in a tree and work
with them as a unit. This could include fetching a new branch in each
from a given source, or possibly checking out a given named branch in
each. Currently, there is no consensus as to what additional commands
should be implemented in the porcelain, requiring all users whose needs
exceed that of git-submodule to do their own scripting. The foreach
command is intended to support such scripting, and in particular does
no error checking and produces no output, thus allowing end users
complete control over any information printed out and over what
constitutes an error. The processing does terminate if the command-list
returns an error, but processing can easily be forced for all
submodules be terminating the list with ';true'.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   15 +++++++++++++++
 git-submodule.sh                |   23 +++++++++++++++++++++--
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index bf33b0c..1e7d352 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
+'git submodule' foreach <command-list>
 
 
 DESCRIPTION
@@ -123,6 +124,20 @@ summary::
 	in the submodule between the given super project commit and the
 	index or working tree (switched by --cached) are shown.
 
+foreach::
+	Executes an arbitrary list of commands in each checked out submodule.
+	$path is the name of the submodule directory relative to the
+	superproject, and $sha1 is the commit as recorded in the superproject.
+	Any submodules defined in the superproject but not checked out are
+	ignored by this command, and an empty command-list provides no output.
+	A non-zero return from the command-list in any submodule causes
+	the processing to terminate. This can be overridden by adding '; true'
+	to the end of the command list.
++
+As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
+show the path and currently checked out commit for each submodule.
+
+
 OPTIONS
 -------
 -q::
diff --git a/git-submodule.sh b/git-submodule.sh
index b40f876..39a19f0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]"
+[--] [<path>...]|[foreach <command-list>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -199,6 +199,25 @@ cmd_add()
 }
 
 #
+# Execute an arbitrary command sequence in each checked out
+# submodule
+#
+# $@ = command to execute
+#
+cmd_foreach()
+{
+	git ls-files --stage | grep '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if test -e "$path"/.git
+		then
+			(cd "$path" && eval "$@") ||
+			die "Error detected evaluating commands in '$path'"
+		fi
+	done
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -583,7 +602,7 @@ cmd_status()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | init | update | status | summary)
+	add | foreach | init | update | status | summary)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.6.0.rc2.44.g974eb

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Balazs Nagy]

Hi,

Sorry for being a bit late, but what if we would have --all for all
submodule commands instead of having another command?

Rgds,
-- 
-jul-

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Johannes Schindelin]

Hi,

On Mon, 11 Aug 2008, Balazs Nagy wrote:

> Sorry for being a bit late, but what if we would have --all for all 
> submodule commands instead of having another command?

and the foreach patch would go into which existing command, exactly?  "git 
--all"?  A new "git foreach --all"?

Just kidding,
Dscho

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Matthias Kestenholz]

On Mon, 2008-08-11 at 08:28 +0200, Balazs Nagy wrote:
> Hi,
> 
> Sorry for being a bit late, but what if we would have --all for all
> submodule commands instead of having another command?
> 
> Rgds,

This would not be very smart. The example in the documentation part of
the patch would be impossible only with --all switches (or, you'd need
to duplicate all git commands in git-submodule instead of reusing them
with git submodule foreach)

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Petr Baudis]

  Hi,

On Sun, Aug 10, 2008 at 07:10:04PM -0400, Mark Levedahl wrote:
> submodule foreach <command-list> will execute the list of commands in
> each currently checked out submodule directory. The list of commands
> is arbitrary as long as it is acceptable to sh. The variables '$path'
> and '$sha1' are availble to the command-list, defining the submodule
> path relative to the superproject and the submodules's commitID as
> recorded in the superproject (this may be different than HEAD in the
> submodule).

  in principle, this looks pretty sensible.

> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index bf33b0c..1e7d352 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -14,6 +14,7 @@ SYNOPSIS
>  'git submodule' [--quiet] init [--] [<path>...]
>  'git submodule' [--quiet] update [--init] [--] [<path>...]
>  'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
> +'git submodule' foreach <command-list>
>  
>  
>  DESCRIPTION

  But as visible here, this is a little bit inconsistent. I think
foreach should be by default verbose about which submodules does it
recurse into - this is what you want in case of casual usage on
command-line. In case you want to have full control on the output within
a script, you can always pass the extra --quiet and it's less obnoxious
this way around.

  I also have a problem with the <command-list> - is this one argument?
Multiple arguments? The semantics is ill-defined. If it is supposed to
be a single argument, please drop the -list bit; silent DWIMmery of
using "$@" internally is acceptable, I guess. If it is supposed to be
multiple arguments, you need to

	(i) Specify that as <command>... instead

	(ii) Either have an eternal annoyance about insane behaviour
here, or use something better than eval "$@". Since

	git submodule foreach cp x\ y z

will simply _not_ work properly.

  So I think it's best to just drop the 'list' part. You're just
evaluating a shell expression passed in a parameter.

> @@ -123,6 +124,20 @@ summary::
>  	in the submodule between the given super project commit and the
>  	index or working tree (switched by --cached) are shown.
>  
> +foreach::
> +	Executes an arbitrary list of commands in each checked out submodule.

I think "evaluates" is a better word here, too.

-- 
				Petr "Pasky" Baudis
The next generation of interesting software will be done
on the Macintosh, not the IBM PC.  -- Bill Gates

Re: [PATCH] git-submodule - Add 'foreach' subcommand

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

>   in principle, this looks pretty sensible.
> ...
> I think "evaluates" is a better word here, too.

I think your refinements all make sense.  Thanks for commenting.

[PATCH] git-submodule foreach - Include output and clarify usage.

[Mark Levedahl]

Petr Baudis provided good feedback on the list, leading to this
patch to modify foreach.

This clarifies that the input to "foreach" is a single shell command,
though internally foreach evaluates "$@" so *may* work even if the
user fails to enclose the command in quotes. Also, this adds output
of each submodule entered unless --quiet is given, providing feedback
at the command line but still allowing a script to precisely control
the output given.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   12 +++++++-----
 git-submodule.sh                |    3 ++-
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 3470795..abbd5b7 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,7 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
-'git submodule' foreach <command-list>
+'git submodule' [--quiet] foreach <command>
 
 
 DESCRIPTION
@@ -125,14 +125,16 @@ summary::
 	index or working tree (switched by --cached) are shown.
 
 foreach::
-	Executes an arbitrary list of commands in each checked out submodule.
+	Evaluates an arbitrary shell command in each checked out submodule.
+	The command has access to the variables $path and $sha1:
 	$path is the name of the submodule directory relative to the
 	superproject, and $sha1 is the commit as recorded in the superproject.
 	Any submodules defined in the superproject but not checked out are
-	ignored by this command, and an empty command-list provides no output.
-	A non-zero return from the command-list in any submodule causes
+	ignored by this command. Unless given --quiet, foreach prints the name
+	of each submodule before evaluating the command.
+	A non-zero return from the command in any submodule causes
 	the processing to terminate. This can be overridden by adding '|| :'
-	to the end of the command list.
+	to the end of the command.
 +
 As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
 show the path and currently checked out commit for each submodule.
diff --git a/git-submodule.sh b/git-submodule.sh
index fc41e59..2d57d60 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]|[foreach <command-list>]"
+[--] [<path>...]|[foreach <command>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -211,6 +211,7 @@ cmd_foreach()
 	do
 		if test -e "$path"/.git
 		then
+			say "Entering '$path'"
 			(cd "$path" && eval "$@") ||
 			die "Stopping at '$path'; script returned non-zero status."
 		fi
-- 
1.6.0.rc2.45.g1363