[PATCH] Let .git/config specify the url for submodules

[PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

This changes git-submodule in a few ways:
a. Section names in .gitmodules are now called 'path', not 'module'.
b. Each 'path' section is required to have a 'submodule' key, used to
   give the submodule a name.
c. During 'git-submodule init', the submodule name and url is found in
   .gitmodules and stored in .git/config under the key 'submodule.$name.url'.
   The 'init' command does not clone the submodule.
d. A new git-submodule command, 'clone', will map submodule path to submodule
   name in .gitmodules and use the name to find a url for the submodule in
   .git/config. This url is then used to clone the submodule before checking
   out the commit named in the index of the containing repository.

The main reason for these changes is to make it easier to specify alternate
urls for a submodule without touching .gitmodules, i.e. like this:

  $ git submodule init
  $ git config submodule.foobar.url /pub/git/foobar.git
  $ git submodule clone

It also might turn out to be nice to have submodules identified by a 'logical
name' instead of by path when a single path is used for different submodules
in different versions of a 'superproject', and if/when the submodules gets
cloned somewhere below .git of the containing repository.

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

This is on top of my previous 'submodule test-script' patch (which was on top
of 'next', sorry for not mentioning this in the mail).


 Documentation/git-submodule.txt |   36 ++++++++-----
 git-submodule.sh                |  109 ++++++++++++++++++++++++++++-----------
 t/t7400-submodule-basic.sh      |   69 ++++++++++++++++--------
 3 files changed, 148 insertions(+), 66 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index cb0424f..c93e40f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,7 +8,7 @@ git-submodule - Initialize, update or inspect submodules
 
 SYNOPSIS
 --------
-'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
+'git-submodule' [--quiet] [--cached] [status|init|clone|update] [--] [<path>...]
 
 
 COMMANDS
@@ -16,22 +16,25 @@ COMMANDS
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
-	submodule path and the output of gitlink:git-describe[1] for the
-	SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is not
-	initialized and `+` if the currently checked out submodule commit
+	submodule name, path and the output of gitlink:git-describe[1] for
+	the SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is
+	not initialized and `+` if the currently checked out submodule commit
 	does not match the SHA-1 found in the index of the containing
 	repository. This command is the default command for git-submodule.
 
 init::
-	Initialize the submodules, i.e. clone the git repositories specified
-	in the .gitmodules file and checkout the submodule commits specified
-	in the index of the containing repository. This will make the
-	submodules HEAD be detached.
+	Initialize the submodules, i.e. register the submodules and their
+	suggested urls in $GIT_DIR/config of the containing repository.
+
+clone::
+	Clone the initialized git repositories and checkout the submodule
+	commits specified in the index of the containing repository. This
+	will make the submodule HEADs be detached.
 
 update::
 	Update the initialized submodules, i.e. checkout the submodule commits
 	specified in the index of the containing repository. This will make
-	the submodules HEAD be detached.
+	the submodule HEADs be detached.
 
 
 OPTIONS
@@ -48,18 +51,25 @@ OPTIONS
 	Path to submodule(s). When specified this will restrict the command
 	to only operate on the submodules found at the specified paths.
 
+
 FILES
 -----
-When cloning submodules, a .gitmodules file in the top-level directory
-of the containing repository is used to find the url of each submodule.
-This file should be formatted in the same way as $GIR_DIR/config. The key
-to each submodule url is "module.$path.url".
+git-submodule uses a file named .gitmodules in the top-level directory of the
+containing repository. This file should be formatted in the same way as
+$GIR_DIR/config.
+
+The .gitmodules file is used to find the name and suggested url for each
+submodule path, using the keys `path.$path.submodule` and 'path.$path.url'.
+The submodule name and url is stored in $GIT_DIR/config by
+`git-submodule init`, using the key `submodule.$name.url`, and this key is
+then used during `git-submodule clone` to lookup the preferred submodule url.
 
 
 AUTHOR
 ------
 Written by Lars Hjemli <hjemli@gmail.com>
 
+
 GIT
 ---
 Part of the gitlink:git[7] suite
diff --git a/git-submodule.sh b/git-submodule.sh
index 6ed5a6c..31de80e 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -4,11 +4,12 @@
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [status|init|clone|update] [--] [<path>...]'
 . git-sh-setup
 require_work_tree
 
 init=
+clone=
 update=
 status=
 quiet=
@@ -26,7 +27,7 @@ say()
 }
 
 #
-# Run clone + checkout on missing submodules
+# Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
 #
@@ -35,9 +36,56 @@ modules_init()
 	git ls-files --stage -- "$@" | grep -e '^160000 ' |
 	while read mode sha1 stage path
 	do
+		# Get the submodule name for this path
+		name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+		test -z "$name" && die "Unnamed submodule at path '$path'"
+
+		# Skip registered submodules
+		url=$(git-config submodule."$name".url)
+		test -z "$url" || continue
+
+		# find suggested url in .gitmodules
+		url=$(GIT_CONFIG=.gitmodules git-config path."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$name' (path '$path') in .gitmodules"
+
+		# Save the url in local config, indicating that this
+		# submodule is now registered (and enabling the user to
+		# override the url before running 'git submodule clone'
+		git-config submodule."$name".url "$url" ||
+		die "Failed to register url '$url' for submodule '$name'"
+
+		say "Submodule '$name' registered for path '$path', using url '$url'"
+	done
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_clone()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		# Get the submodule name for this path
+		name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+		test -z "$name" && die "Unnamed submodule at path '$path'"
+
+		# Get the url for the module from local config
+		url=$(git-config submodule."$name".url)
+		if test -z "$url"
+		then
+			# If this path was specified on the command line,
+			# make sure the user gets a proper errormessage.
+			test "$#" != "$0" &&
+			die "Submodule '$name' (path '$path') not initialized"
+
+			continue
+		fi
+
 		# Skip submodule paths that already contain a .git directory.
-		# This will also trigger if $path is a symlink to a git
-		# repository
 		test -d "$path"/.git && continue
 
 		# If there already is a directory at the submodule path,
@@ -54,28 +102,17 @@ modules_init()
 		test -e "$path" &&
 		die "A file already exist at path '$path'"
 
-		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
-		test -z "$url" &&
-		die "No url found for submodule '$path' in .gitmodules"
-
-		# MAYBE FIXME: this would be the place to check GIT_CONFIG
-		# for a preferred url for this submodule, possibly like this:
-		#
-		# modname=$(GIT_CONFIG=.gitmodules git-config module."$path".name)
-		# alturl=$(git-config module."$modname".url)
-		#
-		# This would let the versioned .gitmodules file use the submodule
-		# path as key, while the unversioned GIT_CONFIG would use the
-		# logical modulename (if present) as key. But this would need
-		# another fallback mechanism if the module wasn't named.
+		# Use the submodule name to find the preferred url in local config
+		url=$(git-config submodule."$name".url)
+		test -z "$url" && die "No url found for submodule '$name' (path '$path')"
 
 		git-clone -n "$url" "$path" ||
-		die "Clone of submodule '$path' failed"
+		die "Clone of submodule '$name' (path '$path') failed"
 
 		(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
-		die "Checkout of submodule '$path' failed"
+		die "Checkout of '$sha1' in submodule '$name' (path '$path') failed"
 
-		say "Submodule '$path' initialized"
+		say "Submodule '$name' (path '$path') cloned and checked out"
 	done
 }
 
@@ -94,27 +131,33 @@ modules_update()
 			# Only mention uninitialized submodules when its
 			# path have been specified
 			test "$#" != "0" &&
-			say "Submodule '$path' not initialized"
+			say "Submodule in directory '$path' not initialized"
 			continue;
 		fi
+
+		# Get the module name for this path
+		name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+		test -z "$name" && die "Unnamed submodule at path '$path'"
+
+		# Get the SHA-1 of the submodule HEAD
 		subsha1=$(unset GIT_DIR && cd "$path" &&
 			git-rev-parse --verify HEAD) ||
-		die "Unable to find current revision of submodule '$path'"
+		die "Unable to find current revision of submodule '$module' (path '$path')"
 
 		if test "$subsha1" != "$sha1"
 		then
 			(unset GIT_DIR && cd "$path" && git-fetch &&
 				git-checkout -q "$sha1") ||
-			die "Unable to checkout '$sha1' in submodule '$path'"
+			die "Unable to checkout '$sha1' for submodule '$name' (path '$path')"
 
-			say "Submodule '$path': checked out '$sha1'"
+			say "Submodule '$name' (path '$path'): checked out '$sha1'"
 		fi
 	done
 }
 
 #
 # List all registered submodules, prefixed with:
-#  - submodule not initialized
+#  - submodule not initialized or cloned
 #  + different revision checked out
 #
 # If --cached was specified the revision in the index will be printed
@@ -153,6 +196,9 @@ do
 	init)
 		init=1
 		;;
+	clone)
+		clone=1
+		;;
 	update)
 		update=1
 		;;
@@ -178,14 +224,17 @@ do
 	shift
 done
 
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$init,$clone,$update,$status,$cached" in
+1,,,,)
 	modules_init "$@"
 	;;
-,1,,)
+,1,,,)
+	modules_clone "$@"
+	;;
+,,1,,)
 	modules_update "$@"
 	;;
-,,*,*)
+,,,*,*)
 	modules_list "$@"
 	;;
 *)
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index a023c87..234f940 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -5,8 +5,8 @@
 
 test_description='Basic submodule support in porcelain
 
-This test tries to run the init, update and status commands of git-submodule
-with a known good setup (and two known bad)
+This test tries to verify basic sanity of the init, clone, update and
+status commands of git-submodule.
 '
 
 . ./test-lib.sh
@@ -24,32 +24,57 @@ cd ..
 echo a >a && echo z >z
 git-add a lib z && git-commit -q -m "super commit 1"
 
-# move submodule to another location, register repo url in .gitmodules
+# move submodule to another location
 mv lib .subrepo
-GIT_CONFIG=.gitmodules git-config module.lib.url ./.subrepo
+
+# register a broken url in .gitmodules
+GIT_CONFIG=.gitmodules git-config path.lib.url ./broken
 
 test_expect_success 'status is "missing"' \
 	'git-submodule status | grep "^-$rev1"'
 
-# make sure 'init' will not overwrite a regular file
-touch lib
-test_expect_failure 'init fails when path is used by a file' \
+# 'init' should fail, since the submodule is unnamed in .gitmodules
+test_expect_failure 'do not initialize unnamed submodules' \
 	'git-submodule init'
 
-# make sure 'init' will not overwrite a nonempty directory
+# give the submodule a logical name
+GIT_CONFIG=.gitmodules git-config path.lib.submodule thelib
+test_expect_success 'init should register the submodule in .git/config' '
+	git-submodule init &&
+	url=$(git-config submodule.thelib.url) &&
+	test "$url" = "./broken"
+'
+
+# specify correct url in local config
+git-config submodule.thelib.url ./.subrepo
+
+# make sure 'clone' will not overwrite a regular file
+touch lib
+test_expect_failure 'clone fails when path is used by a file' \
+	'git-submodule clone'
+
+test_expect_success 'the annoying file is preserved' \
+	'test -f lib'
+
+# make sure 'clone' will not overwrite a nonempty directory
 rm lib
 mkdir -p lib/foo
-test_expect_failure 'init fails when path is used by a nonempty directory' \
-	'git-submodule init'
+test_expect_failure 'clone fails when path is used by a nonempty directory' \
+	'git-submodule clone'
 
-# turn lib into an empty directory, just like git-checkout would do
+test_expect_success 'the annoying directory is preserved' \
+	'test -d lib/foo'
+
+# make lib be an empty directory, just like git-checkout would have done
 rmdir lib/foo
-test_expect_success 'init works when path is an empty dir' \
-	'git-submodule init && test -d lib/.git && git-diff --exit-code'
 
-head=$(cd lib && git-rev-parse HEAD)
-test_expect_success 'submodule HEAD should match rev1' \
-	'test "$head" = "$rev1"'
+test_expect_success 'clone into empty directory should work' \
+	'git-submodule clone'
+
+test_expect_success 'submodule HEAD should match rev1' '
+	head=$(cd lib && git-rev-parse HEAD) &&
+	test "$head" = "$rev1"
+'
 
 test_expect_success 'status is "up-to-date" after init' \
 	'git-submodule status | grep "^ $rev1"'
@@ -66,15 +91,13 @@ test_expect_success 'status is "modified" after submodule commit' \
 test_expect_success 'the --cached sha1 should be rev1' \
 	'git-submodule --cached status | grep "^\+$rev1"'
 
-test_expect_failure 'git-diff --exit-code reports local modifications' \
-	'git-diff --exit-code'
-
-test_expect_success 'update should checkout the correct commit' \
+test_expect_success 'after update, there should be no local modifications' \
 	'git-submodule update && git-diff --exit-code'
 
-head=$(cd lib && git-rev-parse HEAD)
-test_expect_success 'submodule HEAD should match rev1' \
-	'test "$head" = "$rev1"'
+test_expect_success 'submodule HEAD should match rev1' '
+	head=$(cd lib && git-rev-parse HEAD) &&
+	test "$head" = "$rev1"
+'
 
 test_expect_success 'status is "up-to-date" after update' \
 	'git-submodule status | grep "^ $rev1"'
-- 
1.5.2.74.g6b2d

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 5/28/07, Lars Hjemli <hjemli@gmail.com> wrote:
> This changes git-submodule in a few ways:

Please don't apply the "Let .git/config specify the url for
submodules" patch, I'm having second thoughts ;-)

Your design outline in
http://article.gmane.org/gmane.comp.version-control.git/48287 is
obviously superior, and I'd like to take a stab at it with something
like this:

1. 'git-submodule init' saves submodule name and suggested url from
.gitmodules into .git/config (submodule.$name.url)

2. 'git-submodule update' keeps the work tree updated for submodules
with five separate (and optional) operations:
  a) git-clone --bare $url .git/submodules/$name.git
  b) git-clone -l -s .git/submodules/$name.git $path
  c) cd .git/submodules/$name.git && git-fetch
  d) cd $path && git-fetch
  e) cd $path && git-checkout $sha1

3) 'git-submodule push' runs something like 'cd $path && git push
origin $branch', where $branch is found in .gitmodules
(path.$path.branch).

A remaining issue is how to detect if step 2b is necessary if a
submodule is already checked out at the submodule path, but I guess
remote.origin.url in the checked out submodule would be the thing to
peek into.

Also, step 2c/2d should obviously only be performed if the requested
sha1 isn't available, which should be trivial to detect with
'git-cat-file -e'.

Could this turn out to be an acceptable solution?

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> On 5/28/07, Lars Hjemli <hjemli@gmail.com> wrote:
>> This changes git-submodule in a few ways:
>
> Please don't apply the "Let .git/config specify the url for
> submodules" patch, I'm having second thoughts ;-)
>
> Your design outline in
> http://article.gmane.org/gmane.comp.version-control.git/48287 is
> obviously superior, and I'd like to take a stab at it with something
> like this:
>
> 1. 'git-submodule init' saves submodule name and suggested url from
> .gitmodules into .git/config (submodule.$name.url)
>
> 2. 'git-submodule update' keeps the work tree updated for submodules
> with five separate (and optional) operations:
>  a) git-clone --bare $url .git/submodules/$name.git
>  b) git-clone -l -s .git/submodules/$name.git $path
>  c) cd .git/submodules/$name.git && git-fetch
>  d) cd $path && git-fetch
>  e) cd $path && git-checkout $sha1
>
> 3) 'git-submodule push' runs something like 'cd $path && git push
> origin $branch', where $branch is found in .gitmodules
> (path.$path.branch).
>
> A remaining issue is how to detect if step 2b is necessary if a
> submodule is already checked out at the submodule path, but I guess
> remote.origin.url in the checked out submodule would be the thing to
> peek into.
>
> Also, step 2c/2d should obviously only be performed if the requested
> sha1 isn't available, which should be trivial to detect with
> 'git-cat-file -e'.
>
> Could this turn out to be an acceptable solution?

Sounds sane to me, although I have to admit that I cannot really
claim that I have thought things through in that strawman you
quoted.

Re: [PATCH] Let .git/config specify the url for submodules

[Josef Weidendorfer]

On Thursday 31 May 2007, Lars Hjemli wrote:
> On 5/28/07, Lars Hjemli <hjemli@gmail.com> wrote:
> > This changes git-submodule in a few ways:
> 
> Please don't apply the "Let .git/config specify the url for
> submodules" patch, I'm having second thoughts ;-)
> 
> Your design outline in
> http://article.gmane.org/gmane.comp.version-control.git/48287 is
> obviously superior, and I'd like to take a stab at it with something
> like this:
> 
> 1. 'git-submodule init' saves submodule name and suggested url from
> .gitmodules into .git/config (submodule.$name.url)
> 
> 2. 'git-submodule update' keeps the work tree updated for submodules
> with five separate (and optional) operations:
>   a) git-clone --bare $url .git/submodules/$name.git
>   b) git-clone -l -s .git/submodules/$name.git $path
>   c) cd .git/submodules/$name.git && git-fetch
>   d) cd $path && git-fetch
>   e) cd $path && git-checkout $sha1
> 
> 3) 'git-submodule push' runs something like 'cd $path && git push
> origin $branch', where $branch is found in .gitmodules
> (path.$path.branch).

So if you need superproject related corrections in the submodule,
you always have to do it on branch "path.$path.branch" in the
submodule to get it saved?

I would assume that pushing the current branch should be enough.
If you want to play with multiple different "corrections" on
different branches in the submodule, you do not want to force
the branch name to a unique one given in .gitmodules.

Josef

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Thu, May 31, 2007 at 02:17:30AM +0200, Lars Hjemli wrote:
> 1. 'git-submodule init' saves submodule name and suggested url from
> .gitmodules into .git/config (submodule.$name.url)

Could you please document the proposed .gitmodules first?
Since it looks like I'm going to be forced to my submodule URLs there,
I need to know how to do it before I can start using submodules properly.

skimo

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/1/07, Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> > On 5/28/07, Lars Hjemli <hjemli@gmail.com> wrote:
> > 3) 'git-submodule push' runs something like 'cd $path && git push
> > origin $branch', where $branch is found in .gitmodules
> > (path.$path.branch).
>
> So if you need superproject related corrections in the submodule,
> you always have to do it on branch "path.$path.branch" in the
> submodule to get it saved?
>
> I would assume that pushing the current branch should be enough.
> If you want to play with multiple different "corrections" on
> different branches in the submodule, you do not want to force
> the branch name to a unique one given in .gitmodules.
>

The current (and planned) implementation of git-submodule detaches
HEAD in the submodules, so there will not be a current branch unless
the user has done 'cd $path && git-checkout somebranch'.

We might take advantage of that fact:
  * try to exec 'git-symbolic-ref HEAD'
  * if it fails, push to path.$path.branch
  * otherwise, push to the ref pointed to by HEAD

But this will still loose any changes on _other_ branches (that has
not been pushed to origin manually). I'm not sure if/how we can avoid
this, except by making $path be a symlink to .git/submodules/$name.git
(which has other issues...)

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Thu, May 31, 2007 at 02:17:30AM +0200, Lars Hjemli wrote:
> > 1. 'git-submodule init' saves submodule name and suggested url from
> > .gitmodules into .git/config (submodule.$name.url)
>
> Could you please document the proposed .gitmodules first?

The man-page for git-submodule found in 'next' mentions how it uses
.gitmodules, but there isn't (yet) a separate man-page for
.gitmodules(5).

But: when I implement the proposed plan, it will change the current
layout of .gitmodules from

[module '$path']
  url=/some/url

to

[path '$path']
  submodule=modulename
  url=/some/url

Is it ok if gitmodules(5) appears as part of the planned patch-series?

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Fri, Jun 01, 2007 at 11:25:42AM +0200, Lars Hjemli wrote:
> On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> >Could you please document the proposed .gitmodules first?
> 
> The man-page for git-submodule found in 'next' mentions how it uses
> .gitmodules, but there isn't (yet) a separate man-page for
> .gitmodules(5).

That's what I'd like to see first.
.gitmodules(5) will hopefully not change in an incompatible
way after it is accepted, while there is no such constraint
for git-submodule.

> [path '$path']
>  submodule=modulename
>  url=/some/url

Wouldn't it make more sense to have

[path '$path']
	submodule=modulename

and

[submodule '$modulename']
	url=/some/url

in case the same module appears in more than one path?

> Is it ok if gitmodules(5) appears as part of the planned patch-series?

For me it is.

skimo

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Fri, Jun 01, 2007 at 11:25:42AM +0200, Lars Hjemli wrote:
> > On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> > >Could you please document the proposed .gitmodules first?
> >
> > The man-page for git-submodule found in 'next' mentions how it uses
> > .gitmodules, but there isn't (yet) a separate man-page for
> > .gitmodules(5).
>
> That's what I'd like to see first.
> .gitmodules(5) will hopefully not change in an incompatible
> way after it is accepted, while there is no such constraint
> for git-submodule.

True

>
> > [path '$path']
> >  submodule=modulename
> >  url=/some/url
>
> Wouldn't it make more sense to have
>
> [path '$path']
>         submodule=modulename
>
> and
>
> [submodule '$modulename']
>         url=/some/url
>
> in case the same module appears in more than one path?

Yes, that would be a properly normalized model.

Hmm.... Maybe we could allow both variations, with your suggestion
overriding mine if both are present? (I think there would be many
cases where the extra level of [submodule...] wouldn't be needed.
Also, the url in .gitmodules will only function as the initially
suggested url stored in the 'superprojects' .git/config)

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Fri, Jun 01, 2007 at 04:45:06PM +0200, Lars Hjemli wrote:
> On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> >On Fri, Jun 01, 2007 at 11:25:42AM +0200, Lars Hjemli wrote:
> >> [path '$path']
> >>  submodule=modulename
> >>  url=/some/url
> >
> >Wouldn't it make more sense to have
> >
> >[path '$path']
> >        submodule=modulename
> >
> >and
> >
> >[submodule '$modulename']
> >        url=/some/url
> >
> >in case the same module appears in more than one path?
> 
> Yes, that would be a properly normalized model.
> 
> Hmm.... Maybe we could allow both variations, with your suggestion
> overriding mine if both are present? (I think there would be many
> cases where the extra level of [submodule...] wouldn't be needed.

Hmm.... I was thinking that the extra "path" level could be optional,
i.e., if there is no path.$path.submodule, then the name of the
submodule would simply be $path.

skimo

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Fri, Jun 01, 2007 at 04:45:06PM +0200, Lars Hjemli wrote:
> > On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> > >On Fri, Jun 01, 2007 at 11:25:42AM +0200, Lars Hjemli wrote:
> > >> [path '$path']
> > >>  submodule=modulename
> > >>  url=/some/url
> > >
> > >Wouldn't it make more sense to have
> > >
> > >[path '$path']
> > >        submodule=modulename
> > >
> > >and
> > >
> > >[submodule '$modulename']
> > >        url=/some/url
> > >
> > >in case the same module appears in more than one path?
> >
> > Yes, that would be a properly normalized model.
> >
> > Hmm.... Maybe we could allow both variations, with your suggestion
> > overriding mine if both are present? (I think there would be many
> > cases where the extra level of [submodule...] wouldn't be needed.
>
> Hmm.... I was thinking that the extra "path" level could be optional,
> i.e., if there is no path.$path.submodule, then the name of the
> submodule would simply be $path.

Yeah, that should also work out. Time for a quick poll?

-- 
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Linus Torvalds]

On Fri, 1 Jun 2007, Lars Hjemli wrote:

> On 6/1/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> > 
> > Hmm.... I was thinking that the extra "path" level could be optional,
> > i.e., if there is no path.$path.submodule, then the name of the
> > submodule would simply be $path.
> 
> Yeah, that should also work out. Time for a quick poll?

Ack. I think the natural thing for a lot of cases is the trivial "module 
name == path" case, so having to have

	[path "kernel"]
		module = kernel

for that case just sounds unnecessary.

That said, I wonder if it wouldn't be more natural to do things the other 
way around, because quite often a "module" (under CVS conventions) is a 
*set* of directories, so with that in mind, it might be better to have the 
mapping be something like this:

	[module "infrastructure"]
		submodule = lib
		submodule = build

	[submodule "lib"]
		url = git://xyzzy/lib-1.2.3

	[submodule "build"]
		url = git://xyzzy/build-0.61


and make the rule be:
 - submodules are named by their paths (ie "path == submodule")
 - a module is a set of such submodules/paths
 - if no "module" is defined, the default is to just use the 
   path/submodule name

IOW, in the above case, we have *three* modules:

 - module "infrastructure", that is the union of submodules/paths "lib" 
   and "build"
 - module "lib" (== submodule/path "lib")
 - module "build" (== submodule/path "build")

and when you do a 

	git submodule checkout infrastructure

it would be basically equivalent to

	git submodule checkout lib
	git submodule checkout build

Hmm? That's how CVS users use modules (ie the "src" module may be much 
more than a single subdirectory)

		Linus

Re: [PATCH] Let .git/config specify the url for submodules

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Fri, 1 Jun 2007, Lars Hjemli wrote:
> ...
> Ack. I think the natural thing for a lot of cases is the trivial "module 
> name == path" case, so having to have
>
> 	[path "kernel"]
> 		module = kernel
>
> for that case just sounds unnecessary.
>
> That said, I wonder if it wouldn't be more natural to do things the other 
> way around, because quite often a "module" (under CVS conventions) is a 
> *set* of directories, so with that in mind, it might be better to have the 
> mapping be something like this:
>
> 	[module "infrastructure"]
> 		submodule = lib
> 		submodule = build
>
> 	[submodule "lib"]
> 		url = git://xyzzy/lib-1.2.3
>
> 	[submodule "build"]
> 		url = git://xyzzy/build-0.61
>
> and make the rule be:
>  - submodules are named by their paths (ie "path == submodule")
>  - a module is a set of such submodules/paths
>  - if no "module" is defined, the default is to just use the 
>    path/submodule name

I take that if you do want to name a submodule differently from
its (currently in-tree) path, you would do something like...

	[submodule "xyzzylib"]
        	path = lib
                url = git://xyzzy/lib-1.2.3

> IOW, in the above case, we have *three* modules:
>
>  - module "infrastructure", that is the union of submodules/paths "lib" 
>    and "build"
>  - module "lib" (== submodule/path "lib")
>  - module "build" (== submodule/path "build")
>
> and when you do a 
>
> 	git submodule checkout infrastructure
>
> it would be basically equivalent to
>
> 	git submodule checkout lib
> 	git submodule checkout build

Sounds very sensible.

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/1/07, Junio C Hamano <junkio@cox.net> wrote:
> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
> > On Fri, 1 Jun 2007, Lars Hjemli wrote:
> > ...
> > Ack. I think the natural thing for a lot of cases is the trivial "module
> > name == path" case, so having to have
> >
> >       [path "kernel"]
> >               module = kernel
> >
> > for that case just sounds unnecessary.
> >
> > That said, I wonder if it wouldn't be more natural to do things the other
> > way around, because quite often a "module" (under CVS conventions) is a
> > *set* of directories, so with that in mind, it might be better to have the
> > mapping be something like this:
> >
> >       [module "infrastructure"]
> >               submodule = lib
> >               submodule = build
> >
> >       [submodule "lib"]
> >               url = git://xyzzy/lib-1.2.3
> >
> >       [submodule "build"]
> >               url = git://xyzzy/build-0.61
> >
> > and make the rule be:
> >  - submodules are named by their paths (ie "path == submodule")
> >  - a module is a set of such submodules/paths
> >  - if no "module" is defined, the default is to just use the
> >    path/submodule name
>
> I take that if you do want to name a submodule differently from
> its (currently in-tree) path, you would do something like...
>
>         [submodule "xyzzylib"]
>                 path = lib
>                 url = git://xyzzy/lib-1.2.3
>

The reason I wanted to use

[path "lib"]
  submodule=xyzzylib
  url=git://xyzzy/lib-1.2.3

is that the git-submodule command always starts out with the submodule
path, trying to map that to an url or a submodule name. My proposal
allows

  $ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
  $ url=$(GIT_CONFIG=.gitmodules git-config path."$path".url)


Then Sven suggested to create a submodule section for the url, which would allow

  $ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
  $ url=$(GIT_CONFIG=.gitmodules git-config submodule."$name".url)


But I don't see an easy way to do the mapping from path to url/submodule with:

[submodule "xyzzylib"]
  path=lib
  url=git://xyzzy/lib-1.2.3

Suggestions?

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Sat, Jun 02, 2007 at 09:13:55AM +0200, Lars Hjemli wrote:
> Then Sven suggested to create a submodule section for the url, which would 
> allow
> 
>  $ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
>  $ url=$(GIT_CONFIG=.gitmodules git-config submodule."$name".url)
> 
> 
> But I don't see an easy way to do the mapping from path to url/submodule 
> with:
> 
> [submodule "xyzzylib"]
>  path=lib
>  url=git://xyzzy/lib-1.2.3
> 
> Suggestions?

I'm not a shell programmer, but it could look something like this

    $ name=$(git config --get-regexp 'submodule\..*\.path' | while read module modulepath; do if test "$modulepath" = "$path"; then echo $module | sed -e 's/^submodule.//' -e 's/.path//'; fi; done)

skimo

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Fri, Jun 01, 2007 at 09:29:58AM -0700, Linus Torvalds wrote:
> 	[module "infrastructure"]
> 		submodule = lib
> 		submodule = build
> 
> 	[submodule "lib"]
> 		url = git://xyzzy/lib-1.2.3
> 
> 	[submodule "build"]
> 		url = git://xyzzy/build-0.61
> 
> 
> IOW, in the above case, we have *three* modules:
> 
>  - module "infrastructure", that is the union of submodules/paths "lib" 
>    and "build"
>  - module "lib" (== submodule/path "lib")
>  - module "build" (== submodule/path "build")

If there are three modules, then why is one in the "module" section
and the other two in the "submodule" section?
Why not allow a module to both contain smaller modules and be contained
in a bigger module?

skimo

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/2/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Sat, Jun 02, 2007 at 09:13:55AM +0200, Lars Hjemli wrote:
> > Then Sven suggested to create a submodule section for the url, which would
> > allow
> >
> >  $ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
> >  $ url=$(GIT_CONFIG=.gitmodules git-config submodule."$name".url)
> >
> >
> > But I don't see an easy way to do the mapping from path to url/submodule
> > with:
> >
> > [submodule "xyzzylib"]
> >  path=lib
> >  url=git://xyzzy/lib-1.2.3
> >
> > Suggestions?
>
> I'm not a shell programmer,

Me neither ;-)

>but it could look something like this
>
>     $ name=$(git config --get-regexp 'submodule\..*\.path' | while read module modulepath; do if test "$modulepath" = "$path"; then echo $module | sed -e 's/^submodule.//' -e 's/.path//'; fi; done)
>

Ahh, --get-regexp, thanks.

Then this actually works:

name=$(GIT_CONFIG=.gitmodules git-config --get-regexp
'submodule\..*\.path' 'lib' | sed -e 's/^submodule\.\(.*\)\.path
lib$/\1/')


But why would we want to design .gitmodules in a way that makes it
hard to do the mapping from path to url?

-- 
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> But why would we want to design .gitmodules in a way that makes it
> hard to do the mapping from path to url?

Perhaps because "path" is not the "identity" of the subproject, but
the (logical) subproject name is?

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/2/07, Junio C Hamano <junkio@cox.net> wrote:
> "Lars Hjemli" <hjemli@gmail.com> writes:
>
> > But why would we want to design .gitmodules in a way that makes it
> > hard to do the mapping from path to url?
>
> Perhaps because "path" is not the "identity" of the subproject, but
> the (logical) subproject name is?
>
>

Ok.

So I'll make 'git-submodule' expect the .gitmodules layout to be like this:

[submodule 'xyz']
  url=git://example.com/xyz.git
  path=xyz1
  path=xyz2

And submodule.$name.path should be optional: if no submodule.*.path
entry exists for a given $path, submodule $name == $path.

Ok?

--
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> So I'll make 'git-submodule' expect the .gitmodules layout to be like this:
>
> [submodule 'xyz']
>  url=git://example.com/xyz.git
>  path=xyz1
>  path=xyz2
>
> And submodule.$name.path should be optional: if no submodule.*.path
> entry exists for a given $path, submodule $name == $path.
>
> Ok?

I do not think having more than one path for a given submodule
makes much sense, but other than that yes.

Re: [PATCH] Let .git/config specify the url for submodules

[Lars Hjemli]

On 6/2/07, Junio C Hamano <junkio@cox.net> wrote:
> "Lars Hjemli" <hjemli@gmail.com> writes:
>
> > So I'll make 'git-submodule' expect the .gitmodules layout to be like this:
> >
> > [submodule 'xyz']
> >  url=git://example.com/xyz.git
> >  path=xyz1
> >  path=xyz2
> >
> > And submodule.$name.path should be optional: if no submodule.*.path
> > entry exists for a given $path, submodule $name == $path.
> >
> > Ok?
>
> I do not think having more than one path for a given submodule
> makes much sense

Me neither, but it was mentioned as a possibility to have the same
repo checked out in multiple paths simultaneously. It does raise an
issue of how to push changes back into .git/submodules/$name.git tough
(git-submodule push). My initial plan for this was to push into the
branch registered for each path:

[submodule 'xyz']
  url=git://example.com/xyz.git

[path 'xyz1']
  submodule=xyz
  branch=stable

[path 'xyz2']
  submodule=xyz
  branch=bleedingedge

but there is probably other/better solutions.

>but other than that yes.

Thanks

-- 
larsh

Re: [PATCH] Let .git/config specify the url for submodules

[Linus Torvalds]

On Sat, 2 Jun 2007, Sven Verdoolaege wrote:

> On Fri, Jun 01, 2007 at 09:29:58AM -0700, Linus Torvalds wrote:
> > 	[module "infrastructure"]
> > 		submodule = lib
> > 		submodule = build
> > 
> > 	[submodule "lib"]
> > 		url = git://xyzzy/lib-1.2.3
> > 
> > 	[submodule "build"]
> > 		url = git://xyzzy/build-0.61
> > 
> > 
> > IOW, in the above case, we have *three* modules:
> > 
> >  - module "infrastructure", that is the union of submodules/paths "lib" 
> >    and "build"
> >  - module "lib" (== submodule/path "lib")
> >  - module "build" (== submodule/path "build")
> 
> If there are three modules, then why is one in the "module" section
> and the other two in the "submodule" section?

Because there are:

 - *two* actual submodules (== path), namely "lib" and "build".

 - each submodule always is *implicitly* a module too

 - we have a *named* (aka explicit) module "infrastructure" that is a 
   higher-level name for one or more submodules (in this case two).

So the implicit modules could have been written out:

	[module "lib"]
		submodule = "lib" 	# aka 'path = "lib"'

	 [module "build"]
		submodule = "build"	# aka 'path = "build"'

but my suggestion was that if the module name and the path name are the 
same, you don't need to say it.

(And quite frankly, I think it reads better as "submodule" than as "path", 
but maybe that threw you).

> Why not allow a module to both contain smaller modules and be contained
> in a bigger module?

Because the "module" definition is _different_ from the "submodule" 
definition.

The "module" definition is just a level of indirection. It is what allows 
you to call your module "kernel" regardless of where in the tree it is 
(and regardless of whether it's actually built up on *one* directory or 
many). It allows what CVS users have long used the "alias" thing for (or 
whatever it's called in CVSROOT/modules. But it also allows you to name 
single modules *without* having to specify exactly where in the tree they 
are.

In contrast, the "submodule" thing actually would declare where the 
submodule can be found from an URL standpoint. 

And maybe you want to allow the CVS "alias" kind of thing separately, but 
I think it's very common (exactly because quite often you want to cluster 
a few submodules together as "src" or "docs" or something, even if they 
might be technically more than one actual subproject).

		Linus

Re: [PATCH] Let .git/config specify the url for submodules

[Sven Verdoolaege]

On Sat, Jun 02, 2007 at 09:34:26AM -0700, Linus Torvalds wrote:
> On Sat, 2 Jun 2007, Sven Verdoolaege wrote:
> > On Fri, Jun 01, 2007 at 09:29:58AM -0700, Linus Torvalds wrote:
> > > 	[module "infrastructure"]
> > > 		submodule = lib
> > > 		submodule = build
> > > 
> > > 	[submodule "lib"]
> > > 		url = git://xyzzy/lib-1.2.3
> > > 
> > > 	[submodule "build"]
> > > 		url = git://xyzzy/build-0.61
> > > 
> > > 
> > > IOW, in the above case, we have *three* modules:
> > > 
> > >  - module "infrastructure", that is the union of submodules/paths "lib" 
> > >    and "build"
> > >  - module "lib" (== submodule/path "lib")
> > >  - module "build" (== submodule/path "build")
> > 
> > If there are three modules, then why is one in the "module" section
> > and the other two in the "submodule" section?
> 
> Because there are:
> 
>  - *two* actual submodules (== path), namely "lib" and "build".
> 
>  - each submodule always is *implicitly* a module too
> 
>  - we have a *named* (aka explicit) module "infrastructure" that is a 
>    higher-level name for one or more submodules (in this case two).
> 
> So the implicit modules could have been written out:
> 
> 	[module "lib"]
> 		submodule = "lib" 	# aka 'path = "lib"'
> 
> 	 [module "build"]
> 		submodule = "build"	# aka 'path = "build"'
> 
> but my suggestion was that if the module name and the path name are the 
> same, you don't need to say it.
> 
> (And quite frankly, I think it reads better as "submodule" than as "path", 
> but maybe that threw you).

I did indeed glance over the "submodule == path" in your mail (and it
seems Junio did so too).
I thought it was more or less agreed that the URL should be associated
to the logical module rather than the path (which you call "submodule"
here).  See http://article.gmane.org/gmane.comp.version-control.git/47567 .

Personally, I find your section names confusing.
Both your "submodule"s and "module"s are submodules from the point
of view of the project containing the .gitmodules.
And anything in the "submodule" section is only a (proper) submodule
of some other module if there happens to be a "module" containing it.

What's wrong with the following?

	[module "infrastructure"]
		submodule = lib
		submodule = build

	[module "lib"]
		url = git://xyzzy/lib-1.2.3

	[module "build"]
		url = git://xyzzy/build-0.61

A module with a "url" attribute can be cloned from that url.
A module with one or more "submodule" attributes requires
these modules as well.

> > Why not allow a module to both contain smaller modules and be contained
> > in a bigger module?
> 
> Because the "module" definition is _different_ from the "submodule" 
> definition.

>From the point of view of the user, they are the same.
They can both be used as an argument to

	git submodule checkout

> The "module" definition is just a level of indirection.

But why limit yourself to _one_ level of indirection?
If you call everything a "module" (or everything a "submodule")
then you get multiple levels of indirection for free.

skimo

[PATCH] git-submodule fixes for call to git config --get-regexp

[Chris Larson]

Two minor git-submodule fixes:
* Escape !'s in the git config --get-regexp, so submodule paths can
contain them.
* Be more explicit about the value regex, otherwise things get confused if one
  submodule name is a prefix of another (since --get-regexp can return
  multiple values).

Signed-off-by: Chris Larson <clarson@kergoth.com>

--- git-submodule.sh.old	2007-07-20 10:13:22.578125000 -0700
+++ git-submodule.sh	2007-07-20 10:14:56.281250000 -0700
@@ -46,7 +46,8 @@ get_repo_base() {
 #
 module_name()
 {
-       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
'^submodule\..*\.path$' "$1" |
+       path=$(echo "$1" | sed -e 's/\!/\\!/g')
+       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
'^submodule\..*\.path$' "^$path$" |
        sed -nre 's/^submodule\.(.+)\.path .+$/\1/p')
        test -z "$name" &&
        die "No submodule mapping found in .gitmodules for path '$path'"

-- 
Chris Larson - clarson at kergoth dot com
Dedicated Engineer - MontaVista - clarson at mvista dot com
Core Developer/Architect - TSLib, BitBake, OpenEmbedded, OpenZaurus

Re: [PATCH] git-submodule fixes for call to git config --get-regexp

[Junio C Hamano]

"Chris Larson" <clarson@kergoth.com> writes:

> Two minor git-submodule fixes:
> * Escape !'s in the git config --get-regexp, so submodule paths can
> contain them.

> --- git-submodule.sh.old	2007-07-20 10:13:22.578125000 -0700
> +++ git-submodule.sh	2007-07-20 10:14:56.281250000 -0700
> @@ -46,7 +46,8 @@ get_repo_base() {
> #
> module_name()
> {
> -       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
> '^submodule\..*\.path$' "$1" |
> +       path=$(echo "$1" | sed -e 's/\!/\\!/g')

My first reaction was if it shouldn't be done for only the '!'
at the very beginning, to defeat "do_not_match" logic, but it
would not hurt if we have extra "\!" in the middle.

Once you introduce that sed to munge the path string, I suspect
you would also want and can afford to quote extended regular
expression metacharacters as well.

> +       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
> '^submodule\..*\.path$' "^$path$" |
>        sed -nre 's/^submodule\.(.+)\.path .+$/\1/p')
>        test -z "$name" &&
>        die "No submodule mapping found in .gitmodules for path '$path'"
>
> -- 
> Chris Larson - clarson at kergoth dot com
> Dedicated Engineer - MontaVista - clarson at mvista dot com
> Core Developer/Architect - TSLib, BitBake, OpenEmbedded, OpenZaurus

Re: [PATCH] git-submodule fixes for call to git config --get-regexp

[Johannes Schindelin]

Hi,

On Fri, 20 Jul 2007, Junio C Hamano wrote:

> "Chris Larson" <clarson@kergoth.com> writes:
> 
> > +       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
> > '^submodule\..*\.path$' "^$path$" |
> >        sed -nre 's/^submodule\.(.+)\.path .+$/\1/p')

I wonder why it is a regular expression to begin with, since we seem to 
prefer shell patterns on paths.

However, _if_ we already go with regexps, why not just put it into the 
"sed" call, which is _already_ there, and leave "git config" alone?  IOW 
call

	git config --get-regexp '^submodule\..*\.path$' |
		sed -nre 's/^submodule\.(.*$path.*)\.path .+$\1/p'

Hmm?

Ciao,
Dscho

submodule init problem

[Ricky Nite]

Hello,

I encountered this message in "git submodule init":
"No url found for submodule path '<submodule>' in .gitmodules"

But when I look at .gitmodules, the <submodule> url is there.
Subsequently, I cannot do "git submodule update"

Here's the full log:
rnite@linuskarl:~/git_dev/myproj$ ~/bin/git --version
git version 1.5.3.rc2.g11308
rnite@linuskarl:~/git_dev$ mkdir flash
rnite@linuskarl:~/git_dev$ mkdir flashboard
rnite@linuskarl :~/git_dev$ cd flash
rnite@linuskarl:~/git_dev/flash$ echo "foo" > flash.v
rnite@linuskarl:~/git_dev/flash$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/flash$ ~/bin/git add flash.v
rnite@linuskarl:~/git_dev/flash$ ~/bin/git commit -a -m "initial
commit of flash module"
Created initial commit c19874e: initial commit of flash module
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 flash.v
rnite@linuskarl:~/git_dev/flash$ cd ../flashboard
rnite@linuskarl:~/git_dev/flashboard$ echo "foo" > flashboard.v
rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git add flashboard.v
rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git commit -a -m "initial
commit of flashboard module"
Created initial commit 45dff87: initial commit of flashboard module
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 flashboard.v
rnite@linuskarl:~/git_dev/flashboard$ cd ..
rnite@linuskarl :~/git_dev$ mkdir myproj
rnite@linuskarl:~/git_dev$ cd myproj
rnite@linuskarl:~/git_dev/myproj$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule add ~/git_dev/flash
Initialized empty Git repository in /home/rnite/git_dev/myproj/flash/.git/
remote: Generating pack...
remote: Done counting 3 objects.
Deltifying 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0remote:  (delta 0)
Indexing 3 objects...
 100% (3/3) done
rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule add ~/git_dev/flashboard
Initialized empty Git repository in /home/rnite/git_dev/myproj/flashboard/.git/
remote: Generating pack...
remote: Done counting 3 objects.
Deltifying 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0remote:  (delta 0)
Indexing 3 objects...
 100% (3/3) done
rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule init
No url found for submodule path 'flash' in .gitmodules
rnite@linuskarl:~/git_dev/myproj$ cat .gitmodules
[submodule "flash"]
        path = flash
        url = /home/rnite/git_dev/flash/.git
[submodule "flashboard"]
        path = flashboard
        url = /home/rnite/git_dev/flashboard/.git

I can add more submodules, but the message stays the same after "git
submodule init" and I cannot do "git submodule update"

help,
RickyN

Re: submodule init problem

[Ricky Nite]

This problem doesn't seem to occur if I rename my submodules (but I
really don't want to):

rnite@linuskarl:~/git_dev$ mkdir moduleA
rnite@linuskarl:~/git_dev$ mkdir moduleB
rnite@linuskarl:~/git_dev$ cd moduleA
rnite@linuskarl:~/git_dev/moduleA$ echo "foo" > moduleAtop.v
rnite@linuskarl:~/git_dev/moduleA$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/moduleA$ ~/bin/git add moduleAtop
rnite@linuskarl:~/git_dev/moduleA$ ~/bin/git commit -a -m"imitial commit"
Created initial commit f8bc70b: imitial commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 moduleAtop.v
rnite@linuskarl:~/git_dev/moduleA$ cd ../moduleB
rnite@linuskarl:~/git_dev/moduleB$ echo "foo" > moduleBtop.v
rnite@linuskarl:~/git_dev/moduleB$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/moduleB$ ~/bin/git add moduleBtop.v
rnite@linuskarl:~/git_dev/moduleB$ ~/bin/git commit -a -m"imitial commit"
Created initial commit da426f3: imitial commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 moduleBtop.v
rnite@linuskarl:~/git_dev/moduleB$ cd ..
rnite@linuskarl:~/git_dev$ mkdir myproj2
rnite@linuskarl:~/git_dev$ cd myproj2
rnite@linuskarl:~/git_dev/myproj2$ ~/bin/git init
Initialized empty Git repository in .git/
rnite@linuskarl:~/git_dev/myproj2$ ~/bin/git submodule add ~/git_dev/moduleA
Initialized empty Git repository in /home/rnite/git_dev/myproj2/moduleA/.git/
remote: Generating pack...
remote: Done counting 3 objects.
Deltifying 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0remote:  (delta 0)
Indexing 3 objects...
 100% (3/3) done
rnite@linuskarl:~/git_dev/myproj2$ ~/bin/git submodule add ~/git_dev/moduleB
Initialized empty Git repository in /home/rnite/git_dev/myproj2/moduleB/.git/
remote: Generating pack...
remote: Done counting 3 objects.
Deltifying 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0remote:  (delta 0)
Indexing 3 objects...
 100% (3/3) done
rnite@linuskarl:~/git_dev/myproj2$ ~/bin/git submodule init
Submodule 'moduleA' (/home/rnite/git_dev/moduleA/.git) registered for
path 'moduleA'
Submodule 'moduleB' (/home/rnite/git_dev/moduleB/.git) registered for
path 'moduleB'

help,
RickyN

On 7/25/07, Ricky Nite <ricky.nite@gmail.com> wrote:
> Hello,
>
> I encountered this message in "git submodule init":
> "No url found for submodule path '<submodule>' in .gitmodules"
>
> But when I look at .gitmodules, the <submodule> url is there.
> Subsequently, I cannot do "git submodule update"
>
> Here's the full log:
> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git --version
> git version 1.5.3.rc2.g11308
> rnite@linuskarl:~/git_dev$ mkdir flash
> rnite@linuskarl:~/git_dev$ mkdir flashboard
> rnite@linuskarl :~/git_dev$ cd flash
> rnite@linuskarl:~/git_dev/flash$ echo "foo" > flash.v
> rnite@linuskarl:~/git_dev/flash$ ~/bin/git init
> Initialized empty Git repository in .git/
> rnite@linuskarl:~/git_dev/flash$ ~/bin/git add flash.v
> rnite@linuskarl:~/git_dev/flash$ ~/bin/git commit -a -m "initial
> commit of flash module"
> Created initial commit c19874e: initial commit of flash module
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 flash.v
> rnite@linuskarl:~/git_dev/flash$ cd ../flashboard
> rnite@linuskarl:~/git_dev/flashboard$ echo "foo" > flashboard.v
> rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git init
> Initialized empty Git repository in .git/
> rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git add flashboard.v
> rnite@linuskarl:~/git_dev/flashboard$ ~/bin/git commit -a -m "initial
> commit of flashboard module"
> Created initial commit 45dff87: initial commit of flashboard module
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 flashboard.v
> rnite@linuskarl:~/git_dev/flashboard$ cd ..
> rnite@linuskarl :~/git_dev$ mkdir myproj
> rnite@linuskarl:~/git_dev$ cd myproj
> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git init
> Initialized empty Git repository in .git/
> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule add ~/git_dev/flash
> Initialized empty Git repository in /home/rnite/git_dev/myproj/flash/.git/
> remote: Generating pack...
> remote: Done counting 3 objects.
> Deltifying 3 objects...
>  100% (3/3) done
> Total 3 (delta 0), reused 0remote:  (delta 0)
> Indexing 3 objects...
>  100% (3/3) done
> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule add ~/git_dev/flashboard
> Initialized empty Git repository in /home/rnite/git_dev/myproj/flashboard/.git/
> remote: Generating pack...
> remote: Done counting 3 objects.
> Deltifying 3 objects...
>  100% (3/3) done
> Total 3 (delta 0), reused 0remote:  (delta 0)
> Indexing 3 objects...
>  100% (3/3) done
> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git submodule init
> No url found for submodule path 'flash' in .gitmodules
> rnite@linuskarl:~/git_dev/myproj$ cat .gitmodules
> [submodule "flash"]
>         path = flash
>         url = /home/rnite/git_dev/flash/.git
> [submodule "flashboard"]
>         path = flashboard
>         url = /home/rnite/git_dev/flashboard/.git
>
> I can add more submodules, but the message stays the same after "git
> submodule init" and I cannot do "git submodule update"
>
> help,
> RickyN
>

Re: submodule init problem

[Junio C Hamano]

"Ricky Nite" <ricky.nite@gmail.com> writes:

> This problem doesn't seem to occur if I rename my submodules (but I
> really don't want to):
>
> rnite@linuskarl:~/git_dev$ mkdir moduleA
> rnite@linuskarl:~/git_dev$ mkdir moduleB
> ...
> rnite@linuskarl:~/git_dev/myproj2$ ~/bin/git submodule init
> Submodule 'moduleA' (/home/rnite/git_dev/moduleA/.git) registered for
> path 'moduleA'
> Submodule 'moduleB' (/home/rnite/git_dev/moduleB/.git) registered for
> path 'moduleB'
>
> help,
> RickyN
>
> On 7/25/07, Ricky Nite <ricky.nite@gmail.com> wrote:
>> Hello,
>>
>> I encountered this message in "git submodule init":
>> "No url found for submodule path '<submodule>' in .gitmodules"
>>
>> But when I look at .gitmodules, the <submodule> url is there.
>> Subsequently, I cannot do "git submodule update"
>>
>> Here's the full log:
>> rnite@linuskarl:~/git_dev/myproj$ ~/bin/git --version
>> git version 1.5.3.rc2.g11308
>> rnite@linuskarl:~/git_dev$ mkdir flash
>> rnite@linuskarl:~/git_dev$ mkdir flashboard

Ok, this appears it most likely to be related to the fact that
one is a prefix of the other in problematic case.

Lars, Sven?

Re: submodule init problem

[Sven Verdoolaege]

On Tue, Jul 24, 2007 at 06:49:26PM -0700, Junio C Hamano wrote:
> Ok, this appears it most likely to be related to the fact that
> one is a prefix of the other in problematic case.

Yes, this has been noted before and Chris Larson sent in a patch,
but he didn't follow up on it.

On Fri, Jul 20, 2007 at 07:36:43PM +0100, Johannes Schindelin wrote:
> Hi,
> 
> On Fri, 20 Jul 2007, Junio C Hamano wrote:
> 
> > "Chris Larson" <clarson@kergoth.com> writes:
> > 
> > > +       name=$(GIT_CONFIG=.gitmodules git config --get-regexp
> > > '^submodule\..*\.path$' "^$path$" |
> > >        sed -nre 's/^submodule\.(.+)\.path .+$/\1/p')
> 
> I wonder why it is a regular expression to begin with, since we seem to 
> prefer shell patterns on paths.
> 
> However, _if_ we already go with regexps, why not just put it into the 
> "sed" call, which is _already_ there, and leave "git config" alone?  IOW 
> call
> 
> 	git config --get-regexp '^submodule\..*\.path$' |
> 		sed -nre 's/^submodule\.(.*$path.*)\.path .+$\1/p'
>

You would be matching the key (the name of the module) rather than
the value (the path of the module) here.

Anyway, I'm not sure why Lars went for the regexp.
I thought he wanted to match the path exactly, which
is why I originally proposed (the more clunky)

On Sat, Jun 02, 2007 at 09:44:10AM +0200, Sven Verdoolaege wrote:
> On Sat, Jun 02, 2007 at 09:13:55AM +0200, Lars Hjemli wrote:
> > But I don't see an easy way to do the mapping from path to url/submodule 
> > with:
> > 
> > [submodule "xyzzylib"]
> >  path=lib
> >  url=git://xyzzy/lib-1.2.3
> > 
> > Suggestions?
> 
> I'm not a shell programmer, but it could look something like this
> 
>     $ name=$(git config --get-regexp 'submodule\..*\.path' | while read module modulepath; do if test "$modulepath" = "$path"; then echo $module | sed -e 's/^submodule.//' -e 's/.path//'; fi; done)

skimo

Re: submodule init problem

[Lars Hjemli]

On 7/25/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Tue, Jul 24, 2007 at 06:49:26PM -0700, Junio C Hamano wrote:
> > Ok, this appears it most likely to be related to the fact that
> > one is a prefix of the other in problematic case.
>
> Yes, this has been noted before and Chris Larson sent in a patch,
> but he didn't follow up on it.

The following seems to work (in my limitied testing):

  eol='$'
  git config --get-regexp '^submodule\..*\.path$' "^$1$eol"

--
larsh

Re: submodule init problem

[Johannes Schindelin]

Hi,

On Wed, 25 Jul 2007, Lars Hjemli wrote:

> On 7/25/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> > On Tue, Jul 24, 2007 at 06:49:26PM -0700, Junio C Hamano wrote:
> > > Ok, this appears it most likely to be related to the fact that
> > > one is a prefix of the other in problematic case.
> > 
> > Yes, this has been noted before and Chris Larson sent in a patch,
> > but he didn't follow up on it.
> 
> The following seems to work (in my limitied testing):
> 
>  eol='$'
>  git config --get-regexp '^submodule\..*\.path$' "^$1$eol"

Ah, now I get it.  You are looking for the _key_ whose value is "$1".  But 
then you really should use "^$1$", and not just "$1", otherwise you will 
get unintended behaviour when one submodule's name is a substring of 
another submodule's name.

And you do not need the eol hack.

Ciao,
Dscho

Re: submodule init problem

[Lars Hjemli]

On 7/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Wed, 25 Jul 2007, Lars Hjemli wrote:
> >  eol='$'
> >  git config --get-regexp '^submodule\..*\.path$' "^$1$eol"
>
> Ah, now I get it.  You are looking for the _key_ whose value is "$1".

Yes

> And you do not need the eol hack.

Hmm, I tried misc. quoting/escaping without success, care to educate me?

-- 
larsh

Re: submodule init problem

[Johannes Schindelin]

Hi,

On Wed, 25 Jul 2007, Lars Hjemli wrote:

> On 7/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Wed, 25 Jul 2007, Lars Hjemli wrote:
> > >  eol='$'
> > >  git config --get-regexp '^submodule\..*\.path$' "^$1$eol"
> > 
> > Ah, now I get it.  You are looking for the _key_ whose value is "$1".
> 
> Yes
> 
> > And you do not need the eol hack.
> 
> Hmm, I tried misc. quoting/escaping without success, care to educate me?

I already did ;-I  "^$1$"

Ciao,
Dscho

Re: submodule init problem

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

> On Tue, Jul 24, 2007 at 06:49:26PM -0700, Junio C Hamano wrote:
>> Ok, this appears it most likely to be related to the fact that
>> one is a prefix of the other in problematic case.
>
> Yes, this has been noted before and Chris Larson sent in a patch,
> but he didn't follow up on it.

Ok, I re-read the thread and came up with a different solution.
How does this look?

-- >8 --
git-submodule module_name: avoid using unwieldy "value_regexp" feature.

"module_name $path" function wants to look up a configuration
variable "submodule.<modulename>.path" whose value is $path, and
return the <modulename> found.  "git-config --get-regexp" is the
natural thing to use for this, but (1) its value matching has an
unfortunate "feature" that takes leading '!' specially, and (2)
its output needs to be parsed with sed to extract <modulename>
part anyway.

This changes the call to "git-config --get-regexp" not to use
the value-regexp part, and moves the "pick the one whose value
is $path" part to the downstream sed.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 git-submodule.sh |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 1f0cb99..afbaec7 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -46,8 +46,11 @@ get_repo_base() {
 #
 module_name()
 {
-       name=$(GIT_CONFIG=.gitmodules git config --get-regexp '^submodule\..*\.path$' "$1" |
-       sed -nre 's/^submodule\.(.+)\.path .+$/\1/p')
+	# Do we have "submodule.<something>.path = $1" defined in .gitmodules file?
+	re=$(printf '%s' "$1" | sed -e 's/\([^a-zA-Z0-9_]\)/\\\1/g')
+	name=$( GIT_CONFIG=.gitmodules \
+		git config --get-regexp '^submodule\..*\.path$' |
+		sed -n -e 's|^submodule\.\(.*\)\.path '"$re"'$|\1|p' )
        test -z "$name" &&
        die "No submodule mapping found in .gitmodules for path '$path'"
        echo "$name"

Re: submodule init problem

[Ricky Nite]

works for me :)

backgrounder:
I'm currently importing a large cvs repo (multiple projects, shared
modules, shared environment) into cvs-tracking git "superprojects".
Incremental import of the cvs modules using periodic invocations of
"git cvsimport" seems to be working, although there were some messages
(I suspect cvsps - will post later).  I was adding modules into the
superprojects when I hit this submodule init issue.

It will also be nice to have some kind of "superproject browser" in gitk..

On 7/26/07, Junio C Hamano <gitster@pobox.com> wrote:
> Sven Verdoolaege <skimo@kotnet.org> writes:
>
> > On Tue, Jul 24, 2007 at 06:49:26PM -0700, Junio C Hamano wrote:
> >> Ok, this appears it most likely to be related to the fact that
> >> one is a prefix of the other in problematic case.
> >
> > Yes, this has been noted before and Chris Larson sent in a patch,
> > but he didn't follow up on it.
>
> Ok, I re-read the thread and came up with a different solution.
> How does this look?
>

Re: submodule init problem

[Junio C Hamano]

"Ricky Nite" <ricky.nite@gmail.com> writes:

> works for me :)

Thanks, will commit.