[PATCH] Submodules always use a relative path to gitdir

[PATCH] Submodules always use a relative path to gitdir

[Antony Male]

This fixes a problem where moving a git repository with checked-out
submodules would cause a fatal error when commands such as 'git
submodule update' were run.

Git submoule clone uses git clone --separate-git-dir to checkout a
submodule's git repository into <supermodule>/.git/modules, if this
folder does not already exist. git clone --separate-git-dir was
designed for a scenario where the git repository stays in one location
and the working copy can be moved. Therefore the .git file in the
working copy uses an absolute path to specify the location of the
repository.

In the submodules scenario, neither the git repository nor the working
copy will be moved relative to each other. However, the supermodule may
be moved, which moves both the submodule's git repository and its
working copy. This means that the submodule's .git file no longer
points to its repository, causing the error.

Previously, if git submodule clone was called when the submodule's git
repository already existed in <supermodule>/.git/modules, it would
simply re-create the submodule's .git file, using a relative path.
This patch uses the above mechanism to re-write the .git file after git
clone --separate-git-dir is run, replacing the absolute path with a
relative one.

An alternative patch would teach git-clone an option to control whether
an absolute or relative path is used when --separate-git-dir is passed.

Signed-off-by: Antony Male <antony.male@gmail.com>
---
 git-submodule.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 3adab93..18eb5ff 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -159,7 +159,6 @@ module_clone()
 	if test -d "$gitdir"
 	then
 		mkdir -p "$path"
-		echo "gitdir: $rel_gitdir" >"$path/.git"
 		rm -f "$gitdir/index"
 	else
 		mkdir -p "$gitdir_base"
@@ -171,6 +170,7 @@ module_clone()
 		fi ||
 		die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
 	fi
+	echo "gitdir: $rel_gitdir" >"$path/.git"
 }
 
 #
-- 
1.7.8

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Antony Male <antony.male@gmail.com> writes:

> Git submoule clone uses git clone --separate-git-dir to checkout a
> submodule's git repository into <supermodule>/.git/modules,...

This is misleading. The <superproject>/.git/modules/<name> is the location
of the $GIT_DIR for the submodule <name>, not the location of its checkout
at <superproject>/<path> that is outside <superproject>/.git/modules/
hierarchy.

> folder does not already exist. git clone --separate-git-dir was
> designed for a scenario where the git repository stays in one location
> and the working copy can be moved.

Are you sure about this "clone's design"? It sounds like a revisionist
history.

Saying something like "it would be nicer if it also let us use in this new
different scenario" in the proposed commit log message is perfectly fine,
but my understanding is that the --separate-git-dir option and "gitdir: "
support were designed to allow having the $GIT_DIR in a different place
from the working tree that has ".git" in it, nothing more, nothing less. I
do not think we meant to support moving either directory after they are
set up. If you want to move either, you would need to (and you can, like
your patch does) tweak "gitdir:" to adjust.

By-the-way-Nit. We do not use any folders. s/folder/directory/.

> In the submodules scenario, neither the git repository nor the working
> copy will be moved relative to each other. However, the supermodule may
> be moved,...

Again, who said that you are allowed to move the superproject directory in
the first place? I would understand that it might be nicer if it could be
moved but I haven't thought this through thoroughly yet---there may be
other side effects from doing so, other than the relativeness of "gitdir".

> Previously, if git submodule clone was called when the submodule's git
> repository already existed in <supermodule>/.git/modules, it would
> simply re-create the submodule's .git file, using a relative path.

... "to point at the existing <superproject>/.git/modules/<name>".

Overall, I think I can agree with the goal, but the tone of the proposed
commit log message rubs the reader in a wrong way to see clearly what this
patch is proposing to do and where its merit lies. It is probably not a
big deal, and perhaps it may be just the order of explanation.

I would probably explain the goal like this if I were doing this patch,
without triggering any need for revisionist history bias.

    Recent versions of "git submodule" maintain the submodule <name> at
    <path> in the superproject using a "separate git-dir" mechanism. The
    repository data for the submodule is stored in ".git/modules/<name>/"
    directory of the superproject, and its working tree is created at
    "<path>/" directory, with "<path>/.git" file pointing at the
    ".git/modules/<name>/" directory.

    This is so that we can check out an older version of the superproject
    that does not yet have the submodule <name> anywhere without losing
    (and later having to re-clone) the submodule repository. Removing
    "<path>" won't lose ".git/modules/<name>", and a different branch that
    has the submodule at different location in the superproject, say
    "<path2>", can create "<path2>/" and ".git" in it to point at the same
    ".git/modules/<name>".

    When instantiating such a submodule, if ".git/modules/<name>/" does
    not exist in the superproject, the submodule repository needs to be
    cloned there first. Then we only need to create "<path>" directory,
    point ".git/modules/<name>/" in the superproject with "<path>/.git",
    and check out the working tree.

    However, the current code is not structured that way. The codepath to
    deal with newly cloned submodules uses "git clone --separate-git-dir"
    and creates "<path>" and "<path>/.git". This can make the resulting
    submodule working tree at "<path>" different from the codepath for
    existing submodules. An example of such differences is that this
    codepath prepares "<path>/.git" with an absolute path, while the
    normal codepath uses a relative path.

When explained this way, the remedy is quite clear, and the change is more
forward-looking, isn't it?  If we later start doing more in the codepath
to deal with existing submodules, your patch may break without having
extra code to cover the "newly cloned" case, too.

I further wonder if we can get away without using separate-git-dir option
in this codepath, though. IOW using

        git clone $quiet -bare ${reference:+"$reference"} "$url" "$gitdir"

might be a better solution.

For example (this relates to the point I mumbled "haven't thought this
through thoroughly yet"), doesn't the newly cloned repository have
core.worktree that points at the working tree that records the <path>,
which would become meaningless when a commit in the superproject that
binds the submodule at different path <path2>?

 git-submodule.sh |   21 ++++++++-------------
 1 files changed, 8 insertions(+), 13 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 3adab93..9a23e9d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -156,21 +156,16 @@ module_clone()
 		;;
 	esac
 
-	if test -d "$gitdir"
+	if ! test -d "$gitdir"
 	then
-		mkdir -p "$path"
-		echo "gitdir: $rel_gitdir" >"$path/.git"
-		rm -f "$gitdir/index"
-	else
-		mkdir -p "$gitdir_base"
-		if test -n "$reference"
-		then
-			git-clone $quiet "$reference" -n "$url" "$path" --separate-git-dir "$gitdir"
-		else
-			git-clone $quiet -n "$url" "$path" --separate-git-dir "$gitdir"
-		fi ||
-		die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
+		git clone $quiet -n ${reference:+"$reference"} \
+			--separate-git-dir "$gitdir" "$url" "$path" ||
+		die "$(eval_gettext "Clone of '\$url' for submodule '\$name' failed")
 	fi
+
+	mkdir -p "$path"
+	echo "gitdir: $rel_gitdir" >"$path/.git"
+	rm -f "$gitdir/index"
 }
 
 #

Re: [PATCH] Submodules always use a relative path to gitdir

[Phil Hord]

On Thu, Dec 29, 2011 at 5:40 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Antony Male <antony.male@gmail.com> writes:
>
>> Git submoule clone uses git clone --separate-git-dir to checkout a
>> submodule's git repository into <supermodule>/.git/modules,...
>
> This is misleading. The <superproject>/.git/modules/<name> is the location
> of the $GIT_DIR for the submodule <name>, not the location of its checkout
> at <superproject>/<path> that is outside <superproject>/.git/modules/
> hierarchy.

Yes, so I think a simple s/checkout/clone/ should fix it.

[...]

>> In the submodules scenario, neither the git repository nor the working
>> copy will be moved relative to each other. However, the supermodule may
>> be moved,...
>
> Again, who said that you are allowed to move the superproject directory in
> the first place? I would understand that it might be nicer if it could be
> moved but I haven't thought this through thoroughly yet---there may be
> other side effects from doing so, other than the relativeness of "gitdir".

Previously it was accepted practice to clone a local repo with rsync.
This method continues to work well even with submodules before
git-files became the norm.  But now it breaks because of the absolute
paths.

Similarly, clones on network mounts and portable drives where absolute
paths may change from time to time or machine to machine will also
break now but worked before.

So, who said you were NOT allowed to move the superproject directory
directory in the first place?  It seems natural that you should be
able to do so, especially since the submodules are all contained
within the superproject path.


>> Previously, if git submodule clone was called when the submodule's git
>> repository already existed in <supermodule>/.git/modules, it would
>> simply re-create the submodule's .git file, using a relative path.
>
> ... "to point at the existing <superproject>/.git/modules/<name>".
>
> Overall, I think I can agree with the goal, but the tone of the proposed
> commit log message rubs the reader in a wrong way to see clearly what this
> patch is proposing to do and where its merit lies. It is probably not a
> big deal, and perhaps it may be just the order of explanation.
>
> I would probably explain the goal like this if I were doing this patch,
> without triggering any need for revisionist history bias.
>
>    Recent versions of "git submodule" maintain the submodule <name> at
>    <path> in the superproject using a "separate git-dir" mechanism. The
>    repository data for the submodule is stored in ".git/modules/<name>/"
>    directory of the superproject, and its working tree is created at
>    "<path>/" directory, with "<path>/.git" file pointing at the
>    ".git/modules/<name>/" directory.
>
>    This is so that we can check out an older version of the superproject
>    that does not yet have the submodule <name> anywhere without losing
>    (and later having to re-clone) the submodule repository. Removing

Revisionism nit: the real danger here is that you lose local commits.

>    "<path>" won't lose ".git/modules/<name>", and a different branch that
>    has the submodule at different location in the superproject, say
>    "<path2>", can create "<path2>/" and ".git" in it to point at the same
>    ".git/modules/<name>".

This doesn't explain why one path is absolute and one is relative.
But I don't suppose this is the place for historical documentation
anyway.

>    When instantiating such a submodule, if ".git/modules/<name>/" does
>    not exist in the superproject, the submodule repository needs to be
>    cloned there first. Then we only need to create "<path>" directory,
>    point ".git/modules/<name>/" in the superproject with "<path>/.git",
>    and check out the working tree.
>
>    However, the current code is not structured that way. The codepath to
>    deal with newly cloned submodules uses "git clone --separate-git-dir"
>    and creates "<path>" and "<path>/.git". This can make the resulting
>    submodule working tree at "<path>" different from the codepath for
>    existing submodules. An example of such differences is that this
>    codepath prepares "<path>/.git" with an absolute path, while the
>    normal codepath uses a relative path.

I had to read this three times before I understood it. There are some
minor grammatical nits in it, but also the use of nearness and use of
"path" and "codepath" to mean two unrelated things was misleading me.
Here's my attempt to clean it up:

    However, the current code is not structured that way. The code to
    deal with newly cloned submodules is different from the code to
    checkout a workdir for existing submodules.  The "newly cloned
    submodule" code uses "git clone --separate-git-dir" to create
    "<path>" and "<path>/.git". The "existing submodules" code
    simply creates the "<path>/.git" internally, using a relative path.
    This makes the resulting submodule working tree at "<path>" different
    depending on which code is used.  An example of such differences
    is that the "newly cloned submodule" code prepares "<path>/.git"
    with an absolute path, while the "existing submodules" code
    prepares the same file using a relative path.


> When explained this way, the remedy is quite clear, and the change is more
> forward-looking, isn't it?  If we later start doing more in the codepath
> to deal with existing submodules, your patch may break without having
> extra code to cover the "newly cloned" case, too.


> I further wonder if we can get away without using separate-git-dir option
> in this codepath, though. IOW using
>
>        git clone $quiet -bare ${reference:+"$reference"} "$url" "$gitdir"
>
> might be a better solution.

You may be right about this one.  I still think the addition of a
--relative-path option to 'git-checkout --separate-work-dir' could be
useful and also easier to maintain/describe.

> For example (this relates to the point I mumbled "haven't thought this
> through thoroughly yet"), doesn't the newly cloned repository have
> core.worktree that points at the working tree that records the <path>,
> which would become meaningless when a commit in the superproject that
> binds the submodule at different path <path2>?

Ooh, yes it does.  Maybe that should be fixed in this case too.

Because submodule cloning with a separate work-dir is a special case
of git-files and work-dirs because we know that each is relative
(subordinate) to the superproject path.  Therefore, I think in this
special-case version of the "separate work-dir" scenario, we should
use super-project-relative paths for both cases.

How do we codify this so this functionality is reliably retained by
future developers?  I think moving the code into someplace more
explicit would help, but I haven't looked too deeply at the code.

>  git-submodule.sh |   21 ++++++++-------------
>  1 files changed, 8 insertions(+), 13 deletions(-)
>
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 3adab93..9a23e9d 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -156,21 +156,16 @@ module_clone()
>                ;;
>        esac
>
> -       if test -d "$gitdir"
> +       if ! test -d "$gitdir"
>        then
> -               mkdir -p "$path"
> -               echo "gitdir: $rel_gitdir" >"$path/.git"
> -               rm -f "$gitdir/index"
> -       else
> -               mkdir -p "$gitdir_base"
> -               if test -n "$reference"
> -               then
> -                       git-clone $quiet "$reference" -n "$url" "$path" --separate-git-dir "$gitdir"
> -               else
> -                       git-clone $quiet -n "$url" "$path" --separate-git-dir "$gitdir"
> -               fi ||
> -               die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
> +               git clone $quiet -n ${reference:+"$reference"} \
> +                       --separate-git-dir "$gitdir" "$url" "$path" ||
> +               die "$(eval_gettext "Clone of '\$url' for submodule '\$name' failed")
>        fi
> +
> +       mkdir -p "$path"
> +       echo "gitdir: $rel_gitdir" >"$path/.git"
> +       rm -f "$gitdir/index"
>  }

Doesn't this avoid creating core.worktree in the first place?  I'm ok
with that because I assume it's never used in the submodule scenario,
but I also suspect that assumption could be wrong.  Any concerns?

Phil

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Phil Hord <phil.hord@gmail.com> writes:

>> Again, who said that you are allowed to move the superproject directory in
>> the first place? I would understand that it might be nicer if it could be
>> moved but I haven't thought this through thoroughly yet---there may be
>> other side effects from doing so, other than the relativeness of "gitdir".
>
> Previously it was accepted practice to clone a local repo with rsync.
> This method continues to work well even with submodules before
> git-files became the norm.  But now it breaks because of the absolute
> paths.

You are utterly mistaken.

There are 47 million things you can do to your repository outside of the
control of git, and obviously we do not exhaustively enumerate everything
that ought to work (or not work). Anything that is not explicitly allowed
in the documentation is, ehh, not allowed.

Many such things may happen to work, either by accident or as a natural
consequence of the design. Some things needs adjustments after you do them
without telling git. There is a difference between what is not allowed and
what is explicitly forbidden.

Copying with rsync (or cp for that matter) is one good example. Doing so
will cause the cached stat information in the index and the working tree
files go out of sync, and diff-files will give you false differences after
that. You would adjust to that by running "update-index --refresh". So we
do not say "you are allowed to cp and git will guarantee everything will
work as-is", but it is not explicitly forbidden. As long as you make
necessary adjustments, you can keep using the copied repository.

> So, who said you were NOT allowed to move the superproject directory
> directory in the first place?

See above.

And the extent of the design of

    echo "gitdir: $there" >.git && git config core.worktree "$(pwd)"

is to work with the locations of these two places as they are set up.
Moving one or the other or both may or may not work without adjusting to
what you did. If you "mv $there $newlocation" (the repository) behind
Git's back, you may need to update .git to point at the new location of
the repository.  If you move your working tree woth "mv", you may need to
update core.worktree to point at the new location of the working tree.
And until you do so things may not work. That is why we do not explicitly
say "you can move them to arbitrary places without telling git and things
will work"---because that is not the case.

> This doesn't explain why one path is absolute and one is relative.

Exactly. Because absolute/relative does not come into play as the scope of
the design did not include supporting "moving" one, the other, or both to
arbitrary places without telling git.

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ...
> And the extent of the design of
>
>     echo "gitdir: $there" >.git && git config core.worktree "$(pwd)"
>
> is to work with the locations of these two places as they are set up.
> Moving one or the other or both may or may not work without adjusting to
> what you did. If you "mv $there $newlocation" (the repository) behind
> Git's back, you may need to update .git to point at the new location of
> the repository.  If you move your working tree woth "mv", you may need to
> update core.worktree to point at the new location of the working tree.
> And until you do so things may not work. That is why we do not explicitly
> say "you can move them to arbitrary places without telling git and things
> will work"---because that is not the case.

Just to avoid any misunderstanding, I still agree with the overall goal of
the original patch to allow moving the whole superproject tree, including
its submodule repositories in its .git/modules/, and the working trees of
itself and its submodules. It is a narrow special case with a very well
defined relative relationships between the working tree of submodules and
the repositories that control them, and having them point to each other
with relative paths will make any post-move adjustments unnecessary, unlike
more general unconstrained uses of the "gitdir: $there" mechanism.

Re: [PATCH] Submodules always use a relative path to gitdir

[Jens Lehmann]

Am 29.12.2011 23:40, schrieb Junio C Hamano:
> Antony Male <antony.male@gmail.com> writes:
> I further wonder if we can get away without using separate-git-dir option
> in this codepath, though. IOW using
> 
>         git clone $quiet -bare ${reference:+"$reference"} "$url" "$gitdir"
> 
> might be a better solution.

A quick test shows that using a bare repo won't fly because without the
core.worktree setting commands that operate on the work tree can't be
run anymore inside submodules (starting with the initial checkout). If
we could teach setup to take the directory where the gitfile was found
as first guess for the git work tree it looks like we can make that
approach work. I'll see if I can come up with something here ...

> For example (this relates to the point I mumbled "haven't thought this
> through thoroughly yet"), doesn't the newly cloned repository have
> core.worktree that points at the working tree that records the <path>,
> which would become meaningless when a commit in the superproject that
> binds the submodule at different path <path2>?

Yes, and the core.worktree setting also contains an absolute path. So
we must either make that relative too and rewrite it on every "git
submodule add" to record the possibly changed path there or make the
bare clone work with a work tree (which sounds a bit strange ;-).

>  git-submodule.sh |   21 ++++++++-------------
>  1 files changed, 8 insertions(+), 13 deletions(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 3adab93..9a23e9d 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -156,21 +156,16 @@ module_clone()
>  		;;
>  	esac
>  
> -	if test -d "$gitdir"
> +	if ! test -d "$gitdir"
>  	then
> -		mkdir -p "$path"
> -		echo "gitdir: $rel_gitdir" >"$path/.git"
> -		rm -f "$gitdir/index"
> -	else
> -		mkdir -p "$gitdir_base"
> -		if test -n "$reference"
> -		then
> -			git-clone $quiet "$reference" -n "$url" "$path" --separate-git-dir "$gitdir"
> -		else
> -			git-clone $quiet -n "$url" "$path" --separate-git-dir "$gitdir"
> -		fi ||
> -		die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
> +		git clone $quiet -n ${reference:+"$reference"} \
> +			--separate-git-dir "$gitdir" "$url" "$path" ||
> +		die "$(eval_gettext "Clone of '\$url' for submodule '\$name' failed")
>  	fi
> +
> +	mkdir -p "$path"
> +	echo "gitdir: $rel_gitdir" >"$path/.git"
> +	rm -f "$gitdir/index"
>  }
>  
>  #

That broke quite some tests for me (even though I really liked
to get rid of that if ;-)

Here is a patch that solves the first part of the absolute paths
problem (passes all tests; parts of the commit message shamelessly
copied from your proposal). Then another patch can tackle the
core.worktree config setting problem to make superprojects
relocatable gain.
---------8<--------
Subject: [PATCH] submodules: always use a relative path to gitdir

Recent versions of "git submodule" maintain the submodule <name> at
<path> in the superproject using a "separate git-dir" mechanism. The
repository data for the submodule is stored in ".git/modules/<name>/"
directory of the superproject, and its working tree is created at
"<path>/" directory, with "<path>/.git" file pointing at the
".git/modules/<name>/" directory.

This is so that we can check out an older version of the superproject
that does not yet have the submodule <name> anywhere without losing
(and later having to re-clone) the submodule repository. Removing
"<path>" won't lose ".git/modules/<name>", and a different branch that
has the submodule at different location in the superproject, say
"<path2>", can create "<path2>/" and ".git" in it to point at the same
".git/modules/<name>".

When instantiating such a submodule, if ".git/modules/<name>/" does
not exist in the superproject, the submodule repository needs to be
cloned there first. Then we only need to create "<path>" directory,
point ".git/modules/<name>/" in the superproject with "<path>/.git",
and check out the working tree.

However, the current code is not structured that way. The codepath to
deal with newly cloned submodules uses "git clone --separate-git-dir"
and creates "<path>" and "<path>/.git". This can make the resulting
submodule working tree at "<path>" different from the codepath for
existing submodules. An example of such differences is that this
codepath prepares "<path>/.git" with an absolute path, while the
normal codepath uses a relative path.

Fix the latter by always writing the relative path to the git directory
in "<path>/.git". To make that work, the 'name' variable has to be set to
the value of the 'path' variable for newly added submodules.

This is only the first step to make superprojects movable again like they
were before the separate-git-dir approach was introduced. The second step
must be to either use a relative path in core.worktree too or to get rid
of that setting by using a bare repo in "./git/modules/<name>".

While at it also replace an if/else construct evaluating the presence
of the 'reference' option with a single line of bash code.

Reported-by: Antony Male <antony.male@gmail.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
 git-submodule.sh |   12 +++++-------
 1 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 3adab93..2a93c61 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -131,6 +131,7 @@ module_clone()
 	gitdir=
 	gitdir_base=
 	name=$(module_name "$path" 2>/dev/null)
+	test -n "$name" || name="$path"
 	base_path=$(dirname "$path")

 	gitdir=$(git rev-parse --git-dir)
@@ -159,18 +160,15 @@ module_clone()
 	if test -d "$gitdir"
 	then
 		mkdir -p "$path"
-		echo "gitdir: $rel_gitdir" >"$path/.git"
 		rm -f "$gitdir/index"
 	else
 		mkdir -p "$gitdir_base"
-		if test -n "$reference"
-		then
-			git-clone $quiet "$reference" -n "$url" "$path" --separate-git-dir "$gitdir"
-		else
-			git-clone $quiet -n "$url" "$path" --separate-git-dir "$gitdir"
-		fi ||
+		git clone $quiet -n ${reference:+"$reference"} \
+			--separate-git-dir "$gitdir" "$url" "$path" ||
 		die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
 	fi
+
+	echo "gitdir: $rel_gitdir" >"$path/.git"
 }

 #
-- 
1.7.8.2.303.g78a27

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Am 29.12.2011 23:40, schrieb Junio C Hamano:
>> Antony Male <antony.male@gmail.com> writes:
>> I further wonder if we can get away without using separate-git-dir option
>> in this codepath, though. IOW using
>> 
>>         git clone $quiet -bare ${reference:+"$reference"} "$url" "$gitdir"
>> 
>> might be a better solution.
>
> A quick test shows that using a bare repo won't fly because without the
> core.worktree setting commands that operate on the work tree can't be
> run anymore inside submodules (starting with the initial checkout). 

Probably the right thing to do would be to restructure the flow as I
suggested, i.e.

	if we do not have it yet
        then
        	git clone --bare ...
	fi
	# now we have it, make sure they are correct
	git config core.bare false
	git config core.worktree $there
        echo "gitdir: $here" >$there/.git

> Yes, and the core.worktree setting also contains an absolute path. So
> we must either make that relative too and rewrite it on every "git
> submodule add" to record the possibly changed path there or make the
> bare clone work with a work tree (which sounds a bit strange ;-).

Update of core.worktree has to be done regardless of the absolute/relative
differences anyway, no?

The first version of the superproject you trigger module_clone for
submodule $name may happen to have it at $path, module_clone notices that
you do not have it, and the initial "clone --separate-git-dir" will set
the core.worktree to $superproject/$path.  Nobody will update it after
that, even when we check out different version of superproject that has
the same submodule $name at a different location in the superproject.

Re: [PATCH] Submodules always use a relative path to gitdir

[Jens Lehmann]

Am 03.01.2012 19:27, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
>> Am 29.12.2011 23:40, schrieb Junio C Hamano:
>>> I further wonder if we can get away without using separate-git-dir option
>>> in this codepath, though. IOW using
>>>
>>>         git clone $quiet -bare ${reference:+"$reference"} "$url" "$gitdir"
>>>
>>> might be a better solution.
>>
>> A quick test shows that using a bare repo won't fly because without the
>> core.worktree setting commands that operate on the work tree can't be
>> run anymore inside submodules (starting with the initial checkout). 
> 
> Probably the right thing to do would be to restructure the flow as I
> suggested, i.e.
> 
> 	if we do not have it yet
>         then
>         	git clone --bare ...
> 	fi
> 	# now we have it, make sure they are correct
> 	git config core.bare false

Ah, I forgot to set core.bare to false when trying this. But even then
a dozen tests fail, no matter if I set core.worktree or not. A cursory
glance indicates problems with branches ... I'll have to dig deeper
here.

> 	git config core.worktree $there

Please see below.

>         echo "gitdir: $here" >$there/.git
> 
>> Yes, and the core.worktree setting also contains an absolute path. So
>> we must either make that relative too and rewrite it on every "git
>> submodule add" to record the possibly changed path there or make the
>> bare clone work with a work tree (which sounds a bit strange ;-).
> 
> Update of core.worktree has to be done regardless of the absolute/relative
> differences anyway, no?

Not if we would implement a "if no worktree is set but we came here via
a gitfile, then take the directory the gitfile was found in as worktree"
heuristic. And that heuristic looks quite sane to me, as a gitfile can
only be found in a work tree, or am I missing something obvious here?

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Not if we would implement a "if no worktree is set but we came here via
> a gitfile, then take the directory the gitfile was found in as worktree"
> heuristic. And that heuristic looks quite sane to me, as a gitfile can
> only be found in a work tree, or am I missing something obvious here?

Like it wouldn't work without changes to the core side?

Re: [PATCH] Submodules always use a relative path to gitdir

[Jens Lehmann]

Am 03.01.2012 23:17, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Not if we would implement a "if no worktree is set but we came here via
>> a gitfile, then take the directory the gitfile was found in as worktree"
>> heuristic. And that heuristic looks quite sane to me, as a gitfile can
>> only be found in a work tree, or am I missing something obvious here?
> 
> Like it wouldn't work without changes to the core side?

I totally agree that when just talking about being able to move the
superproject around that approach is more invasive than just adding
a relative core.worktree setting and is just not worth the hassle.

But I was also thinking about moving the submodule around inside the
superproject. Until the gitfile was used that meant just mv'ing the
submodule and changing the path in .gitmodules accordingly. Now you
also have to adjust the core.worktree setting and maybe also the
gitfile content (if you move the submodule out of the directory level
it lived in before).

One solution I can think of is to teach "git mv" about submodules and
let it do the necessary changes to .gitmodules (which seems to be a
good idea anyways), core.worktree and the gitfile. The manipulation of
core.worktree could be obsoleted by not using that setting but instead
implementing the heuristic I described above. And if the gitfile could
be taught somehow that a path in there is relative to the superprojects
root directory, then it would never have to be changed either, restoring
the behavior we had before introducing the gitfile.

So in the long run I suspect we might have to change core git anyways
to make moving submodules easy for the user (surely "git mv" and maybe
also the setup and gitfile code). Does that make more sense?

If not I'm fine with just setting core.worktree to a relative path in
the git-submodule.sh script (like I did for the gitfile). And I'll look
into teaching "git mv" about submodules right after that.

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> So in the long run I suspect we might have to change core git anyways
> to make moving submodules easy for the user (surely "git mv" and maybe
> also the setup and gitfile code). Does that make more sense?

If you need to change "git mv" anyway to help moving submodule checkout,
then how gitfile points into .git/modules/ hierarchy of the superproject
becomes an implementation detail the end users should not have to care
about.

What does "if we reached thru a gitfile, then the working tree is where
you found that gitfile" really solve? The way you found that gitfile is by
traversing the directory hierarchy upwards from a subdirectory of a
working tree of a submodule, and you already know where the top of that
working tree is, no?

And the heuristics would not work if somebody goes into the $GIT_DIR/ that
governs the submodule as going upwards from there will not hit gitfile, so
we would need help from core.worktree anyway. A non-submodule setting that
uses gitfile would need to worry about core.worktree, too, so I'd rather
avoid loading more heuristics to gitfile handling unless there is a clear
advantage for doing so, which I am not really seeing here.

That is not really a "If not" below (i.e. I am not saying it is _not_ OK.
I am saying I don't know what the advantage of that approach is), but ...

> If not I'm fine with just setting core.worktree to a relative path in
> the git-submodule.sh script (like I did for the gitfile). And I'll look
> into teaching "git mv" about submodules right after that.

... teaching "git mv" may be a good move, I would think. I do think keeping
core.worktree pointing at the right directory is necessary, but I do not
see much point in making it a relative path, though.

Re: [PATCH] Submodules always use a relative path to gitdir

[Phil Hord]

On Thu, Jan 5, 2012 at 7:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
>> If not I'm fine with just setting core.worktree to a relative path in
>> the git-submodule.sh script (like I did for the gitfile). And I'll look
>> into teaching "git mv" about submodules right after that.
>
> ... teaching "git mv" may be a good move, I would think. I do think keeping
> core.worktree pointing at the right directory is necessary, but I do not
> see much point in making it a relative path, though.

I do, in the case of submodules, as already discussed.

Do you see any _problem_ with making core.worktree a relative
directory in the specific case of git submodules?

Phil

Re: [PATCH] Submodules always use a relative path to gitdir

[Nguyen Thai Ngoc Duy]

On Fri, Jan 6, 2012 at 9:26 PM, Phil Hord <phil.hord@gmail.com> wrote:
> Do you see any _problem_ with making core.worktree a relative
> directory in the specific case of git submodules?

Not a problem per se, but you should look at the comment at the top of
t1510 to see where it is relative to. Two interesting rules:

2. .git file is relative to parent directory. .git file is basically
   symlink in disguise. The directory where .git file points to will
   become new git_dir.

3. core.worktree is relative to git_dir.
-- 
Duy

Re: [PATCH] Submodules always use a relative path to gitdir

[Junio C Hamano]

Phil Hord <phil.hord@gmail.com> writes:

> On Thu, Jan 5, 2012 at 7:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>> If not I'm fine with just setting core.worktree to a relative path in
>>> the git-submodule.sh script (like I did for the gitfile). And I'll look
>>> into teaching "git mv" about submodules right after that.
>>
>> ... teaching "git mv" may be a good move, I would think. I do think keeping
>> core.worktree pointing at the right directory is necessary, but I do not
>> see much point in making it a relative path, though.
>
> I do, in the case of submodules, as already discussed.

Of course you are right.

Re: [PATCH] Submodules always use a relative path to gitdir

[Fredrik Gustafsson]

2011/12/29 Antony Male <antony.male@gmail.com>:
<snip>
>                die "$(eval_gettext "Clone of '\$url' into submodule path '\$path' failed")"
>        fi
> +       echo "gitdir: $rel_gitdir" >"$path/.git"

This will replace an already created file. Is it really the best
solution to create a gitfile with an absolute path and after that
replace it with a relative path. Why not write the relative path from
the beginning?

The patch also breaks two tests:
t7406 and t5526.

Regards
Fredrik

Re: [PATCH] Submodules always use a relative path to gitdir

[Phil Hord]

On Thu, Dec 29, 2011 at 4:00 PM, Antony Male <antony.male@gmail.com> wrote:
> This fixes a problem where moving a git repository with checked-out
> submodules would cause a fatal error when commands such as 'git
> submodule update' were run.

Thanks.  I noticed this itch when looking at git-files a few months
ago.  It bothered me, but not enough to fix it;  just enough to note
it as a problem area to avoid in the future.

> An alternative patch would teach git-clone an option to control whether
> an absolute or relative path is used when --separate-git-dir is passed.

I think I like this option better. Did you look at what it would take?

Phil