Problems with new location of submodule's GIT_DIR

Problems with new location of submodule's GIT_DIR

[Daniel Milde]

Hi all,
I have encountered some difficulties with the new location of the
metainformation directory (GIT_DIR) of submodules.

The change of location happened in 1.7.8:

 * When populating a new submodule directory with "git submodule init",
   the $GIT_DIR metainformation directory for submodules is created
inside
   $GIT_DIR/modules/<name>/ directory of the superproject and referenced
   via the gitfile mechanism. This is to make it possible to switch
   between commits in the superproject that has and does not have the
   submodule in the tree without re-cloning.

Which problems it causes:

1) You cannot move submodule easily (change the path to submodule). You
can either remove the submodule and then add it again to different path,
or move the submodule directory and then edit path in its .git, change
path in superproject's .git/modules dir and in .git/config. This is
quite difficult in comparisson with previous behaviour when you can just
move the submodule and then stash changes.

2) This change introduced many difficulties on Windows platform, because
there is limitation to path length and the way how paths are now
composed leads to strange errors when project with submodules is in
user's dir. 

3) Submodules became dependent on superproject. This is quite bad if you
are using submodules a lot (e.g. versioning the whole working space),
because you cannot move or copy them out of the superproject now.

I honestly don't know why this change was introduced because I have
never seen git removing any submodule while walking through the history.
I have seen only the message saying that directory is not empty and so
git will not remove it. Or is there another reason why this change was
added to git?

Thanks

Cheers
Daniel

Re: Problems with new location of submodule's GIT_DIR

[Jens Lehmann]

Am 17.08.2012 17:28, schrieb Daniel Milde:
> Hi all,
> I have encountered some difficulties with the new location of the
> metainformation directory (GIT_DIR) of submodules.
> 
> The change of location happened in 1.7.8:
> 
>  * When populating a new submodule directory with "git submodule init",
>    the $GIT_DIR metainformation directory for submodules is created
> inside
>    $GIT_DIR/modules/<name>/ directory of the superproject and referenced
>    via the gitfile mechanism. This is to make it possible to switch
>    between commits in the superproject that has and does not have the
>    submodule in the tree without re-cloning.
> 
> Which problems it causes:
> 
> 1) You cannot move submodule easily (change the path to submodule). You
> can either remove the submodule and then add it again to different path,
> or move the submodule directory and then edit path in its .git, change
> path in superproject's .git/modules dir and in .git/config. This is
> quite difficult in comparisson with previous behaviour when you can just
> move the submodule and then stash changes.

Unfortunately that is currently the case. Plans are to have something
like a "git submodule mv", which will do all the magic and make moving
submodules easy again.

> 2) This change introduced many difficulties on Windows platform, because
> there is limitation to path length and the way how paths are now
> composed leads to strange errors when project with submodules is in
> user's dir. 

Hmm, this sounds more like the problems we had with "c:" and "/c",
which are fixed in current git. What version are you using?

> 3) Submodules became dependent on superproject. This is quite bad if you
> are using submodules a lot (e.g. versioning the whole working space),
> because you cannot move or copy them out of the superproject now.

Okay, that use case is new to me. What is the intention when you move a
submodule out of the work tree? Couldn't you just as easily clone the
upstream repo again at the new place?

> I honestly don't know why this change was introduced because I have
> never seen git removing any submodule while walking through the history.
> I have seen only the message saying that directory is not empty and so
> git will not remove it. Or is there another reason why this change was
> added to git?

But that is just the same as when a submodules appears and you check
out a commit before the submodule was added. It should be removed but
currently isn't, which we can change in a future git version now the
git directory lives in a safe place. Other reasons are:

*) Converting a directory tree into a submodules (or the other way
   round) is currently virtually impossible to handle. But that'll
   work just fine in the near future, which will make a lot of users
   happy.

*) It is easy to accidentally throw away the submodules history and
   unpushed changes when removing it from the work tree (e.g. because
   you don't work on that part of the tree anymore), which should
   only contain tracked content and not repository data like it did.

So we did that change to make submodule handling easier and to be able
to support valid use cases which we couldn't. Sorry for the hassle,
but we believe it is worth it in the long run.

Re: Problems with new location of submodule's GIT_DIR

[Phil Hord]

On Fri, Aug 17, 2012 at 12:40 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 17.08.2012 17:28, schrieb Daniel Milde:
>> Hi all,
>> I have encountered some difficulties with the new location of the
>> metainformation directory (GIT_DIR) of submodules.
>>
>> The change of location happened in 1.7.8:
>>
>>  * When populating a new submodule directory with "git submodule init",
>>    the $GIT_DIR metainformation directory for submodules is created
>> inside
>>    $GIT_DIR/modules/<name>/ directory of the superproject and referenced
>>    via the gitfile mechanism. This is to make it possible to switch
>>    between commits in the superproject that has and does not have the
>>    submodule in the tree without re-cloning.
>>
>> Which problems it causes:

>> 3) Submodules became dependent on superproject. This is quite bad if you
>> are using submodules a lot (e.g. versioning the whole working space),
>> because you cannot move or copy them out of the superproject now.
>
> Okay, that use case is new to me. What is the intention when you move a
> submodule out of the work tree? Couldn't you just as easily clone the
> upstream repo again at the new place?

Not all git projects have upstreams.  Maybe this is a local project only.

In any case, cloning the upstream anew does not preserve local history.

Maybe the future 'git submodule mv' can be smart about moving a
submodule to a new location outside of a superproject.  Maybe it can
do so explicitly with the --export switch.

Phil