contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Antonin Hildebrand]

Hi there,

I'm just solving same problem as described here in the question:
http://stackoverflow.com/questions/4115817/duplicate-submodules-with-git

I wanted to try proposed solution, but git-new-workdir does not work
in latest release 1.7.10.

The problem are plaintext .git files pointing to the root
"superproject" .git directory. The script has not been updated to deal
with this new situation.

thanks,
Antonin

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Jens Lehmann]

Am 20.04.2012 22:16, schrieb Antonin Hildebrand:
> Hi there,
> 
> I'm just solving same problem as described here in the question:
> http://stackoverflow.com/questions/4115817/duplicate-submodules-with-git
> 
> I wanted to try proposed solution, but git-new-workdir does not work
> in latest release 1.7.10.
> 
> The problem are plaintext .git files pointing to the root
> "superproject" .git directory. The script has not been updated to deal
> with this new situation.

The problem is not the gitfile, but the core.worktree setting. And
I don't see how that script can be updated to deal with the new
situation. Since 1.7.8 the core.worktree setting is used to point
from the submodules .git directory to its work tree. And as
git-new-worktree just links the config file, it inherits the
core.worktree setting too, so that will always point to the first
work tree even when in the new workdir. And if git-new-worktree
would change core.worktree, the first work tree will stop working.

I'm not sure why git-new-workdir is used here anyways, the only reason
I can think of is to save some disk space. So use a regular submodule
there and everything should work for you (at the expense of having the
same object store on disk twice).

But it seems like a check is missing in the git-new-workdir script if
the core.worktree setting is used, as it will never do what it is meant
to when that is set and doesn't point to the target directory. Maybe
something like the patch below?

What I worry about is that after this change it will give users who
follow the above mentioned recipe a misleading advice, as following it
will make the first submodule stop working. But from the perspective of
git-new-workdir this change makes sense, and maybe we have to fix the
stackoverflow recipe instead ... on the other hand, problems like this
could be the avoided if we'd use the "if we reached thru a gitfile,
then the working tree is where you found that gitfile" setup logic
outlined in
http://permalink.gmane.org/gmane.comp.version-control.git/188007

Opinions?

----8<-----
Subject: [PATCH] git-new-workdir: Suggest unsetting the core.worktree setting

Linking to a repository that has the core.worktree option set can only
work when that core.worktree setting already points to the new-workdir.
In all other cases strange things will happen, as new-workdir will be
overridden by that setting.

So just die when that setting is found and tell the user why and that he
should remove it.

Reported-by: Antonin Hildebrand <antonin@hildebrand.cz>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
 contrib/workdir/git-new-workdir |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..90cc207 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -41,6 +41,14 @@ then
 		" remove from \"$git_dir/config\" to use $0"
 fi

+# don't link to a repository that has core.worktree defined
+coreworktree=$(git --git-dir="$git_dir" config --get core.worktree)
+if test -n "$coreworktree"
+then
+	die "\"$git_dir\" has core.worktree set to \"$coreworktree\"," \
+		" remove from \"$git_dir/config\" to use $0"
+fi
+
 # don't link to a workdir
 if test -h "$git_dir/config"
 then
-- 
1.7.10.208.gb5e6d

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Opinions?
>
> ----8<-----
> Subject: [PATCH] git-new-workdir: Suggest unsetting the core.worktree setting
>
> Linking to a repository that has the core.worktree option set can only
> work when that core.worktree setting already points to the new-workdir.
> In all other cases strange things will happen, as new-workdir will be
> overridden by that setting.

As you analyzed correctly, core.worktree lets a GIT_DIR to declare that
there is a single working tree associated with it. It fundamentally is
incompatible with new-workdir, which is a hack to let more than one
working tree associated with a single GIT_DIR.

I however do not think a simplistic "unset core.worktree" is a good
suggestion, though, as we do not know why the original repository has
that variable set pointing at somewhere.  Blindly removing it will break
the use of the original repository.  If somebody _really_ wants to use
new-workdir for whatever reason in such a setting, I would imagine that
doing something like this:

 - Create a new repository somewhere that is an identical copy of the
   original repository's GIT_DIR, except for core.worktree dropped;

 - Turn the working tree original repository pointed with core.worktree
   into a "new-workdir" off of that new repository; and

 - When you want more "new-workdir"s, create them off of that new copy.

may work.  But honestly speaking, "Do not use this hack---having more
than one working tree is fundamentally incompatible with it", may be a
more sensible message.

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Antonin Hildebrand]

Quite frankly, the discussion got pretty technical for me and I don't follow it.

I have just posted my current solution involving hard links instead
new-workdir script:
http://stackoverflow.com/a/10265084/84283

Also it links this discussion as a warning that new-workdir may not be
the right solution.

Anyways, thanks for your great job on git! Much appreciated.

On Sat, Apr 21, 2012 at 12:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
>
>> Opinions?
>>
>> ----8<-----
>> Subject: [PATCH] git-new-workdir: Suggest unsetting the core.worktree setting
>>
>> Linking to a repository that has the core.worktree option set can only
>> work when that core.worktree setting already points to the new-workdir.
>> In all other cases strange things will happen, as new-workdir will be
>> overridden by that setting.
>
> As you analyzed correctly, core.worktree lets a GIT_DIR to declare that
> there is a single working tree associated with it. It fundamentally is
> incompatible with new-workdir, which is a hack to let more than one
> working tree associated with a single GIT_DIR.
>
> I however do not think a simplistic "unset core.worktree" is a good
> suggestion, though, as we do not know why the original repository has
> that variable set pointing at somewhere.  Blindly removing it will break
> the use of the original repository.  If somebody _really_ wants to use
> new-workdir for whatever reason in such a setting, I would imagine that
> doing something like this:
>
>  - Create a new repository somewhere that is an identical copy of the
>   original repository's GIT_DIR, except for core.worktree dropped;
>
>  - Turn the working tree original repository pointed with core.worktree
>   into a "new-workdir" off of that new repository; and
>
>  - When you want more "new-workdir"s, create them off of that new copy.
>
> may work.  But honestly speaking, "Do not use this hack---having more
> than one working tree is fundamentally incompatible with it", may be a
> more sensible message.
>
>

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> As you analyzed correctly, core.worktree lets a GIT_DIR to declare that
> there is a single working tree associated with it. It fundamentally is
> incompatible with new-workdir, which is a hack to let more than one
> working tree associated with a single GIT_DIR.
>
> I however do not think a simplistic "unset core.worktree" is a good
> suggestion, though, as we do not know why the original repository has
> that variable set pointing at somewhere.  Blindly removing it will break
> the use of the original repository.  If somebody _really_ wants to use
> new-workdir for whatever reason in such a setting, I would imagine that
> doing something like this:
> ...
> may work.

I am too lazy to try it out myself, but a hack something along the line
of the attached patch _might_ turn out to work well.

At least, it gives an incentive to people to update to more recent
versions of git ;-)  I dunno.

-- >8 --
Subject: new-workdir: use its own config file

Instead of letting a new workdir share the same config, we simply
include the original config and override core.worktree in it.  This
obviously changes the behaviour from the traditional workdir, by making
any update to the config in a workdir private to that workdir and not
reflected back to the original repository.  Because a workdir is
supposed to be just a peek only window to check out a branch that is
different from the main working tree, and you are not expected to modify
the config file in any way (e.g. you do not create a new branch with
remote configuration in a workdir), it may not be a huge issue.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 contrib/workdir/git-new-workdir |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..1d49258 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -63,7 +63,7 @@ mkdir -p "$new_workdir/.git" || die "unable to create \"$new_workdir\"!"
 # create the links to the original repo.  explicitly exclude index, HEAD and
 # logs/HEAD from the list since they are purely related to the current working
 # directory, and should not be shared.
-for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache svn
+for x in refs logs/refs objects info hooks packed-refs remotes rr-cache svn
 do
 	case $x in
 	*/*)
@@ -77,6 +77,12 @@ done
 cd "$new_workdir"
 # copy the HEAD from the original repository as a default branch
 cp "$git_dir/HEAD" .git/HEAD
+cat >".git/config" <<EOF
+[include]
+	path = "$git_dir/config"
+[core]
+	worktree = "$(pwd)"
+EOF
 # checkout the branch (either the same as HEAD from the original repository, or
 # the one that was asked for)
 git checkout -f $branch

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Jens Lehmann]

Am 22.04.2012 06:41, schrieb Junio C Hamano:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> As you analyzed correctly, core.worktree lets a GIT_DIR to declare that
>> there is a single working tree associated with it. It fundamentally is
>> incompatible with new-workdir, which is a hack to let more than one
>> working tree associated with a single GIT_DIR.
>>
>> I however do not think a simplistic "unset core.worktree" is a good
>> suggestion, though, as we do not know why the original repository has
>> that variable set pointing at somewhere.  Blindly removing it will break
>> the use of the original repository.  If somebody _really_ wants to use
>> new-workdir for whatever reason in such a setting, I would imagine that
>> doing something like this:
>> ...
>> may work.
> 
> I am too lazy to try it out myself, but a hack something along the line
> of the attached patch _might_ turn out to work well.

A quick test worked just fine. Looks like you fixed git-new-workdir to
be usable with the new gitfile submodules ;-)

Re: contrib/workdir/git-new-workdir broken in 1.7.10 after introducing gitfiles

[Mark Levedahl]

On 04/22/2012 12:41 AM, Junio C Hamano wrote:
> Junio C Hamano<gitster@pobox.com>  writes:
>
>> As you analyzed correctly, core.worktree lets a GIT_DIR to declare that
>> there is a single working tree associated with it. It fundamentally is
>> incompatible with new-workdir, which is a hack to let more than one
>> working tree associated with a single GIT_DIR.
>>
>> I however do not think a simplistic "unset core.worktree" is a good
>> suggestion, though, as we do not know why the original repository has
>> that variable set pointing at somewhere.  Blindly removing it will break
>> the use of the original repository.  If somebody _really_ wants to use
>> new-workdir for whatever reason in such a setting, I would imagine that
>> doing something like this:
>> ...
>> may work.
>
> I am too lazy to try it out myself, but a hack something along the line
> of the attached patch _might_ turn out to work well.
>
> At least, it gives an incentive to people to update to more recent
> versions of git ;-)  I dunno.
>
> -- >8 --
> Subject: new-workdir: use its own config file
>
> Instead of letting a new workdir share the same config, we simply
> include the original config and override core.worktree in it.  This
> obviously changes the behaviour from the traditional workdir, by making
> any update to the config in a workdir private to that workdir and not
> reflected back to the original repository.  Because a workdir is
> supposed to be just a peek only window to check out a branch that is
> different from the main working tree, and you are not expected to modify
> the config file in any way (e.g. you do not create a new branch with
> remote configuration in a workdir), it may not be a huge issue.
>

This change will break my use of new-workdir, which is maintaining 
multiple checked out branches in separate directories, some with 
embedded modules, and being able to share changes and remote branches 
between them. These directories are time-consuming to set up, so the 
usually suggested approach of "git-checkout $other_branch" is not very 
useful. All of the workdirs are set up off of a common set of bare repos 
kept elsewhere.

There are other ways to share between multiple independent repositories 
on the same machine, but new-workdir is the simplest as there is no push 
/ pull / patch / am involved.

I think the "peek-only" use case is better supported by clone, which on 
a local machine does not copy the object store, and is of course 
well-documented and in core-git. Perhaps a better patch is to just 
suggest "clone" to the user who has core.worktree and/or submodules in use?

Mark