[PATCH 0/3] nd/multiple-work-trees updates

[PATCH 0/3] nd/multiple-work-trees updates

[Nguyễn Thái Ngọc Duy]

These patches are on top of what's in 'pu'. They add
--ignore-other-worktrees and make a note about current submodule
support status. I don't think submodule support is ready yet even
with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
though, so only 01/03 and 02/03 are new.

[1] http://thread.gmane.org/gmane.comp.version-control.git/261107

Nguyễn Thái Ngọc Duy (3):
  checkout: pass whole struct to parse_branchname_arg instead of individual flags
  checkout: add --ignore-other-wortrees
  git-checkout.txt: a note about multiple checkout support for submodules

 Documentation/git-checkout.txt |  9 +++++++++
 builtin/checkout.c             | 19 +++++++++++--------
 t/t2025-checkout-to.sh         |  7 +++++++
 3 files changed, 27 insertions(+), 8 deletions(-)

-- 
2.2.0.84.ge9c7a8a

[PATCH 1/3] checkout: pass whole struct to parse_branchname_arg instead of individual flags

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 953b763..d8717ef 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1079,11 +1079,12 @@ static void check_linked_checkouts(struct branch_info *new)
 static int parse_branchname_arg(int argc, const char **argv,
 				int dwim_new_local_branch_ok,
 				struct branch_info *new,
-				struct tree **source_tree,
-				unsigned char rev[20],
-				const char **new_branch,
-				int force_detach)
+				struct checkout_opts *opts,
+				unsigned char rev[20])
 {
+	struct tree **source_tree = &opts->source_tree;
+	const char **new_branch = &opts->new_branch;
+	int force_detach = opts->force_detach;
 	int argcount = 0;
 	unsigned char branch_rev[20];
 	const char *arg;
@@ -1420,9 +1421,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			opts.track == BRANCH_TRACK_UNSPECIFIED &&
 			!opts.new_branch;
 		int n = parse_branchname_arg(argc, argv, dwim_ok,
-					     &new, &opts.source_tree,
-					     rev, &opts.new_branch,
-					     opts.force_detach);
+					     &new, &opts, rev);
 		argv += n;
 		argc -= n;
 	}
-- 
2.2.0.84.ge9c7a8a

[PATCH 2/3] checkout: add --ignore-other-wortrees

[Nguyễn Thái Ngọc Duy]

Noticed-by: Mark Levedahl <mlevedahl@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt | 6 ++++++
 builtin/checkout.c             | 6 +++++-
 t/t2025-checkout-to.sh         | 7 +++++++
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 0c13825..52eaa48 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -232,6 +232,12 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	specific files such as HEAD, index... See "MULTIPLE WORKING
 	TREES" section for more information.
 
+--ignore-other-worktrees::
+	`git checkout` refuses when the wanted ref is already checked
+	out by another worktree. This option makes it check the ref
+	out anyway. In other words, the ref can be held by more than one
+	worktree.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/builtin/checkout.c b/builtin/checkout.c
index d8717ef..8b2bf20 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
 	int writeout_stage;
 	int overwrite_ignore;
 	int ignore_skipworktree;
+	int ignore_other_worktrees;
 
 	const char *new_branch;
 	const char *new_branch_force;
@@ -1210,7 +1211,8 @@ static int parse_branchname_arg(int argc, const char **argv,
 		int flag;
 		char *head_ref = resolve_refdup("HEAD", 0, sha1, &flag);
 		if (head_ref &&
-		    (!(flag & REF_ISSYMREF) || strcmp(head_ref, new->path)))
+		    (!(flag & REF_ISSYMREF) || strcmp(head_ref, new->path)) &&
+		    !opts->ignore_other_worktrees)
 			check_linked_checkouts(new);
 		free(head_ref);
 	}
@@ -1341,6 +1343,8 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 				N_("second guess 'git checkout no-such-branch'")),
 		OPT_FILENAME(0, "to", &opts.new_worktree,
 			   N_("check a branch out in a separate working directory")),
+		OPT_BOOL(0, "ignore-other-worktrees", &opts.ignore_other_worktrees,
+			 N_("do not check if another worktree is holding the given ref")),
 		OPT_END(),
 	};
 
diff --git a/t/t2025-checkout-to.sh b/t/t2025-checkout-to.sh
index 915b506..f8e4df4 100755
--- a/t/t2025-checkout-to.sh
+++ b/t/t2025-checkout-to.sh
@@ -79,6 +79,13 @@ test_expect_success 'die the same branch is already checked out' '
 	)
 '
 
+test_expect_success 'not die the same branch is already checked out' '
+	(
+		cd here &&
+		git checkout --ignore-other-worktrees --to anothernewmaster newmaster
+	)
+'
+
 test_expect_success 'not die on re-checking out current branch' '
 	(
 		cd there &&
-- 
2.2.0.84.ge9c7a8a

Re: [PATCH 2/3] checkout: add --ignore-other-wortrees

[Mark Levedahl]

On 01/03/2015 04:41 AM, Nguyễn Thái Ngọc Duy wrote:
> Noticed-by: Mark Levedahl <mlevedahl@gmail.com>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>   Documentation/git-checkout.txt | 6 ++++++
>   builtin/checkout.c             | 6 +++++-
>   t/t2025-checkout-to.sh         | 7 +++++++
>   3 files changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> index 0c13825..52eaa48 100644
> --- a/Documentation/git-checkout.txt
> +++ b/Documentation/git-checkout.txt
> @@ -232,6 +232,12 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
>   	specific files such as HEAD, index... See "MULTIPLE WORKING
>   	TREES" section for more information.
>   
> +--ignore-other-worktrees::
> +	`git checkout` refuses when the wanted ref is already checked
> +	out by another worktree. This option makes it check the ref
> +	out anyway. In other words, the ref can be held by more than one
> +	worktree.
> +
>
Thanks for adding this, I haven't had a chance to test but by reading 
this solves the problem I raised.

Mark

[PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

[Nguyễn Thái Ngọc Duy]

The goal seems to be using multiple checkouts to reduce disk space.
But we have not reached an agreement how things should be. There are a
couple options.

 - You may want to keep $SUB repos elsewhere (perhaps in a central
   place) outside $SUPER. This is also true for nested submodules
   where a superproject may be a submodule of another superproject.

 - You may want to keep all $SUB repos in $SUPER/modules (or some
   other place in $SUPER)

 - We could even push it further and merge all $SUB repos into $SUPER
   instead of storing them separately. But that would at least require
   ref namespace enabled.

On top of that, git-submodule.sh expects $GIT_DIR/config to be
per-worktree, at least for the submodule.* part. Here I think we have
two options, either update config.c to also read
$GIT_DIR/config.worktree (which is per worktree) in addition to
$GIT_DIR/config (shared) and store worktree-specific vars in the new
place, or update git-submodule.sh to read/write submodule.* directly
from $GIT_DIR/config.submodule (per worktree).

These take time to address properly. Meanwhile, make a note to the
user that they should not use multiple worktrees in submodule context.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 52eaa48..72def5b 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -463,6 +463,9 @@ to `/path/main/.git/worktrees/test-next` then a file named
 `test-next` entry from being pruned.  See
 linkgit:gitrepository-layout[5] for details.
 
+Multiple checkout support for submodules is incomplete. It is NOT
+recommended to make multiple checkouts of a superproject.
+
 EXAMPLES
 --------
 
-- 
2.2.0.84.ge9c7a8a

Re: [PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

[Mark Levedahl]

On 01/03/2015 04:41 AM, Nguyễn Thái Ngọc Duy wrote:
> The goal seems to be using multiple checkouts to reduce disk space.
> But we have not reached an agreement how things should be. There are a
> couple options.
>
>   - You may want to keep $SUB repos elsewhere (perhaps in a central
>     place) outside $SUPER. This is also true for nested submodules
>     where a superproject may be a submodule of another superproject.
>
>
This is my preference: I keep a tree of bare git repos outside of all 
work areas, and use new-workdir to create trees of workdirs as needed. I 
explored trying to keep $SUB repos in others (including mods to 
submodule / new-workdir to manage this), found this really leads to too 
much complication compared to just having a set of bare repos elsewhere. 
This bare repo approach also has the advantage that no particular 
workdir is special, all workdirs that point to the same gitdir are equal.

Mark

Re: [PATCH 3/3] git-checkout.txt: a note about multiple checkout support for submodules

[Max Kirillov]

On Sat, Jan 03, 2015 at 04:41:27PM +0700, Nguyễn Thái Ngọc Duy wrote:
> The goal seems to be using multiple checkouts to reduce disk space.
> But we have not reached an agreement how things should be. There are a
> couple options.
> 
>  - You may want to keep $SUB repos elsewhere (perhaps in a central
>    place) outside $SUPER. This is also true for nested submodules
>    where a superproject may be a submodule of another superproject.
> 
>  - You may want to keep all $SUB repos in $SUPER/modules (or some
>    other place in $SUPER)
> 
>  - We could even push it further and merge all $SUB repos into $SUPER
>    instead of storing them separately. But that would at least require
>    ref namespace enabled.
> 
> On top of that, git-submodule.sh expects $GIT_DIR/config to be
> per-worktree, at least for the submodule.* part. Here I think we have
> two options, either update config.c to also read
> $GIT_DIR/config.worktree (which is per worktree) in addition to
> $GIT_DIR/config (shared) and store worktree-specific vars in the new
> place, or update git-submodule.sh to read/write submodule.* directly
> from $GIT_DIR/config.submodule (per worktree).
> 
> These take time to address properly. Meanwhile, make a note to the
> user that they should not use multiple worktrees in submodule context.

I'd like to describe though how much the glass is full:

If all submodules exist in same paths in all checked-out
commits, and there is no special settings, only
`submodule.<name>.url` is set, then user can inspect, commit
and update the submodule content. Submodules can be either
independedntly initalized with `submodule update --init`
(being a fully independet clone), or checked-out with
`checkout --to` to the submodule worktree.

It is even verified by tests, so does it worth mentioning in
some manpage or at least commit message?

-- 
Max

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> These patches are on top of what's in 'pu'. They add
> --ignore-other-worktrees and make a note about current submodule
> support status. I don't think submodule support is ready yet even
> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> though, so only 01/03 and 02/03 are new.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107

With the understanding (perhaps a strongly-worded paragraph in the
release notes) that this is not suitable for submodule users yet,
is this in a good enough shape to go to 'next'?

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Jens Lehmann]

Am 13.02.2015 um 11:57 schrieb Junio C Hamano:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> These patches are on top of what's in 'pu'. They add
>> --ignore-other-worktrees and make a note about current submodule
>> support status. I don't think submodule support is ready yet even
>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>> though, so only 01/03 and 02/03 are new.
>>
>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?

No objections from my side (and maybe we should also add a warning
that *all* worktree-related configuration - e.g. EOL options - are
currently always shared between all worktrees).

Adding submodule support can then be done in another series (and
renaming core.worktree to something else is definitely *not* the
way to do that! ;-).

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Mark Levedahl]

On 02/12/2015 06:51 PM, Jens Lehmann wrote:
> Am 13.02.2015 um 11:57 schrieb Junio C Hamano:
>> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
>>
>>> These patches are on top of what's in 'pu'. They add
>>> --ignore-other-worktrees and make a note about current submodule
>>> support status. I don't think submodule support is ready yet even
>>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>>> though, so only 01/03 and 02/03 are new.
>>>
>>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>>
>> With the understanding (perhaps a strongly-worded paragraph in the
>> release notes) that this is not suitable for submodule users yet,
>> is this in a good enough shape to go to 'next'?
>
> No objections from my side (and maybe we should also add a warning
> that *all* worktree-related configuration - e.g. EOL options - are
> currently always shared between all worktrees).
>
> Adding submodule support can then be done in another series (and
> renaming core.worktree to something else is definitely *not* the
> way to do that! ;-).
>
I concur the patch series is good enough for next. Better multiple 
worktree support for submodules is, I think, a sizeable topic that will 
take a while to settle, so should be worked after this base is integrated.

Mark

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Dennis Kaarsemaker]

On do, 2015-02-12 at 14:57 -0800, Junio C Hamano wrote:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> 
> > These patches are on top of what's in 'pu'. They add
> > --ignore-other-worktrees and make a note about current submodule
> > support status. I don't think submodule support is ready yet even
> > with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> > though, so only 01/03 and 02/03 are new.
> >
> > [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
> 
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?

I've been using this for a while and really like it. However, it needs a
few fixups to merge with next as there are a few merge conflicts.

(A version of this branch that I stuck on top of next last week can be
found at https://github.com/seveas/git/tree/nd/multiple-work-trees )
-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Duy Nguyen]

On Fri, Feb 13, 2015 at 5:57 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> These patches are on top of what's in 'pu'. They add
>> --ignore-other-worktrees and make a note about current submodule
>> support status. I don't think submodule support is ready yet even
>> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
>> though, so only 01/03 and 02/03 are new.
>>
>> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> With the understanding (perhaps a strongly-worded paragraph in the
> release notes) that this is not suitable for submodule users yet,
> is this in a good enough shape to go to 'next'?

I'm not aware of any problems on this series (except submodules). So, yes.
-- 
Duy

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Ephrim Khong]

Without having looked into this and nd/multiple-work-trees, but with 
"make multiple checkouts aware of each other" in mind: Could this 
mechanism be re-used to make alternates aware of each other, to mitigate 
the dangers of having  git gc  on an alternate remove objects that are 
used by a referencing repository?

Thanks
- Eph

On 03.01.2015 10:41, Nguyễn Thái Ngọc Duy wrote:
> These patches are on top of what's in 'pu'. They add
> --ignore-other-worktrees and make a note about current submodule
> support status. I don't think submodule support is ready yet even
> with Max Kirillov's series [1]. His 03/03 is already fixed in 'pu'
> though, so only 01/03 and 02/03 are new.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/261107
>
> Nguyễn Thái Ngọc Duy (3):
>    checkout: pass whole struct to parse_branchname_arg instead of individual flags
>    checkout: add --ignore-other-wortrees
>    git-checkout.txt: a note about multiple checkout support for submodules
>
>   Documentation/git-checkout.txt |  9 +++++++++
>   builtin/checkout.c             | 19 +++++++++++--------
>   t/t2025-checkout-to.sh         |  7 +++++++
>   3 files changed, 27 insertions(+), 8 deletions(-)
>

Re: [PATCH 0/3] nd/multiple-work-trees updates

[Duy Nguyen]

On Wed, Mar 18, 2015 at 3:04 PM, Ephrim Khong <dr.khong@gmail.com> wrote:
> Without having looked into this and nd/multiple-work-trees, but with "make
> multiple checkouts aware of each other" in mind: Could this mechanism be
> re-used to make alternates aware of each other, to mitigate the dangers of
> having  git gc  on an alternate remove objects that are used by a
> referencing repository?

If we can turn on ref namespace and make $GIT_DIR/config and hooks per
worktree, I think it may have a chance of replacing alternate object
mechanism entirely: one object database, one ref database (but refs of
each worktree is namespaced so no conflicts), multiple worktrees,
multiple config files, multiple hooks.

Because some config keys affect object database, having
multiple/conflicting config keys imply that this worktree may change
object database in a way trhat  impacts performance (not correctness)
of another worktree. Later on when we have multiple ref backends, if
config keys can change ref backend behavior (or even choose the
backend), we may run into other problems. This problem might go away
if we define that those "global" config keys can't be per-worktree..

In short, I am good at confusing people.
-- 
Duy