Git development
 help / color / mirror / Atom feed
* [RFC PATCHv2 16/17] completion: add '--recurse-submodules' to checkout
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 contrib/completion/git-completion.bash | 2 +-
 t/t9902-completion.sh                  | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 21016bf8df..28acfdb377 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1068,7 +1068,7 @@ _git_checkout ()
 	--*)
 		__gitcomp "
 			--quiet --ours --theirs --track --no-track --merge
-			--conflict= --orphan --patch
+			--conflict= --orphan --patch --recurse-submodules
 			"
 		;;
 	*)
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 2ba62fbc17..d2d1102a3d 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -447,6 +447,7 @@ test_expect_success 'double dash "git checkout"' '
 	--conflict=
 	--orphan Z
 	--patch Z
+	--recurse-submodules Z
 	EOF
 '
 
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 03/17] update submodules: move up prepare_submodule_repo_env
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

In a later patch we need to prepare the submodule environment with
another git directory, so split up the function.

Also move it up in the file such that we do not need to declare the
function later before using it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)

diff --git a/submodule.c b/submodule.c
index eb80b0c5ad..78b69b5a55 100644
--- a/submodule.c
+++ b/submodule.c
@@ -307,6 +307,22 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 	strbuf_release(&sb);
 }
 
+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)
+{
+	const char * const *var;
+
+	for (var = local_repo_env; *var; var++) {
+		if (strcmp(*var, CONFIG_DATA_ENVIRONMENT))
+			argv_array_push(out, *var);
+	}
+}
+
+void prepare_submodule_repo_env(struct argv_array *out)
+{
+	prepare_submodule_repo_env_no_git_dir(out);
+	argv_array_push(out, "GIT_DIR=.git");
+}
+
 /* Helper function to display the submodule header line prior to the full
  * summary output. If it can locate the submodule objects directory it will
  * attempt to lookup both the left and right commits and put them into the
@@ -1246,14 +1262,3 @@ int parallel_submodules(void)
 {
 	return parallel_jobs;
 }
-
-void prepare_submodule_repo_env(struct argv_array *out)
-{
-	const char * const *var;
-
-	for (var = local_repo_env; *var; var++) {
-		if (strcmp(*var, CONFIG_DATA_ENVIRONMENT))
-			argv_array_push(out, *var);
-	}
-	argv_array_push(out, "GIT_DIR=.git");
-}
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Allow checkout to recurse into submodules via
the command line option --[no-]recurse-submodules.

The flag for recurse-submodules in its current form
could be an OPT_BOOL, but eventually we may want to have
it as:

    git checkout --recurse-submodules=rebase|merge| \
			cherry-pick|checkout|none

which resembles the submodule.<name>.update options,
so naturally a value such as
"as-configured-or-X-as-fallback" would also come in handy.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-checkout.txt |   7 ++
 builtin/checkout.c             |  31 ++++-
 t/t2013-checkout-submodule.sh  | 277 +++++++++++++++++++++++++++++++++++++++--
 3 files changed, 306 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 8e2c0662dd..a0ea2c5651 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	out anyway. In other words, the ref can be held by more than one
 	worktree.
 
+--[no-]recurse-submodules::
+	Using --recurse-submodules will update the content of all initialized
+	submodules according to the commit recorded in the superproject. If
+	local modifications in a submodule would be overwritten the checkout
+	will fail until `-f` is used. If nothing (or --no-recurse-submodules)
+	is used, the work trees of submodules will not be updated.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 512492aad9..5db0d933d1 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -21,12 +21,31 @@
 #include "submodule-config.h"
 #include "submodule.h"
 
+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
+
 static const char * const checkout_usage[] = {
 	N_("git checkout [<options>] <branch>"),
 	N_("git checkout [<options>] [<branch>] -- <file>..."),
 	NULL,
 };
 
+int option_parse_recurse_submodules(const struct option *opt,
+				    const char *arg, int unset)
+{
+	if (unset) {
+		recurse_submodules = RECURSE_SUBMODULES_OFF;
+		return 0;
+	}
+	if (arg)
+		recurse_submodules =
+			parse_update_recurse_submodules_arg(opt->long_name,
+							    arg);
+	else
+		recurse_submodules = RECURSE_SUBMODULES_ON;
+
+	return 0;
+}
+
 struct checkout_opts {
 	int patch_mode;
 	int quiet;
@@ -826,7 +845,8 @@ static int switch_branches(const struct checkout_opts *opts,
 		parse_commit_or_die(new->commit);
 	}
 
-	ret = merge_working_tree(opts, &old, new, &writeout_error);
+	ret = merge_working_tree(opts, &old, new, &writeout_error) ||
+	      update_submodules(opts->force);
 	if (ret) {
 		free(path_to_free);
 		return ret;
@@ -1160,6 +1180,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 				N_("second guess 'git checkout <no-such-branch>'")),
 		OPT_BOOL(0, "ignore-other-worktrees", &opts.ignore_other_worktrees,
 			 N_("do not check if another worktree is holding the given ref")),
+		{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules,
+			    "checkout", "control recursive updating of submodules",
+			    PARSE_OPT_OPTARG, option_parse_recurse_submodules },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
 		OPT_END(),
 	};
@@ -1190,6 +1213,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
 	}
 
+	if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
+		git_config(submodule_config, NULL);
+		if (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)
+			set_config_update_recurse_submodules(recurse_submodules);
+	}
+
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));
 
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 6847f75822..33fb2f5de3 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -7,15 +7,21 @@ test_description='checkout can handle submodules'
 
 test_expect_success 'setup' '
 	mkdir submodule &&
-	(cd submodule &&
-	 git init &&
-	 test_commit first) &&
-	git add submodule &&
+	(
+		cd submodule &&
+		git init &&
+		test_commit first
+	) &&
+	echo first >file &&
+	git add file submodule &&
 	test_tick &&
 	git commit -m superproject &&
-	(cd submodule &&
-	 test_commit second) &&
-	git add submodule &&
+	(
+		cd submodule &&
+		test_commit second
+	) &&
+	echo second > file &&
+	git add file submodule &&
 	test_tick &&
 	git commit -m updated.superproject
 '
@@ -37,7 +43,8 @@ test_expect_success '"checkout <submodule>" updates the index only' '
 	git checkout HEAD^ submodule &&
 	test_must_fail git diff-files --quiet &&
 	git checkout HEAD submodule &&
-	git diff-files --quiet
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD
 '
 
 test_expect_success '"checkout <submodule>" honors diff.ignoreSubmodules' '
@@ -63,6 +70,260 @@ test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/
 	! test -s actual
 '
 
+submodule_creation_must_succeed() {
+	# checkout base ($1)
+	git checkout -f --recurse-submodules $1 &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached $1 &&
+
+	# checkout target ($2)
+	if test -d submodule; then
+		echo change >>submodule/first.t &&
+		test_must_fail git checkout --recurse-submodules $2 &&
+		git checkout -f --recurse-submodules $2
+	else
+		git checkout --recurse-submodules $2
+	fi &&
+	test -e submodule/.git &&
+	test -f submodule/first.t &&
+	test -f submodule/second.t &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached $2
+}
+
+test_expect_success 'setup the submodule config' '
+	git config -f .gitmodules submodule.submodule.path submodule &&
+	git config -f .gitmodules submodule.submodule.url ./submodule.bare &&
+	git -C submodule clone --bare . ../submodule.bare &&
+	echo submodule.bare >>.gitignore &&
+	git add .gitignore .gitmodules submodule &&
+	git commit -m "submodule registered with a gitmodules file" &&
+	git config submodule.submodule.url ./submodule.bare
+'
+
+test_expect_success '"checkout --recurse-submodules" migrates submodule git dir before deleting' '
+	git checkout -b base &&
+	git checkout -b delete_submodule &&
+	git update-index --force-remove submodule &&
+	git config -f .gitmodules --unset submodule.submodule.path &&
+	git config -f .gitmodules --unset submodule.submodule.url &&
+	git add .gitmodules &&
+	git commit -m "submodule deleted" &&
+	git checkout base &&
+	git checkout --recurse-submodules delete_submodule 2>output.err 1>output.out &&
+	test_i18ngrep "Migrating git directory" output.out &&
+	! test -d submodule
+'
+
+test_expect_success '"check --recurse-submodules" removes deleted submodule' '
+	# Make sure we have the submodule here and ready.
+	git checkout base &&
+	git submodule embedgitdirs &&
+	git submodule update -f . &&
+	test -e submodule/.git &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached base &&
+
+	# Check if the checkout deletes the submodule.
+	echo change >>submodule/first.t &&
+	test_must_fail git checkout --recurse-submodules delete_submodule &&
+	git checkout -f --recurse-submodules delete_submodule &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached delete_submodule &&
+	! test -d submodule
+'
+
+test_expect_success '"checkout --recurse-submodules" repopulates submodule' '
+	submodule_creation_must_succeed delete_submodule base
+'
+
+test_expect_success '"checkout --recurse-submodules" repopulates submodule in existing directory' '
+	git checkout --recurse-submodules delete_submodule &&
+	mkdir submodule &&
+	submodule_creation_must_succeed delete_submodule base
+'
+
+test_expect_success '"checkout --recurse-submodules" replaces submodule with files' '
+	git checkout -f base &&
+	git checkout -b replace_submodule_with_file &&
+	git rm -f submodule &&
+	echo "file instead" >submodule &&
+	git add submodule &&
+	git commit -m "submodule replaced" &&
+	git checkout -f base &&
+	git submodule update -f . &&
+	git checkout --recurse-submodules replace_submodule_with_file &&
+	test -f submodule
+'
+
+test_expect_success '"checkout --recurse-submodules" removes files and repopulates submodule' '
+	submodule_creation_must_succeed replace_submodule_with_file base
+'
+
+test_expect_success '"checkout --recurse-submodules" replaces submodule with a directory' '
+	git checkout -f base &&
+	git checkout -b replace_submodule_with_dir &&
+	git rm -f submodule &&
+	mkdir -p submodule/dir &&
+	echo content >submodule/dir/file &&
+	git add submodule &&
+	git commit -m "submodule replaced with a directory (file inside)" &&
+	git checkout -f base &&
+	git submodule update -f . &&
+	git checkout --recurse-submodules replace_submodule_with_dir &&
+	test -d submodule &&
+	! test -e submodule/.git &&
+	! test -f submodule/first.t &&
+	! test -f submodule/second.t &&
+	test -d submodule/dir
+'
+
+test_expect_success '"checkout --recurse-submodules" removes the directory and repopulates submodule' '
+	submodule_creation_must_succeed replace_submodule_with_dir base
+'
+
+test_expect_success SYMLINKS '"checkout --recurse-submodules" replaces submodule with a link' '
+	git checkout -f base &&
+	git checkout -b replace_submodule_with_link &&
+	git rm -f submodule &&
+	ln -s submodule &&
+	git add submodule &&
+	git commit -m "submodule replaced with a link" &&
+	git checkout -f base &&
+	git submodule update -f . &&
+	git checkout --recurse-submodules replace_submodule_with_link &&
+	test -L submodule
+'
+
+test_expect_success SYMLINKS '"checkout --recurse-submodules" removes the link and repopulates submodule' '
+	submodule_creation_must_succeed replace_submodule_with_link base
+'
+
+test_expect_success '"checkout --recurse-submodules" updates the submodule' '
+	git checkout --recurse-submodules base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD &&
+	git checkout -b updated_submodule &&
+	(
+		cd submodule &&
+		echo x >>first.t &&
+		git add first.t &&
+		test_commit third
+	) &&
+	git add submodule &&
+	test_tick &&
+	git commit -m updated.superproject &&
+	git checkout --recurse-submodules base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD
+'
+
+# In 293ab15eea34 we considered untracked ignored files in submodules
+# expendable, we may want to revisit this decision by adding user as
+# well as command specific configuration for it.
+# When building in-tree the untracked ignored files are probably ok to remove
+# in a case as tested here, but e.g. when git.git is a submodule, then a user
+# may not want to lose a well crafted (but ignored by default) "config.mak"
+# Commands like "git rm" may care less about untracked files in a submodule
+# as the checkout command that removes a submodule as well.
+test_expect_failure 'untracked file is not deleted' '
+	git checkout --recurse-submodules base &&
+	echo important >submodule/untracked &&
+	test_must_fail git checkout --recurse-submodules delete_submodule &&
+	git checkout -f --recurse-submodules delete_submodule
+'
+
+test_expect_success 'ignored file works just fine' '
+	git checkout --recurse-submodules base &&
+	echo important >submodule/ignored &&
+	echo ignored >.git/modules/submodule/info/exclude &&
+	git checkout --recurse-submodules delete_submodule
+'
+
+test_expect_success 'dirty file file is not deleted' '
+	git checkout --recurse-submodules base &&
+	echo important >submodule/first.t &&
+	test_must_fail git checkout --recurse-submodules delete_submodule &&
+	git checkout -f --recurse-submodules delete_submodule
+'
+
+test_expect_success 'added to index is not deleted' '
+	git checkout --recurse-submodules base &&
+	echo important >submodule/to_index &&
+	git -C submodule add to_index &&
+	test_must_fail git checkout --recurse-submodules delete_submodule &&
+	git checkout -f --recurse-submodules delete_submodule
+'
+
+# This is ok in theory, we just need to make sure
+# the garbage collection doesn't eat the commit.
+test_expect_success 'different commit prevents from deleting' '
+	git checkout --recurse-submodules base &&
+	echo important >submodule/to_index &&
+	git -C submodule add to_index &&
+	test_must_fail git checkout --recurse-submodules delete_submodule &&
+	git checkout -f --recurse-submodules delete_submodule
+'
+
+test_expect_failure '"checkout --recurse-submodules" needs -f to update a modifed submodule commit' '
+	git -C submodule checkout --recurse-submodules HEAD^ &&
+	test_must_fail git checkout --recurse-submodules master &&
+	test_must_fail git diff-files --quiet submodule &&
+	git diff-files --quiet file &&
+	git checkout --recurse-submodules -f master &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD
+'
+
+test_expect_failure '"checkout --recurse-submodules" needs -f to update modifed submodule content' '
+	echo modified >submodule/second.t &&
+	test_must_fail git checkout --recurse-submodules HEAD^ &&
+	test_must_fail git diff-files --quiet submodule &&
+	git diff-files --quiet file &&
+	git checkout --recurse-submodules -f HEAD^ &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD &&
+	git checkout --recurse-submodules -f master &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD
+'
+
+test_expect_failure '"checkout --recurse-submodules" ignores modified submodule content that would not be changed' '
+	echo modified >expected &&
+	cp expected submodule/first.t &&
+	git checkout --recurse-submodules HEAD^ &&
+	test_cmp expected submodule/first.t &&
+	test_must_fail git diff-files --quiet submodule &&
+	git diff-index --quiet --cached HEAD &&
+	git checkout --recurse-submodules -f master &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached HEAD
+'
+
+test_expect_failure '"checkout --recurse-submodules" does not care about untracked submodule content' '
+	echo untracked >submodule/untracked &&
+	git checkout --recurse-submodules master &&
+	git diff-files --quiet --ignore-submodules=untracked &&
+	git diff-index --quiet --cached HEAD &&
+	rm submodule/untracked
+'
+
+test_expect_failure '"checkout --recurse-submodules" needs -f when submodule commit is not present (but does fail anyway)' '
+	git checkout --recurse-submodules -b bogus_commit master &&
+	git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 submodule &&
+	BOGUS_TREE=$(git write-tree) &&
+	BOGUS_COMMIT=$(echo "bogus submodule commit" | git commit-tree $BOGUS_TREE) &&
+	git commit -m "bogus submodule commit" &&
+	git checkout --recurse-submodules -f master &&
+	test_must_fail git checkout --recurse-submodules bogus_commit &&
+	git diff-files --quiet &&
+	test_must_fail git checkout --recurse-submodules -f bogus_commit &&
+	test_must_fail git diff-files --quiet submodule &&
+	git diff-files --quiet file &&
+	git diff-index --quiet --cached HEAD &&
+	git checkout --recurse-submodules -f master
+'
+
 test_submodule_switch "git checkout"
 
 test_submodule_forced_switch "git checkout -f"
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 05/17] update submodules: add submodule config parsing
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Similar to b33a15b08 (push: add recurseSubmodules config option,
2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the
fetchRecurseSubmodules option, 2015-08-17), we add submodule-config code
that is later used to parse whether we are interested in updating
submodules.

We need the `die_on_error` parameter to be able to call this parsing
function for the config file as well, which if incorrect lets Git die.

As we're just touching the header file, also mark all functions extern.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule-config.c | 22 ++++++++++++++++++++++
 submodule-config.h | 17 +++++++++--------
 2 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/submodule-config.c b/submodule-config.c
index 098085be69..4b5297e31d 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)
 	return parse_fetch_recurse(opt, arg, 1);
 }
 
+static int parse_update_recurse(const char *opt, const char *arg,
+				int die_on_error)
+{
+	switch (git_config_maybe_bool(opt, arg)) {
+	case 1:
+		return RECURSE_SUBMODULES_ON;
+	case 0:
+		return RECURSE_SUBMODULES_OFF;
+	default:
+		if (!strcmp(arg, "checkout"))
+			return RECURSE_SUBMODULES_ON;
+		if (die_on_error)
+			die("bad %s argument: %s", opt, arg);
+		return RECURSE_SUBMODULES_ERROR;
+	}
+}
+
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)
+{
+	return parse_update_recurse(opt, arg, 1);
+}
+
 static int parse_push_recurse(const char *opt, const char *arg,
 			       int die_on_error)
 {
diff --git a/submodule-config.h b/submodule-config.h
index d05c542d2c..992bfbebc0 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -22,13 +22,14 @@ struct submodule {
 	int recommend_shallow;
 };
 
-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
-int parse_submodule_config_option(const char *var, const char *value);
-const struct submodule *submodule_from_name(const unsigned char *commit_sha1,
-		const char *name);
-const struct submodule *submodule_from_path(const unsigned char *commit_sha1,
-		const char *path);
-void submodule_free(void);
+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_submodule_config_option(const char *var, const char *value);
+extern const struct submodule *submodule_from_name(
+		const unsigned char *commit_sha1, const char *name);
+extern const struct submodule *submodule_from_path(
+		const unsigned char *commit_sha1, const char *path);
+extern void submodule_free(void);
 
 #endif /* SUBMODULE_CONFIG_H */
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 10/17] update submodules: is_submodule_checkout_safe
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

In later patches we introduce the options and flag for commands
that modify the working directory, e.g. git-checkout.

This piece of code will answer the question:
"Is it safe to change the submodule to this new state?"
e.g. is it overwriting untracked files or are there local
changes that would be overwritten?

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 37 +++++++++++++++++++++++++++++++++++++
 submodule.h |  2 ++
 2 files changed, 39 insertions(+)

diff --git a/submodule.c b/submodule.c
index 02c28ef56b..4253f7f044 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1155,6 +1155,43 @@ int ok_to_remove_submodule(const char *path)
 	return ok_to_remove;
 }
 
+/**
+ * Check if a submodule update to a given sha1 is safe.
+ * Return 1 if it is safe, 0 when it is not.
+ *
+ * If the submodule is not populated, we need to check
+ */
+int is_submodule_checkout_safe(const char *path,
+			       const struct object_id *new_hash)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_pushl(&cp.args, "read-tree", "-n", "-m", "HEAD",
+			sha1_to_hex(new_hash->hash), NULL);
+
+	if (!is_submodule_populated(path)) {
+		const struct submodule *sub;
+
+		/* See if we have the submodule configured already: */
+		sub = submodule_from_path(null_sha1, path);
+
+		prepare_submodule_repo_env_no_git_dir(&cp.env_array);
+		argv_array_pushf(&cp.env_array, "GIT_DIR=%s",
+				 sub ? sub->name : path);
+		argv_array_pushf(&cp.env_array, "GIT_WORK_TREE=%s", path);
+	} else {
+		prepare_submodule_repo_env(&cp.env_array);
+	}
+
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.no_stdout = 1;
+	cp.no_stderr = 1;
+	cp.dir = path;
+
+	return run_command(&cp) == 0;
+}
+
 static int find_first_merges(struct object_array *result, const char *path,
 		struct commit *a, struct commit *b)
 {
diff --git a/submodule.h b/submodule.h
index 74df8b93d5..1dfcd6939b 100644
--- a/submodule.h
+++ b/submodule.h
@@ -72,6 +72,8 @@ extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
 extern int is_submodule_populated(const char *path);
 extern int submodule_uses_gitfile(const char *path);
 extern int ok_to_remove_submodule(const char *path);
+extern int is_submodule_checkout_safe(const char *path,
+				      const struct object_id *new_hash);
 extern int merge_submodule(unsigned char result[20], const char *path,
 			   const unsigned char base[20],
 			   const unsigned char a[20],
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 04/17] update submodules: add is_submodule_populated
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

See if a submodule is populated by checking if there
is a .git file or directory at the given path.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 11 +++++++++++
 submodule.h |  1 +
 2 files changed, 12 insertions(+)

diff --git a/submodule.c b/submodule.c
index 78b69b5a55..c29153e9ff 100644
--- a/submodule.c
+++ b/submodule.c
@@ -930,6 +930,17 @@ int fetch_populated_submodules(const struct argv_array *options,
 	return spf.result;
 }
 
+int is_submodule_populated(const char *path)
+{
+	int retval = 0;
+	struct strbuf gitdir = STRBUF_INIT;
+	strbuf_addf(&gitdir, "%s/.git", path);
+	if (resolve_gitdir(gitdir.buf))
+		retval = 1;
+	strbuf_release(&gitdir);
+	return retval;
+}
+
 unsigned is_submodule_modified(const char *path, int ignore_untracked)
 {
 	ssize_t len;
diff --git a/submodule.h b/submodule.h
index 5db0b53b3f..a9eabcc3d0 100644
--- a/submodule.h
+++ b/submodule.h
@@ -58,6 +58,7 @@ extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
 			       int quiet, int max_parallel_jobs);
 extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
+extern int is_submodule_populated(const char *path);
 extern int submodule_uses_gitfile(const char *path);
 extern int ok_to_remove_submodule(const char *path);
 extern int merge_submodule(unsigned char result[20], const char *path,
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 07/17] update submodules: introduce submodule_is_interesting
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

In later patches we introduce the --recurse-submodule flag for commands
that modify the working directory, e.g. git-checkout.

It is potentially expensive to check if a submodule needs an update,
because a common theme to interact with submodules is to spawn a child
process for each interaction.

So let's introduce a function that pre checks if a submodule needs
to be checked for an update.

I am not particular happy with the name `submodule_is_interesting`,
in internal iterations I had `submodule_requires_check_for_update`
and `submodule_needs_update`, but I was even less happy with those
names. Maybe `submodule_interesting_for_update`?

Generally this is to answer "Am I allowed to touch the submodule
at all?" or: "Does the user expect me to touch it?"
which includes all of creation/deletion/update.

This patch is based off a prior attempt by Jens Lehmann to add
submodules to checkout.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 23 +++++++++++++++++++++++
 submodule.h |  9 +++++++++
 2 files changed, 32 insertions(+)

diff --git a/submodule.c b/submodule.c
index 1ba398ba3b..62e9ef3872 100644
--- a/submodule.c
+++ b/submodule.c
@@ -516,6 +516,29 @@ void set_config_update_recurse_submodules(int value)
 	config_update_recurse_submodules = value;
 }
 
+int submodules_interesting_for_update(void)
+{
+	/*
+	 * Update can't be "none", "merge" or "rebase",
+	 * treat any value as OFF, except an explicit ON.
+	 */
+	return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;
+}
+
+int submodule_is_interesting(const char *path)
+{
+	const struct submodule *sub;
+
+	if (!submodules_interesting_for_update())
+		return 0;
+
+	sub = submodule_from_path(null_sha1, path);
+	if (!sub)
+		return 0;
+
+	return sub->update_strategy.type != SM_UPDATE_NONE;
+}
+
 static int has_remote(const char *refname, const struct object_id *oid,
 		      int flags, void *cb_data)
 {
diff --git a/submodule.h b/submodule.h
index 21236b095c..7d890e0464 100644
--- a/submodule.h
+++ b/submodule.h
@@ -54,6 +54,15 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const struct diff_options *opt);
 extern void set_config_fetch_recurse_submodules(int value);
 extern void set_config_update_recurse_submodules(int value);
+
+/**
+ * When updating the working tree, we need to check if the submodule needs
+ * updating. We do not require a check if we are already sure that the
+ * submodule doesn't need updating, e.g. when we are not interested in submodules
+ * or the submodule is marked uninteresting by being not initialized.
+ */
+extern int submodule_is_interesting(const char *path);
+extern int submodules_interesting_for_update(void);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 08/17] update submodules: add depopulate_submodule
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Implement the functionality needed to enable work tree manipulating
commands so that a deleted submodule should not only affect the index
(leaving all the files of the submodule in the work tree) but also to
remove the work tree of the superproject (including any untracked
files).

To do so, we need an equivalent of "rm -rf", which is already found in
entry.c, so expose that and for clarity add a suffix "_or_dir" to it.

That will only work properly when the submodule uses a gitfile instead of
a .git directory and no untracked files are present. Otherwise the removal
will fail with a warning (which is just what happened until now).

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h     |  2 ++
 entry.c     |  5 +++++
 submodule.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h |  1 +
 4 files changed, 54 insertions(+)

diff --git a/cache.h b/cache.h
index a50a61a197..b645ca2f9a 100644
--- a/cache.h
+++ b/cache.h
@@ -2018,4 +2018,6 @@ void sleep_millisec(int millisec);
  */
 void safe_create_dir(const char *dir, int share);
 
+extern void remove_directory_or_die(struct strbuf *path);
+
 #endif /* CACHE_H */
diff --git a/entry.c b/entry.c
index c6eea240b6..02c4ac9f22 100644
--- a/entry.c
+++ b/entry.c
@@ -73,6 +73,11 @@ static void remove_subtree(struct strbuf *path)
 		die_errno("cannot rmdir '%s'", path->buf);
 }
 
+void remove_directory_or_die(struct strbuf *path)
+{
+	remove_subtree(path);
+}
+
 static int create_file(const char *path, unsigned int mode)
 {
 	mode = (mode & 0100) ? 0777 : 0666;
diff --git a/submodule.c b/submodule.c
index 62e9ef3872..7bb64d6c69 100644
--- a/submodule.c
+++ b/submodule.c
@@ -324,6 +324,52 @@ void prepare_submodule_repo_env(struct argv_array *out)
 	argv_array_push(out, "GIT_DIR=.git");
 }
 
+int depopulate_submodule(const char *path)
+{
+	int ret = 0;
+	struct strbuf pathbuf = STRBUF_INIT;
+	char *dot_git = xstrfmt("%s/.git", path);
+
+	/* Is it populated? */
+	if (!resolve_gitdir(dot_git))
+		goto out;
+
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+
+		prepare_submodule_repo_env(&cp.env_array);
+		argv_array_pushl(&cp.args, "submodule--helper",
+				 "embed-git-dirs", path, NULL);
+		cp.git_cmd = 1;
+		if (run_command(&cp)) {
+			warning(_("Cannot remove submodule '%s'\n"
+				  "because it (or one of its nested submodules) has a git \n"
+				  "directory in the working tree, which could not be embedded\n"
+				  "the superprojects git directory automatically."), path);
+			ret = -1;
+			goto out;
+		}
+
+		if (!submodule_uses_gitfile(path)) {
+			/*
+			 * We should be using a gitfile by now, let's double
+			 * check as loosing the git dir would be fatal.
+			 */
+			die("BUG: \"git submodule--helper embed git-dirs '%s'\" "
+			    "did not embed the git-dirs recursively for '%s'",
+			    path, path);
+		}
+	}
+
+	strbuf_addstr(&pathbuf, path);
+	remove_directory_or_die(&pathbuf);
+out:
+	strbuf_release(&pathbuf);
+	free(dot_git);
+	return ret;
+}
+
 /* Helper function to display the submodule header line prior to the full
  * summary output. If it can locate the submodule objects directory it will
  * attempt to lookup both the left and right commits and put them into the
diff --git a/submodule.h b/submodule.h
index 7d890e0464..d8bb1d4baf 100644
--- a/submodule.h
+++ b/submodule.h
@@ -63,6 +63,7 @@ extern void set_config_update_recurse_submodules(int value);
  */
 extern int submodule_is_interesting(const char *path);
 extern int submodules_interesting_for_update(void);
+extern int depopulate_submodule(const char *path);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 09/17] update submodules: add scheduling to update submodules
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

The walker of a tree is only expected to call `schedule_submodule_for_update`
and once done, to run `update_submodules`. This avoids directory/file
conflicts and later we can parallelize all submodule actions if needed.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h |   5 +++
 2 files changed, 137 insertions(+)

diff --git a/submodule.c b/submodule.c
index 7bb64d6c69..02c28ef56b 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1348,3 +1348,135 @@ int parallel_submodules(void)
 {
 	return parallel_jobs;
 }
+
+static struct scheduled_submodules_update_type {
+	const char *path;
+	const struct object_id *oid;
+	/*
+	 * Do we need to perform a complete checkout or just incremental
+	 * update?
+	 */
+	unsigned is_new:1;
+} *scheduled_submodules;
+#define SCHEDULED_SUBMODULES_INIT {NULL, NULL, 0}
+
+static int scheduled_submodules_nr, scheduled_submodules_alloc;
+
+void schedule_submodule_for_update(const struct cache_entry *ce, int is_new)
+{
+	struct scheduled_submodules_update_type *ssu;
+	ALLOC_GROW(scheduled_submodules,
+		   scheduled_submodules_nr + 1,
+		   scheduled_submodules_alloc);
+	ssu = &scheduled_submodules[scheduled_submodules_nr++];
+	ssu->path = ce->name;
+	ssu->oid = &ce->oid;
+	ssu->is_new = !!is_new;
+}
+
+static int update_submodule(const char *path, const struct object_id *oid,
+			    int force, int is_new)
+{
+	const char *git_dir;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	const struct submodule *sub = submodule_from_path(null_sha1, path);
+
+	if (!sub || !sub->name)
+		return -1;
+
+	git_dir = resolve_gitdir(git_common_path("modules/%s", sub->name));
+
+	if (!git_dir)
+		return -1;
+
+	if (is_new)
+		connect_work_tree_and_git_dir(path, git_dir);
+
+	/* update index via `read-tree --reset sha1` */
+	argv_array_pushl(&cp.args, "read-tree",
+				   force ? "--reset" : "-m",
+				   "-u", sha1_to_hex(oid->hash), NULL);
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (run_command(&cp)) {
+		warning(_("reading the index in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* write index to working dir */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout-index", "-a", NULL);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (force)
+		argv_array_push(&cp.args, "-f");
+
+	if (run_command(&cp)) {
+		warning(_("populating the working directory in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* get the HEAD right */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout", "--recurse-submodules", NULL);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (force)
+		argv_array_push(&cp.args, "-f");
+	argv_array_push(&cp.args, sha1_to_hex(oid->hash));
+
+	if (run_command(&cp)) {
+		warning(_("setting the HEAD in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	child_process_clear(&cp);
+	return 0;
+}
+
+int update_submodules(int force)
+{
+	int i;
+	gitmodules_config();
+
+	/*
+	 * NEEDSWORK: As submodule updates can potentially take some
+	 * time each and they do not overlap (i.e. no d/f conflicts;
+	 * this can be parallelized using the run_commands.h API.
+	 */
+	for (i = 0; i < scheduled_submodules_nr; i++) {
+		struct submodule *sub;
+		struct scheduled_submodules_update_type *ssu =
+			&scheduled_submodules[i];
+
+		if (!submodule_is_interesting(ssu->path))
+			continue;
+
+		sub = submodule_from_path(null_sha1, ssu->path);
+
+		switch (sub->update_strategy) {
+		case SM_UPDATE_UNSPECIFIED: /* fall thru */
+		case SM_UPDATE_CHECKOUT:
+			update_submodule(ssu->path, ssu->oid,
+					 force, ssu->is_new);
+			break;
+		case SM_UPDATE_REBASE:
+		case SM_UPDATE_MERGE:
+		case SM_UPDATE_NONE:
+		case SM_UPDATE_COMMAND:
+			warning("update strategy for submodule '%s' not supported", ssu->path);
+		}
+	}
+
+	scheduled_submodules_nr = 0;
+	return 0;
+}
diff --git a/submodule.h b/submodule.h
index d8bb1d4baf..74df8b93d5 100644
--- a/submodule.h
+++ b/submodule.h
@@ -90,4 +90,9 @@ extern int parallel_submodules(void);
  * retaining any config in the environment.
  */
 extern void prepare_submodule_repo_env(struct argv_array *out);
+
+extern void schedule_submodule_for_update(const struct cache_entry *ce,
+					  int new);
+extern int update_submodules(int force);
+
 #endif
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 12/17] unpack-trees: remove submodule contents if interesting
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Currently when a unlink_entry() is called on a submodule, this fails
as remove_or_warn detects it needs to delete a directory via rmdir.
However rmdir only works on empty directories, such that the
"_or_warn" part kicks in, and we get a warning message.

In case the submodule is of no interest we're not going to delete the
submodule, so a warning like that is useful.

If the submodule is interesting then we need to depopulate it properly,
there is no need to react on its return code the depopulation method
handles proper warnings nor do we need to schedule the directory for
removal later, such that we can return early.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 unpack-trees.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 22e32eca96..db03293347 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -214,7 +214,12 @@ static void unlink_entry(const struct cache_entry *ce)
 {
 	if (!check_leading_path(ce->name, ce_namelen(ce)))
 		return;
-	if (remove_or_warn(ce->ce_mode, ce->name))
+
+	if (S_ISGITLINK(ce->ce_mode) &&
+	    submodule_is_interesting(ce->name)) {
+		depopulate_submodule(ce->name);
+		return;
+	} else if (remove_or_warn(ce->ce_mode, ce->name))
 		return;
 	schedule_dir_for_removal(ce->name, ce_namelen(ce));
 }
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [RFC PATCHv2 00/17] Checkout aware of Submodules!
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller

v2:
* based on top of the series sent out an hour ago
  "[PATCHv4 0/5] submodule embedgitdirs"
* Try to embed a submodule if we need to remove it.
* Strictly do not change behavior if not giving the new flag.
* I think I missed some review comments from v1, but I'd like to get
  the current state out over the weekend, as a lot has changed so far.
  On Monday I'll go through the previous discussion with a comb to see
  if I missed something.
  
v1:
When working with submodules, nearly anytime after checking out
a different state of the projects, that has submodules changed
you'd run "git submodule update" with a current version of Git.

There are two problems with this approach:

* The "submodule update" command is dangerous as it
  doesn't check for work that may be lost in the submodule
  (e.g. a dangling commit).
* you may forget to run the command as checkout is supposed
  to do all the work for you.

Integrate updating the submodules into git checkout, with the same
safety promises that git-checkout has, i.e. not throw away data unless
asked to. This is done by first checking if the submodule is at the same
sha1 as it is recorded in the superproject. If there are changes we stop
proceeding the checkout just like it is when checking out a file that
has local changes.

The integration happens in the code that is also used in other commands
such that it will be easier in the future to make other commands aware
of submodule.

This also solves d/f conflicts in case you replace a file/directory
with a submodule or vice versa.

The patches are still a bit rough, but the overall series seems
promising enough to me that I want to put it out here.

Any review, specifically on the design level welcome!

Thanks,
Stefan


Stefan Beller (17):
  submodule.h: add extern keyword to functions
  submodule: modernize ok_to_remove_submodule to use argv_array
  update submodules: move up prepare_submodule_repo_env
  update submodules: add is_submodule_populated
  update submodules: add submodule config parsing
  update submodules: add a config option to determine if submodules are
    updated
  update submodules: introduce submodule_is_interesting
  update submodules: add depopulate_submodule
  update submodules: add scheduling to update submodules
  update submodules: is_submodule_checkout_safe
  unpack-trees: teach verify_clean_submodule to inspect submodules
  unpack-trees: remove submodule contents if interesting
  entry: write_entry to write populate submodules
  submodule: teach unpack_trees() to update submodules
  checkout: recurse into submodules if asked to
  completion: add '--recurse-submodules' to checkout
  checkout: add config option to recurse into submodules by default

 Documentation/config.txt               |   6 +
 Documentation/git-checkout.txt         |   8 +
 builtin/checkout.c                     |  31 +++-
 cache.h                                |   2 +
 contrib/completion/git-completion.bash |   2 +-
 entry.c                                |  14 +-
 submodule-config.c                     |  22 +++
 submodule-config.h                     |  17 +-
 submodule.c                            | 292 +++++++++++++++++++++++++++++---
 submodule.h                            |  74 ++++++---
 t/t2013-checkout-submodule.sh          | 293 ++++++++++++++++++++++++++++++++-
 t/t9902-completion.sh                  |   1 +
 unpack-trees.c                         |  92 ++++++++---
 unpack-trees.h                         |   1 +
 14 files changed, 768 insertions(+), 87 deletions(-)

-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply

* [RFC PATCHv2 02/17] submodule: modernize ok_to_remove_submodule to use argv_array
From: Stefan Beller @ 2016-12-03  0:30 UTC (permalink / raw)
  To: bmwill, David.Turner; +Cc: git, sandals, hvoigt, gitster, Stefan Beller
In-Reply-To: <20161203003022.29797-1-sbeller@google.com>

Instead of constructing the NULL terminated array ourselves, we
should make use of the argv_array infrastructure.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/submodule.c b/submodule.c
index 66c5ce5a24..eb80b0c5ad 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1022,13 +1022,6 @@ int ok_to_remove_submodule(const char *path)
 {
 	ssize_t len;
 	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *argv[] = {
-		"status",
-		"--porcelain",
-		"-u",
-		"--ignore-submodules=none",
-		NULL,
-	};
 	struct strbuf buf = STRBUF_INIT;
 	int ok_to_remove = 1;
 
@@ -1038,14 +1031,15 @@ int ok_to_remove_submodule(const char *path)
 	if (!submodule_uses_gitfile(path))
 		return 0;
 
-	cp.argv = argv;
+	argv_array_pushl(&cp.args, "status", "--porcelain", "-u",
+				   "--ignore-submodules=none", NULL);
 	prepare_submodule_repo_env(&cp.env_array);
 	cp.git_cmd = 1;
 	cp.no_stdin = 1;
 	cp.out = -1;
 	cp.dir = path;
 	if (start_command(&cp))
-		die("Could not run 'git status --porcelain -uall --ignore-submodules=none' in submodule %s", path);
+		die("Could not run 'git status --porcelain -u --ignore-submodules=none' in submodule %s", path);
 
 	len = strbuf_read(&buf, cp.out, 1024);
 	if (len > 2)
@@ -1053,7 +1047,7 @@ int ok_to_remove_submodule(const char *path)
 	close(cp.out);
 
 	if (finish_command(&cp))
-		die("'git status --porcelain -uall --ignore-submodules=none' failed in submodule %s", path);
+		die("'git status --porcelain -u --ignore-submodules=none' failed in submodule %s", path);
 
 	strbuf_release(&buf);
 	return ok_to_remove;
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* Re: [PATCH v6 1/6] submodules: add helper functions to determine presence of submodules
From: Brandon Williams @ 2016-12-03  0:16 UTC (permalink / raw)
  To: Jeff King
  Cc: Stefan Beller, Jacob Keller, Junio C Hamano, git@vger.kernel.org,
	Jonathan Tan
In-Reply-To: <20161202214529.mjekdaixrdoyroxq@sigill.intra.peff.net>

On 12/02, Jeff King wrote:
> On Fri, Dec 02, 2016 at 11:28:49AM -0800, Stefan Beller wrote:
> 
> > I just reviewed 2 libc implementations (glibc and an Android libc) and
> > both of them
> > do not use chdir internally, but use readlink and compose the path 'manually'
> > c.f. http://osxr.org:8080/glibc/source/stdlib/canonicalize.c?v=glibc-2.13
> 
> Interesting. It might be worth updating our implementation. The original
> comes all the way from 54f4b8745 (Library code for user-relative paths,
> take three., 2005-11-17). That references a suggestion which I think
> comes from:
> 
>   http://public-inbox.org/git/Pine.LNX.4.64.0510181728490.3369@g5.osdl.org/
> 
> where it's claimed to be simpler and more efficient (which sounds
> plausible to me).  But back then it was _just_ git-daemon doing a
> canonicalization, and nobody cared about things like thread safety.
> 
> Looking at the glibc implementation, it's really not that bad. We
> _could_ even rely on the system realpath() and just provide our own
> fallback for systems without it, but I think ours might be a little more
> featureful (at the very least, it handles arbitrary-sized paths via
> strbufs).

I've actually been working on updating our implementation of realpath
today.  Its slow going but we'll see if it works when i'm done :)

Also we can just drop in realpath since it requires that all path
components are valid, while ours allows for the final component to be
invalid.

-- 
Brandon Williams

^ permalink raw reply

* [PATCHv4 2/5] submodule helper: support super prefix
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller
In-Reply-To: <20161202234220.24664-1-sbeller@google.com>

Just like main commands in Git, the submodule helper needs
access to the superproject prefix. Enable this in the git.c
but have its own fuse in the helper code by having a flag to
turn on the super prefix.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 builtin/submodule--helper.c | 31 ++++++++++++++++++++-----------
 git.c                       |  2 +-
 2 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 4beeda5f9f..806e29ce4e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1076,21 +1076,24 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
 	return 0;
 }
 
+#define SUPPORT_SUPER_PREFIX (1<<0)
+
 struct cmd_struct {
 	const char *cmd;
 	int (*fn)(int, const char **, const char *);
+	int option;
 };
 
 static struct cmd_struct commands[] = {
-	{"list", module_list},
-	{"name", module_name},
-	{"clone", module_clone},
-	{"update-clone", update_clone},
-	{"relative-path", resolve_relative_path},
-	{"resolve-relative-url", resolve_relative_url},
-	{"resolve-relative-url-test", resolve_relative_url_test},
-	{"init", module_init},
-	{"remote-branch", resolve_remote_submodule_branch}
+	{"list", module_list, 0},
+	{"name", module_name, 0},
+	{"clone", module_clone, 0},
+	{"update-clone", update_clone, 0},
+	{"relative-path", resolve_relative_path, 0},
+	{"resolve-relative-url", resolve_relative_url, 0},
+	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"init", module_init, 0},
+	{"remote-branch", resolve_remote_submodule_branch, 0}
 };
 
 int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
@@ -1100,9 +1103,15 @@ int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
 		die(_("submodule--helper subcommand must be "
 		      "called with a subcommand"));
 
-	for (i = 0; i < ARRAY_SIZE(commands); i++)
-		if (!strcmp(argv[1], commands[i].cmd))
+	for (i = 0; i < ARRAY_SIZE(commands); i++) {
+		if (!strcmp(argv[1], commands[i].cmd)) {
+			if (get_super_prefix() &&
+			    !(commands[i].option & SUPPORT_SUPER_PREFIX))
+				die("%s doesn't support --super-prefix",
+				    commands[i].cmd);
 			return commands[i].fn(argc - 1, argv + 1, prefix);
+		}
+	}
 
 	die(_("'%s' is not a valid submodule--helper "
 	      "subcommand"), argv[1]);
diff --git a/git.c b/git.c
index efa1059fe0..98dcf6c518 100644
--- a/git.c
+++ b/git.c
@@ -493,7 +493,7 @@ static struct cmd_struct commands[] = {
 	{ "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE },
 	{ "status", cmd_status, RUN_SETUP | NEED_WORK_TREE },
 	{ "stripspace", cmd_stripspace },
-	{ "submodule--helper", cmd_submodule__helper, RUN_SETUP },
+	{ "submodule--helper", cmd_submodule__helper, RUN_SETUP | SUPPORT_SUPER_PREFIX},
 	{ "symbolic-ref", cmd_symbolic_ref, RUN_SETUP },
 	{ "tag", cmd_tag, RUN_SETUP },
 	{ "unpack-file", cmd_unpack_file, RUN_SETUP },
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [PATCHv4 3/5] test-lib-functions.sh: teach test_commit -C <dir>
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller
In-Reply-To: <20161202234220.24664-1-sbeller@google.com>

Specifically when setting up submodule tests, it comes in handy if
we can create commits in repositories that are not at the root of
the tested trash dir. Add "-C <dir>" similar to gits -C parameter
that will perform the operation in the given directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/test-lib-functions.sh | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index fdaeb3a96b..579e812506 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -157,16 +157,21 @@ debug () {
 	 GIT_TEST_GDB=1 "$@"
 }
 
-# Call test_commit with the arguments "<message> [<file> [<contents> [<tag>]]]"
+# Call test_commit with the arguments
+# [-C <directory>] <message> [<file> [<contents> [<tag>]]]"
 #
 # This will commit a file with the given contents and the given commit
 # message, and tag the resulting commit with the given tag name.
 #
 # <file>, <contents>, and <tag> all default to <message>.
+#
+# If the first argument is "-C", the second argument is used as a path for
+# the git invocations.
 
 test_commit () {
 	notick= &&
 	signoff= &&
+	indir= &&
 	while test $# != 0
 	do
 		case "$1" in
@@ -176,21 +181,26 @@ test_commit () {
 		--signoff)
 			signoff="$1"
 			;;
+		-C)
+			indir="$2"
+			shift
+			;;
 		*)
 			break
 			;;
 		esac
 		shift
 	done &&
+	indir=${indir:+"$indir"/} &&
 	file=${2:-"$1.t"} &&
-	echo "${3-$1}" > "$file" &&
-	git add "$file" &&
+	echo "${3-$1}" > "$indir$file" &&
+	git ${indir:+ -C "$indir"} add "$file" &&
 	if test -z "$notick"
 	then
 		test_tick
 	fi &&
-	git commit $signoff -m "$1" &&
-	git tag "${4:-$1}"
+	git ${indir:+ -C "$indir"} commit $signoff -m "$1" &&
+	git ${indir:+ -C "$indir"} tag "${4:-$1}"
 }
 
 # Call test_merge with the arguments "<message> <commit>", where <commit>
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [PATCHv4 5/5] submodule: add embed-git-dir function
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller
In-Reply-To: <20161202234220.24664-1-sbeller@google.com>

When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.

Add functionality to migrate the git directory to be embedded
into the superprojects git directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-submodule.txt   |  14 ++++++
 builtin/submodule--helper.c       |  62 ++++++++++++++++++++++-
 dir.c                             |  81 ++++++++++++++++++++++++++++++
 dir.h                             |   4 ++
 git-submodule.sh                  |   7 ++-
 t/t7412-submodule-embedgitdirs.sh | 101 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 267 insertions(+), 2 deletions(-)
 create mode 100755 t/t7412-submodule-embedgitdirs.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index d841573475..34791cfc65 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -22,6 +22,7 @@ SYNOPSIS
 	      [commit] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
 'git submodule' [--quiet] sync [--recursive] [--] [<path>...]
+'git submodule' [--quiet] embedgitdirs [--] [<path>...]
 
 
 DESCRIPTION
@@ -245,6 +246,19 @@ sync::
 If `--recursive` is specified, this command will recurse into the
 registered submodules, and sync any nested submodules within.
 
+embedgitdirs::
+	Move the git directory of submodules into its superprojects
+	`$GIT_DIR/modules` path and then connect the git directory and
+	its working directory by setting the `core.worktree` and adding
+	a .git file pointing to the git directory interned into the
+	superproject.
++
+A repository that was cloned independently and later added as a submodule or
+old setups have the submodules git directory inside the submodule instead of
+embedded into the superprojects git directory.
++
+This command is recursive by default.
+
 OPTIONS
 -------
 -q::
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 806e29ce4e..10df69c86a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1076,6 +1076,65 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
 	return 0;
 }
 
+static int embed_git_dir(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	unsigned flags = RELOCATE_GITDIR_RECURSE_SUBMODULES;
+
+	struct option embed_gitdir_options[] = {
+		OPT_STRING(0, "prefix", &prefix,
+			   N_("path"),
+			   N_("path into the working tree")),
+		OPT_BIT(0, "--recursive", &flags, N_("recurse into submodules"),
+			RELOCATE_GITDIR_RECURSE_SUBMODULES),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper embed-git-dir [<path>...]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, embed_gitdir_options,
+			     git_submodule_helper_usage, 0);
+
+	gitmodules_config();
+	git_config(submodule_config, NULL);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	for (i = 0; i < list.nr; i++) {
+		const char *path = list.entries[i]->name, *sub_git_dir, *v;
+		char *real_sub_git_dir = NULL, *real_common_git_dir = NULL;
+		struct strbuf gitdir = STRBUF_INIT;
+
+		strbuf_addf(&gitdir, "%s/.git", path);
+		sub_git_dir = resolve_gitdir(gitdir.buf);
+
+		/* not populated? */
+		if (!sub_git_dir)
+			goto free_and_continue;
+
+		/* Is it already embedded? */
+		real_sub_git_dir = xstrdup(real_path(sub_git_dir));
+		real_common_git_dir = xstrdup(real_path(get_git_common_dir()));
+		if (skip_prefix(real_sub_git_dir, real_common_git_dir, &v), NULL)
+			goto free_and_continue;
+
+		relocate_gitdir(prefix, path, flags);
+
+free_and_continue:
+		strbuf_release(&gitdir);
+		free(real_sub_git_dir);
+		free(real_common_git_dir);
+	}
+
+	return 0;
+}
+
 #define SUPPORT_SUPER_PREFIX (1<<0)
 
 struct cmd_struct {
@@ -1093,7 +1152,8 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"init", module_init, 0},
-	{"remote-branch", resolve_remote_submodule_branch, 0}
+	{"remote-branch", resolve_remote_submodule_branch, 0},
+	{"embed-git-dirs", embed_git_dir, SUPPORT_SUPER_PREFIX}
 };
 
 int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
diff --git a/dir.c b/dir.c
index bfa8c8a9a5..d2f60b5abf 100644
--- a/dir.c
+++ b/dir.c
@@ -15,6 +15,9 @@
 #include "utf8.h"
 #include "varint.h"
 #include "ewah/ewok.h"
+#include "submodule-config.h"
+#include "run-command.h"
+#include "worktree.h"
 
 struct path_simplify {
 	int len;
@@ -2748,3 +2751,81 @@ void untracked_cache_add_to_index(struct index_state *istate,
 {
 	untracked_cache_invalidate_path(istate, path);
 }
+
+/*
+ * Migrate the given submodule (and all its submodules recursively) from
+ * having its git directory within the working tree to the git dir nested
+ * in its superprojects git dir under modules/.
+ */
+void relocate_gitdir(const char *prefix, const char *path, unsigned flags)
+{
+	char *old_git_dir;
+	const char *new_git_dir;
+	const struct submodule *sub;
+	struct worktree **worktrees;
+	int i;
+
+	worktrees = get_submodule_worktrees(path, 0);
+	if (worktrees) {
+		for (i = 0; worktrees[i]; i++)
+			;
+		free_worktrees(worktrees);
+		if (i > 1)
+			die(_("relocate_gitdir for submodule with more than one worktree not supported"));
+	}
+
+	old_git_dir = xstrfmt("%s/.git", path);
+	if (read_gitfile(old_git_dir))
+		/* If it is an actual gitfile, it doesn't need migration. */
+		goto out;
+
+	sub = submodule_from_path(null_sha1, path);
+	if (!sub)
+		die(_("Could not lookup name for submodule '%s'"),
+		      path);
+
+	new_git_dir = git_path("modules/%s", sub->name);
+	if (safe_create_leading_directories_const(new_git_dir) < 0)
+		die(_("could not create directory '%s'"), new_git_dir);
+
+	if (!prefix)
+		prefix = get_super_prefix();
+	printf("Migrating git directory of %s%s from\n'%s' to\n'%s'\n",
+		prefix ? prefix : "", path,
+		real_path(old_git_dir), new_git_dir);
+
+	if (rename(old_git_dir, new_git_dir) < 0)
+		die_errno(_("Could not migrate git directory from '%s' to '%s'"),
+			old_git_dir, new_git_dir);
+
+	connect_work_tree_and_git_dir(path, new_git_dir);
+
+out:
+	if (flags & RELOCATE_GITDIR_RECURSE_SUBMODULES) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (flags & ~RELOCATE_GITDIR_RECURSE_SUBMODULES)
+			die("BUG: we don't know how to pass the flags down?");
+
+		if (get_super_prefix())
+			strbuf_addstr(&sb, get_super_prefix());
+		strbuf_addstr(&sb, path);
+		strbuf_addch(&sb, '/');
+
+		cp.dir = path;
+		cp.git_cmd = 1;
+		cp.no_stdin = 1;
+		argv_array_pushl(&cp.args, "--super-prefix", sb.buf,
+					    "submodule--helper",
+					   "embed-git-dirs", NULL);
+		prepare_submodule_repo_env(&cp.env_array);
+		if (run_command(&cp))
+			die(_("Could not migrate git directory in submodule '%s'"),
+			    path);
+
+		strbuf_release(&sb);
+	}
+
+	free(old_git_dir);
+}
diff --git a/dir.h b/dir.h
index 97c83bb383..0b5e99b21d 100644
--- a/dir.h
+++ b/dir.h
@@ -335,4 +335,8 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long
 void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked);
 void add_untracked_cache(struct index_state *istate);
 void remove_untracked_cache(struct index_state *istate);
+
+#define RELOCATE_GITDIR_RECURSE_SUBMODULES (1<<0)
+extern void relocate_gitdir(const char *prefix, const char *path, unsigned flags);
+
 #endif
diff --git a/git-submodule.sh b/git-submodule.sh
index a024a135d6..b7e124f340 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1131,6 +1131,11 @@ cmd_sync()
 	done
 }
 
+cmd_embedgitdirs()
+{
+	git submodule--helper embed-git-dirs --prefix "$wt_prefix" "$@"
+}
+
 # This loop parses the command line arguments to find the
 # subcommand name to dispatch.  Parsing of the subcommand specific
 # options are primarily done by the subcommand implementations.
@@ -1140,7 +1145,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | deinit | update | status | summary | sync)
+	add | foreach | init | deinit | update | status | summary | sync | embedgitdirs)
 		command=$1
 		;;
 	-q|--quiet)
diff --git a/t/t7412-submodule-embedgitdirs.sh b/t/t7412-submodule-embedgitdirs.sh
new file mode 100755
index 0000000000..a02e7c447a
--- /dev/null
+++ b/t/t7412-submodule-embedgitdirs.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='Test submodule embedgitdirs
+
+This test verifies that `git submodue embedgitdirs` moves a submodules git
+directory into the superproject.
+'
+
+. ./test-lib.sh
+
+test_expect_success 'setup a real submodule' '
+	git init sub1 &&
+	test_commit -C sub1 first &&
+	git submodule add ./sub1 &&
+	test_tick &&
+	git commit -m superproject
+'
+
+test_expect_success 'embed the git dir' '
+	>expect.1 &&
+	>expect.2 &&
+	>actual.1 &&
+	>actual.2 &&
+	git status >expect.1 &&
+	git -C sub1 rev-parse HEAD >expect.2 &&
+	git submodule embedgitdirs &&
+	git fsck &&
+	test -f sub1/.git &&
+	test -d .git/modules/sub1 &&
+	git status >actual.1 &&
+	git -C sub1 rev-parse HEAD >actual.2 &&
+	test_cmp expect.1 actual.1 &&
+	test_cmp expect.2 actual.2
+'
+
+test_expect_success 'embedding does not fail for deinitalized submodules' '
+	test_when_finished "git submodule update --init" &&
+	git submodule deinit --all &&
+	git submodule embedgitdirs &&
+	test -d .git/modules/sub1 &&
+	! test -f sub1/.git &&
+	test -d sub1
+'
+
+test_expect_success 'setup nested submodule' '
+	git init sub1/nested &&
+	test_commit -C sub1/nested first_nested &&
+	git -C sub1 submodule add ./nested &&
+	test_tick &&
+	git -C sub1 commit -m "add nested" &&
+	git add sub1 &&
+	git commit -m "sub1 to include nested submodule"
+'
+
+test_expect_success 'embed the git dir in a nested submodule' '
+	git status >expect.1 &&
+	git -C sub1/nested rev-parse HEAD >expect.2 &&
+	git submodule embedgitdirs &&
+	test -f sub1/nested/.git &&
+	test -d .git/modules/sub1/modules/nested &&
+	git status >actual.1 &&
+	git -C sub1/nested rev-parse HEAD >actual.2 &&
+	test_cmp expect.1 actual.1 &&
+	test_cmp expect.2 actual.2
+'
+
+test_expect_success 'setup a gitlink with missing .gitmodules entry' '
+	git init sub2 &&
+	test_commit -C sub2 first &&
+	git add sub2 &&
+	git commit -m superproject
+'
+
+test_expect_success 'embedding the git dir fails for incomplete submodules' '
+	git status >expect.1 &&
+	git -C sub2 rev-parse HEAD >expect.2 &&
+	test_must_fail git submodule embedgitdirs &&
+	git -C sub2 fsck &&
+	test -d sub2/.git &&
+	git status >actual &&
+	git -C sub2 rev-parse HEAD >actual.2 &&
+	test_cmp expect.1 actual.1 &&
+	test_cmp expect.2 actual.2
+'
+
+test_expect_success 'setup a submodule with multiple worktrees' '
+	# first create another unembedded git dir in a new submodule
+	git init sub3 &&
+	test_commit -C sub3 first &&
+	git submodule add ./sub3 &&
+	test_tick &&
+	git commit -m "add another submodule" &&
+	git -C sub3 worktree add ../sub3_second_work_tree
+'
+
+test_expect_success 'embed a submodule with multiple worktrees' '
+	test_must_fail git submodule embedgitdirs sub3 2>error &&
+	test_i18ngrep "not supported" error
+'
+
+test_done
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [PATCHv4 4/5] worktree: get worktrees from submodules
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller
In-Reply-To: <20161202234220.24664-1-sbeller@google.com>

In a later patch we want to move around the the git directory of
a submodule. Both submodules as well as worktrees are involved in
placing git directories at unusual places, so their functionality
may collide. To react appropriately to situations where worktrees
in submodules are in use, offer a new function to query the
worktrees for submodules.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 worktree.c | 47 +++++++++++++++++++++++++++++++++++++----------
 worktree.h |  6 ++++++
 2 files changed, 43 insertions(+), 10 deletions(-)

diff --git a/worktree.c b/worktree.c
index eb6121263b..fa2b6dfa9a 100644
--- a/worktree.c
+++ b/worktree.c
@@ -72,7 +72,7 @@ static void add_head_info(struct strbuf *head_ref, struct worktree *worktree)
 /**
  * get the main worktree
  */
-static struct worktree *get_main_worktree(void)
+static struct worktree *get_main_worktree(const char *git_common_dir)
 {
 	struct worktree *worktree = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -81,12 +81,12 @@ static struct worktree *get_main_worktree(void)
 	int is_bare = 0;
 	int is_detached = 0;
 
-	strbuf_add_absolute_path(&worktree_path, get_git_common_dir());
+	strbuf_add_absolute_path(&worktree_path, git_common_dir);
 	is_bare = !strbuf_strip_suffix(&worktree_path, "/.git");
 	if (is_bare)
 		strbuf_strip_suffix(&worktree_path, "/.");
 
-	strbuf_addf(&path, "%s/HEAD", get_git_common_dir());
+	strbuf_addf(&path, "%s/HEAD", git_common_dir);
 
 	worktree = xcalloc(1, sizeof(*worktree));
 	worktree->path = strbuf_detach(&worktree_path, NULL);
@@ -101,7 +101,8 @@ static struct worktree *get_main_worktree(void)
 	return worktree;
 }
 
-static struct worktree *get_linked_worktree(const char *id)
+static struct worktree *get_linked_worktree(const char *git_common_dir,
+					    const char *id)
 {
 	struct worktree *worktree = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -112,7 +113,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	if (!id)
 		die("Missing linked worktree name");
 
-	strbuf_git_common_path(&path, "worktrees/%s/gitdir", id);
+	strbuf_addf(&path, "%s/worktrees/%s/gitdir", git_common_dir, id);
 	if (strbuf_read_file(&worktree_path, path.buf, 0) <= 0)
 		/* invalid gitdir file */
 		goto done;
@@ -125,7 +126,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	}
 
 	strbuf_reset(&path);
-	strbuf_addf(&path, "%s/worktrees/%s/HEAD", get_git_common_dir(), id);
+	strbuf_addf(&path, "%s/worktrees/%s/HEAD", git_common_dir, id);
 
 	if (parse_ref(path.buf, &head_ref, &is_detached) < 0)
 		goto done;
@@ -167,7 +168,8 @@ static int compare_worktree(const void *a_, const void *b_)
 	return fspathcmp((*a)->path, (*b)->path);
 }
 
-struct worktree **get_worktrees(unsigned flags)
+static struct worktree **get_worktrees_internal(const char *git_common_dir,
+						unsigned flags)
 {
 	struct worktree **list = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -177,9 +179,9 @@ struct worktree **get_worktrees(unsigned flags)
 
 	list = xmalloc(alloc * sizeof(struct worktree *));
 
-	list[counter++] = get_main_worktree();
+	list[counter++] = get_main_worktree(git_common_dir);
 
-	strbuf_addf(&path, "%s/worktrees", get_git_common_dir());
+	strbuf_addf(&path, "%s/worktrees", git_common_dir);
 	dir = opendir(path.buf);
 	strbuf_release(&path);
 	if (dir) {
@@ -188,7 +190,7 @@ struct worktree **get_worktrees(unsigned flags)
 			if (is_dot_or_dotdot(d->d_name))
 				continue;
 
-			if ((linked = get_linked_worktree(d->d_name))) {
+			if ((linked = get_linked_worktree(git_common_dir, d->d_name))) {
 				ALLOC_GROW(list, counter + 1, alloc);
 				list[counter++] = linked;
 			}
@@ -209,6 +211,31 @@ struct worktree **get_worktrees(unsigned flags)
 	return list;
 }
 
+struct worktree **get_worktrees(unsigned flags)
+{
+	return get_worktrees_internal(get_git_common_dir(), flags);
+}
+
+struct worktree **get_submodule_worktrees(const char *path, unsigned flags)
+{
+	char *submodule_gitdir;
+	const char *submodule_common_dir;
+	struct strbuf sb = STRBUF_INIT;
+	struct worktree **ret;
+
+	submodule_gitdir = git_pathdup_submodule(path, "%s", "");
+	if (!submodule_gitdir)
+		return NULL;
+
+	/* the env would be set for the superproject */
+	get_common_dir_noenv(&sb, submodule_gitdir);
+	submodule_common_dir = strbuf_detach(&sb, NULL);
+	ret = get_worktrees_internal(submodule_common_dir, flags);
+
+	free(submodule_gitdir);
+	return ret;
+}
+
 const char *get_worktree_git_dir(const struct worktree *wt)
 {
 	if (!wt)
diff --git a/worktree.h b/worktree.h
index d59ce1fee8..157fbc4a66 100644
--- a/worktree.h
+++ b/worktree.h
@@ -27,6 +27,12 @@ struct worktree {
  */
 extern struct worktree **get_worktrees(unsigned flags);
 
+/*
+ * Get the worktrees of a submodule named by `path`.
+ */
+extern struct worktree **get_submodule_worktrees(const char *path,
+						 unsigned flags);
+
 /*
  * Return git dir of the worktree. Note that the path may be relative.
  * If wt is NULL, git dir of current worktree is returned.
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [PATCHv4 1/5] submodule: use absolute path for computing relative path connecting
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller
In-Reply-To: <20161202234220.24664-1-sbeller@google.com>

The current caller of connect_work_tree_and_git_dir passes
an absolute path for the `git_dir` parameter. In the future patch
we will also pass in relative path for `git_dir`. Extend the functionality
of connect_work_tree_and_git_dir to take relative paths for parameters.

We could work around this in the future patch by computing the absolute
path for the git_dir in the calling site, however accepting relative
paths for either parameter makes the API for this function much harder
to misuse.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 submodule.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/submodule.c b/submodule.c
index 6f7d883de9..66c5ce5a24 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1227,23 +1227,25 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)
 {
 	struct strbuf file_name = STRBUF_INIT;
 	struct strbuf rel_path = STRBUF_INIT;
-	const char *real_work_tree = xstrdup(real_path(work_tree));
+	char *real_git_dir = xstrdup(real_path(git_dir));
+	char *real_work_tree = xstrdup(real_path(work_tree));
 
 	/* Update gitfile */
 	strbuf_addf(&file_name, "%s/.git", work_tree);
 	write_file(file_name.buf, "gitdir: %s",
-		   relative_path(git_dir, real_work_tree, &rel_path));
+		   relative_path(real_git_dir, real_work_tree, &rel_path));
 
 	/* Update core.worktree setting */
 	strbuf_reset(&file_name);
-	strbuf_addf(&file_name, "%s/config", git_dir);
+	strbuf_addf(&file_name, "%s/config", real_git_dir);
 	git_config_set_in_file(file_name.buf, "core.worktree",
-			       relative_path(real_work_tree, git_dir,
+			       relative_path(real_work_tree, real_git_dir,
 					     &rel_path));
 
 	strbuf_release(&file_name);
 	strbuf_release(&rel_path);
-	free((void *)real_work_tree);
+	free(real_work_tree);
+	free(real_git_dir);
 }
 
 int parallel_submodules(void)
-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply related

* [PATCHv4 0/5] submodule embedgitdirs
From: Stefan Beller @ 2016-12-02 23:42 UTC (permalink / raw)
  To: pclouds, bmwill, gitster; +Cc: git, Stefan Beller

v4:
* rebuilt on top of nd/worktree-list-fixup
* fix and test behavior for un-init submodules (don't crash, rather do nothing)
* incorporated a "static" as pointed out by Ramsay
* use internal functions instead of duplicating code in worktree.c
  (use get_common_dir_noenv for the submodule to actually get the common dir)
* fixed a memory leak in relocate_gitdir 

Stefan Beller (5):
  submodule: use absolute path for computing relative path connecting
  submodule helper: support super prefix
  test-lib-functions.sh: teach test_commit -C <dir>
  worktree: get worktrees from submodules
  submodule: add embed-git-dir function

 Documentation/git-submodule.txt   |  14 ++++++
 builtin/submodule--helper.c       |  91 +++++++++++++++++++++++++++++-----
 dir.c                             |  81 ++++++++++++++++++++++++++++++
 dir.h                             |   4 ++
 git-submodule.sh                  |   7 ++-
 git.c                             |   2 +-
 submodule.c                       |  12 +++--
 t/t7412-submodule-embedgitdirs.sh | 101 ++++++++++++++++++++++++++++++++++++++
 t/test-lib-functions.sh           |  20 ++++++--
 worktree.c                        |  47 ++++++++++++++----
 worktree.h                        |   6 +++
 11 files changed, 352 insertions(+), 33 deletions(-)
 create mode 100755 t/t7412-submodule-embedgitdirs.sh

v3:
* have a slightly more generic function "relocate_gitdir".
  The recursion is strictly related to submodules, though.
* bail out if a submodule is using worktrees.
  This also lays the groundwork for later doing the proper thing,
  as worktree.h offers a function `get_submodule_worktrees(path)`
* nit by duy: use git_path instead of git_common_dir

* diff to v2 (as queued by Junio) below.

v2:
* fixed commit message for patch:
 "submodule: use absolute path for computing relative path connecting"
* a new patch "submodule helper: support super prefix"
* redid the final patch with more tests and fixing bugs along the way
* "test-lib-functions.sh: teach test_commit -C <dir>" unchanged

v1:
The discussion of the submodule checkout series revealed to me that a command
is needed to move the git directory from the submodules working tree to be
embedded into the superprojects git directory.

So I wrote the code to intern the submodules git dir into the superproject,
but whilst writing the code I realized this could be valueable for our use
in testing too. So I exposed it via the submodule--helper. But as the
submodule helper ought to be just an internal API, we could also
offer it via the proper submodule command.

The command as it is has little value to the end user for now, but
breaking it out of the submodule checkout series hopefully makes review easier.

Thanks,
Stefan

-- 
2.11.0.rc2.28.g2673dad


^ permalink raw reply

* difftool -d not populating left correctly when not in git root
From: Frank Becker @ 2016-12-02 23:04 UTC (permalink / raw)
  To: git

Hi,

looks like this broke between 2.9.2 and 2.9.3

cat ~/.gitconfig
[difftool "diff"]
     cmd = ls -l ${LOCAL}/* ${REMOTE}/*
     #cmd = diff -r ${LOCAL} ${REMOTE} | less

~/stuff/gittest> ls -l *
d1:
total 8
-rw-r--r--  1 frank  staff  16  2 Dec 14:30 test.txt

d2:
total 8
-rw-r--r--  1 frank  staff  18  2 Dec 14:30 anothertest.tst


~/stuff/gittest> git status
On branch master
Changes not staged for commit:
   (use "git add <file>..." to update what will be committed)
   (use "git checkout -- <file>..." to discard changes in working directory)

     modified:   d1/test.txt
     modified:   d2/anothertest.tst


~/stuff/gittest> ~/stuff/git_tmp/bin/git --version
git version 2.11.0

~/stuff/gittest> ~/stuff/git_tmp/bin/git difftool -d -t diff
/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.0oGRF/left/d1:
total 8
-rw-r--r--  1 frank  staff  6  2 Dec 14:52 test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.0oGRF/left/d2:
total 8
-rw-r--r--  1 frank  staff  7  2 Dec 14:52 anothertest.tst

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.0oGRF/right/d1:
total 8
lrwxr-xr-x  1 frank  staff  38  2 Dec 14:52 test.txt -> 
/Users/frank/stuff/gittest/d1/test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.0oGRF/right/d2:
total 8
lrwxr-xr-x  1 frank  staff  45  2 Dec 14:52 anothertest.tst -> 
/Users/frank/stuff/gittest/d2/anothertest.tst


cd d2
~/stuff/gittest/d2> ~/stuff/git_tmp/bin/git difftool -d -t diff
/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.eRXhB/left/d2:
total 8
-rw-r--r--  1 frank  staff  7  2 Dec 14:52 anothertest.tst

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.eRXhB/right/d1:
total 8
lrwxr-xr-x  1 frank  staff  38  2 Dec 14:52 test.txt -> 
/Users/frank/stuff/gittest/d1/test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.eRXhB/right/d2:
total 8
lrwxr-xr-x  1 frank  staff  45  2 Dec 14:52 anothertest.tst -> 
/Users/frank/stuff/gittest/d2/anothertest.tst


Note that left does not contain d1



~/stuff/gittest/d2> ~/stuff/git_tmp/bin/git --version
git version 2.9.2
~/stuff/gittest/d2> ~/stuff/git_tmp/bin/git difftool -d -t diff
/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.YxtVw/left/d1:
total 8
-rw-r--r--  1 frank  staff  6  2 Dec 15:02 test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.YxtVw/left/d2:
total 8
-rw-r--r--  1 frank  staff  7  2 Dec 15:02 anothertest.tst

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.YxtVw/right/d1:
total 8
lrwxr-xr-x  1 frank  staff  38  2 Dec 15:02 test.txt -> 
/Users/frank/stuff/gittest/d1/test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.YxtVw/right/d2:
total 8
lrwxr-xr-x  1 frank  staff  45  2 Dec 15:02 anothertest.tst -> 
/Users/frank/stuff/gittest/d2/anothertest.tst



~/stuff/gittest/d2> ~/stuff/git_tmp/bin/git --version
git version 2.9.3
~/stuff/gittest/d2> ~/stuff/git_tmp/bin/git difftool -d -t diff
/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.TpJ5u/left/d2:
total 8
-rw-r--r--  1 frank  staff  7  2 Dec 15:01 anothertest.tst

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.TpJ5u/right/d1:
total 8
lrwxr-xr-x  1 frank  staff  38  2 Dec 15:01 test.txt -> 
/Users/frank/stuff/gittest/d1/test.txt

/var/folders/0j/3pk3pdsx7rzb9_njdpyjwm000000gn/T/git-difftool.TpJ5u/right/d2:
total 8
lrwxr-xr-x  1 frank  staff  45  2 Dec 15:01 anothertest.tst -> 
/Users/frank/stuff/gittest/d2/anothertest.tst



Cheers,
Frank.





^ permalink raw reply

* [PATCHv1 2/2] git-p4: support updating an existing shelved changelist
From: Luke Diamand @ 2016-12-02 22:43 UTC (permalink / raw)
  To: git; +Cc: Vinicius Kursancew, larsxschneider, Luke Diamand
In-Reply-To: <20161202224319.5385-1-luke@diamand.org>

Adds new option "--update-shelve CHANGELIST" which updates
an existing shelved changelist.

The original changelist must have been created by the current user.

This allows workflow something like:

   hack hack hack
   git commit
   git p4 submit --shelve
   $mail interested parties about shelved changelist
   make corrections
   git commit --amend
   git p4 submit --update-shelve $CHANGELIST
   $mail interested parties about shelved changelist
   etc

Signed-off-by: Luke Diamand <luke@diamand.org>
---
 Documentation/git-p4.txt |  4 ++++
 git-p4.py                | 33 +++++++++++++++++++++++++++++----
 t/t9807-git-p4-submit.sh | 38 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt
index 1bbf43d15..ce40b9a54 100644
--- a/Documentation/git-p4.txt
+++ b/Documentation/git-p4.txt
@@ -308,6 +308,10 @@ These options can be used to modify 'git p4 submit' behavior.
 	After creating each shelve, the relevant files are reverted/deleted.
 	If you have multiple commits pending multiple shelves will be created.
 
+--update-shelve CHANGELIST::
+	Update an existing shelved changelist with this commit. Implies
+	--shelve.
+
 --conflict=(ask|skip|quit)::
 	Conflicts can occur when applying a commit to p4.  When this
 	happens, the default behavior ("ask") is to prompt whether to
diff --git a/git-p4.py b/git-p4.py
index 5e2db1919..36242d384 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -262,6 +262,10 @@ def p4_revert(f):
 def p4_reopen(type, f):
     p4_system(["reopen", "-t", type, wildcard_encode(f)])
 
+def p4_reopen_in_change(changelist, files):
+    cmd = ["reopen", "-c", str(changelist)] + files
+    p4_system(cmd)
+
 def p4_move(src, dest):
     p4_system(["move", "-k", wildcard_encode(src), wildcard_encode(dest)])
 
@@ -1298,6 +1302,9 @@ class P4Submit(Command, P4UserMap):
                 optparse.make_option("--shelve", dest="shelve", action="store_true",
                                      help="Shelve instead of submit. Shelved files are reverted, "
                                      "restoring the workspace to the state before the shelve"),
+                optparse.make_option("--update-shelve", dest="update_shelve", action="store", type="int",
+                                     metavar="CHANGELIST",
+                                     help="update an existing shelved changelist, implies --shelve")
         ]
         self.description = "Submit changes from git to the perforce depot."
         self.usage += " [name of git branch to submit into perforce depot]"
@@ -1306,6 +1313,7 @@ class P4Submit(Command, P4UserMap):
         self.preserveUser = gitConfigBool("git-p4.preserveUser")
         self.dry_run = False
         self.shelve = False
+        self.update_shelve = None
         self.prepare_p4_only = False
         self.conflict_behavior = None
         self.isWindows = (platform.system() == "Windows")
@@ -1474,7 +1482,7 @@ class P4Submit(Command, P4UserMap):
                     return 1
         return 0
 
-    def prepareSubmitTemplate(self):
+    def prepareSubmitTemplate(self, changelist=None):
         """Run "p4 change -o" to grab a change specification template.
            This does not use "p4 -G", as it is nice to keep the submission
            template in original order, since a human might edit it.
@@ -1486,7 +1494,11 @@ class P4Submit(Command, P4UserMap):
 
         template = ""
         inFilesSection = False
-        for line in p4_read_pipe_lines(['change', '-o']):
+        args = ['change', '-o']
+        if changelist:
+            args.append(str(changelist))
+
+        for line in p4_read_pipe_lines(args):
             if line.endswith("\r\n"):
                 line = line[:-2] + "\n"
             if inFilesSection:
@@ -1585,11 +1597,14 @@ class P4Submit(Command, P4UserMap):
         editedFiles = set()
         pureRenameCopy = set()
         filesToChangeExecBit = {}
+        all_files = list()
 
         for line in diff:
             diff = parseDiffTreeEntry(line)
             modifier = diff['status']
             path = diff['src']
+            all_files.append(path)
+
             if modifier == "M":
                 p4_edit(path)
                 if isModeExecChanged(diff['src_mode'], diff['dst_mode']):
@@ -1715,6 +1730,10 @@ class P4Submit(Command, P4UserMap):
             mode = filesToChangeExecBit[f]
             setP4ExecBit(f, mode)
 
+        if self.update_shelve:
+            print("all_files = %s" % str(all_files))
+            p4_reopen_in_change(self.update_shelve, all_files)
+
         #
         # Build p4 change description, starting with the contents
         # of the git commit message.
@@ -1723,7 +1742,7 @@ class P4Submit(Command, P4UserMap):
         logMessage = logMessage.strip()
         (logMessage, jobs) = self.separate_jobs_from_description(logMessage)
 
-        template = self.prepareSubmitTemplate()
+        template = self.prepareSubmitTemplate(self.update_shelve)
         submitTemplate = self.prepareLogMessage(template, logMessage, jobs)
 
         if self.preserveUser:
@@ -1795,7 +1814,10 @@ class P4Submit(Command, P4UserMap):
                 if self.isWindows:
                     message = message.replace("\r\n", "\n")
                 submitTemplate = message[:message.index(separatorLine)]
-                if self.shelve:
+
+                if self.update_shelve:
+                    p4_write_pipe(['shelve', '-r', '-i'], submitTemplate)
+                elif self.shelve:
                     p4_write_pipe(['shelve', '-i'], submitTemplate)
                 else:
                     p4_write_pipe(['submit', '-i'], submitTemplate)
@@ -1921,6 +1943,9 @@ class P4Submit(Command, P4UserMap):
         if len(self.origin) == 0:
             self.origin = upstream
 
+        if self.update_shelve:
+            self.shelve = True
+
         if self.preserveUser:
             if not self.canChangeChangelists():
                 die("Cannot preserve user names without p4 super-user or admin permissions")
diff --git a/t/t9807-git-p4-submit.sh b/t/t9807-git-p4-submit.sh
index 42a5fada5..e37239e65 100755
--- a/t/t9807-git-p4-submit.sh
+++ b/t/t9807-git-p4-submit.sh
@@ -444,6 +444,44 @@ test_expect_success 'submit --shelve' '
 	)
 '
 
+# Update an existing shelved changelist
+
+test_expect_success 'submit --update-shelve' '
+	test_when_finished cleanup_git &&
+	git p4 clone --dest="$git" //depot &&
+	(
+		cd "$cli" &&
+		p4 revert ... &&
+		cd "$git" &&
+		git config git-p4.skipSubmitEdit true &&
+		test_commit "test-update-shelved-change" &&
+		git p4 submit --origin=HEAD^ --shelve &&
+
+		shelf_cl=$(p4 -G changes -s shelved -m 1 |\
+			marshal_dump change) &&
+		test -n $shelf_cl &&
+		echo "updating shelved change list $shelf_cl" &&
+
+		echo "updated-line" >>shelf.t &&
+		echo added-file.t >added-file.t &&
+		git add shelf.t added-file.t &&
+		git rm -f test-update-shelved-change.t &&
+		git commit --amend -C HEAD &&
+		git show --stat HEAD &&
+		git p4 submit -v --origin HEAD^ --update-shelve $shelf_cl &&
+		echo "done git p4 submit"
+	) &&
+	(
+		cd "$cli" &&
+		change=$(p4 -G changes -s shelved -m 1 //depot/... | \
+			 marshal_dump change) &&
+		p4 unshelve -c $change -s $change &&
+		grep -q updated-line shelf.t &&
+		p4 describe -S $change | grep added-file.t &&
+		test_path_is_missing test-update-shelved-change.t
+	)
+'
+
 test_expect_success 'kill p4d' '
 	kill_p4d
 '
-- 
2.11.0.274.g0ea315c


^ permalink raw reply related

* [PATCHv1 1/2] git-p4: support git-workspaces
From: Luke Diamand @ 2016-12-02 22:43 UTC (permalink / raw)
  To: git; +Cc: Vinicius Kursancew, larsxschneider, Luke Diamand
In-Reply-To: <20161202224319.5385-1-luke@diamand.org>

Teach git-p4 about git-workspaces.

Signed-off-by: Luke Diamand <luke@diamand.org>
---
 git-p4.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/git-p4.py b/git-p4.py
index 0c4f2afd2..5e2db1919 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -566,6 +566,12 @@ def isValidGitDir(path):
     if (os.path.exists(path + "/HEAD")
         and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
         return True;
+
+    # git workspace directory?
+    if (os.path.exists(path + "/HEAD")
+        and os.path.exists(path + "/gitdir")):
+        return True
+
     return False
 
 def parseRevision(ref):
-- 
2.11.0.274.g0ea315c


^ permalink raw reply related

* [PATCHv1 0/2] git-p4 patches
From: Luke Diamand @ 2016-12-02 22:43 UTC (permalink / raw)
  To: git; +Cc: Vinicius Kursancew, larsxschneider, Luke Diamand

This is a couple of small patches for git-p4.

The first just teaches git-p4 about git workspaces, so that you can
do "git p4 submit" from within a workspace (P.S. workspaces
are completely *awesome*).

The second follows on from the work by Vinicius for shelving
changelists, by letting you update an existing shelved changelist.

This is a bit like using "git commit --amend" only far more painful....

Luke Diamand (2):
  git-p4: support git-workspaces
  git-p4: support updating an existing shelved changelist

 Documentation/git-p4.txt |  4 ++++
 git-p4.py                | 39 +++++++++++++++++++++++++++++++++++----
 t/t9807-git-p4-submit.sh | 38 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 77 insertions(+), 4 deletions(-)

-- 
2.11.0.274.g0ea315c


^ permalink raw reply

* Re: git 2.11.0 error when pushing to remote located on a windows share
From: Jeff King @ 2016-12-02 22:37 UTC (permalink / raw)
  To: thomas.attwood; +Cc: git
In-Reply-To: <AABB04BF1441D24CB4E9FCF46394F17D666F34E1@exchmbx01>

On Fri, Dec 02, 2016 at 06:02:16PM +0000, thomas.attwood@stfc.ac.uk wrote:

> After updating git from 2.10.0 to 2.11.0 when trying to push any
> changes to a repo located in a windows share, the following error
> occurs:
> 
> $ git push origin test
> Counting objects: 2, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (2/2), done.
> Writing objects: 100% (2/2), 284 bytes | 0 bytes/s, done.
> Total 2 (delta 1), reused 1 (delta 0)
> remote: error: object directory /path/to/dir/objects does not exist; check .git/objects/info/alternates.
> remote: fatal: unresolved deltas left after unpacking
> error: unpack failed: unpack-objects abnormal exit
> To //path/to/dir
>  ! [remote rejected] test -> test (unpacker error)
> error: failed to push some refs to '//path/to/dir'

Hmm. This is probably related to the quarantine-push change in v2.11;
the receiving end will write the objects into a temporary directory but
point to the original via GIT_ALTERNATE_OBJECT_DIRECTORIES. That pointer
isn't working for some reason, so the receiver can't resolve the deltas
it needs.

As you noted, the extra "/" is missing in the error message, and that
sounds like a plausible cause for what you're seeing. I'm not sure where
the slash is getting dropped, though. The value in the environment comes
from calling absolute_path(get_object_directory()), so I suspect the
real problem is not in the quarantine code, but it's just triggering a
latent bug elsewhere (either in absolute_path(), or in the code which
generates the objdir path).

> No error occurs if pushing to the same repo (a direct copy into a local directory) using 2.11.0.
> 
> $ git push local_test test
> Counting objects: 2, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (2/2), done.
> Writing objects: 100% (2/2), 284 bytes | 0 bytes/s, done.
> Total 2 (delta 1), reused 1 (delta 0)
> To C:/path/to/dir
>  * [new branch]      test -> test

The fact that it works using the non-UNC path reinforces my feeling that
something is normalizing the absolute path incorrectly.

> Using `git fsck --full` in both 2.11.0 and 2.10.0, it doesn't reveal any additional problems.

Yeah, I don't think there is anything wrong with your repo. It's just a
path-building issue internal to the receiving process.

-Peff

^ permalink raw reply

* Re: Where is Doc to configure Git + Apache + kerberos for Project level access in repo?
From: Jeff King @ 2016-12-02 22:21 UTC (permalink / raw)
  To: ken edward; +Cc: git
In-Reply-To: <CAAqgmoNG4vOqLnOqmrUvwTNJpqGBckfN-y=Fc99TrvjPhz7h0w@mail.gmail.com>

On Fri, Dec 02, 2016 at 01:15:02PM -0500, ken edward wrote:

> Where is Doc to configure Git + Apache + kerberos for Project level
> access in repo?

I don't know about Kerberos, but all of the documentation in git for
configuring Apache is found in "git help http-backend".

-Peff

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox