* [PATCH 0/9] Encode submodule gitdir names to avoid conflicts
@ 2025-08-16 21:36 Adrian Ratiu
2025-08-16 21:36 ` [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (15 more replies)
0 siblings, 16 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
Hello,
This is a continuation of work done back in 2018 [1], so a big thank you to
everyone who participated in the initial thread, especially Brandon on whose
code this is partially based upon. Hope you are still around and doing well. :)
It's mostly a rewrite from scratch addressig open feedback. I decided to
iterate upen Brandon's url-encoding design instead of pursuing alternatives
like a custom encoding, name hashing or round-trip encoding/decoding using
an in-memory git mapping (we'd still have to encode/hash the paths to avoid
colflicts so IIUC this last one is more complicated for little gain).
I tried to organize and explain the commits in a logical way which is also
easy to review, keeping the encoding parts, new tests, code moving around
and path update churn as clearly separated as possible.
This is based on master and I've merged and succesfully run all tests in
both the next and seen branches.
P.S. I plan to give a short talk at the mini-summit in 2 weesks based on this
series and some other patches I wish to propose on the ML, so if any of you
are attending and wish to connect in person, see you there!
Link: https://lore.kernel.org/git/20180807230637.247200-1-bmwill@google.com/ [1]
Adrian Ratiu (9):
submodule--helper: use submodule_name_to_gitdir in add_submodule
submodule: create new gitdirs under submodules path
submodule: add gitdir path config override
t: submodules: add basic mixed gitdir path tests
strbuf: bring back is_rfc3986_unreserved
submodule: encode gitdir paths to avoid conflicts
submodule: remove validate_submodule_git_dir()
t: move nested gitdir tests to proper location
t: add gitdir encoding tests
Documentation/fetch-options.adoc | 2 +-
Documentation/git-fetch.adoc | 2 +-
Documentation/git-submodule.adoc | 2 +-
Documentation/gitsubmodules.adoc | 8 +-
builtin/credential-store.c | 6 -
builtin/submodule--helper.c | 49 +++--
setup.c | 2 +-
strbuf.c | 6 +
strbuf.h | 2 +
submodule.c | 158 +++++++---------
submodule.h | 5 -
t/lib-submodule-update.sh | 50 ++---
t/lib-verify-submodule-gitdir-path.sh | 15 ++
t/meson.build | 1 +
t/t0035-safe-bare-repository.sh | 4 +-
t/t1600-index.sh | 4 +-
t/t2405-worktree-submodule.sh | 8 +-
t/t2501-cwd-empty.sh | 2 +-
t/t3600-rm.sh | 8 +-
t/t5526-fetch-submodules.sh | 2 +-
t/t5619-clone-local-ambiguous-transport.sh | 4 +-
t/t6120-describe.sh | 4 +-
t/t7001-mv.sh | 4 +-
t/t7400-submodule-basic.sh | 33 +++-
t/t7406-submodule-update.sh | 14 +-
t/t7407-submodule-foreach.sh | 6 +-
t/t7408-submodule-reference.sh | 22 +--
t/t7412-submodule-absorbgitdirs.sh | 22 +--
t/t7423-submodule-symlinks.sh | 8 +-
t/t7425-submodule-mixed-gitdir-paths.sh | 207 +++++++++++++++++++++
t/t7450-bad-git-dotfiles.sh | 73 +-------
t/t7527-builtin-fsmonitor.sh | 4 +-
32 files changed, 451 insertions(+), 286 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-mixed-gitdir-paths.sh
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-20 19:04 ` Josh Steadmon
2025-08-16 21:36 ` [PATCH 2/9] submodule: create new gitdirs under submodules path Adrian Ratiu
` (14 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded name-based path leading to test failures, so
convert it to the common helper function introduced by commit ce125d431a
("submodule: extract path to submodule gitdir func") and used in other
locations accross the source tree.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 07a1935cbe..7243429c6f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3213,10 +3213,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3225,8 +3226,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3244,7 +3245,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 2/9] submodule: create new gitdirs under submodules path
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-08-16 21:36 ` [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-09-08 14:24 ` Phillip Wood
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
` (13 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
This is in preparation for encoding the submodule names to avoid conflicts
like submodules named foo and foo/bar together with case-insensitive file-
system handling and other corner cases like reserved filenames on Windows.
Backward compatibility is kept with plain-name modules already existing at
paths like .git/modules/<name>, however a clear separation between legacy
(plain) and new (encoded) namespaces is desirable, to avoid situations like
an existing plain-name module containing the encoding escape character/
Thus we split the new-style (encoded) gitdir name paths to .git/submodules,
while legacy-style paths remain under .git/modules.
This is just a default directory change with the accompanying test updates,
in preparation for the actual encoding additions in future commits.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/fetch-options.adoc | 2 +-
Documentation/git-fetch.adoc | 2 +-
Documentation/git-submodule.adoc | 2 +-
Documentation/gitsubmodules.adoc | 8 ++--
setup.c | 2 +-
submodule.c | 28 +++++++++---
t/lib-submodule-update.sh | 50 +++++++++++-----------
t/t0035-safe-bare-repository.sh | 4 +-
t/t1600-index.sh | 4 +-
t/t2405-worktree-submodule.sh | 8 ++--
t/t2501-cwd-empty.sh | 2 +-
t/t3600-rm.sh | 8 ++--
t/t5526-fetch-submodules.sh | 2 +-
t/t5619-clone-local-ambiguous-transport.sh | 4 +-
t/t6120-describe.sh | 4 +-
t/t7001-mv.sh | 4 +-
t/t7400-submodule-basic.sh | 18 ++++----
t/t7406-submodule-update.sh | 10 ++---
t/t7407-submodule-foreach.sh | 6 +--
t/t7408-submodule-reference.sh | 22 +++++-----
t/t7412-submodule-absorbgitdirs.sh | 22 +++++-----
t/t7423-submodule-symlinks.sh | 8 ++--
t/t7450-bad-git-dotfiles.sh | 32 +++++++-------
t/t7527-builtin-fsmonitor.sh | 4 +-
24 files changed, 136 insertions(+), 120 deletions(-)
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
index b01372e4b3..f8d3f65009 100644
--- a/Documentation/fetch-options.adoc
+++ b/Documentation/fetch-options.adoc
@@ -210,7 +210,7 @@ ifndef::git-pull[]
submodule that has commits that are referenced by a newly fetched
superproject commit but are missing in the local submodule clone. A
changed submodule can be fetched as long as it is present locally e.g.
- in `$GIT_DIR/modules/` (see linkgit:gitsubmodules[7]); if the upstream
+ in `$GIT_DIR/submodules/` (see linkgit:gitsubmodules[7]); if the upstream
adds a new submodule, that submodule cannot be fetched until it is
cloned e.g. by `git submodule update`.
+
diff --git a/Documentation/git-fetch.adoc b/Documentation/git-fetch.adoc
index 16f5d9d69a..2923a29bef 100644
--- a/Documentation/git-fetch.adoc
+++ b/Documentation/git-fetch.adoc
@@ -304,7 +304,7 @@ include::config/fetch.adoc[]
BUGS
----
Using --recurse-submodules can only fetch new commits in submodules that are
-present locally e.g. in `$GIT_DIR/modules/`. If the upstream adds a new
+present locally e.g. in `$GIT_DIR/submodules/`. If the upstream adds a new
submodule, that submodule cannot be fetched until it is cloned e.g. by `git
submodule update`. This is expected to be fixed in a future Git version.
diff --git a/Documentation/git-submodule.adoc b/Documentation/git-submodule.adoc
index 503c84a200..9389862208 100644
--- a/Documentation/git-submodule.adoc
+++ b/Documentation/git-submodule.adoc
@@ -266,7 +266,7 @@ registered submodules, and sync any nested submodules within.
absorbgitdirs::
If a git directory of a submodule is inside the submodule,
move the git directory of the submodule into its superproject's
- `$GIT_DIR/modules` path and then connect the git directory and
+ `$GIT_DIR/submodules` path and then connect the git directory and
its working directory by setting the `core.worktree` and adding
a .git file pointing to the git directory embedded in the
superprojects git directory.
diff --git a/Documentation/gitsubmodules.adoc b/Documentation/gitsubmodules.adoc
index f7b5a25a0c..061e24f316 100644
--- a/Documentation/gitsubmodules.adoc
+++ b/Documentation/gitsubmodules.adoc
@@ -21,12 +21,12 @@ The submodule has its own history; the repository it is embedded
in is called a superproject.
On the filesystem, a submodule usually (but not always - see FORMS below)
-consists of (i) a Git directory located under the `$GIT_DIR/modules/`
+consists of (i) a Git directory located under the `$GIT_DIR/submodules/`
directory of its superproject, (ii) a working directory inside the
superproject's working directory, and a `.git` file at the root of
the submodule's working directory pointing to (i).
-Assuming the submodule has a Git directory at `$GIT_DIR/modules/foo/`
+Assuming the submodule has a Git directory at `$GIT_DIR/submodules/foo/`
and a working directory at `path/to/bar/`, the superproject tracks the
submodule via a `gitlink` entry in the tree at `path/to/bar` and an entry
in its `.gitmodules` file (see linkgit:gitmodules[5]) of the form
@@ -137,7 +137,7 @@ using older versions of Git.
It is possible to construct these old form repositories manually.
+
When deinitialized or deleted (see below), the submodule's Git
-directory is automatically moved to `$GIT_DIR/modules/<name>/`
+directory is automatically moved to `$GIT_DIR/submodules/<name>/`
of the superproject.
* Deinitialized submodule: A `gitlink`, and a `.gitmodules` entry,
@@ -162,7 +162,7 @@ possible to checkout past commits without requiring fetching
from another repository.
+
To completely remove a submodule, manually delete
-`$GIT_DIR/modules/<name>/`.
+`$GIT_DIR/submodules/<name>/`.
ACTIVE SUBMODULES
-----------------
diff --git a/setup.c b/setup.c
index 98ddbf377f..d054dafa6a 100644
--- a/setup.c
+++ b/setup.c
@@ -1416,7 +1416,7 @@ static int is_implicit_bare_repo(const char *path)
* we are inside $GIT_DIR of a worktree of a non-embedded
* submodule, whose superproject is not a bare repository.
*/
- if (strstr(path, "/.git/modules/"))
+ if (strstr(path, "/.git/modules/") || strstr(path, "/.git/submodules/"))
return 1;
return 0;
diff --git a/submodule.c b/submodule.c
index fff3c75570..dbf2244e60 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1278,22 +1278,29 @@ void check_for_new_submodule_commits(struct object_id *oid)
/*
* Returns 1 if there is at least one submodule gitdir in
- * $GIT_DIR/modules and 0 otherwise. This follows
+ * $GIT_DIR/(sub)modules and 0 otherwise. This follows
* submodule_name_to_gitdir(), which looks for submodules in
- * $GIT_DIR/modules, not $GIT_COMMON_DIR.
+ * $GIT_DIR/(sub)modules, not $GIT_COMMON_DIR.
*
- * A submodule can be moved to $GIT_DIR/modules manually by running "git
- * submodule absorbgitdirs", or it may be initialized there by "git
- * submodule update".
+ * A submodule can be moved to $GIT_DIR/(sub)modules manually by running
+ * "git submodule absorbgitdirs", or it may be initialized there by
+ * "git submodule update".
*/
static int repo_has_absorbed_submodules(struct repository *r)
{
int ret;
struct strbuf buf = STRBUF_INIT;
+ /* check legacy path */
repo_git_path_append(r, &buf, "modules/");
ret = file_exists(buf.buf) && !is_empty_dir(buf.buf);
+ strbuf_reset(&buf);
+
+ /* new (encoded name) path */
+ repo_git_path_append(r, &buf, "submodules/");
+ ret |= file_exists(buf.buf) && !is_empty_dir(buf.buf);
strbuf_release(&buf);
+
return ret;
}
@@ -2273,7 +2280,7 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
*
* Example: having a submodule named `hippo` and another one named
* `hippo/hooks` would result in the git directories
- * `.git/modules/hippo/` and `.git/modules/hippo/hooks/`, respectively,
+ * `.git/submodules/hippo/` and `.git/submodules/hippo/hooks/`, respectively,
* but the latter directory is already designated to contain the hooks
* of the former.
*/
@@ -2604,6 +2611,15 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
* administrators can explicitly set. Nothing has been decided,
* so for now, just append the name at the end of the path.
*/
+
+ /* Legacy behavior: allow existing paths under modules/<name>. */
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+ if (!access(buf->buf, F_OK))
+ return;
+
+ /* New style (encoded) paths go under submodules/<encoded>. */
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "submodules/");
+ strbuf_addstr(buf, submodule_name);
}
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 36f767cb74..b6b2be1df5 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -161,7 +161,7 @@ replace_gitfile_with_git_dir () {
}
# Test that the .git directory in the submodule is unchanged (except for the
-# core.worktree setting, which appears only in $GIT_DIR/modules/$1/config).
+# core.worktree setting, which appears only in $GIT_DIR/submodules/$1/config).
# Call this function before test_submodule_content as the latter might
# write the index file leading to false positive index differences.
#
@@ -170,23 +170,23 @@ replace_gitfile_with_git_dir () {
test_git_directory_is_unchanged () {
# does core.worktree point at the right place?
echo "../../../$1" >expect &&
- git -C ".git/modules/$1" config core.worktree >actual &&
+ git -C ".git/submodules/$1" config core.worktree >actual &&
test_cmp expect actual &&
# remove it temporarily before comparing, as
# "$1/.git/config" lacks it...
- git -C ".git/modules/$1" config --unset core.worktree &&
- diff -r ".git/modules/$1" "$1/.git" &&
+ git -C ".git/submodules/$1" config --unset core.worktree &&
+ diff -r ".git/submodules/$1" "$1/.git" &&
# ... and then restore.
- git -C ".git/modules/$1" config core.worktree "../../../$1"
+ git -C ".git/submodules/$1" config core.worktree "../../../$1"
}
test_git_directory_exists () {
- test -e ".git/modules/$1" &&
+ test -e ".git/submodules/$1" &&
if test -f sub1/.git
then
# does core.worktree point at the right place?
echo "../../../$1" >expect &&
- git -C ".git/modules/$1" config core.worktree >actual &&
+ git -C ".git/submodules/$1" config core.worktree >actual &&
test_cmp expect actual
fi
}
@@ -225,22 +225,22 @@ reset_work_tree_to () {
reset_work_tree_to_interested () {
reset_work_tree_to $1 &&
# make the submodule git dirs available
- if ! test -d submodule_update/.git/modules/sub1
+ if ! test -d submodule_update/.git/submodules/sub1
then
- mkdir -p submodule_update/.git/modules &&
- cp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1
- GIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree
+ mkdir -p submodule_update/.git/submodules &&
+ cp -r submodule_update_repo/.git/submodules/sub1 submodule_update/.git/submodules/sub1
+ GIT_WORK_TREE=. git -C submodule_update/.git/submodules/sub1 config --unset core.worktree
fi &&
- if ! test -d submodule_update/.git/modules/sub1/modules/sub2
+ if ! test -d submodule_update/.git/submodules/sub1/submodules/sub2
then
- mkdir -p submodule_update/.git/modules/sub1/modules &&
- cp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2
+ mkdir -p submodule_update/.git/submodules/sub1/submodules &&
+ cp -r submodule_update_repo/.git/submodules/sub1/submodules/sub2 submodule_update/.git/submodules/sub1/submodules/sub2
# core.worktree is unset for sub2 as it is not checked out
fi &&
# indicate we are interested in the submodule:
git -C submodule_update config submodule.sub1.url "bogus" &&
# sub1 might not be checked out, so use the git dir
- git -C submodule_update/.git/modules/sub1 config submodule.sub2.url "bogus"
+ git -C submodule_update/.git/submodules/sub1 config submodule.sub2.url "bogus"
}
# Test that the superproject contains the content according to commit "$1"
@@ -742,7 +742,7 @@ test_submodule_recursing_with_args_common () {
$command remove_sub1 &&
test_superproject_content origin/remove_sub1 &&
! test -e sub1 &&
- test_must_fail git config -f .git/modules/sub1/config core.worktree
+ test_must_fail git config -f .git/submodules/sub1/config core.worktree
)
'
# ... absorbing a .git directory along the way.
@@ -753,7 +753,7 @@ test_submodule_recursing_with_args_common () {
cd submodule_update &&
git branch -t remove_sub1 origin/remove_sub1 &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
$command remove_sub1 &&
test_superproject_content origin/remove_sub1 &&
! test -e sub1 &&
@@ -803,8 +803,8 @@ test_submodule_recursing_with_args_common () {
$command no_submodule &&
test_superproject_content origin/no_submodule &&
test_path_is_missing sub1 &&
- test_must_fail git config -f .git/modules/sub1/config core.worktree &&
- test_must_fail git config -f .git/modules/sub1/modules/sub2/config core.worktree
+ test_must_fail git config -f .git/submodules/sub1/config core.worktree &&
+ test_must_fail git config -f .git/submodules/sub1/submodules/sub2/config core.worktree
)
'
@@ -937,7 +937,7 @@ test_submodule_switch_recursing_with_args () {
cd submodule_update &&
git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
$command replace_sub1_with_directory &&
test_superproject_content origin/replace_sub1_with_directory &&
test_git_directory_exists sub1
@@ -946,15 +946,15 @@ test_submodule_switch_recursing_with_args () {
# ... and ignored files are ignored
test_expect_success "$command: replace submodule with a file works ignores ignored files in submodule" '
- test_when_finished "rm submodule_update/.git/modules/sub1/info/exclude" &&
+ test_when_finished "rm submodule_update/.git/submodules/sub1/info/exclude" &&
prolog &&
reset_work_tree_to_interested add_sub1 &&
(
cd submodule_update &&
- rm -rf .git/modules/sub1/info &&
+ rm -rf .git/submodules/sub1/info &&
git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
- mkdir .git/modules/sub1/info &&
- echo ignored >.git/modules/sub1/info/exclude &&
+ mkdir .git/submodules/sub1/info &&
+ echo ignored >.git/submodules/sub1/info/exclude &&
: >sub1/ignored &&
$command replace_sub1_with_file &&
test_superproject_content origin/replace_sub1_with_file &&
@@ -1034,7 +1034,7 @@ test_submodule_forced_switch_recursing_with_args () {
cd submodule_update &&
git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules/sub1 &&
+ rm -rf .git/submodules/sub1 &&
$command replace_sub1_with_directory &&
test_superproject_content origin/replace_sub1_with_directory &&
test_git_directory_exists sub1
diff --git a/t/t0035-safe-bare-repository.sh b/t/t0035-safe-bare-repository.sh
index ae7ef092ab..a480ddf8d6 100755
--- a/t/t0035-safe-bare-repository.sh
+++ b/t/t0035-safe-bare-repository.sh
@@ -41,7 +41,7 @@ test_expect_success 'setup an embedded bare repo, secondary worktree and submodu
submodule add --name subn -- ./bare-repo subd
) &&
test_path_is_dir outer-repo/.git/worktrees/outer-secondary &&
- test_path_is_dir outer-repo/.git/modules/subn
+ test_path_is_dir outer-repo/.git/submodules/subn
'
test_expect_success 'safe.bareRepository unset' '
@@ -100,7 +100,7 @@ test_expect_success 'no trace in $GIT_DIR of secondary worktree' '
'
test_expect_success 'no trace in $GIT_DIR of a submodule' '
- expect_accepted_implicit -C outer-repo/.git/modules/subn
+ expect_accepted_implicit -C outer-repo/.git/submodules/subn
'
test_done
diff --git a/t/t1600-index.sh b/t/t1600-index.sh
index 03239e9faa..0e5e8efb20 100755
--- a/t/t1600-index.sh
+++ b/t/t1600-index.sh
@@ -87,10 +87,10 @@ test_expect_success 'index.skipHash config option' '
git -c protocol.file.allow=always submodule add ./ sub &&
git config index.skipHash false &&
git -C sub config index.skipHash true &&
- rm -f .git/modules/sub/index &&
+ rm -f .git/submodules/sub/index &&
>sub/file &&
git -C sub add a &&
- test_trailing_hash .git/modules/sub/index >hash &&
+ test_trailing_hash .git/submodules/sub/index >hash &&
test_cmp expect hash &&
git -C sub fsck
'
diff --git a/t/t2405-worktree-submodule.sh b/t/t2405-worktree-submodule.sh
index 11018f37c7..c18c2efca5 100755
--- a/t/t2405-worktree-submodule.sh
+++ b/t/t2405-worktree-submodule.sh
@@ -62,7 +62,7 @@ test_expect_success 'submodule is checked out after manually adding submodule wo
test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules in a linked worktree' '
git -C main worktree add "$base_path/checkout-recurse" --detach &&
git -C checkout-recurse submodule update --init &&
- echo "gitdir: ../../main/.git/worktrees/checkout-recurse/modules/sub" >expect-gitfile &&
+ echo "gitdir: ../../main/.git/worktrees/checkout-recurse/submodules/sub" >expect-gitfile &&
cat checkout-recurse/sub/.git >actual-gitfile &&
test_cmp expect-gitfile actual-gitfile &&
git -C main/sub rev-parse HEAD >expect-head-main &&
@@ -73,7 +73,7 @@ test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules
test_cmp expect-head-main actual-head-main
'
-test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config, not in $GIT_COMMON_DIR/modules/<name>/config' '
+test_expect_success 'core.worktree is removed in $GIT_DIR/submodules/<name>/config, not in $GIT_COMMON_DIR/submodules/<name>/config' '
echo "../../../sub" >expect-main &&
git -C main/sub config --get core.worktree >actual-main &&
test_cmp expect-main actual-main &&
@@ -81,14 +81,14 @@ test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config,
git -C checkout-recurse/sub config --get core.worktree >actual-linked &&
test_cmp expect-linked actual-linked &&
git -C checkout-recurse checkout --recurse-submodules first &&
- test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/modules/sub config --get core.worktree >linked-config &&
+ test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/submodules/sub config --get core.worktree >linked-config &&
test_must_be_empty linked-config &&
git -C main/sub config --get core.worktree >actual-main &&
test_cmp expect-main actual-main
'
test_expect_success 'unsetting core.worktree does not prevent running commands directly against the submodule repository' '
- git -C main/.git/worktrees/checkout-recurse/modules/sub log
+ git -C main/.git/worktrees/checkout-recurse/submodules/sub log
'
test_done
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index be9140bbaa..bb8751433f 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -239,7 +239,7 @@ test_submodule_removal () {
test "$path_status" = dir && test_status=test_must_fail
test_when_finished "git reset --hard HEAD~1" &&
- test_when_finished "rm -rf .git/modules/my_submodule" &&
+ test_when_finished "rm -rf .git/submodules/my_submodule" &&
git checkout foo/bar/baz &&
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 1f16e6b522..5b8ed57538 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -582,7 +582,7 @@ test_expect_success 'rm of a conflicted populated submodule with a .git director
(
cd submod &&
rm .git &&
- cp -R ../.git/modules/sub .git &&
+ cp -R ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
test_must_fail git merge conflict2 &&
@@ -617,9 +617,9 @@ test_expect_success 'rm of a populated submodule with a .git directory migrates
(
cd submod &&
rm .git &&
- cp -R ../.git/modules/sub .git &&
+ cp -R ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree &&
- rm -r ../.git/modules/sub
+ rm -r ../.git/submodules/sub
) &&
git rm submod 2>output.err &&
test_path_is_missing submod &&
@@ -709,7 +709,7 @@ test_expect_success "rm absorbs submodule's nested .git directory" '
(
cd submod/subsubmod &&
rm .git &&
- mv ../../.git/modules/sub/modules/sub .git &&
+ mv ../../.git/submodules/sub/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
git rm submod 2>output.err &&
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 5e566205ba..b7385bc088 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -1143,7 +1143,7 @@ test_expect_success 'fetch --recurse-submodules updates name-conflicted, unpopul
head1=$(git -C same-name-1/submodule rev-parse HEAD) &&
head2=$(git -C same-name-2/submodule rev-parse HEAD) &&
(
- cd same-name-downstream/.git/modules/submodule &&
+ cd same-name-downstream/.git/submodules/submodule &&
# The submodule has core.worktree pointing to the "git
# rm"-ed directory, overwrite the invalid value. See
# comment in get_fetch_task_from_changed() for more
diff --git a/t/t5619-clone-local-ambiguous-transport.sh b/t/t5619-clone-local-ambiguous-transport.sh
index cce62bf78d..cf2d5e7bfb 100755
--- a/t/t5619-clone-local-ambiguous-transport.sh
+++ b/t/t5619-clone-local-ambiguous-transport.sh
@@ -38,7 +38,7 @@ test_expect_success 'setup' '
ln -s "$(cd .. && pwd)/sensitive" repo/objects &&
mkdir -p "$HTTPD_URL/dumb" &&
- ln -s "../../../.git/modules/sub/../../../repo/" "$URI" &&
+ ln -s "../../../.git/submodules/sub/../../../repo/" "$URI" &&
git add . &&
git commit -m "initial commit"
@@ -57,7 +57,7 @@ test_expect_success 'ambiguous transport does not lead to arbitrary file-inclusi
git clone malicious clone &&
test_must_fail git -C clone submodule update --init 2>err &&
- test_path_is_missing clone/.git/modules/sub/objects/secret &&
+ test_path_is_missing clone/.git/submodules/sub/objects/secret &&
# We would actually expect "transport .file. not allowed" here,
# but due to quirks of the URL detection in Git, we mis-parse
# the absolute path as a bogus URL and die before that step.
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 256ccaefb7..56460ae8b5 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -357,7 +357,7 @@ test_expect_success 'setup and absorb a submodule' '
'
test_expect_success 'describe chokes on severely broken submodules' '
- mv .git/modules/sub1/ .git/modules/sub_moved &&
+ mv .git/submodules/sub1/ .git/submodules/sub_moved &&
test_must_fail git describe --dirty
'
@@ -371,7 +371,7 @@ test_expect_success 'describe with --work-tree ignoring a broken submodule' '
cd "$TEST_DIRECTORY" &&
git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --broken >"$TRASH_DIRECTORY/out"
) &&
- test_when_finished "mv .git/modules/sub_moved .git/modules/sub1" &&
+ test_when_finished "mv .git/submodules/sub_moved .git/submodules/sub1" &&
grep broken out
'
diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
index 920479e925..89b06ae3c1 100755
--- a/t/t7001-mv.sh
+++ b/t/t7001-mv.sh
@@ -360,7 +360,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and no .gitm
(
cd sub &&
rm -f .git &&
- cp -R -P -p ../.git/modules/sub .git &&
+ cp -R -P -p ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
mkdir mod &&
@@ -380,7 +380,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
(
cd sub &&
rm -f .git &&
- cp -R -P -p ../.git/modules/sub .git &&
+ cp -R -P -p ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
mkdir mod &&
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index fd3e7e355e..178c386212 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -163,7 +163,7 @@ test_expect_success 'submodule add' '
cd addtest &&
git submodule add -q "$submodurl" submod >actual &&
test_must_be_empty actual &&
- echo "gitdir: ../.git/modules/submod" >expect &&
+ echo "gitdir: ../.git/submodules/submod" >expect &&
test_cmp expect submod/.git &&
(
cd submod &&
@@ -976,21 +976,21 @@ test_expect_success 'submodule add --name allows to replace a submodule with ano
echo "$submodurl/repo" >expect &&
git config remote.origin.url >actual &&
test_cmp expect actual &&
- echo "gitdir: ../.git/modules/repo" >expect &&
+ echo "gitdir: ../.git/submodules/repo" >expect &&
test_cmp expect .git
) &&
rm -rf repo &&
git rm repo &&
git submodule add -q --name repo_new "$submodurl/bare.git" repo >actual &&
test_must_be_empty actual &&
- echo "gitdir: ../.git/modules/submod" >expect &&
+ echo "gitdir: ../.git/submodules/submod" >expect &&
test_cmp expect submod/.git &&
(
cd repo &&
echo "$submodurl/bare.git" >expect &&
git config remote.origin.url >actual &&
test_cmp expect actual &&
- echo "gitdir: ../.git/modules/repo_new" >expect &&
+ echo "gitdir: ../.git/submodules/repo_new" >expect &&
test_cmp expect .git
) &&
echo "repo" >expect &&
@@ -1045,8 +1045,8 @@ test_expect_success 'recursive relative submodules stay relative' '
(
cd clone2 &&
git submodule update --init --recursive &&
- echo "gitdir: ../.git/modules/sub3" >./sub3/.git_expect &&
- echo "gitdir: ../../../.git/modules/sub3/modules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
+ echo "gitdir: ../.git/submodules/sub3" >./sub3/.git_expect &&
+ echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
) &&
test_cmp clone2/sub3/.git_expect clone2/sub3/.git &&
test_cmp clone2/sub3/dirdir/subsub/.git_expect clone2/sub3/dirdir/subsub/.git
@@ -1108,8 +1108,8 @@ test_expect_success 'submodule deinit should remove the whole submodule section
'
test_expect_success 'submodule deinit should unset core.worktree' '
- test_path_is_file .git/modules/example/config &&
- test_must_fail git config -f .git/modules/example/config core.worktree
+ test_path_is_file .git/submodules/example/config &&
+ test_must_fail git config -f .git/submodules/example/config core.worktree
'
test_expect_success 'submodule deinit from subdirectory' '
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule deinit absorbs .git directory if .git is a direct
(
cd init &&
rm .git &&
- mv ../.git/modules/example .git &&
+ mv ../.git/submodules/example .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
git submodule deinit init &&
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 3adab12091..f0c4da1ffa 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -864,7 +864,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/modules/deeper/submodule &&
+ (cd .git/submodules/deeper/submodule &&
git log > ../../../../actual
) &&
test_cmp expected actual
@@ -882,7 +882,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/modules/deeper/submodule &&
+ (cd .git/submodules/deeper/submodule &&
git log > ../../../../actual
) &&
test_cmp expected actual
@@ -899,7 +899,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir recur
git commit -m "added subsubmodule" &&
git push origin :
) &&
- (cd .git/modules/deeper/submodule/modules/subsubmodule &&
+ (cd .git/submodules/deeper/submodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
git add deeper/submodule &&
@@ -949,7 +949,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir re
(cd submodule/subsubmodule &&
git log > ../../expected
) &&
- (cd .git/modules/submodule/modules/subsubmodule &&
+ (cd .git/submodules/submodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
test_cmp expected actual
@@ -1298,7 +1298,7 @@ test_expect_success CASE_INSENSITIVE_FS,SYMLINKS \
git init captain &&
(
cd captain &&
- git submodule add --name x/y "$hook_repo_path" A/modules/x &&
+ git submodule add --name x/y "$hook_repo_path" A/submodules/x &&
test_tick &&
git commit -m add-submodule &&
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 77b6d0040e..75ba826968 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -368,9 +368,9 @@ test_expect_success 'test "update --recursive" with a flag with spaces' '
git rev-parse --resolve-git-dir nested1/.git &&
git rev-parse --resolve-git-dir nested1/nested2/.git &&
git rev-parse --resolve-git-dir nested1/nested2/nested3/.git &&
- test -f .git/modules/nested1/objects/info/alternates &&
- test -f .git/modules/nested1/modules/nested2/objects/info/alternates &&
- test -f .git/modules/nested1/modules/nested2/modules/nested3/objects/info/alternates
+ test -f .git/submodules/nested1/objects/info/alternates &&
+ test -f .git/submodules/nested1/submodules/nested2/objects/info/alternates &&
+ test -f .git/submodules/nested1/submodules/nested2/submodules/nested3/objects/info/alternates
)
'
diff --git a/t/t7408-submodule-reference.sh b/t/t7408-submodule-reference.sh
index f860e7bbf4..25f4aec57e 100755
--- a/t/t7408-submodule-reference.sh
+++ b/t/t7408-submodule-reference.sh
@@ -61,7 +61,7 @@ test_expect_success 'submodule add --reference uses alternates' '
git commit -m B-super-added &&
git repack -ad
) &&
- test_alternate_is_used super/.git/modules/sub/objects/info/alternates super/sub
+ test_alternate_is_used super/.git/submodules/sub/objects/info/alternates super/sub
'
test_expect_success 'submodule add --reference with --dissociate does not use alternates' '
@@ -71,7 +71,7 @@ test_expect_success 'submodule add --reference with --dissociate does not use al
git commit -m B-super-added &&
git repack -ad
) &&
- test_path_is_missing super/.git/modules/sub-dissociate/objects/info/alternates
+ test_path_is_missing super/.git/submodules/sub-dissociate/objects/info/alternates
'
test_expect_success 'that reference gets used with add' '
@@ -94,14 +94,14 @@ test_expect_success 'updating superproject keeps alternates' '
test_when_finished "rm -rf super-clone" &&
git clone super super-clone &&
git -C super-clone submodule update --init --reference ../B &&
- test_alternate_is_used super-clone/.git/modules/sub/objects/info/alternates super-clone/sub
+ test_alternate_is_used super-clone/.git/submodules/sub/objects/info/alternates super-clone/sub
'
test_expect_success 'updating superproject with --dissociate does not keep alternates' '
test_when_finished "rm -rf super-clone" &&
git clone super super-clone &&
git -C super-clone submodule update --init --reference ../B --dissociate &&
- test_path_is_missing super-clone/.git/modules/sub/objects/info/alternates
+ test_path_is_missing super-clone/.git/submodules/sub/objects/info/alternates
'
test_expect_success 'submodules use alternates when cloning a superproject' '
@@ -112,7 +112,7 @@ test_expect_success 'submodules use alternates when cloning a superproject' '
# test superproject has alternates setup correctly
test_alternate_is_used .git/objects/info/alternates . &&
# test submodule has correct setup
- test_alternate_is_used .git/modules/sub/objects/info/alternates sub
+ test_alternate_is_used .git/submodules/sub/objects/info/alternates sub
)
'
@@ -127,7 +127,7 @@ test_expect_success 'missing submodule alternate fails clone and submodule updat
# update of the submodule succeeds
test_must_fail git submodule update --init &&
# and we have no alternates:
- test_path_is_missing .git/modules/sub/objects/info/alternates &&
+ test_path_is_missing .git/submodules/sub/objects/info/alternates &&
test_path_is_missing sub/file1
)
'
@@ -142,7 +142,7 @@ test_expect_success 'ignoring missing submodule alternates passes clone and subm
# update of the submodule succeeds
git submodule update --init &&
# and we have no alternates:
- test_path_is_missing .git/modules/sub/objects/info/alternates &&
+ test_path_is_missing .git/submodules/sub/objects/info/alternates &&
test_path_is_file sub/file1
)
'
@@ -176,18 +176,18 @@ test_expect_success 'nested submodule alternate in works and is actually used' '
# test superproject has alternates setup correctly
test_alternate_is_used .git/objects/info/alternates . &&
# immediate submodule has alternate:
- test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
+ test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
# nested submodule also has alternate:
- test_alternate_is_used .git/modules/subwithsub/modules/sub/objects/info/alternates subwithsub/sub
+ test_alternate_is_used .git/submodules/subwithsub/submodules/sub/objects/info/alternates subwithsub/sub
)
'
check_that_two_of_three_alternates_are_used() {
test_alternate_is_used .git/objects/info/alternates . &&
# immediate submodule has alternate:
- test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
+ test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
# but nested submodule has no alternate:
- test_path_is_missing .git/modules/subwithsub/modules/sub/objects/info/alternates
+ test_path_is_missing .git/submodules/subwithsub/submodules/sub/objects/info/alternates
}
diff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh
index 0490499573..dbaca9c69f 100755
--- a/t/t7412-submodule-absorbgitdirs.sh
+++ b/t/t7412-submodule-absorbgitdirs.sh
@@ -29,13 +29,13 @@ test_expect_success 'absorb the git dir' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1'\'' from
'\''$cwd/sub1/.git'\'' to
- '\''$cwd/.git/modules/sub1'\''
+ '\''$cwd/.git/submodules/sub1'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
git fsck &&
test -f sub1/.git &&
- test -d .git/modules/sub1 &&
+ test -d .git/submodules/sub1 &&
git status >actual.1 &&
git -C sub1 rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -47,7 +47,7 @@ test_expect_success 'absorbing does not fail for deinitialized submodules' '
git submodule deinit --all &&
git submodule absorbgitdirs 2>err &&
test_must_be_empty err &&
- test -d .git/modules/sub1 &&
+ test -d .git/submodules/sub1 &&
test -d sub1 &&
! test -e sub1/.git
'
@@ -68,12 +68,12 @@ test_expect_success 'absorb the git dir in a nested submodule' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1/nested'\'' from
'\''$cwd/sub1/nested/.git'\'' to
- '\''$cwd/.git/modules/sub1/modules/nested'\''
+ '\''$cwd/.git/submodules/sub1/submodules/nested'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
test -f sub1/nested/.git &&
- test -d .git/modules/sub1/modules/nested &&
+ test -d .git/submodules/sub1/submodules/nested &&
git status >actual.1 &&
git -C sub1/nested rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -84,11 +84,11 @@ test_expect_success 're-setup nested submodule' '
# un-absorb the direct submodule, to test if the nested submodule
# is still correct (needs a rewrite of the gitfile only)
rm -rf sub1/.git &&
- mv .git/modules/sub1 sub1/.git &&
+ mv .git/submodules/sub1 sub1/.git &&
GIT_WORK_TREE=. git -C sub1 config --unset core.worktree &&
# fixup the nested submodule
- echo "gitdir: ../.git/modules/nested" >sub1/nested/.git &&
- GIT_WORK_TREE=../../../nested git -C sub1/.git/modules/nested config \
+ echo "gitdir: ../.git/submodules/nested" >sub1/nested/.git &&
+ GIT_WORK_TREE=../../../nested git -C sub1/.git/submodules/nested config \
core.worktree "../../../nested" &&
# make sure this re-setup is correct
git status --ignore-submodules=none &&
@@ -105,13 +105,13 @@ test_expect_success 'absorb the git dir in a nested submodule' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1'\'' from
'\''$cwd/sub1/.git'\'' to
- '\''$cwd/.git/modules/sub1'\''
+ '\''$cwd/.git/submodules/sub1'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
test -f sub1/.git &&
test -f sub1/nested/.git &&
- test -d .git/modules/sub1/modules/nested &&
+ test -d .git/submodules/sub1/submodules/nested &&
git status >actual.1 &&
git -C sub1/nested rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -133,7 +133,7 @@ test_expect_success 'absorb the git dir outside of primary worktree' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub2'\'' from
'\''$cwd/repo-wt/sub2/.git'\'' to
- '\''$cwd/repo-bare.git/worktrees/repo-wt/modules/sub2'\''
+ '\''$cwd/repo-bare.git/worktrees/repo-wt/submodules/sub2'\''
EOF
git -C repo-wt submodule absorbgitdirs 2>actual &&
test_cmp expect actual
diff --git a/t/t7423-submodule-symlinks.sh b/t/t7423-submodule-symlinks.sh
index 3d3c7af3ce..a51235136d 100755
--- a/t/t7423-submodule-symlinks.sh
+++ b/t/t7423-submodule-symlinks.sh
@@ -49,19 +49,19 @@ test_expect_success SYMLINKS 'git restore --recurse-submodules must not be confu
test_expect_success SYMLINKS 'git restore --recurse-submodules must not migrate git dir of symlinked repo' '
prepare_symlink_to_repo &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
test_must_fail git restore --recurse-submodules a/sm &&
test_path_is_dir a/target/.git &&
- test_path_is_missing .git/modules/a/sm &&
+ test_path_is_missing .git/submodules/a/sm &&
test_path_is_missing a/target/submodule_file
'
test_expect_success SYMLINKS 'git checkout -f --recurse-submodules must not migrate git dir of symlinked repo when removing submodule' '
prepare_symlink_to_repo &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
test_must_fail git checkout -f --recurse-submodules initial &&
test_path_is_dir a/target/.git &&
- test_path_is_missing .git/modules/a/sm
+ test_path_is_missing .git/submodules/a/sm
'
test_done
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index f512eed278..4e2ced3636 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -77,28 +77,28 @@ test_expect_success 'create innocent subrepo' '
test_expect_success 'submodule add refuses invalid names' '
test_must_fail \
- git submodule add --name ../../modules/evil "$PWD/innocent" evil
+ git submodule add --name ../../submodules/evil "$PWD/innocent" evil
'
test_expect_success 'add evil submodule' '
git submodule add "$PWD/innocent" evil &&
- mkdir modules &&
- cp -r .git/modules/evil modules &&
- write_script modules/evil/hooks/post-checkout <<-\EOF &&
+ mkdir submodules &&
+ cp -r .git/submodules/evil submodules &&
+ write_script submodules/evil/hooks/post-checkout <<-\EOF &&
echo >&2 "RUNNING POST CHECKOUT"
EOF
git config -f .gitmodules submodule.evil.update checkout &&
git config -f .gitmodules --rename-section \
- submodule.evil submodule.../../modules/evil &&
- git add modules &&
+ submodule.evil submodule.../../submodules/evil &&
+ git add submodules &&
git commit -am evil
'
# This step seems like it shouldn't be necessary, since the payload is
# contained entirely in the evil submodule. But due to the vagaries of the
-# submodule code, checking out the evil module will fail unless ".git/modules"
+# submodule code, checking out the evil module will fail unless ".git/submodules"
# exists. Adding another submodule (with a name that sorts before "evil") is an
# easy way to make sure this is the case in the victim clone.
test_expect_success 'add other submodule' '
@@ -350,8 +350,8 @@ test_expect_success 'submodule git dir nesting detection must work with parallel
cat err &&
grep -E "(already exists|is inside git dir|not a git repository)" err &&
{
- test_path_is_missing .git/modules/hippo/HEAD ||
- test_path_is_missing .git/modules/hippo/hooks/HEAD
+ test_path_is_missing .git/submodules/hippo/HEAD ||
+ test_path_is_missing .git/submodules/hippo/hooks/HEAD
}
'
@@ -361,10 +361,10 @@ test_expect_success 'checkout -f --recurse-submodules must not use a nested gitd
cd nested_checkout &&
git submodule init &&
git submodule update thing1 &&
- mkdir -p .git/modules/hippo/hooks/refs &&
- mkdir -p .git/modules/hippo/hooks/objects/info &&
- echo "../../../../objects" >.git/modules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/modules/hippo/hooks/HEAD
+ mkdir -p .git/submodules/hippo/hooks/refs &&
+ mkdir -p .git/submodules/hippo/hooks/objects/info &&
+ echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD
) &&
test_must_fail git -C nested_checkout checkout -f --recurse-submodules HEAD 2>err &&
cat err &&
@@ -390,13 +390,13 @@ test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into d
git config unset -f repo/.gitmodules submodule.sub.path &&
printf "\tpath = \"sub\r\"\n" >>repo/.gitmodules &&
- git config unset -f repo/.git/modules/sub/config core.worktree &&
+ git config unset -f repo/.git/submodules/sub/config core.worktree &&
{
printf "[core]\n" &&
printf "\tworktree = \"../../../sub\r\"\n"
- } >>repo/.git/modules/sub/config &&
+ } >>repo/.git/submodules/sub/config &&
- ln -s .git/modules/sub/hooks repo/sub &&
+ ln -s .git/submodules/sub/hooks repo/sub &&
git -C repo add -A &&
git -C repo commit -m submodule &&
diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh
index 409cd0cd12..ded482fdf2 100755
--- a/t/t7527-builtin-fsmonitor.sh
+++ b/t/t7527-builtin-fsmonitor.sh
@@ -866,7 +866,7 @@ test_expect_success 'submodule always visited' '
'
# If a submodule has a `sub/.git/` directory (rather than a file
-# pointing to the super's `.git/modules/sub`) and `core.fsmonitor`
+# pointing to the super's `.git/submodules/sub`) and `core.fsmonitor`
# turned on in the submodule and the daemon is not yet started in
# the submodule, and someone does a `git submodule absorbgitdirs`
# in the super, Git will recursively invoke `git submodule--helper`
@@ -895,7 +895,7 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" '
cat >expect <<-EOF &&
Migrating git directory of '\''dir_1/dir_2/sub'\'' from
'\''$cwd/dir_1/dir_2/sub/.git'\'' to
- '\''$cwd/.git/modules/dir_1/dir_2/sub'\''
+ '\''$cwd/.git/submodules/dir_1/dir_2/sub'\''
EOF
GIT_TRACE2_EVENT="$PWD/super-sub.trace" \
git -C super submodule absorbgitdirs >out 2>actual &&
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 3/9] submodule: add gitdir path config override
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-08-16 21:36 ` [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-08-16 21:36 ` [PATCH 2/9] submodule: create new gitdirs under submodules path Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-20 19:37 ` Josh Steadmon
` (3 more replies)
2025-08-16 21:36 ` [PATCH 4/9] t: submodules: add basic mixed gitdir path tests Adrian Ratiu
` (12 subsequent siblings)
15 siblings, 4 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu, Brandon Williams
This adds an ability to override gitdir paths via config files
(not .gitmodules), such that any encoding scheme can be changed
and JGit & co don't need to exactly match the default encoding.
A new test and a helper are added. The helper will be used by
further tests exercising gitdir paths & encodings.
Based-on-patch-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 17 +++++++++++++++++
submodule.c | 11 +++++++++++
t/lib-verify-submodule-gitdir-path.sh | 15 +++++++++++++++
t/t7400-submodule-basic.sh | 15 +++++++++++++++
4 files changed, 58 insertions(+)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7243429c6f..30e40d6c79 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1214,6 +1214,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo UNUSED)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, the_repository, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3597,6 +3613,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
diff --git a/submodule.c b/submodule.c
index dbf2244e60..bf78636195 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2611,6 +2611,17 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
* administrators can explicitly set. Nothing has been decided,
* so for now, just append the name at the end of the path.
*/
+ char *gitdir_path, *key;
+
+ /* Allow config override. */
+ key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
+ if (!repo_config_get_string(r, key, &gitdir_path)) {
+ strbuf_addstr(buf, gitdir_path);
+ free(key);
+ free(gitdir_path);
+ return;
+ }
+ free(key);
/* Legacy behavior: allow existing paths under modules/<name>. */
repo_git_path_append(r, buf, "modules/");
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..fb5cb8eea4
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,15 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir in path $3
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ cat >expect <<-EOF &&
+ $(git rev-parse --git-common-dir)/$path
+ EOF
+ git submodule--helper gitdir "$name" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 178c386212..f4d4fb8397 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup - enable local submodules' '
git config --global protocol.file.allow always
@@ -1505,4 +1506,18 @@ test_expect_success 'submodule add fails when name is reused' '
)
'
+test_expect_success 'submodule helper gitdir config overrides' '
+ verify_submodule_gitdir_path test-submodule child submodules/child &&
+ (
+ cd test-submodule &&
+ git config submodule.child.gitdirpath ".git/submodules/custom-child"
+ ) &&
+ verify_submodule_gitdir_path test-submodule child submodules/custom-child &&
+ (
+ cd test-submodule &&
+ git config --unset submodule.child.gitdirpath
+ ) &&
+ verify_submodule_gitdir_path test-submodule child submodules/child
+'
+
test_done
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 4/9] t: submodules: add basic mixed gitdir path tests
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (2 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-20 22:07 ` Josh Steadmon
2025-09-02 23:02 ` Junio C Hamano
2025-08-16 21:36 ` [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
` (11 subsequent siblings)
15 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
Add some basic submodule tests for mixed gitdir path handling of
legacy (.git/modules) and new-style (.git/submodule) paths.
For now these just test the coexistence, creation and push/pull of
submodules using mixed paths.
More tests will be added later, especially for new-style encoding.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/meson.build | 1 +
t/t7425-submodule-mixed-gitdir-paths.sh | 101 ++++++++++++++++++++++++
2 files changed, 102 insertions(+)
create mode 100755 t/t7425-submodule-mixed-gitdir-paths.sh
diff --git a/t/meson.build b/t/meson.build
index bbeba1a8d5..ffd74f1d3b 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -874,6 +874,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-mixed-gitdir-paths.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
new file mode 100755
index 0000000000..801e90522a
--- /dev/null
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed new and legacy submodules' '
+ git init legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init new-sub &&
+ test_commit -C new-sub new-initial &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init main &&
+ (
+ cd main &&
+
+ git config receive.denyCurrentBranch updateInstead &&
+
+ git submodule add ../new-sub new &&
+ test_commit new-sub &&
+
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ # simulate legacy .git/modules path by moving submodule
+ mkdir -p .git/modules &&
+ mv .git/submodules/legacy .git/modules/ &&
+ echo "gitdir: ../.git/modules/legacy" > legacy/.git
+ )
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned &&
+ (
+ cd cloned &&
+
+ # At this point, .git/modules/<name> should not exist as
+ # submodules are checked out into the new path
+ test_path_is_dir .git/submodules/legacy &&
+ test_path_is_dir .git/submodules/new &&
+
+ git submodule status >list &&
+ grep "$legacy_rev legacy" list &&
+ grep "$new_rev new" list
+ )
+'
+
+test_expect_success 'commit and push changes to submodules' '
+ (
+ cd cloned &&
+
+ git -C legacy switch --track -C master origin/master &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C new switch --track -C master origin/master &&
+ test_commit -C new second-commit &&
+ git -C new push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C master origin/master &&
+ git add legacy new &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/submodules/new &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ grep "$legacy_rev legacy" list &&
+ grep "$new_rev new" list
+ )
+'
+
+test_done
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (3 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 4/9] t: submodules: add basic mixed gitdir path tests Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-16 21:56 ` Ben Knoble
2025-08-16 21:36 ` [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
` (10 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
Commit f89854362c ("credential-store: move related functions to...")
moved the function inside credential-store.c, making it static under
the correct assumption (at the time) that it's the only place used.
However now we need it to apply url encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 6 ------
strbuf.c | 6 ++++++
strbuf.h | 2 ++
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..0acaf1cc82 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -76,12 +76,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/strbuf.c b/strbuf.c
index 6c3851a7f8..e8d84cbb6d 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -817,6 +817,12 @@ void strbuf_addstr_xml_quoted(struct strbuf *buf, const char *s)
}
}
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
static void strbuf_add_urlencode(struct strbuf *sb, const char *s, size_t len,
char_predicate allow_unencoded_fn)
{
diff --git a/strbuf.h b/strbuf.h
index a580ac6084..5139269039 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -640,6 +640,8 @@ static inline void strbuf_complete_line(struct strbuf *sb)
typedef int (*char_predicate)(char ch);
+int is_rfc3986_unreserved(char ch);
+
void strbuf_addstr_urlencode(struct strbuf *sb, const char *name,
char_predicate allow_unencoded_fn);
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (4 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-20 19:29 ` Jeff King
2025-08-16 21:36 ` [PATCH 7/9] submodule: remove validate_submodule_git_dir() Adrian Ratiu
` (9 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu, Brandon Williams
This follows up commit ce125d431a ("submodule: extract path to submodule
gitdir func") to resolve gitdir path conflicts, eg. names "a" and "a/b",
by encoding the submodule names before the paths are built.
Based on previous work by Brandon & all [1].
A custom encoding can become unnecesarily complex, while url-encoding is
relatively well-known, however it needs some extending to support case
insensitive filesystems and quirks like Windows reserving "COM1" names.
Hence why I opted to encode A as _a, B as _b and so on, which also fixes
the COM1 case, as suggested in [1].
The current implementation errors out if the encoded name is too long.
This can be improved, for e.g. the encoded name could be trimmed as the
conflict probability is low at the NAME_MAX length mark, or long names
could be sharded into subdirectories in the future.
This also fixes the existing affected tests and adds a TODO to cleanup
the short-circuit in validate_submodule_git_dir().
A further commit will some more tests to exercise these codepaths.
Link: https://lore.kernel.org/git/20180807230637.247200-1-bmwill@google.com/ [1]
Based-on-patch-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 65 ++++++++++++++++++++++++-------------
t/t7400-submodule-basic.sh | 2 +-
t/t7406-submodule-update.sh | 10 +++---
t/t7450-bad-git-dotfiles.sh | 39 ++++++++++++----------
4 files changed, 69 insertions(+), 47 deletions(-)
diff --git a/submodule.c b/submodule.c
index bf78636195..722a8e4f2a 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2271,8 +2271,13 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
- BUG("submodule name '%s' not a suffix of git dir '%s'",
- submodule_name, git_dir);
+ /*
+ * TODO: revisit and cleanup this test short-circuit, because
+ * submodules with encoded names are expected to take this path.
+ * Likely just move the invariants to submodule_name_to_gitdir()
+ * and delete this entire function in a future commit.
+ */
+ return 0;
/*
* We prevent the contents of sibling submodules' git directories to
@@ -2588,30 +2593,26 @@ int submodule_to_gitdir(struct repository *repo,
return ret;
}
+static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
+{
+ for (; *src; src++) {
+ unsigned char c = *src;
+ if (c >= 'A' && c <= 'Z') {
+ strbuf_addch(dst, '_');
+ strbuf_addch(dst, c - 'A' + 'a');
+ } else {
+ strbuf_addch(dst, c);
+ }
+ }
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
- /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
- */
+ struct strbuf encoded_sub_name = STRBUF_INIT, tmp = STRBUF_INIT;
+ size_t base_len, encoded_len;
char *gitdir_path, *key;
+ long name_max;
/* Allow config override. */
key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
@@ -2632,5 +2633,23 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
/* New style (encoded) paths go under submodules/<encoded>. */
strbuf_reset(buf);
repo_git_path_append(r, buf, "submodules/");
- strbuf_addstr(buf, submodule_name);
+ base_len = buf->len;
+
+ /* URL-encode then case case-encode A to _a, B to _b and so on */
+ strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
+ strbuf_addstr_case_encode(&encoded_sub_name, tmp.buf);
+ strbuf_release(&tmp);
+ strbuf_addbuf(buf, &encoded_sub_name);
+
+ /* Ensure final path length is below NAME_MAX after encoding */
+ name_max = pathconf(buf->buf, _PC_NAME_MAX);
+ if (name_max == -1)
+ name_max = NAME_MAX;
+
+ encoded_len = buf->len - base_len;
+ if (encoded_len >= name_max)
+ die(_("encoded submodule name '%s' is too long (%zu bytes, limit is %ld)"),
+ encoded_sub_name.buf, encoded_len, name_max);
+
+ strbuf_release(&encoded_sub_name);
}
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index f4d4fb8397..607fdd26bd 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1047,7 +1047,7 @@ test_expect_success 'recursive relative submodules stay relative' '
cd clone2 &&
git submodule update --init --recursive &&
echo "gitdir: ../.git/submodules/sub3" >./sub3/.git_expect &&
- echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
+ echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir%2fsubsub" >./sub3/dirdir/subsub/.git_expect
) &&
test_cmp clone2/sub3/.git_expect clone2/sub3/.git &&
test_cmp clone2/sub3/dirdir/subsub/.git_expect clone2/sub3/dirdir/subsub/.git
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index f0c4da1ffa..c44a7e9513 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -864,8 +864,8 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/submodules/deeper/submodule &&
- git log > ../../../../actual
+ (cd .git/submodules/deeper%2fsubmodule &&
+ git log > ../../../actual
) &&
test_cmp expected actual
)
@@ -882,8 +882,8 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/submodules/deeper/submodule &&
- git log > ../../../../actual
+ (cd .git/submodules/deeper%2fsubmodule &&
+ git log > ../../../actual
) &&
test_cmp expected actual
)
@@ -899,7 +899,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir recur
git commit -m "added subsubmodule" &&
git push origin :
) &&
- (cd .git/submodules/deeper/submodule/submodules/subsubmodule &&
+ (cd .git/submodules/deeper%2fsubmodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
git add deeper/submodule &&
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index 4e2ced3636..27254300f8 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -15,6 +15,7 @@ Such as:
. ./test-lib.sh
. "$TEST_DIRECTORY"/lib-pack.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup' '
git config --global protocol.file.allow always
@@ -319,6 +320,8 @@ test_expect_success WINDOWS 'prevent git~1 squatting on Windows' '
fi
'
+# TODO: move these nested gitdir tests to another location in a later commit because
+# they are not pathological cases anymore: by encoding the gitdir paths do not conflict.
test_expect_success 'setup submodules with nested git dirs' '
git init nested &&
test_commit -C nested nested &&
@@ -341,35 +344,35 @@ test_expect_success 'setup submodules with nested git dirs' '
'
test_expect_success 'git dirs of sibling submodules must not be nested' '
- test_must_fail git clone --recurse-submodules nested clone 2>err &&
- test_grep "is inside git dir" err
+ git clone --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
- test_must_fail git clone --recurse-submodules --jobs=2 nested clone_parallel 2>err &&
- cat err &&
- grep -E "(already exists|is inside git dir|not a git repository)" err &&
- {
- test_path_is_missing .git/submodules/hippo/HEAD ||
- test_path_is_missing .git/submodules/hippo/hooks/HEAD
- }
+ git clone --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
-test_expect_success 'checkout -f --recurse-submodules must not use a nested gitdir' '
- git clone nested nested_checkout &&
+test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
+ git clone nested clone_recursive_checkout &&
(
- cd nested_checkout &&
+ cd clone_recursive_checkout &&
+
git submodule init &&
- git submodule update thing1 &&
+ git submodule update thing1 thing2 &&
+
+ # simulate a malicious nested alternate which git should not follow
mkdir -p .git/submodules/hippo/hooks/refs &&
mkdir -p .git/submodules/hippo/hooks/objects/info &&
echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
+
+ git checkout -f --recurse-submodules HEAD
) &&
- test_must_fail git -C nested_checkout checkout -f --recurse-submodules HEAD 2>err &&
- cat err &&
- grep "is inside git dir" err &&
- test_path_is_missing nested_checkout/thing2/.git
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into different directory' '
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 7/9] submodule: remove validate_submodule_git_dir()
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (5 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-09-08 14:23 ` Phillip Wood
2025-08-16 21:36 ` [PATCH 8/9] t: move nested gitdir tests to proper location Adrian Ratiu
` (8 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
The validate_submodule_git_dir test is not very useful anymore, after
submodule names are encoded to resolve gitdir path conflicts.
In other words, the purpouse of gitdir path encoding is precisely to
avoid such conflicts as this function tries to also prevent.
The first test from the function can be kept though, because it just
verifies invariants which should always be true and raise a BUG if:
- no "/" separator is between dirs/names.
- len(full_gitdir) < len(name).
- name does not match the gitdir path suffix.
Thus we move the invariant checks to submodule_name_to_gitdir() and
clean up the rest of validate_submodule_git_dir() and its uses.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 21 -----------
submodule.c | 74 ++++---------------------------------
submodule.h | 5 ---
3 files changed, 7 insertions(+), 93 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 30e40d6c79..d1ae864e8f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1725,10 +1725,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
clone_data_path = to_free = xstrfmt("%s/%s", repo_get_work_tree(the_repository),
clone_data->path);
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
-
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
@@ -1802,23 +1798,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
free(path);
}
- /*
- * We already performed this check at the beginning of this function,
- * before cloning the objects. This tries to detect racy behavior e.g.
- * in parallel clones, where another process could easily have made the
- * gitdir nested _after_ it was created.
- *
- * To prevent further harm coming from this unintentionally-nested
- * gitdir, let's disable it by deleting the `HEAD` file.
- */
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
- char *head = xstrfmt("%s/HEAD", sm_gitdir);
- unlink(head);
- free(head);
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
- }
-
connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
p = repo_submodule_path(the_repository, clone_data_path, "config");
diff --git a/submodule.c b/submodule.c
index 722a8e4f2a..cfb45a8f9f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2163,27 +2163,10 @@ int submodule_move_head(const char *path, const char *super_prefix,
if (!submodule_uses_gitfile(path))
absorb_git_dir_into_superproject(path,
super_prefix);
- else {
- char *dotgit = xstrfmt("%s/.git", path);
- char *git_dir = xstrdup(read_gitfile(dotgit));
-
- free(dotgit);
- if (validate_submodule_git_dir(git_dir,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in "
- "another submodule's git dir"),
- git_dir);
- free(git_dir);
- }
} else {
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
- if (validate_submodule_git_dir(gitdir.buf,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in another "
- "submodule's git dir"),
- gitdir.buf);
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
@@ -2263,52 +2246,6 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
-{
- size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
- char *p;
- int ret = 0;
-
- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
- strcmp(p, submodule_name))
- /*
- * TODO: revisit and cleanup this test short-circuit, because
- * submodules with encoded names are expected to take this path.
- * Likely just move the invariants to submodule_name_to_gitdir()
- * and delete this entire function in a future commit.
- */
- return 0;
-
- /*
- * We prevent the contents of sibling submodules' git directories to
- * clash.
- *
- * Example: having a submodule named `hippo` and another one named
- * `hippo/hooks` would result in the git directories
- * `.git/submodules/hippo/` and `.git/submodules/hippo/hooks/`, respectively,
- * but the latter directory is already designated to contain the hooks
- * of the former.
- */
- for (; *p; p++) {
- if (is_dir_sep(*p)) {
- char c = *p;
-
- *p = '\0';
- if (is_git_directory(git_dir))
- ret = -1;
- *p = c;
-
- if (ret < 0)
- return error(_("submodule git dir '%s' is "
- "inside git dir '%.*s'"),
- git_dir,
- (int)(p - git_dir), git_dir);
- }
- }
-
- return 0;
-}
-
int validate_submodule_path(const char *path)
{
char *p = xstrdup(path);
@@ -2367,9 +2304,6 @@ static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
submodule_name_to_gitdir(&new_gitdir, the_repository, sub->name);
- if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
- die(_("refusing to move '%s' into an existing git dir"),
- real_old_git_dir);
if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
die(_("could not create directory '%s'"), new_gitdir.buf);
real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
@@ -2611,7 +2545,7 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
{
struct strbuf encoded_sub_name = STRBUF_INIT, tmp = STRBUF_INIT;
size_t base_len, encoded_len;
- char *gitdir_path, *key;
+ char *gitdir_path, *key, *p;
long name_max;
/* Allow config override. */
@@ -2651,5 +2585,11 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
die(_("encoded submodule name '%s' is too long (%zu bytes, limit is %ld)"),
encoded_sub_name.buf, encoded_len, name_max);
+ /* Trigger a BUG if these invariants do not hold */
+ p = buf->buf + buf->len - encoded_len;
+ if (buf->len <= encoded_len || p[-1] != '/' || strcmp(p, encoded_sub_name.buf))
+ BUG("encoded submodule name '%s' is not a suffix of git dir '%s'",
+ encoded_sub_name.buf, buf->buf);
+
strbuf_release(&encoded_sub_name);
}
diff --git a/submodule.h b/submodule.h
index b10e16e6c0..0b7692bc20 100644
--- a/submodule.h
+++ b/submodule.h
@@ -137,11 +137,6 @@ int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name);
-/*
- * Make sure that no submodule's git dir is nested in a sibling submodule's.
- */
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
-
/*
* Make sure that the given submodule path does not follow symlinks.
*/
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 8/9] t: move nested gitdir tests to proper location
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (6 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 7/9] submodule: remove validate_submodule_git_dir() Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 9/9] t: add gitdir encoding tests Adrian Ratiu
` (7 subsequent siblings)
15 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
Now that we are encoding gitdir paths, these tests are not handling
pathological cases anymore, because nested git dirs shouldn't cause
conflicts, so move them from t7450-bad-git-dotfiles.sh to a more
appropriate location where we test mixed gitdir path & encoding use.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/t7425-submodule-mixed-gitdir-paths.sh | 54 ++++++++++++++++++++++++
t/t7450-bad-git-dotfiles.sh | 56 -------------------------
2 files changed, 54 insertions(+), 56 deletions(-)
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
index 801e90522a..902b2560ca 100755
--- a/t/t7425-submodule-mixed-gitdir-paths.sh
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -3,6 +3,7 @@
test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup: allow file protocol' '
git config --global protocol.file.allow always
@@ -98,4 +99,57 @@ test_expect_success 'fetch mixed submodule changes and verify updates' '
)
'
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of sibling submodules must not be nested' '
+ git clone --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
+test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
+ git clone nested clone_recursive_checkout &&
+ (
+ cd clone_recursive_checkout &&
+
+ git submodule init &&
+ git submodule update thing1 thing2 &&
+
+ # simulate a malicious nested alternate which git should not follow
+ mkdir -p .git/submodules/hippo/hooks/refs &&
+ mkdir -p .git/submodules/hippo/hooks/objects/info &&
+ echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
+
+ git checkout -f --recurse-submodules HEAD
+ ) &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
test_done
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index 27254300f8..18624fabc4 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -15,7 +15,6 @@ Such as:
. ./test-lib.sh
. "$TEST_DIRECTORY"/lib-pack.sh
-. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup' '
git config --global protocol.file.allow always
@@ -320,61 +319,6 @@ test_expect_success WINDOWS 'prevent git~1 squatting on Windows' '
fi
'
-# TODO: move these nested gitdir tests to another location in a later commit because
-# they are not pathological cases anymore: by encoding the gitdir paths do not conflict.
-test_expect_success 'setup submodules with nested git dirs' '
- git init nested &&
- test_commit -C nested nested &&
- (
- cd nested &&
- cat >.gitmodules <<-EOF &&
- [submodule "hippo"]
- url = .
- path = thing1
- [submodule "hippo/hooks"]
- url = .
- path = thing2
- EOF
- git clone . thing1 &&
- git clone . thing2 &&
- git add .gitmodules thing1 thing2 &&
- test_tick &&
- git commit -m nested
- )
-'
-
-test_expect_success 'git dirs of sibling submodules must not be nested' '
- git clone --recurse-submodules nested clone_nested &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
-test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
- git clone --recurse-submodules --jobs=2 nested clone_parallel &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
-test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
- git clone nested clone_recursive_checkout &&
- (
- cd clone_recursive_checkout &&
-
- git submodule init &&
- git submodule update thing1 thing2 &&
-
- # simulate a malicious nested alternate which git should not follow
- mkdir -p .git/submodules/hippo/hooks/refs &&
- mkdir -p .git/submodules/hippo/hooks/objects/info &&
- echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
-
- git checkout -f --recurse-submodules HEAD
- ) &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into different directory' '
test_when_finished "rm -rf sub repo bad-clone" &&
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH 9/9] t: add gitdir encoding tests
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (7 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 8/9] t: move nested gitdir tests to proper location Adrian Ratiu
@ 2025-08-16 21:36 ` Adrian Ratiu
2025-08-18 22:06 ` Junio C Hamano
2025-08-17 13:01 ` [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (6 subsequent siblings)
15 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-16 21:36 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Adrian Ratiu
Add some tests to further exercise the gitdir encoding functionality
alongside the existing mixed directory and nested gitdir tests.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/t7425-submodule-mixed-gitdir-paths.sh | 52 +++++++++++++++++++++++++
1 file changed, 52 insertions(+)
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
index 902b2560ca..cfdf487a56 100755
--- a/t/t7425-submodule-mixed-gitdir-paths.sh
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -152,4 +152,56 @@ test_expect_success 'checkout -f --recurse-submodules must corectly handle neste
verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
+test_expect_success 'new style submodule gitdir paths are properly encoded' '
+ (
+ cd main &&
+
+ # add new-style submodule name containing /
+ git submodule add ../new-sub foo/bar &&
+ git commit -m "add foo/bar" &&
+
+ # simulate existing legacy submodule name containing escaping char %
+ git clone --separate-git-dir .git/modules/foo%bar ../legacy-sub foo%bar &&
+ cat >>.gitmodules <<-EOF &&
+ [submodule "foo%bar"]
+ path = foo%bar
+ url = ../legacy-sub
+ EOF
+ git add .gitmodules &&
+ git commit -m "add foo%bar" &&
+
+ # add new style submodule name containing escaping char %
+ git submodule add ../new-sub fooish%bar &&
+ git commit -m "add fooish%bar" &&
+
+ # add a mixed case submdule name
+ git submodule add ../new-sub FooBar &&
+ git commit -m "add FooBar" &&
+
+ # add a reserved name on Windows
+ git submodule add ../new-sub COM1 &&
+ git commit -m "add COM1"
+ ) &&
+ verify_submodule_gitdir_path main foo/bar submodules/foo%2fbar &&
+ verify_submodule_gitdir_path main foo%bar modules/foo%bar &&
+ verify_submodule_gitdir_path main fooish%bar submodules/fooish%25bar &&
+ verify_submodule_gitdir_path main FooBar submodules/_foo_bar &&
+ verify_submodule_gitdir_path main COM1 submodules/_c_o_m1
+'
+
+test_expect_success 'submodule encoded name exceeds max name limit' '
+ (
+ cd main &&
+
+ # find the system NAME_MAX (fall back to 255 if unknown)
+ name_max=$(getconf NAME_MAX . 2>/dev/null || echo 255) &&
+
+ # each "%" char encodes to "%25" (3 chars), ensure we exceed NAME_MAX
+ count=$((name_max + 10)) &&
+ longname=$(printf "%%%0.s" $(seq 1 $count)) &&
+
+ test_must_fail git submodule add ../new-sub "$longname"
+ )
+'
+
test_done
--
2.50.1.679.gbf363a8fbb.dirty
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved
2025-08-16 21:36 ` [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
@ 2025-08-16 21:56 ` Ben Knoble
2025-08-21 13:08 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Ben Knoble @ 2025-08-16 21:56 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
> Le 16 août 2025 à 17:39, Adrian Ratiu <adrian.ratiu@collabora.com> a écrit :
>
> Commit f89854362c ("credential-store: move related functions to...")
Here and elsewhere, we refer to commits by the output of “git show -s --format=reference <object>”
Best,
Ben
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 0/9] Encode submodule gitdir names to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (8 preceding siblings ...)
2025-08-16 21:36 ` [PATCH 9/9] t: add gitdir encoding tests Adrian Ratiu
@ 2025-08-17 13:01 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (5 subsequent siblings)
15 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-17 13:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
---- On Sun, 17 Aug 2025 00:36:33 +0300 Adrian Ratiu <adrian.ratiu@collabora.com> wrote ---
> Hello,
>
> This is a continuation of work done back in 2018 [1], so a big thank you to
> everyone who participated in the initial thread, especially Brandon on whose
> code this is partially based upon. Hope you are still around and doing well. :)
>
> It's mostly a rewrite from scratch addressig open feedback. I decided to
> iterate upen Brandon's url-encoding design instead of pursuing alternatives
> like a custom encoding, name hashing or round-trip encoding/decoding using
> an in-memory git mapping (we'd still have to encode/hash the paths to avoid
> colflicts so IIUC this last one is more complicated for little gain).
>
> I tried to organize and explain the commits in a logical way which is also
> easy to review, keeping the encoding parts, new tests, code moving around
> and path update churn as clearly separated as possible.
>
> This is based on master and I've merged and succesfully run all tests in
> both the next and seen branches.
I also ran the GitHub CI pipeline and noticed there are failures on Win + Mac.
I will address those in v2.
In the meantime I'll leave v1 for a while on the ML to gather more feedback.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 9/9] t: add gitdir encoding tests
2025-08-16 21:36 ` [PATCH 9/9] t: add gitdir encoding tests Adrian Ratiu
@ 2025-08-18 22:06 ` Junio C Hamano
2025-08-21 13:17 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-08-18 22:06 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Stefan Beller, Patrick Steinhardt
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> Add some tests to further exercise the gitdir encoding functionality
> alongside the existing mixed directory and nested gitdir tests.
>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> t/t7425-submodule-mixed-gitdir-paths.sh | 52 +++++++++++++++++++++++++
> 1 file changed, 52 insertions(+)
>
> diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
> index 902b2560ca..cfdf487a56 100755
> --- a/t/t7425-submodule-mixed-gitdir-paths.sh
> +++ b/t/t7425-submodule-mixed-gitdir-paths.sh
> @@ -152,4 +152,56 @@ test_expect_success 'checkout -f --recurse-submodules must corectly handle neste
> ...
> + longname=$(printf "%%%0.s" $(seq 1 $count)) &&
Use of 'seq' gets complaint from
$ make -C t test-lint-shell-syntax
See the commit message of d17cf5f3 (tests: Introduce test_seq,
2012-08-04) and b32c7ec0 (test-lib: teach test_seq the -f option,
2025-06-23). I think you should be able to do something like
longname=$(test_seq -f "%%%0.s" 1 $count) &&
but I haven't even run the test with such a fix, so take it with a
grain of salt, please.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-08-16 21:36 ` [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-08-20 19:04 ` Josh Steadmon
2025-08-21 11:26 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-08-20 19:04 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
On 2025.08.17 00:36, Adrian Ratiu wrote:
> While testing submodule gitdir path encoding, I noticed submodule--helper
> is still using a hardcoded name-based path leading to test failures, so
> convert it to the common helper function introduced by commit ce125d431a
> ("submodule: extract path to submodule gitdir func") and used in other
> locations accross the source tree.
>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> builtin/submodule--helper.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 07a1935cbe..7243429c6f 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -3213,10 +3213,11 @@ static int add_submodule(const struct add_data *add_data)
> free(submod_gitdir_path);
> } else {
> struct child_process cp = CHILD_PROCESS_INIT;
> + struct strbuf submod_gitdir = STRBUF_INIT;
>
> - submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
> + submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
I believe submod_gitdir_path is now only used in the `if (...) {...}`
side corresponding to this `else` branch, so perhaps we should make it
local to that block?
> - if (is_directory(submod_gitdir_path)) {
> + if (is_directory(submod_gitdir.buf)) {
> if (!add_data->force) {
> struct strbuf msg = STRBUF_INIT;
> char *die_msg;
> @@ -3225,8 +3226,8 @@ static int add_submodule(const struct add_data *add_data)
> "locally with remote(s):\n"),
> add_data->sm_name);
>
> - append_fetch_remotes(&msg, submod_gitdir_path);
> - free(submod_gitdir_path);
> + append_fetch_remotes(&msg, submod_gitdir.buf);
> + strbuf_release(&submod_gitdir);
>
> strbuf_addf(&msg, _("If you want to reuse this local git "
> "directory instead of cloning again from\n"
> @@ -3244,7 +3245,7 @@ static int add_submodule(const struct add_data *add_data)
> "submodule '%s'\n"), add_data->sm_name);
> }
> }
> - free(submod_gitdir_path);
> + strbuf_release(&submod_gitdir);
>
> clone_data.prefix = add_data->prefix;
> clone_data.path = add_data->sm_path;
> --
> 2.50.1.679.gbf363a8fbb.dirty
>
>
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts
2025-08-16 21:36 ` [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-08-20 19:29 ` Jeff King
2025-08-21 13:14 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Jeff King @ 2025-08-20 19:29 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Junio C Hamano,
Aaron Schrab, Jonathan Nieder, Stefan Beller, Patrick Steinhardt,
Brandon Williams
On Sun, Aug 17, 2025 at 12:36:39AM +0300, Adrian Ratiu wrote:
> @@ -2632,5 +2633,23 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> /* New style (encoded) paths go under submodules/<encoded>. */
> strbuf_reset(buf);
> repo_git_path_append(r, buf, "submodules/");
> - strbuf_addstr(buf, submodule_name);
> + base_len = buf->len;
> +
> + /* URL-encode then case case-encode A to _a, B to _b and so on */
> + strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
> + strbuf_addstr_case_encode(&encoded_sub_name, tmp.buf);
> + strbuf_release(&tmp);
> + strbuf_addbuf(buf, &encoded_sub_name);
> +
> + /* Ensure final path length is below NAME_MAX after encoding */
> + name_max = pathconf(buf->buf, _PC_NAME_MAX);
> + if (name_max == -1)
> + name_max = NAME_MAX;
This patch seems to break the Windows CI builds, as they don't have
pathconf() there. I guess we'd need a compat wrapper that returns -1 in
this case. And likewise protects _PC_NAME_MAX from being seen on systems
that don't have it.
> + encoded_len = buf->len - base_len;
> + if (encoded_len >= name_max)
> + die(_("encoded submodule name '%s' is too long (%zu bytes, limit is %ld)"),
> + encoded_sub_name.buf, encoded_len, name_max);
It also complained about %z here. I think you have to use PRIuMAX
instead. Likewise size_t is a "long long" on Windows (LLP64). So "%ld"
probably also needs to be PRIuMAX.
I also saw failures on the osx jobs for t7527.62 (submodule
absorbgitdirs implicitly starts daemon). I didn't dig in, but I can
guess they may be related to this series.
-Peff
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
@ 2025-08-20 19:37 ` Josh Steadmon
2025-08-21 12:18 ` Adrian Ratiu
2025-08-20 21:38 ` Josh Steadmon
` (2 subsequent siblings)
3 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-08-20 19:37 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Brandon Williams
On 2025.08.17 00:36, Adrian Ratiu wrote:
[snip]
> diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
> new file mode 100644
> index 0000000000..fb5cb8eea4
> --- /dev/null
> +++ b/t/lib-verify-submodule-gitdir-path.sh
> @@ -0,0 +1,15 @@
> +# Helper to verify if repo $1 contains a submodule named $2 with gitdir in path $3
This comment is a bit inaccurate, right? If I'm reading correctly, we
only verify that the submodule's gitdir actually exists in the "legacy"
.git/modules/$path case. If we don't see anything there, we fall through
to .git/submodules/$encoded_path, but we never verify it actually
exists.
> +
> +verify_submodule_gitdir_path() {
> + repo="$1" &&
> + name="$2" &&
> + path="$3" &&
> + (
> + cd "$repo" &&
> + cat >expect <<-EOF &&
> + $(git rev-parse --git-common-dir)/$path
> + EOF
> + git submodule--helper gitdir "$name" >actual &&
> + test_cmp expect actual
> + )
> +}
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 178c386212..f4d4fb8397 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>
> test_expect_success 'setup - enable local submodules' '
> git config --global protocol.file.allow always
> @@ -1505,4 +1506,18 @@ test_expect_success 'submodule add fails when name is reused' '
> )
> '
>
> +test_expect_success 'submodule helper gitdir config overrides' '
> + verify_submodule_gitdir_path test-submodule child submodules/child &&
> + (
> + cd test-submodule &&
> + git config submodule.child.gitdirpath ".git/submodules/custom-child"
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/custom-child &&
> + (
> + cd test-submodule &&
> + git config --unset submodule.child.gitdirpath
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/child
> +'
> +
> test_done
> --
> 2.50.1.679.gbf363a8fbb.dirty
>
>
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
2025-08-20 19:37 ` Josh Steadmon
@ 2025-08-20 21:38 ` Josh Steadmon
2025-08-21 13:04 ` Adrian Ratiu
2025-08-20 21:50 ` Josh Steadmon
2025-09-08 14:23 ` Phillip Wood
3 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-08-20 21:38 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Brandon Williams
On 2025.08.17 00:36, Adrian Ratiu wrote:
[snip]
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 178c386212..f4d4fb8397 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>
> test_expect_success 'setup - enable local submodules' '
> git config --global protocol.file.allow always
> @@ -1505,4 +1506,18 @@ test_expect_success 'submodule add fails when name is reused' '
> )
> '
>
> +test_expect_success 'submodule helper gitdir config overrides' '
> + verify_submodule_gitdir_path test-submodule child submodules/child &&
> + (
> + cd test-submodule &&
> + git config submodule.child.gitdirpath ".git/submodules/custom-child"
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/custom-child &&
> + (
> + cd test-submodule &&
> + git config --unset submodule.child.gitdirpath
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/child
> +'
> +
Rather than `( cd test-submodule && git config ... )` here, you should
use `test_config -C test-submodule ...` and `test_unconfig -C
test-submodule ...`
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
2025-08-20 19:37 ` Josh Steadmon
2025-08-20 21:38 ` Josh Steadmon
@ 2025-08-20 21:50 ` Josh Steadmon
2025-08-21 13:05 ` Adrian Ratiu
2025-09-08 14:23 ` Phillip Wood
3 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-08-20 21:50 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On 2025.08.17 00:36, Adrian Ratiu wrote:
> This adds an ability to override gitdir paths via config files
> (not .gitmodules), such that any encoding scheme can be changed
> and JGit & co don't need to exactly match the default encoding.
>
> A new test and a helper are added. The helper will be used by
> further tests exercising gitdir paths & encodings.
>
> Based-on-patch-by: Brandon Williams <bmwill@google.com>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> builtin/submodule--helper.c | 17 +++++++++++++++++
> submodule.c | 11 +++++++++++
> t/lib-verify-submodule-gitdir-path.sh | 15 +++++++++++++++
> t/t7400-submodule-basic.sh | 15 +++++++++++++++
> 4 files changed, 58 insertions(+)
> create mode 100644 t/lib-verify-submodule-gitdir-path.sh
Sorry to keep sending piecemeal feedback. You should also document the
new config option in `Documentation/config/submodule.adoc`
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 4/9] t: submodules: add basic mixed gitdir path tests
2025-08-16 21:36 ` [PATCH 4/9] t: submodules: add basic mixed gitdir path tests Adrian Ratiu
@ 2025-08-20 22:07 ` Josh Steadmon
2025-09-02 23:02 ` Junio C Hamano
1 sibling, 0 replies; 179+ messages in thread
From: Josh Steadmon @ 2025-08-20 22:07 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
On 2025.08.17 00:36, Adrian Ratiu wrote:
> Add some basic submodule tests for mixed gitdir path handling of
> legacy (.git/modules) and new-style (.git/submodule) paths.
>
> For now these just test the coexistence, creation and push/pull of
> submodules using mixed paths.
>
> More tests will be added later, especially for new-style encoding.
>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> t/meson.build | 1 +
> t/t7425-submodule-mixed-gitdir-paths.sh | 101 ++++++++++++++++++++++++
> 2 files changed, 102 insertions(+)
> create mode 100755 t/t7425-submodule-mixed-gitdir-paths.sh
>
> diff --git a/t/meson.build b/t/meson.build
> index bbeba1a8d5..ffd74f1d3b 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -874,6 +874,7 @@ integration_tests = [
> 't7422-submodule-output.sh',
> 't7423-submodule-symlinks.sh',
> 't7424-submodule-mixed-ref-formats.sh',
> + 't7425-submodule-mixed-gitdir-paths.sh',
> 't7450-bad-git-dotfiles.sh',
> 't7500-commit-template-squash-signoff.sh',
> 't7501-commit-basic-functionality.sh',
> diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
> new file mode 100755
> index 0000000000..801e90522a
> --- /dev/null
> +++ b/t/t7425-submodule-mixed-gitdir-paths.sh
> @@ -0,0 +1,101 @@
> +#!/bin/sh
> +
> +test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup: allow file protocol' '
> + git config --global protocol.file.allow always
> +'
> +
> +test_expect_success 'create repo with mixed new and legacy submodules' '
> + git init legacy-sub &&
> + test_commit -C legacy-sub legacy-initial &&
> + git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
> + legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
> +
> + git init new-sub &&
> + test_commit -C new-sub new-initial &&
> + git -C new-sub config receive.denyCurrentBranch updateInstead &&
> + new_rev=$(git -C new-sub rev-parse HEAD) &&
> +
> + git init main &&
> + (
> + cd main &&
> +
> + git config receive.denyCurrentBranch updateInstead &&
> +
> + git submodule add ../new-sub new &&
> + test_commit new-sub &&
> +
> + git submodule add ../legacy-sub legacy &&
> + test_commit legacy-sub &&
> +
> + # simulate legacy .git/modules path by moving submodule
> + mkdir -p .git/modules &&
> + mv .git/submodules/legacy .git/modules/ &&
> + echo "gitdir: ../.git/modules/legacy" > legacy/.git
> + )
> +'
> +
> +test_expect_success 'clone from repo with both legacy and new-style submodules' '
> + git clone --recurse-submodules main cloned &&
> + (
> + cd cloned &&
> +
> + # At this point, .git/modules/<name> should not exist as
> + # submodules are checked out into the new path
> + test_path_is_dir .git/submodules/legacy &&
> + test_path_is_dir .git/submodules/new &&
> +
> + git submodule status >list &&
> + grep "$legacy_rev legacy" list &&
> + grep "$new_rev new" list
> + )
You probably want to use `test_grep` here and below.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-08-20 19:04 ` Josh Steadmon
@ 2025-08-21 11:26 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 11:26 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
On Wed, 20 Aug 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.08.17 00:36, Adrian Ratiu wrote:
>> While testing submodule gitdir path encoding, I noticed submodule--helper
>> is still using a hardcoded name-based path leading to test failures, so
>> convert it to the common helper function introduced by commit ce125d431a
>> ("submodule: extract path to submodule gitdir func") and used in other
>> locations accross the source tree.
>>
>> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
>> ---
>> builtin/submodule--helper.c | 11 ++++++-----
>> 1 file changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index 07a1935cbe..7243429c6f 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -3213,10 +3213,11 @@ static int add_submodule(const struct add_data *add_data)
>> free(submod_gitdir_path);
>> } else {
>> struct child_process cp = CHILD_PROCESS_INIT;
>> + struct strbuf submod_gitdir = STRBUF_INIT;
>>
>> - submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
>> + submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
>
> I believe submod_gitdir_path is now only used in the `if (...) {...}`
> side corresponding to this `else` branch, so perhaps we should make it
> local to that block?
Yes, I'll move it in v2. Thanks!
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-20 19:37 ` Josh Steadmon
@ 2025-08-21 12:18 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 12:18 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Brandon Williams
On Wed, 20 Aug 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.08.17 00:36, Adrian Ratiu wrote: [snip]
>> diff --git a/t/lib-verify-submodule-gitdir-path.sh
>> b/t/lib-verify-submodule-gitdir-path.sh new file mode 100644
>> index 0000000000..fb5cb8eea4 --- /dev/null +++
>> b/t/lib-verify-submodule-gitdir-path.sh @@ -0,0 +1,15 @@ +#
>> Helper to verify if repo $1 contains a submodule named $2 with
>> gitdir in path $3
>
> This comment is a bit inaccurate, right? If I'm reading
> correctly, we only verify that the submodule's gitdir actually
> exists in the "legacy" .git/modules/$path case. If we don't see
> anything there, we fall through to
> .git/submodules/$encoded_path, but we never verify it actually
> exists.
>
It was not my intention to imply gitdir existence is verified
here. :)
We just verify where the gitdirs are expected vs configured,
regardless of existence (eg when adding a new submodule it won't
exist beforehand).
The behavior you describe is correct: we only verify existence in
the legacy case, yes, in submodule_name_to_gitdir(), because only
those must exist beforehand and that's how we know we're in the
legacy (backwards compatibility) case, otherwise we default to the
newly encoded paths.
Hope this makes sense. I'll expand the description to clarify in v2.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-20 21:38 ` Josh Steadmon
@ 2025-08-21 13:04 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 13:04 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Wed, 20 Aug 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.08.17 00:36, Adrian Ratiu wrote: [snip]
>> diff --git a/t/t7400-submodule-basic.sh
>> b/t/t7400-submodule-basic.sh index 178c386212..f4d4fb8397
>> 100755 --- a/t/t7400-submodule-basic.sh +++
>> b/t/t7400-submodule-basic.sh @@ -13,6 +13,7 @@
>> GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
>> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh
>> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>> test_expect_success 'setup - enable local submodules' ' git
>> config --global protocol.file.allow always
>> @@ -1505,4 +1506,18 @@ test_expect_success 'submodule add fails
>> when name is reused' '
>> ) '
>> +test_expect_success 'submodule helper gitdir config overrides'
>> ' + verify_submodule_gitdir_path test-submodule child
>> submodules/child && + ( + cd test-submodule
>> && + git config submodule.child.gitdirpath
>> ".git/submodules/custom-child" + ) && +
>> verify_submodule_gitdir_path test-submodule child
>> submodules/custom-child && + ( + cd test-submodule
>> && + git config --unset submodule.child.gitdirpath + )
>> && + verify_submodule_gitdir_path test-submodule child
>> submodules/child +' +
>
> Rather than `( cd test-submodule && git config ... )` here, you
> should use `test_config -C test-submodule ...` and
> `test_unconfig -C test-submodule ...`
ack, will do in v2.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-20 21:50 ` Josh Steadmon
@ 2025-08-21 13:05 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 13:05 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Wed, 20 Aug 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.08.17 00:36, Adrian Ratiu wrote:
>> This adds an ability to override gitdir paths via config files
>> (not .gitmodules), such that any encoding scheme can be changed
>> and JGit & co don't need to exactly match the default encoding.
>> A new test and a helper are added. The helper will be used by
>> further tests exercising gitdir paths & encodings.
>> Based-on-patch-by: Brandon Williams <bmwill@google.com>
>> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> ---
>> builtin/submodule--helper.c | 17 +++++++++++++++++
>> submodule.c | 11 +++++++++++
>> t/lib-verify-submodule-gitdir-path.sh | 15 +++++++++++++++
>> t/t7400-submodule-basic.sh | 15 +++++++++++++++ 4
>> files changed, 58 insertions(+) create mode 100644
>> t/lib-verify-submodule-gitdir-path.sh
>
> Sorry to keep sending piecemeal feedback. You should also
> document the new config option in
> `Documentation/config/submodule.adoc`
No problem, thank you for taking the time to review & give
feedback.
Will do in v2.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved
2025-08-16 21:56 ` Ben Knoble
@ 2025-08-21 13:08 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 13:08 UTC (permalink / raw)
To: Ben Knoble
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Sat, 16 Aug 2025, Ben Knoble <ben.knoble@gmail.com> wrote:
>> Le 16 août 2025 à 17:39, Adrian Ratiu
>> <adrian.ratiu@collabora.com> a écrit : Commit f89854362c
>> ("credential-store: move related functions to...")
>
> Here and elsewhere, we refer to commits by the output of “git
> show -s --format=reference <object>”
>
> Best, Ben
Ack, will fix in v2.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts
2025-08-20 19:29 ` Jeff King
@ 2025-08-21 13:14 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 13:14 UTC (permalink / raw)
To: Jeff King
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Junio C Hamano,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Wed, 20 Aug 2025, Jeff King <peff@peff.net> wrote:
> On Sun, Aug 17, 2025 at 12:36:39AM +0300, Adrian Ratiu wrote:
>
>> @@ -2632,5 +2633,23 @@ void submodule_name_to_gitdir(struct
>> strbuf *buf, struct repository *r,
>> /* New style (encoded) paths go under
>> submodules/<encoded>. */ strbuf_reset(buf);
>> repo_git_path_append(r, buf, "submodules/");
>> - strbuf_addstr(buf, submodule_name); + base_len =
>> buf->len; + + /* URL-encode then case case-encode A to
>> _a, B to _b and so on */ + strbuf_addstr_urlencode(&tmp,
>> submodule_name, is_rfc3986_unreserved); +
>> strbuf_addstr_case_encode(&encoded_sub_name, tmp.buf); +
>> strbuf_release(&tmp); + strbuf_addbuf(buf,
>> &encoded_sub_name); + + /* Ensure final path length is
>> below NAME_MAX after encoding */ + name_max =
>> pathconf(buf->buf, _PC_NAME_MAX); + if (name_max == -1) +
>> name_max = NAME_MAX;
>
> This patch seems to break the Windows CI builds, as they don't
> have pathconf() there. I guess we'd need a compat wrapper that
> returns -1 in this case. And likewise protects _PC_NAME_MAX from
> being seen on systems that don't have it.
>
Ack, will fix in v2.
>> + encoded_len = buf->len - base_len; + if (encoded_len >=
>> name_max) + die(_("encoded submodule name '%s' is too
>> long (%zu bytes, limit is %ld)"), +
>> encoded_sub_name.buf, encoded_len, name_max);
>
> It also complained about %z here. I think you have to use
> PRIuMAX instead. Likewise size_t is a "long long" on Windows
> (LLP64). So "%ld" probably also needs to be PRIuMAX.
>
> I also saw failures on the osx jobs for t7527.62 (submodule
> absorbgitdirs implicitly starts daemon). I didn't dig in, but I
> can guess they may be related to this series.
It's possible. I'm so sorry for these breakages and will address
them in v2. I'll use the GitHub CI since I don't have access to
win+mac systems.
Also thank you for your patience, this is my first git project patch. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 9/9] t: add gitdir encoding tests
2025-08-18 22:06 ` Junio C Hamano
@ 2025-08-21 13:17 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-08-21 13:17 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Mon, 18 Aug 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> Add some tests to further exercise the gitdir encoding
>> functionality alongside the existing mixed directory and nested
>> gitdir tests.
>>
>> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> ---
>> t/t7425-submodule-mixed-gitdir-paths.sh | 52
>> +++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
>>
>> diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh
>> b/t/t7425-submodule-mixed-gitdir-paths.sh index
>> 902b2560ca..cfdf487a56 100755 ---
>> a/t/t7425-submodule-mixed-gitdir-paths.sh +++
>> b/t/t7425-submodule-mixed-gitdir-paths.sh @@ -152,4 +152,56 @@
>> test_expect_success 'checkout -f --recurse-submodules must
>> corectly handle neste ... + longname=$(printf "%%%0.s"
>> $(seq 1 $count)) &&
>
> Use of 'seq' gets complaint from
>
> $ make -C t test-lint-shell-syntax
>
> See the commit message of d17cf5f3 (tests: Introduce test_seq,
> 2012-08-04) and b32c7ec0 (test-lib: teach test_seq the -f
> option, 2025-06-23). I think you should be able to do something
> like
>
> longname=$(test_seq -f "%%%0.s" 1 $count) &&
>
> but I haven't even run the test with such a fix, so take it with
> a grain of salt, please.
Ack and thank you for the pointers. Will address in v2.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 4/9] t: submodules: add basic mixed gitdir path tests
2025-08-16 21:36 ` [PATCH 4/9] t: submodules: add basic mixed gitdir path tests Adrian Ratiu
2025-08-20 22:07 ` Josh Steadmon
@ 2025-09-02 23:02 ` Junio C Hamano
1 sibling, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-09-02 23:02 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Stefan Beller, Patrick Steinhardt
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> +test_expect_success 'commit and push changes to submodules' '
> + (
> + cd cloned &&
> +
> + git -C legacy switch --track -C master origin/master &&
This test needs to future-proof itself, perhaps with something like
to force the initial branch name to a known value.
t/t7425-submodule-mixed-gitdir-paths.sh | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
index 8a2d2e917f..02bd48aeeb 100755
--- a/t/t7425-submodule-mixed-gitdir-paths.sh
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -2,9 +2,13 @@
test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
+
. ./test-lib.sh
. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
test_expect_success 'setup: allow file protocol' '
git config --global protocol.file.allow always
'
--
2.51.0-302-ga83f9e55f9
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 00/10] Encode submodule gitdir names to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (9 preceding siblings ...)
2025-08-17 13:01 ` [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (9 more replies)
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (4 subsequent siblings)
15 siblings, 10 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
Hello all,
This is v2 of the submodule encoding topic which fixes filesystem conflicts
due to plain-text module name clashes (nested dirs, case insensitive FS).
This is based on the master branch and I've merged and ensured all CI passes
with both next and seen branches, with all combinations (linux, win, osx, meson
makefile, etc).
Changes between v1 -> v2:
* Added test name prefixes to commits modifying tests (eg "t7454: modify ...")
* Fixed all GitHub CI failures, including Win+Linux+Mac for both make and meson
* Test branch names now use main, which also fixes linux-breaking-changes (Junio)
* Replaced seq with test_seq for portability (Junio)
* Moved submod_gitdir_path variable declaration inside if block (Josh)
* Clarify FS existence check inside lib-verify-submodule-gitdir-path.sh (Josh)
* Replaced (cd submod && git config) with test_config -C submod (Josh)
* Document the new submodule--helper gitdir path option (Josh)
* Replaced grep -> test_grep (Josh)
* Reworded commit messages to use the proper commit reference format (Ben)
* Removed claim this fixes the Windows reserved file names (Mark)
* Split NAME_MAX logic into separate commit & added compat stub for pathconf (Peff)
Many thanks to all who have reviewed v1.
Adrian Ratiu (10):
submodule--helper: use submodule_name_to_gitdir in add_submodule
submodule: create new gitdirs under submodules path
submodule: add gitdir path config override
t7425: add basic mixed submodule gitdir path tests
strbuf: bring back is_rfc3986_unreserved
submodule: encode gitdir paths to avoid conflicts
submodule: error out if gitdir name is too long
submodule: remove validate_submodule_git_dir()
t7450: move nested gitdir tests to t7425
t7425: add gitdir encoding tests
Documentation/config/submodule.adoc | 4 +
Documentation/fetch-options.adoc | 2 +-
Documentation/git-fetch.adoc | 2 +-
Documentation/git-submodule.adoc | 2 +-
Documentation/gitsubmodules.adoc | 8 +-
Makefile | 5 +
builtin/credential-store.c | 6 -
builtin/submodule--helper.c | 51 +++---
compat/pathconf.c | 10 +
compat/posix.h | 8 +
config.mak.uname | 2 +
meson.build | 1 +
setup.c | 2 +-
strbuf.c | 6 +
strbuf.h | 2 +
submodule.c | 162 ++++++++---------
submodule.h | 5 -
t/lib-submodule-update.sh | 50 ++---
t/lib-verify-submodule-gitdir-path.sh | 20 ++
t/meson.build | 1 +
t/t0035-safe-bare-repository.sh | 4 +-
t/t1600-index.sh | 4 +-
t/t2405-worktree-submodule.sh | 8 +-
t/t2501-cwd-empty.sh | 2 +-
t/t3600-rm.sh | 8 +-
t/t5526-fetch-submodules.sh | 2 +-
t/t5619-clone-local-ambiguous-transport.sh | 4 +-
t/t6120-describe.sh | 4 +-
t/t7001-mv.sh | 4 +-
t/t7400-submodule-basic.sh | 27 ++-
t/t7406-submodule-update.sh | 14 +-
t/t7407-submodule-foreach.sh | 6 +-
t/t7408-submodule-reference.sh | 22 +--
t/t7412-submodule-absorbgitdirs.sh | 22 +--
t/t7423-submodule-symlinks.sh | 8 +-
t/t7425-submodule-mixed-gitdir-paths.sh | 202 +++++++++++++++++++++
t/t7450-bad-git-dotfiles.sh | 73 +-------
t/t7527-builtin-fsmonitor.sh | 4 +-
t/t9902-completion.sh | 1 +
39 files changed, 481 insertions(+), 287 deletions(-)
create mode 100644 compat/pathconf.c
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-mixed-gitdir-paths.sh
--
2.51.GIT
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-30 13:37 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 02/10] submodule: create new gitdirs under submodules path Adrian Ratiu
` (8 subsequent siblings)
9 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded name-based path leading to test failures, so
convert it to the common helper function introduced by commit ce125d431a
(submodule: extract path to submodule gitdir func, 2021-09-15) and used
in other locations accross the source tree.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 07a1935cbe..d06e2fe265 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3193,13 +3193,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3213,10 +3213,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3225,8 +3226,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3244,7 +3245,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 02/10] submodule: create new gitdirs under submodules path
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-09 7:40 ` Patrick Steinhardt
2025-09-08 14:01 ` [PATCH v2 03/10] submodule: add gitdir path config override Adrian Ratiu
` (7 subsequent siblings)
9 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
This is in preparation for encoding the submodule names to avoid conflicts
like submodules named foo and foo/bar together with case-insensitive file-
system handling and other corner cases like reserved filenames on Windows.
Backward compatibility is kept with plain-name modules already existing at
paths like .git/modules/<name>, however a clear separation between legacy
(plain) and new (encoded) namespaces is desirable, to avoid situations like
an existing plain-name module containing the encoding escape character/
Thus we split the new-style (encoded) gitdir name paths to .git/submodules,
while legacy-style paths remain under .git/modules.
This is just a default directory change with the accompanying test updates,
in preparation for the actual encoding additions in future commits.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/fetch-options.adoc | 2 +-
Documentation/git-fetch.adoc | 2 +-
Documentation/git-submodule.adoc | 2 +-
Documentation/gitsubmodules.adoc | 8 ++--
setup.c | 2 +-
submodule.c | 28 +++++++++---
t/lib-submodule-update.sh | 50 +++++++++++-----------
t/t0035-safe-bare-repository.sh | 4 +-
t/t1600-index.sh | 4 +-
t/t2405-worktree-submodule.sh | 8 ++--
t/t2501-cwd-empty.sh | 2 +-
t/t3600-rm.sh | 8 ++--
t/t5526-fetch-submodules.sh | 2 +-
t/t5619-clone-local-ambiguous-transport.sh | 4 +-
t/t6120-describe.sh | 4 +-
t/t7001-mv.sh | 4 +-
t/t7400-submodule-basic.sh | 18 ++++----
t/t7406-submodule-update.sh | 10 ++---
t/t7407-submodule-foreach.sh | 6 +--
t/t7408-submodule-reference.sh | 22 +++++-----
t/t7412-submodule-absorbgitdirs.sh | 22 +++++-----
t/t7423-submodule-symlinks.sh | 8 ++--
t/t7450-bad-git-dotfiles.sh | 32 +++++++-------
t/t7527-builtin-fsmonitor.sh | 4 +-
24 files changed, 136 insertions(+), 120 deletions(-)
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
index d3ac31f4e2..2605a58b72 100644
--- a/Documentation/fetch-options.adoc
+++ b/Documentation/fetch-options.adoc
@@ -215,7 +215,7 @@ ifndef::git-pull[]
submodule that has commits that are referenced by a newly fetched
superproject commit but are missing in the local submodule clone. A
changed submodule can be fetched as long as it is present locally e.g.
- in `$GIT_DIR/modules/` (see linkgit:gitsubmodules[7]); if the upstream
+ in `$GIT_DIR/submodules/` (see linkgit:gitsubmodules[7]); if the upstream
adds a new submodule, that submodule cannot be fetched until it is
cloned e.g. by `git submodule update`.
+
diff --git a/Documentation/git-fetch.adoc b/Documentation/git-fetch.adoc
index 16f5d9d69a..2923a29bef 100644
--- a/Documentation/git-fetch.adoc
+++ b/Documentation/git-fetch.adoc
@@ -304,7 +304,7 @@ include::config/fetch.adoc[]
BUGS
----
Using --recurse-submodules can only fetch new commits in submodules that are
-present locally e.g. in `$GIT_DIR/modules/`. If the upstream adds a new
+present locally e.g. in `$GIT_DIR/submodules/`. If the upstream adds a new
submodule, that submodule cannot be fetched until it is cloned e.g. by `git
submodule update`. This is expected to be fixed in a future Git version.
diff --git a/Documentation/git-submodule.adoc b/Documentation/git-submodule.adoc
index 95beaee561..08008eeb62 100644
--- a/Documentation/git-submodule.adoc
+++ b/Documentation/git-submodule.adoc
@@ -266,7 +266,7 @@ registered submodules, and sync any nested submodules within.
absorbgitdirs::
If a git directory of a submodule is inside the submodule,
move the git directory of the submodule into its superproject's
- `$GIT_DIR/modules` path and then connect the git directory and
+ `$GIT_DIR/submodules` path and then connect the git directory and
its working directory by setting the `core.worktree` and adding
a .git file pointing to the git directory embedded in the
superprojects git directory.
diff --git a/Documentation/gitsubmodules.adoc b/Documentation/gitsubmodules.adoc
index 2082296199..c6b07847bb 100644
--- a/Documentation/gitsubmodules.adoc
+++ b/Documentation/gitsubmodules.adoc
@@ -22,12 +22,12 @@ The submodule has its own history; the repository it is embedded
in is called a superproject.
On the filesystem, a submodule usually (but not always - see FORMS below)
-consists of (i) a Git directory located under the `$GIT_DIR/modules/`
+consists of (i) a Git directory located under the `$GIT_DIR/submodules/`
directory of its superproject, (ii) a working directory inside the
superproject's working directory, and a `.git` file at the root of
the submodule's working directory pointing to (i).
-Assuming the submodule has a Git directory at `$GIT_DIR/modules/foo/`
+Assuming the submodule has a Git directory at `$GIT_DIR/submodules/foo/`
and a working directory at `path/to/bar/`, the superproject tracks the
submodule via a `gitlink` entry in the tree at `path/to/bar` and an entry
in its `.gitmodules` file (see linkgit:gitmodules[5]) of the form
@@ -138,7 +138,7 @@ using older versions of Git.
It is possible to construct these old form repositories manually.
+
When deinitialized or deleted (see below), the submodule's Git
-directory is automatically moved to `$GIT_DIR/modules/<name>/`
+directory is automatically moved to `$GIT_DIR/submodules/<name>/`
of the superproject.
* Deinitialized submodule: A `gitlink`, and a `.gitmodules` entry,
@@ -163,7 +163,7 @@ possible to checkout past commits without requiring fetching
from another repository.
+
To completely remove a submodule, manually delete
-`$GIT_DIR/modules/<name>/`.
+`$GIT_DIR/submodules/<name>/`.
ACTIVE SUBMODULES
-----------------
diff --git a/setup.c b/setup.c
index 7086741e6c..433beebc51 100644
--- a/setup.c
+++ b/setup.c
@@ -1416,7 +1416,7 @@ static int is_implicit_bare_repo(const char *path)
* we are inside $GIT_DIR of a worktree of a non-embedded
* submodule, whose superproject is not a bare repository.
*/
- if (strstr(path, "/.git/modules/"))
+ if (strstr(path, "/.git/modules/") || strstr(path, "/.git/submodules/"))
return 1;
return 0;
diff --git a/submodule.c b/submodule.c
index fff3c75570..dbf2244e60 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1278,22 +1278,29 @@ void check_for_new_submodule_commits(struct object_id *oid)
/*
* Returns 1 if there is at least one submodule gitdir in
- * $GIT_DIR/modules and 0 otherwise. This follows
+ * $GIT_DIR/(sub)modules and 0 otherwise. This follows
* submodule_name_to_gitdir(), which looks for submodules in
- * $GIT_DIR/modules, not $GIT_COMMON_DIR.
+ * $GIT_DIR/(sub)modules, not $GIT_COMMON_DIR.
*
- * A submodule can be moved to $GIT_DIR/modules manually by running "git
- * submodule absorbgitdirs", or it may be initialized there by "git
- * submodule update".
+ * A submodule can be moved to $GIT_DIR/(sub)modules manually by running
+ * "git submodule absorbgitdirs", or it may be initialized there by
+ * "git submodule update".
*/
static int repo_has_absorbed_submodules(struct repository *r)
{
int ret;
struct strbuf buf = STRBUF_INIT;
+ /* check legacy path */
repo_git_path_append(r, &buf, "modules/");
ret = file_exists(buf.buf) && !is_empty_dir(buf.buf);
+ strbuf_reset(&buf);
+
+ /* new (encoded name) path */
+ repo_git_path_append(r, &buf, "submodules/");
+ ret |= file_exists(buf.buf) && !is_empty_dir(buf.buf);
strbuf_release(&buf);
+
return ret;
}
@@ -2273,7 +2280,7 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
*
* Example: having a submodule named `hippo` and another one named
* `hippo/hooks` would result in the git directories
- * `.git/modules/hippo/` and `.git/modules/hippo/hooks/`, respectively,
+ * `.git/submodules/hippo/` and `.git/submodules/hippo/hooks/`, respectively,
* but the latter directory is already designated to contain the hooks
* of the former.
*/
@@ -2604,6 +2611,15 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
* administrators can explicitly set. Nothing has been decided,
* so for now, just append the name at the end of the path.
*/
+
+ /* Legacy behavior: allow existing paths under modules/<name>. */
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+ if (!access(buf->buf, F_OK))
+ return;
+
+ /* New style (encoded) paths go under submodules/<encoded>. */
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "submodules/");
+ strbuf_addstr(buf, submodule_name);
}
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 36f767cb74..b6b2be1df5 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -161,7 +161,7 @@ replace_gitfile_with_git_dir () {
}
# Test that the .git directory in the submodule is unchanged (except for the
-# core.worktree setting, which appears only in $GIT_DIR/modules/$1/config).
+# core.worktree setting, which appears only in $GIT_DIR/submodules/$1/config).
# Call this function before test_submodule_content as the latter might
# write the index file leading to false positive index differences.
#
@@ -170,23 +170,23 @@ replace_gitfile_with_git_dir () {
test_git_directory_is_unchanged () {
# does core.worktree point at the right place?
echo "../../../$1" >expect &&
- git -C ".git/modules/$1" config core.worktree >actual &&
+ git -C ".git/submodules/$1" config core.worktree >actual &&
test_cmp expect actual &&
# remove it temporarily before comparing, as
# "$1/.git/config" lacks it...
- git -C ".git/modules/$1" config --unset core.worktree &&
- diff -r ".git/modules/$1" "$1/.git" &&
+ git -C ".git/submodules/$1" config --unset core.worktree &&
+ diff -r ".git/submodules/$1" "$1/.git" &&
# ... and then restore.
- git -C ".git/modules/$1" config core.worktree "../../../$1"
+ git -C ".git/submodules/$1" config core.worktree "../../../$1"
}
test_git_directory_exists () {
- test -e ".git/modules/$1" &&
+ test -e ".git/submodules/$1" &&
if test -f sub1/.git
then
# does core.worktree point at the right place?
echo "../../../$1" >expect &&
- git -C ".git/modules/$1" config core.worktree >actual &&
+ git -C ".git/submodules/$1" config core.worktree >actual &&
test_cmp expect actual
fi
}
@@ -225,22 +225,22 @@ reset_work_tree_to () {
reset_work_tree_to_interested () {
reset_work_tree_to $1 &&
# make the submodule git dirs available
- if ! test -d submodule_update/.git/modules/sub1
+ if ! test -d submodule_update/.git/submodules/sub1
then
- mkdir -p submodule_update/.git/modules &&
- cp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1
- GIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree
+ mkdir -p submodule_update/.git/submodules &&
+ cp -r submodule_update_repo/.git/submodules/sub1 submodule_update/.git/submodules/sub1
+ GIT_WORK_TREE=. git -C submodule_update/.git/submodules/sub1 config --unset core.worktree
fi &&
- if ! test -d submodule_update/.git/modules/sub1/modules/sub2
+ if ! test -d submodule_update/.git/submodules/sub1/submodules/sub2
then
- mkdir -p submodule_update/.git/modules/sub1/modules &&
- cp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2
+ mkdir -p submodule_update/.git/submodules/sub1/submodules &&
+ cp -r submodule_update_repo/.git/submodules/sub1/submodules/sub2 submodule_update/.git/submodules/sub1/submodules/sub2
# core.worktree is unset for sub2 as it is not checked out
fi &&
# indicate we are interested in the submodule:
git -C submodule_update config submodule.sub1.url "bogus" &&
# sub1 might not be checked out, so use the git dir
- git -C submodule_update/.git/modules/sub1 config submodule.sub2.url "bogus"
+ git -C submodule_update/.git/submodules/sub1 config submodule.sub2.url "bogus"
}
# Test that the superproject contains the content according to commit "$1"
@@ -742,7 +742,7 @@ test_submodule_recursing_with_args_common () {
$command remove_sub1 &&
test_superproject_content origin/remove_sub1 &&
! test -e sub1 &&
- test_must_fail git config -f .git/modules/sub1/config core.worktree
+ test_must_fail git config -f .git/submodules/sub1/config core.worktree
)
'
# ... absorbing a .git directory along the way.
@@ -753,7 +753,7 @@ test_submodule_recursing_with_args_common () {
cd submodule_update &&
git branch -t remove_sub1 origin/remove_sub1 &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
$command remove_sub1 &&
test_superproject_content origin/remove_sub1 &&
! test -e sub1 &&
@@ -803,8 +803,8 @@ test_submodule_recursing_with_args_common () {
$command no_submodule &&
test_superproject_content origin/no_submodule &&
test_path_is_missing sub1 &&
- test_must_fail git config -f .git/modules/sub1/config core.worktree &&
- test_must_fail git config -f .git/modules/sub1/modules/sub2/config core.worktree
+ test_must_fail git config -f .git/submodules/sub1/config core.worktree &&
+ test_must_fail git config -f .git/submodules/sub1/submodules/sub2/config core.worktree
)
'
@@ -937,7 +937,7 @@ test_submodule_switch_recursing_with_args () {
cd submodule_update &&
git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
$command replace_sub1_with_directory &&
test_superproject_content origin/replace_sub1_with_directory &&
test_git_directory_exists sub1
@@ -946,15 +946,15 @@ test_submodule_switch_recursing_with_args () {
# ... and ignored files are ignored
test_expect_success "$command: replace submodule with a file works ignores ignored files in submodule" '
- test_when_finished "rm submodule_update/.git/modules/sub1/info/exclude" &&
+ test_when_finished "rm submodule_update/.git/submodules/sub1/info/exclude" &&
prolog &&
reset_work_tree_to_interested add_sub1 &&
(
cd submodule_update &&
- rm -rf .git/modules/sub1/info &&
+ rm -rf .git/submodules/sub1/info &&
git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
- mkdir .git/modules/sub1/info &&
- echo ignored >.git/modules/sub1/info/exclude &&
+ mkdir .git/submodules/sub1/info &&
+ echo ignored >.git/submodules/sub1/info/exclude &&
: >sub1/ignored &&
$command replace_sub1_with_file &&
test_superproject_content origin/replace_sub1_with_file &&
@@ -1034,7 +1034,7 @@ test_submodule_forced_switch_recursing_with_args () {
cd submodule_update &&
git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
replace_gitfile_with_git_dir sub1 &&
- rm -rf .git/modules/sub1 &&
+ rm -rf .git/submodules/sub1 &&
$command replace_sub1_with_directory &&
test_superproject_content origin/replace_sub1_with_directory &&
test_git_directory_exists sub1
diff --git a/t/t0035-safe-bare-repository.sh b/t/t0035-safe-bare-repository.sh
index ae7ef092ab..a480ddf8d6 100755
--- a/t/t0035-safe-bare-repository.sh
+++ b/t/t0035-safe-bare-repository.sh
@@ -41,7 +41,7 @@ test_expect_success 'setup an embedded bare repo, secondary worktree and submodu
submodule add --name subn -- ./bare-repo subd
) &&
test_path_is_dir outer-repo/.git/worktrees/outer-secondary &&
- test_path_is_dir outer-repo/.git/modules/subn
+ test_path_is_dir outer-repo/.git/submodules/subn
'
test_expect_success 'safe.bareRepository unset' '
@@ -100,7 +100,7 @@ test_expect_success 'no trace in $GIT_DIR of secondary worktree' '
'
test_expect_success 'no trace in $GIT_DIR of a submodule' '
- expect_accepted_implicit -C outer-repo/.git/modules/subn
+ expect_accepted_implicit -C outer-repo/.git/submodules/subn
'
test_done
diff --git a/t/t1600-index.sh b/t/t1600-index.sh
index 03239e9faa..0e5e8efb20 100755
--- a/t/t1600-index.sh
+++ b/t/t1600-index.sh
@@ -87,10 +87,10 @@ test_expect_success 'index.skipHash config option' '
git -c protocol.file.allow=always submodule add ./ sub &&
git config index.skipHash false &&
git -C sub config index.skipHash true &&
- rm -f .git/modules/sub/index &&
+ rm -f .git/submodules/sub/index &&
>sub/file &&
git -C sub add a &&
- test_trailing_hash .git/modules/sub/index >hash &&
+ test_trailing_hash .git/submodules/sub/index >hash &&
test_cmp expect hash &&
git -C sub fsck
'
diff --git a/t/t2405-worktree-submodule.sh b/t/t2405-worktree-submodule.sh
index 11018f37c7..c18c2efca5 100755
--- a/t/t2405-worktree-submodule.sh
+++ b/t/t2405-worktree-submodule.sh
@@ -62,7 +62,7 @@ test_expect_success 'submodule is checked out after manually adding submodule wo
test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules in a linked worktree' '
git -C main worktree add "$base_path/checkout-recurse" --detach &&
git -C checkout-recurse submodule update --init &&
- echo "gitdir: ../../main/.git/worktrees/checkout-recurse/modules/sub" >expect-gitfile &&
+ echo "gitdir: ../../main/.git/worktrees/checkout-recurse/submodules/sub" >expect-gitfile &&
cat checkout-recurse/sub/.git >actual-gitfile &&
test_cmp expect-gitfile actual-gitfile &&
git -C main/sub rev-parse HEAD >expect-head-main &&
@@ -73,7 +73,7 @@ test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules
test_cmp expect-head-main actual-head-main
'
-test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config, not in $GIT_COMMON_DIR/modules/<name>/config' '
+test_expect_success 'core.worktree is removed in $GIT_DIR/submodules/<name>/config, not in $GIT_COMMON_DIR/submodules/<name>/config' '
echo "../../../sub" >expect-main &&
git -C main/sub config --get core.worktree >actual-main &&
test_cmp expect-main actual-main &&
@@ -81,14 +81,14 @@ test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config,
git -C checkout-recurse/sub config --get core.worktree >actual-linked &&
test_cmp expect-linked actual-linked &&
git -C checkout-recurse checkout --recurse-submodules first &&
- test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/modules/sub config --get core.worktree >linked-config &&
+ test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/submodules/sub config --get core.worktree >linked-config &&
test_must_be_empty linked-config &&
git -C main/sub config --get core.worktree >actual-main &&
test_cmp expect-main actual-main
'
test_expect_success 'unsetting core.worktree does not prevent running commands directly against the submodule repository' '
- git -C main/.git/worktrees/checkout-recurse/modules/sub log
+ git -C main/.git/worktrees/checkout-recurse/submodules/sub log
'
test_done
diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
index be9140bbaa..bb8751433f 100755
--- a/t/t2501-cwd-empty.sh
+++ b/t/t2501-cwd-empty.sh
@@ -239,7 +239,7 @@ test_submodule_removal () {
test "$path_status" = dir && test_status=test_must_fail
test_when_finished "git reset --hard HEAD~1" &&
- test_when_finished "rm -rf .git/modules/my_submodule" &&
+ test_when_finished "rm -rf .git/submodules/my_submodule" &&
git checkout foo/bar/baz &&
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index 1f16e6b522..5b8ed57538 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -582,7 +582,7 @@ test_expect_success 'rm of a conflicted populated submodule with a .git director
(
cd submod &&
rm .git &&
- cp -R ../.git/modules/sub .git &&
+ cp -R ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
test_must_fail git merge conflict2 &&
@@ -617,9 +617,9 @@ test_expect_success 'rm of a populated submodule with a .git directory migrates
(
cd submod &&
rm .git &&
- cp -R ../.git/modules/sub .git &&
+ cp -R ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree &&
- rm -r ../.git/modules/sub
+ rm -r ../.git/submodules/sub
) &&
git rm submod 2>output.err &&
test_path_is_missing submod &&
@@ -709,7 +709,7 @@ test_expect_success "rm absorbs submodule's nested .git directory" '
(
cd submod/subsubmod &&
rm .git &&
- mv ../../.git/modules/sub/modules/sub .git &&
+ mv ../../.git/submodules/sub/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
git rm submod 2>output.err &&
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 5e566205ba..b7385bc088 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -1143,7 +1143,7 @@ test_expect_success 'fetch --recurse-submodules updates name-conflicted, unpopul
head1=$(git -C same-name-1/submodule rev-parse HEAD) &&
head2=$(git -C same-name-2/submodule rev-parse HEAD) &&
(
- cd same-name-downstream/.git/modules/submodule &&
+ cd same-name-downstream/.git/submodules/submodule &&
# The submodule has core.worktree pointing to the "git
# rm"-ed directory, overwrite the invalid value. See
# comment in get_fetch_task_from_changed() for more
diff --git a/t/t5619-clone-local-ambiguous-transport.sh b/t/t5619-clone-local-ambiguous-transport.sh
index cce62bf78d..cf2d5e7bfb 100755
--- a/t/t5619-clone-local-ambiguous-transport.sh
+++ b/t/t5619-clone-local-ambiguous-transport.sh
@@ -38,7 +38,7 @@ test_expect_success 'setup' '
ln -s "$(cd .. && pwd)/sensitive" repo/objects &&
mkdir -p "$HTTPD_URL/dumb" &&
- ln -s "../../../.git/modules/sub/../../../repo/" "$URI" &&
+ ln -s "../../../.git/submodules/sub/../../../repo/" "$URI" &&
git add . &&
git commit -m "initial commit"
@@ -57,7 +57,7 @@ test_expect_success 'ambiguous transport does not lead to arbitrary file-inclusi
git clone malicious clone &&
test_must_fail git -C clone submodule update --init 2>err &&
- test_path_is_missing clone/.git/modules/sub/objects/secret &&
+ test_path_is_missing clone/.git/submodules/sub/objects/secret &&
# We would actually expect "transport .file. not allowed" here,
# but due to quirks of the URL detection in Git, we mis-parse
# the absolute path as a bogus URL and die before that step.
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 2c70cc561a..e7e8127130 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -357,7 +357,7 @@ test_expect_success 'setup and absorb a submodule' '
'
test_expect_success 'describe chokes on severely broken submodules' '
- mv .git/modules/sub1/ .git/modules/sub_moved &&
+ mv .git/submodules/sub1/ .git/submodules/sub_moved &&
test_must_fail git describe --dirty
'
@@ -371,7 +371,7 @@ test_expect_success 'describe with --work-tree ignoring a broken submodule' '
cd "$TEST_DIRECTORY" &&
git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --broken >"$TRASH_DIRECTORY/out"
) &&
- test_when_finished "mv .git/modules/sub_moved .git/modules/sub1" &&
+ test_when_finished "mv .git/submodules/sub_moved .git/submodules/sub1" &&
grep broken out
'
diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
index 920479e925..89b06ae3c1 100755
--- a/t/t7001-mv.sh
+++ b/t/t7001-mv.sh
@@ -360,7 +360,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and no .gitm
(
cd sub &&
rm -f .git &&
- cp -R -P -p ../.git/modules/sub .git &&
+ cp -R -P -p ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
mkdir mod &&
@@ -380,7 +380,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
(
cd sub &&
rm -f .git &&
- cp -R -P -p ../.git/modules/sub .git &&
+ cp -R -P -p ../.git/submodules/sub .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
mkdir mod &&
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index fd3e7e355e..178c386212 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -163,7 +163,7 @@ test_expect_success 'submodule add' '
cd addtest &&
git submodule add -q "$submodurl" submod >actual &&
test_must_be_empty actual &&
- echo "gitdir: ../.git/modules/submod" >expect &&
+ echo "gitdir: ../.git/submodules/submod" >expect &&
test_cmp expect submod/.git &&
(
cd submod &&
@@ -976,21 +976,21 @@ test_expect_success 'submodule add --name allows to replace a submodule with ano
echo "$submodurl/repo" >expect &&
git config remote.origin.url >actual &&
test_cmp expect actual &&
- echo "gitdir: ../.git/modules/repo" >expect &&
+ echo "gitdir: ../.git/submodules/repo" >expect &&
test_cmp expect .git
) &&
rm -rf repo &&
git rm repo &&
git submodule add -q --name repo_new "$submodurl/bare.git" repo >actual &&
test_must_be_empty actual &&
- echo "gitdir: ../.git/modules/submod" >expect &&
+ echo "gitdir: ../.git/submodules/submod" >expect &&
test_cmp expect submod/.git &&
(
cd repo &&
echo "$submodurl/bare.git" >expect &&
git config remote.origin.url >actual &&
test_cmp expect actual &&
- echo "gitdir: ../.git/modules/repo_new" >expect &&
+ echo "gitdir: ../.git/submodules/repo_new" >expect &&
test_cmp expect .git
) &&
echo "repo" >expect &&
@@ -1045,8 +1045,8 @@ test_expect_success 'recursive relative submodules stay relative' '
(
cd clone2 &&
git submodule update --init --recursive &&
- echo "gitdir: ../.git/modules/sub3" >./sub3/.git_expect &&
- echo "gitdir: ../../../.git/modules/sub3/modules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
+ echo "gitdir: ../.git/submodules/sub3" >./sub3/.git_expect &&
+ echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
) &&
test_cmp clone2/sub3/.git_expect clone2/sub3/.git &&
test_cmp clone2/sub3/dirdir/subsub/.git_expect clone2/sub3/dirdir/subsub/.git
@@ -1108,8 +1108,8 @@ test_expect_success 'submodule deinit should remove the whole submodule section
'
test_expect_success 'submodule deinit should unset core.worktree' '
- test_path_is_file .git/modules/example/config &&
- test_must_fail git config -f .git/modules/example/config core.worktree
+ test_path_is_file .git/submodules/example/config &&
+ test_must_fail git config -f .git/submodules/example/config core.worktree
'
test_expect_success 'submodule deinit from subdirectory' '
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule deinit absorbs .git directory if .git is a direct
(
cd init &&
rm .git &&
- mv ../.git/modules/example .git &&
+ mv ../.git/submodules/example .git &&
GIT_WORK_TREE=. git config --unset core.worktree
) &&
git submodule deinit init &&
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 3adab12091..f0c4da1ffa 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -864,7 +864,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/modules/deeper/submodule &&
+ (cd .git/submodules/deeper/submodule &&
git log > ../../../../actual
) &&
test_cmp expected actual
@@ -882,7 +882,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/modules/deeper/submodule &&
+ (cd .git/submodules/deeper/submodule &&
git log > ../../../../actual
) &&
test_cmp expected actual
@@ -899,7 +899,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir recur
git commit -m "added subsubmodule" &&
git push origin :
) &&
- (cd .git/modules/deeper/submodule/modules/subsubmodule &&
+ (cd .git/submodules/deeper/submodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
git add deeper/submodule &&
@@ -949,7 +949,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir re
(cd submodule/subsubmodule &&
git log > ../../expected
) &&
- (cd .git/modules/submodule/modules/subsubmodule &&
+ (cd .git/submodules/submodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
test_cmp expected actual
@@ -1298,7 +1298,7 @@ test_expect_success CASE_INSENSITIVE_FS,SYMLINKS \
git init captain &&
(
cd captain &&
- git submodule add --name x/y "$hook_repo_path" A/modules/x &&
+ git submodule add --name x/y "$hook_repo_path" A/submodules/x &&
test_tick &&
git commit -m add-submodule &&
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 77b6d0040e..75ba826968 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -368,9 +368,9 @@ test_expect_success 'test "update --recursive" with a flag with spaces' '
git rev-parse --resolve-git-dir nested1/.git &&
git rev-parse --resolve-git-dir nested1/nested2/.git &&
git rev-parse --resolve-git-dir nested1/nested2/nested3/.git &&
- test -f .git/modules/nested1/objects/info/alternates &&
- test -f .git/modules/nested1/modules/nested2/objects/info/alternates &&
- test -f .git/modules/nested1/modules/nested2/modules/nested3/objects/info/alternates
+ test -f .git/submodules/nested1/objects/info/alternates &&
+ test -f .git/submodules/nested1/submodules/nested2/objects/info/alternates &&
+ test -f .git/submodules/nested1/submodules/nested2/submodules/nested3/objects/info/alternates
)
'
diff --git a/t/t7408-submodule-reference.sh b/t/t7408-submodule-reference.sh
index f860e7bbf4..25f4aec57e 100755
--- a/t/t7408-submodule-reference.sh
+++ b/t/t7408-submodule-reference.sh
@@ -61,7 +61,7 @@ test_expect_success 'submodule add --reference uses alternates' '
git commit -m B-super-added &&
git repack -ad
) &&
- test_alternate_is_used super/.git/modules/sub/objects/info/alternates super/sub
+ test_alternate_is_used super/.git/submodules/sub/objects/info/alternates super/sub
'
test_expect_success 'submodule add --reference with --dissociate does not use alternates' '
@@ -71,7 +71,7 @@ test_expect_success 'submodule add --reference with --dissociate does not use al
git commit -m B-super-added &&
git repack -ad
) &&
- test_path_is_missing super/.git/modules/sub-dissociate/objects/info/alternates
+ test_path_is_missing super/.git/submodules/sub-dissociate/objects/info/alternates
'
test_expect_success 'that reference gets used with add' '
@@ -94,14 +94,14 @@ test_expect_success 'updating superproject keeps alternates' '
test_when_finished "rm -rf super-clone" &&
git clone super super-clone &&
git -C super-clone submodule update --init --reference ../B &&
- test_alternate_is_used super-clone/.git/modules/sub/objects/info/alternates super-clone/sub
+ test_alternate_is_used super-clone/.git/submodules/sub/objects/info/alternates super-clone/sub
'
test_expect_success 'updating superproject with --dissociate does not keep alternates' '
test_when_finished "rm -rf super-clone" &&
git clone super super-clone &&
git -C super-clone submodule update --init --reference ../B --dissociate &&
- test_path_is_missing super-clone/.git/modules/sub/objects/info/alternates
+ test_path_is_missing super-clone/.git/submodules/sub/objects/info/alternates
'
test_expect_success 'submodules use alternates when cloning a superproject' '
@@ -112,7 +112,7 @@ test_expect_success 'submodules use alternates when cloning a superproject' '
# test superproject has alternates setup correctly
test_alternate_is_used .git/objects/info/alternates . &&
# test submodule has correct setup
- test_alternate_is_used .git/modules/sub/objects/info/alternates sub
+ test_alternate_is_used .git/submodules/sub/objects/info/alternates sub
)
'
@@ -127,7 +127,7 @@ test_expect_success 'missing submodule alternate fails clone and submodule updat
# update of the submodule succeeds
test_must_fail git submodule update --init &&
# and we have no alternates:
- test_path_is_missing .git/modules/sub/objects/info/alternates &&
+ test_path_is_missing .git/submodules/sub/objects/info/alternates &&
test_path_is_missing sub/file1
)
'
@@ -142,7 +142,7 @@ test_expect_success 'ignoring missing submodule alternates passes clone and subm
# update of the submodule succeeds
git submodule update --init &&
# and we have no alternates:
- test_path_is_missing .git/modules/sub/objects/info/alternates &&
+ test_path_is_missing .git/submodules/sub/objects/info/alternates &&
test_path_is_file sub/file1
)
'
@@ -176,18 +176,18 @@ test_expect_success 'nested submodule alternate in works and is actually used' '
# test superproject has alternates setup correctly
test_alternate_is_used .git/objects/info/alternates . &&
# immediate submodule has alternate:
- test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
+ test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
# nested submodule also has alternate:
- test_alternate_is_used .git/modules/subwithsub/modules/sub/objects/info/alternates subwithsub/sub
+ test_alternate_is_used .git/submodules/subwithsub/submodules/sub/objects/info/alternates subwithsub/sub
)
'
check_that_two_of_three_alternates_are_used() {
test_alternate_is_used .git/objects/info/alternates . &&
# immediate submodule has alternate:
- test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
+ test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
# but nested submodule has no alternate:
- test_path_is_missing .git/modules/subwithsub/modules/sub/objects/info/alternates
+ test_path_is_missing .git/submodules/subwithsub/submodules/sub/objects/info/alternates
}
diff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh
index 0490499573..dbaca9c69f 100755
--- a/t/t7412-submodule-absorbgitdirs.sh
+++ b/t/t7412-submodule-absorbgitdirs.sh
@@ -29,13 +29,13 @@ test_expect_success 'absorb the git dir' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1'\'' from
'\''$cwd/sub1/.git'\'' to
- '\''$cwd/.git/modules/sub1'\''
+ '\''$cwd/.git/submodules/sub1'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
git fsck &&
test -f sub1/.git &&
- test -d .git/modules/sub1 &&
+ test -d .git/submodules/sub1 &&
git status >actual.1 &&
git -C sub1 rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -47,7 +47,7 @@ test_expect_success 'absorbing does not fail for deinitialized submodules' '
git submodule deinit --all &&
git submodule absorbgitdirs 2>err &&
test_must_be_empty err &&
- test -d .git/modules/sub1 &&
+ test -d .git/submodules/sub1 &&
test -d sub1 &&
! test -e sub1/.git
'
@@ -68,12 +68,12 @@ test_expect_success 'absorb the git dir in a nested submodule' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1/nested'\'' from
'\''$cwd/sub1/nested/.git'\'' to
- '\''$cwd/.git/modules/sub1/modules/nested'\''
+ '\''$cwd/.git/submodules/sub1/submodules/nested'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
test -f sub1/nested/.git &&
- test -d .git/modules/sub1/modules/nested &&
+ test -d .git/submodules/sub1/submodules/nested &&
git status >actual.1 &&
git -C sub1/nested rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -84,11 +84,11 @@ test_expect_success 're-setup nested submodule' '
# un-absorb the direct submodule, to test if the nested submodule
# is still correct (needs a rewrite of the gitfile only)
rm -rf sub1/.git &&
- mv .git/modules/sub1 sub1/.git &&
+ mv .git/submodules/sub1 sub1/.git &&
GIT_WORK_TREE=. git -C sub1 config --unset core.worktree &&
# fixup the nested submodule
- echo "gitdir: ../.git/modules/nested" >sub1/nested/.git &&
- GIT_WORK_TREE=../../../nested git -C sub1/.git/modules/nested config \
+ echo "gitdir: ../.git/submodules/nested" >sub1/nested/.git &&
+ GIT_WORK_TREE=../../../nested git -C sub1/.git/submodules/nested config \
core.worktree "../../../nested" &&
# make sure this re-setup is correct
git status --ignore-submodules=none &&
@@ -105,13 +105,13 @@ test_expect_success 'absorb the git dir in a nested submodule' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub1'\'' from
'\''$cwd/sub1/.git'\'' to
- '\''$cwd/.git/modules/sub1'\''
+ '\''$cwd/.git/submodules/sub1'\''
EOF
git submodule absorbgitdirs 2>actual &&
test_cmp expect actual &&
test -f sub1/.git &&
test -f sub1/nested/.git &&
- test -d .git/modules/sub1/modules/nested &&
+ test -d .git/submodules/sub1/submodules/nested &&
git status >actual.1 &&
git -C sub1/nested rev-parse HEAD >actual.2 &&
test_cmp expect.1 actual.1 &&
@@ -133,7 +133,7 @@ test_expect_success 'absorb the git dir outside of primary worktree' '
cat >expect <<-EOF &&
Migrating git directory of '\''sub2'\'' from
'\''$cwd/repo-wt/sub2/.git'\'' to
- '\''$cwd/repo-bare.git/worktrees/repo-wt/modules/sub2'\''
+ '\''$cwd/repo-bare.git/worktrees/repo-wt/submodules/sub2'\''
EOF
git -C repo-wt submodule absorbgitdirs 2>actual &&
test_cmp expect actual
diff --git a/t/t7423-submodule-symlinks.sh b/t/t7423-submodule-symlinks.sh
index 3d3c7af3ce..a51235136d 100755
--- a/t/t7423-submodule-symlinks.sh
+++ b/t/t7423-submodule-symlinks.sh
@@ -49,19 +49,19 @@ test_expect_success SYMLINKS 'git restore --recurse-submodules must not be confu
test_expect_success SYMLINKS 'git restore --recurse-submodules must not migrate git dir of symlinked repo' '
prepare_symlink_to_repo &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
test_must_fail git restore --recurse-submodules a/sm &&
test_path_is_dir a/target/.git &&
- test_path_is_missing .git/modules/a/sm &&
+ test_path_is_missing .git/submodules/a/sm &&
test_path_is_missing a/target/submodule_file
'
test_expect_success SYMLINKS 'git checkout -f --recurse-submodules must not migrate git dir of symlinked repo when removing submodule' '
prepare_symlink_to_repo &&
- rm -rf .git/modules &&
+ rm -rf .git/submodules &&
test_must_fail git checkout -f --recurse-submodules initial &&
test_path_is_dir a/target/.git &&
- test_path_is_missing .git/modules/a/sm
+ test_path_is_missing .git/submodules/a/sm
'
test_done
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index f512eed278..4e2ced3636 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -77,28 +77,28 @@ test_expect_success 'create innocent subrepo' '
test_expect_success 'submodule add refuses invalid names' '
test_must_fail \
- git submodule add --name ../../modules/evil "$PWD/innocent" evil
+ git submodule add --name ../../submodules/evil "$PWD/innocent" evil
'
test_expect_success 'add evil submodule' '
git submodule add "$PWD/innocent" evil &&
- mkdir modules &&
- cp -r .git/modules/evil modules &&
- write_script modules/evil/hooks/post-checkout <<-\EOF &&
+ mkdir submodules &&
+ cp -r .git/submodules/evil submodules &&
+ write_script submodules/evil/hooks/post-checkout <<-\EOF &&
echo >&2 "RUNNING POST CHECKOUT"
EOF
git config -f .gitmodules submodule.evil.update checkout &&
git config -f .gitmodules --rename-section \
- submodule.evil submodule.../../modules/evil &&
- git add modules &&
+ submodule.evil submodule.../../submodules/evil &&
+ git add submodules &&
git commit -am evil
'
# This step seems like it shouldn't be necessary, since the payload is
# contained entirely in the evil submodule. But due to the vagaries of the
-# submodule code, checking out the evil module will fail unless ".git/modules"
+# submodule code, checking out the evil module will fail unless ".git/submodules"
# exists. Adding another submodule (with a name that sorts before "evil") is an
# easy way to make sure this is the case in the victim clone.
test_expect_success 'add other submodule' '
@@ -350,8 +350,8 @@ test_expect_success 'submodule git dir nesting detection must work with parallel
cat err &&
grep -E "(already exists|is inside git dir|not a git repository)" err &&
{
- test_path_is_missing .git/modules/hippo/HEAD ||
- test_path_is_missing .git/modules/hippo/hooks/HEAD
+ test_path_is_missing .git/submodules/hippo/HEAD ||
+ test_path_is_missing .git/submodules/hippo/hooks/HEAD
}
'
@@ -361,10 +361,10 @@ test_expect_success 'checkout -f --recurse-submodules must not use a nested gitd
cd nested_checkout &&
git submodule init &&
git submodule update thing1 &&
- mkdir -p .git/modules/hippo/hooks/refs &&
- mkdir -p .git/modules/hippo/hooks/objects/info &&
- echo "../../../../objects" >.git/modules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/modules/hippo/hooks/HEAD
+ mkdir -p .git/submodules/hippo/hooks/refs &&
+ mkdir -p .git/submodules/hippo/hooks/objects/info &&
+ echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD
) &&
test_must_fail git -C nested_checkout checkout -f --recurse-submodules HEAD 2>err &&
cat err &&
@@ -390,13 +390,13 @@ test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into d
git config unset -f repo/.gitmodules submodule.sub.path &&
printf "\tpath = \"sub\r\"\n" >>repo/.gitmodules &&
- git config unset -f repo/.git/modules/sub/config core.worktree &&
+ git config unset -f repo/.git/submodules/sub/config core.worktree &&
{
printf "[core]\n" &&
printf "\tworktree = \"../../../sub\r\"\n"
- } >>repo/.git/modules/sub/config &&
+ } >>repo/.git/submodules/sub/config &&
- ln -s .git/modules/sub/hooks repo/sub &&
+ ln -s .git/submodules/sub/hooks repo/sub &&
git -C repo add -A &&
git -C repo commit -m submodule &&
diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh
index 409cd0cd12..ded482fdf2 100755
--- a/t/t7527-builtin-fsmonitor.sh
+++ b/t/t7527-builtin-fsmonitor.sh
@@ -866,7 +866,7 @@ test_expect_success 'submodule always visited' '
'
# If a submodule has a `sub/.git/` directory (rather than a file
-# pointing to the super's `.git/modules/sub`) and `core.fsmonitor`
+# pointing to the super's `.git/submodules/sub`) and `core.fsmonitor`
# turned on in the submodule and the daemon is not yet started in
# the submodule, and someone does a `git submodule absorbgitdirs`
# in the super, Git will recursively invoke `git submodule--helper`
@@ -895,7 +895,7 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" '
cat >expect <<-EOF &&
Migrating git directory of '\''dir_1/dir_2/sub'\'' from
'\''$cwd/dir_1/dir_2/sub/.git'\'' to
- '\''$cwd/.git/modules/dir_1/dir_2/sub'\''
+ '\''$cwd/.git/submodules/dir_1/dir_2/sub'\''
EOF
GIT_TRACE2_EVENT="$PWD/super-sub.trace" \
git -C super submodule absorbgitdirs >out 2>actual &&
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 03/10] submodule: add gitdir path config override
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 02/10] submodule: create new gitdirs under submodules path Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-09 7:40 ` Patrick Steinhardt
2025-09-08 14:01 ` [PATCH v2 04/10] t7425: add basic mixed submodule gitdir path tests Adrian Ratiu
` (6 subsequent siblings)
9 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu, Brandon Williams
This adds an ability to override gitdir paths via config files
(not .gitmodules), such that any encoding scheme can be changed
and JGit & co don't need to exactly match the default encoding.
A new test and a helper are added. The helper will be used by
further tests exercising gitdir paths & encodings.
Based-on-patch-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/submodule.adoc | 4 ++++
builtin/submodule--helper.c | 17 +++++++++++++++++
submodule.c | 11 +++++++++++
t/lib-verify-submodule-gitdir-path.sh | 20 ++++++++++++++++++++
t/t7400-submodule-basic.sh | 9 +++++++++
t/t9902-completion.sh | 1 +
6 files changed, 62 insertions(+)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..8f64adfbe3 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,10 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This option sets the gitdir path for submodule <name>, allowing users
+ to override the default path or change the default path name encoding.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d06e2fe265..564f7aadf8 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1214,6 +1214,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo UNUSED)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, the_repository, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3597,6 +3613,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
diff --git a/submodule.c b/submodule.c
index dbf2244e60..bf78636195 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2611,6 +2611,17 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
* administrators can explicitly set. Nothing has been decided,
* so for now, just append the name at the end of the path.
*/
+ char *gitdir_path, *key;
+
+ /* Allow config override. */
+ key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
+ if (!repo_config_get_string(r, key, &gitdir_path)) {
+ strbuf_addstr(buf, gitdir_path);
+ free(key);
+ free(gitdir_path);
+ return;
+ }
+ free(key);
/* Legacy behavior: allow existing paths under modules/<name>. */
repo_git_path_append(r, buf, "modules/");
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..ef2a8a47a7
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,20 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (eg. when adding a new submodule), so this only checks
+# the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ cat >expect <<-EOF &&
+ $(git rev-parse --git-common-dir)/$path
+ EOF
+ git submodule--helper gitdir "$name" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 178c386212..a632f47b73 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup - enable local submodules' '
git config --global protocol.file.allow always
@@ -1505,4 +1506,12 @@ test_expect_success 'submodule add fails when name is reused' '
)
'
+test_expect_success 'submodule helper gitdir config overrides' '
+ verify_submodule_gitdir_path test-submodule child submodules/child &&
+ test_config -C test-submodule submodule.child.gitdirpath ".git/submodules/custom-child" &&
+ verify_submodule_gitdir_path test-submodule child submodules/custom-child &&
+ test_unconfig -C test-submodule submodule.child.gitdirpath &&
+ verify_submodule_gitdir_path test-submodule child submodules/child
+'
+
test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 6650d33fba..928e519267 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 04/10] t7425: add basic mixed submodule gitdir path tests
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (2 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 03/10] submodule: add gitdir path config override Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 05/10] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
` (5 subsequent siblings)
9 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
Add some basic submodule tests for mixed gitdir path handling of
legacy (.git/modules) and new-style (.git/submodule) paths.
For now these just test the coexistence, creation and push/pull of
submodules using mixed paths.
More tests will be added later, especially for new-style encoding.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/meson.build | 1 +
t/t7425-submodule-mixed-gitdir-paths.sh | 101 ++++++++++++++++++++++++
2 files changed, 102 insertions(+)
create mode 100755 t/t7425-submodule-mixed-gitdir-paths.sh
diff --git a/t/meson.build b/t/meson.build
index baeeba2ce6..28456158a0 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -879,6 +879,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-mixed-gitdir-paths.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
new file mode 100755
index 0000000000..31f16d7741
--- /dev/null
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed new and legacy submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+
+ git config receive.denyCurrentBranch updateInstead &&
+
+ git submodule add ../new-sub new &&
+ test_commit new-sub &&
+
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ # simulate legacy .git/modules path by moving submodule
+ mkdir -p .git/modules &&
+ mv .git/submodules/legacy .git/modules/ &&
+ echo "gitdir: ../.git/modules/legacy" > legacy/.git
+ )
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned &&
+ (
+ cd cloned &&
+
+ # At this point, .git/modules/<name> should not exist as
+ # submodules are checked out into the new path
+ test_path_is_dir .git/submodules/legacy &&
+ test_path_is_dir .git/submodules/new &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev new" list
+ )
+'
+
+test_expect_success 'commit and push changes to submodules' '
+ (
+ cd cloned &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C new switch --track -C main origin/main &&
+ test_commit -C new second-commit &&
+ git -C new push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy new &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/submodules/new &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev new" list
+ )
+'
+
+test_done
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 05/10] strbuf: bring back is_rfc3986_unreserved
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (3 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 04/10] t7425: add basic mixed submodule gitdir path tests Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
` (4 subsequent siblings)
9 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it's the only place used.
However now we need it to apply url encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 6 ------
strbuf.c | 6 ++++++
strbuf.h | 2 ++
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..0acaf1cc82 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -76,12 +76,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/strbuf.c b/strbuf.c
index 6c3851a7f8..e8d84cbb6d 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -817,6 +817,12 @@ void strbuf_addstr_xml_quoted(struct strbuf *buf, const char *s)
}
}
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
static void strbuf_add_urlencode(struct strbuf *sb, const char *s, size_t len,
char_predicate allow_unencoded_fn)
{
diff --git a/strbuf.h b/strbuf.h
index a580ac6084..5139269039 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -640,6 +640,8 @@ static inline void strbuf_complete_line(struct strbuf *sb)
typedef int (*char_predicate)(char ch);
+int is_rfc3986_unreserved(char ch);
+
void strbuf_addstr_urlencode(struct strbuf *sb, const char *name,
char_predicate allow_unencoded_fn);
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (4 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 05/10] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-10 18:15 ` SZEDER Gábor
` (2 more replies)
2025-09-08 14:01 ` [PATCH v2 07/10] submodule: error out if gitdir name is too long Adrian Ratiu
` (3 subsequent siblings)
9 siblings, 3 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu, Brandon Williams
Based on previous work by Brandon & all [1].
This encodes submodule gitdir names to avoid colisions like nested gitdirs
due to names like "foo" and "foo/bar".
A custom encoding can become unnecesarily complex, while url-encoding is
relatively well-known, however it needs some extending to support case
insensitive filesystems and quirks like Windows reserving "COM1" names.
Hence why I opted to encode A as _a, B as _b and so on.
Unfortunately encoding A -> _a (...) is not enough to fix the reserved
Windows file names (eg COM1) because workdirs still use the name COM1
even though gitdirs paths encoded, so future work will be needed to
fully address that case (or just use a different name).
This affected tests are fixed and a TODO is added to cleanup a hack /
short-circuit in validate_submodule_git_dir().
A further commit will add more tests to exercise these codepaths.
Link: https://lore.kernel.org/git/20180807230637.247200-1-bmwill@google.com/ [1]
Based-on-patch-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 55 +++++++++++++++++++++---------------
t/t7400-submodule-basic.sh | 2 +-
t/t7406-submodule-update.sh | 10 +++----
t/t7450-bad-git-dotfiles.sh | 39 +++++++++++++------------
t/t7527-builtin-fsmonitor.sh | 2 +-
5 files changed, 60 insertions(+), 48 deletions(-)
diff --git a/submodule.c b/submodule.c
index bf78636195..8e0fd077db 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2271,8 +2271,13 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
- BUG("submodule name '%s' not a suffix of git dir '%s'",
- submodule_name, git_dir);
+ /*
+ * TODO: revisit and cleanup this test short-circuit, because
+ * submodules with encoded names are expected to take this path.
+ * Likely just move the invariants to submodule_name_to_gitdir()
+ * and delete this entire function in a future commit.
+ */
+ return 0;
/*
* We prevent the contents of sibling submodules' git directories to
@@ -2588,30 +2593,26 @@ int submodule_to_gitdir(struct repository *repo,
return ret;
}
+static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
+{
+ for (; *src; src++) {
+ unsigned char c = *src;
+ if (c >= 'A' && c <= 'Z') {
+ strbuf_addch(dst, '_');
+ strbuf_addch(dst, c - 'A' + 'a');
+ } else {
+ strbuf_addch(dst, c);
+ }
+ }
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
- /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
- */
+ struct strbuf encoded_sub_name = STRBUF_INIT, tmp = STRBUF_INIT;
+ size_t base_len, encoded_len;
char *gitdir_path, *key;
+ long name_max;
/* Allow config override. */
key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
@@ -2632,5 +2633,13 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
/* New style (encoded) paths go under submodules/<encoded>. */
strbuf_reset(buf);
repo_git_path_append(r, buf, "submodules/");
- strbuf_addstr(buf, submodule_name);
+ base_len = buf->len;
+
+ /* URL-encode then case case-encode A to _a, B to _b and so on */
+ strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
+ strbuf_addstr_case_encode(&encoded_sub_name, tmp.buf);
+ strbuf_release(&tmp);
+ strbuf_addbuf(buf, &encoded_sub_name);
+
+ strbuf_release(&encoded_sub_name);
}
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index a632f47b73..fe772fa244 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1047,7 +1047,7 @@ test_expect_success 'recursive relative submodules stay relative' '
cd clone2 &&
git submodule update --init --recursive &&
echo "gitdir: ../.git/submodules/sub3" >./sub3/.git_expect &&
- echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
+ echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir%2fsubsub" >./sub3/dirdir/subsub/.git_expect
) &&
test_cmp clone2/sub3/.git_expect clone2/sub3/.git &&
test_cmp clone2/sub3/dirdir/subsub/.git_expect clone2/sub3/dirdir/subsub/.git
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index f0c4da1ffa..c44a7e9513 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -864,8 +864,8 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/submodules/deeper/submodule &&
- git log > ../../../../actual
+ (cd .git/submodules/deeper%2fsubmodule &&
+ git log > ../../../actual
) &&
test_cmp expected actual
)
@@ -882,8 +882,8 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir' '
(cd deeper/submodule &&
git log > ../../expected
) &&
- (cd .git/submodules/deeper/submodule &&
- git log > ../../../../actual
+ (cd .git/submodules/deeper%2fsubmodule &&
+ git log > ../../../actual
) &&
test_cmp expected actual
)
@@ -899,7 +899,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir recur
git commit -m "added subsubmodule" &&
git push origin :
) &&
- (cd .git/submodules/deeper/submodule/submodules/subsubmodule &&
+ (cd .git/submodules/deeper%2fsubmodule/submodules/subsubmodule &&
git log > ../../../../../actual
) &&
git add deeper/submodule &&
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index 4e2ced3636..27254300f8 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -15,6 +15,7 @@ Such as:
. ./test-lib.sh
. "$TEST_DIRECTORY"/lib-pack.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup' '
git config --global protocol.file.allow always
@@ -319,6 +320,8 @@ test_expect_success WINDOWS 'prevent git~1 squatting on Windows' '
fi
'
+# TODO: move these nested gitdir tests to another location in a later commit because
+# they are not pathological cases anymore: by encoding the gitdir paths do not conflict.
test_expect_success 'setup submodules with nested git dirs' '
git init nested &&
test_commit -C nested nested &&
@@ -341,35 +344,35 @@ test_expect_success 'setup submodules with nested git dirs' '
'
test_expect_success 'git dirs of sibling submodules must not be nested' '
- test_must_fail git clone --recurse-submodules nested clone 2>err &&
- test_grep "is inside git dir" err
+ git clone --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
- test_must_fail git clone --recurse-submodules --jobs=2 nested clone_parallel 2>err &&
- cat err &&
- grep -E "(already exists|is inside git dir|not a git repository)" err &&
- {
- test_path_is_missing .git/submodules/hippo/HEAD ||
- test_path_is_missing .git/submodules/hippo/hooks/HEAD
- }
+ git clone --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
-test_expect_success 'checkout -f --recurse-submodules must not use a nested gitdir' '
- git clone nested nested_checkout &&
+test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
+ git clone nested clone_recursive_checkout &&
(
- cd nested_checkout &&
+ cd clone_recursive_checkout &&
+
git submodule init &&
- git submodule update thing1 &&
+ git submodule update thing1 thing2 &&
+
+ # simulate a malicious nested alternate which git should not follow
mkdir -p .git/submodules/hippo/hooks/refs &&
mkdir -p .git/submodules/hippo/hooks/objects/info &&
echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
+
+ git checkout -f --recurse-submodules HEAD
) &&
- test_must_fail git -C nested_checkout checkout -f --recurse-submodules HEAD 2>err &&
- cat err &&
- grep "is inside git dir" err &&
- test_path_is_missing nested_checkout/thing2/.git
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into different directory' '
diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh
index ded482fdf2..15e44a6979 100755
--- a/t/t7527-builtin-fsmonitor.sh
+++ b/t/t7527-builtin-fsmonitor.sh
@@ -895,7 +895,7 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" '
cat >expect <<-EOF &&
Migrating git directory of '\''dir_1/dir_2/sub'\'' from
'\''$cwd/dir_1/dir_2/sub/.git'\'' to
- '\''$cwd/.git/submodules/dir_1/dir_2/sub'\''
+ '\''$cwd/.git/submodules/dir_1%2fdir_2%2fsub'\''
EOF
GIT_TRACE2_EVENT="$PWD/super-sub.trace" \
git -C super submodule absorbgitdirs >out 2>actual &&
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 07/10] submodule: error out if gitdir name is too long
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (5 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-08 15:51 ` Jeff King
2025-09-30 13:35 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 08/10] submodule: remove validate_submodule_git_dir() Adrian Ratiu
` (2 subsequent siblings)
9 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
Encoding submodule names increases their name size, so there is an
increased risk to hit the max filename length in the gitdir path.
(the likelihood is still rather small, so it's an acceptable risk)
This gitdir file-name-too-long corner case can be be addressed in
multiple ways, including sharding or trimming, however for now, just
add the portable logic (suggested by Peff) to detect the corner case
then error out to avoid comitting to a specific policy (or policies).
In the future, instead of throwing an error (which we do now anyway
without submodule encoding), we could maybe let the user specify via
configs how to address this case, eg pick trimming or sharding.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Makefile | 5 +++++
compat/pathconf.c | 10 ++++++++++
compat/posix.h | 8 ++++++++
config.mak.uname | 2 ++
meson.build | 1 +
submodule.c | 14 ++++++++++++++
6 files changed, 40 insertions(+)
create mode 100644 compat/pathconf.c
diff --git a/Makefile b/Makefile
index 555b7f4dc3..1a98eac8a5 100644
--- a/Makefile
+++ b/Makefile
@@ -2212,6 +2212,11 @@ ifndef HAVE_PLATFORM_PROCINFO
COMPAT_OBJS += compat/stub/procinfo.o
endif
+ifdef NO_PATHCONF
+ COMPAT_CFLAGS += -DNO_PATHCONF
+ COMPAT_OBJS += compat/pathconf.o
+endif
+
ifdef RUNTIME_PREFIX
ifdef HAVE_BSD_KERN_PROC_SYSCTL
diff --git a/compat/pathconf.c b/compat/pathconf.c
new file mode 100644
index 0000000000..37500cfa0d
--- /dev/null
+++ b/compat/pathconf.c
@@ -0,0 +1,10 @@
+#include "git-compat-util.h"
+
+/*
+ * Minimal stub for platforms without pathconf() (e.g. Windows),
+ * to fall back to NAME_MAX from limits.h or compat/posix.h.
+ */
+long git_pathconf(const char *path UNUSED, int name UNUSED)
+{
+ return -1;
+}
diff --git a/compat/posix.h b/compat/posix.h
index 067a00f33b..aa050fd58c 100644
--- a/compat/posix.h
+++ b/compat/posix.h
@@ -250,6 +250,14 @@ char *gitdirname(char *);
#define NAME_MAX 255
#endif
+#ifdef NO_PATHCONF
+#ifndef _PC_NAME_MAX
+#define _PC_NAME_MAX 1 /* dummy value, only used for git_pathconf */
+#endif
+#define pathconf(a,b) git_pathconf(a,b)
+long git_pathconf(const char *path, int name);
+#endif
+
typedef uintmax_t timestamp_t;
#define PRItime PRIuMAX
#define parse_timestamp strtoumax
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..49ba3de39d 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -473,6 +473,7 @@ ifeq ($(uname_S),Windows)
NEEDS_CRYPTO_WITH_SSL = YesPlease
NO_LIBGEN_H = YesPlease
NO_POLL = YesPlease
+ NO_PATHCONF = YesPlease
NO_SYMLINK_HEAD = YesPlease
NO_IPV6 = YesPlease
NO_SETENV = YesPlease
@@ -688,6 +689,7 @@ ifeq ($(uname_S),MINGW)
NEEDS_CRYPTO_WITH_SSL = YesPlease
NO_LIBGEN_H = YesPlease
NO_POLL = YesPlease
+ NO_PATHCONF = YesPlease
NO_SYMLINK_HEAD = YesPlease
NO_SETENV = YesPlease
NO_STRCASESTR = YesPlease
diff --git a/meson.build b/meson.build
index e8ec0eca16..1fb9300ab1 100644
--- a/meson.build
+++ b/meson.build
@@ -1393,6 +1393,7 @@ checkfuncs = {
'initgroups' : [],
'strtoumax' : ['strtoumax.c', 'strtoimax.c'],
'pread' : ['pread.c'],
+ 'pathconf' : ['pathconf.c'],
}
if host_machine.system() == 'windows'
diff --git a/submodule.c b/submodule.c
index 8e0fd077db..016509806e 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2641,5 +2641,19 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
strbuf_release(&tmp);
strbuf_addbuf(buf, &encoded_sub_name);
+ /* Ensure final path length is below NAME_MAX after encoding */
+ name_max = pathconf(buf->buf, _PC_NAME_MAX);
+ if (name_max == -1)
+ name_max = NAME_MAX;
+
+ encoded_len = buf->len - base_len;
+ if (encoded_len > name_max)
+ /*
+ * TODO: make this smarter; instead of erroring out, maybe we could trim or
+ * shard the gitdir names to make them fit under NAME_MAX.
+ */
+ die(_("encoded submodule name '%s' is too long (%"PRIuMAX" bytes, limit %"PRIuMAX")"),
+ encoded_sub_name.buf, (uintmax_t)encoded_len, (uintmax_t)name_max);
+
strbuf_release(&encoded_sub_name);
}
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 08/10] submodule: remove validate_submodule_git_dir()
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (6 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 07/10] submodule: error out if gitdir name is too long Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-30 13:35 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 09/10] t7450: move nested gitdir tests to t7425 Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 10/10] t7425: add gitdir encoding tests Adrian Ratiu
9 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
The validate_submodule_git_dir test is not very useful anymore, after
submodule names are encoded to resolve gitdir path conflicts.
In other words, the purpouse of gitdir path encoding is precisely to
avoid such conflicts as this function tries to also prevent.
The first test from the function can be kept though, because it just
verifies invariants which should always be true and raise a BUG if:
- no "/" separator is between dirs/names.
- len(full_gitdir) < len(name).
- name does not match the gitdir path suffix.
Thus we move the invariant checks to submodule_name_to_gitdir() and
clean up the rest of validate_submodule_git_dir() and its uses.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 21 -----------
submodule.c | 74 ++++---------------------------------
submodule.h | 5 ---
3 files changed, 7 insertions(+), 93 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 564f7aadf8..8af7062ff2 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1725,10 +1725,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
clone_data_path = to_free = xstrfmt("%s/%s", repo_get_work_tree(the_repository),
clone_data->path);
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
-
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
@@ -1802,23 +1798,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
free(path);
}
- /*
- * We already performed this check at the beginning of this function,
- * before cloning the objects. This tries to detect racy behavior e.g.
- * in parallel clones, where another process could easily have made the
- * gitdir nested _after_ it was created.
- *
- * To prevent further harm coming from this unintentionally-nested
- * gitdir, let's disable it by deleting the `HEAD` file.
- */
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
- char *head = xstrfmt("%s/HEAD", sm_gitdir);
- unlink(head);
- free(head);
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
- }
-
connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
p = repo_submodule_path(the_repository, clone_data_path, "config");
diff --git a/submodule.c b/submodule.c
index 016509806e..dbe1a7b091 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2163,27 +2163,10 @@ int submodule_move_head(const char *path, const char *super_prefix,
if (!submodule_uses_gitfile(path))
absorb_git_dir_into_superproject(path,
super_prefix);
- else {
- char *dotgit = xstrfmt("%s/.git", path);
- char *git_dir = xstrdup(read_gitfile(dotgit));
-
- free(dotgit);
- if (validate_submodule_git_dir(git_dir,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in "
- "another submodule's git dir"),
- git_dir);
- free(git_dir);
- }
} else {
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
- if (validate_submodule_git_dir(gitdir.buf,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in another "
- "submodule's git dir"),
- gitdir.buf);
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
@@ -2263,52 +2246,6 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
-{
- size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
- char *p;
- int ret = 0;
-
- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
- strcmp(p, submodule_name))
- /*
- * TODO: revisit and cleanup this test short-circuit, because
- * submodules with encoded names are expected to take this path.
- * Likely just move the invariants to submodule_name_to_gitdir()
- * and delete this entire function in a future commit.
- */
- return 0;
-
- /*
- * We prevent the contents of sibling submodules' git directories to
- * clash.
- *
- * Example: having a submodule named `hippo` and another one named
- * `hippo/hooks` would result in the git directories
- * `.git/submodules/hippo/` and `.git/submodules/hippo/hooks/`, respectively,
- * but the latter directory is already designated to contain the hooks
- * of the former.
- */
- for (; *p; p++) {
- if (is_dir_sep(*p)) {
- char c = *p;
-
- *p = '\0';
- if (is_git_directory(git_dir))
- ret = -1;
- *p = c;
-
- if (ret < 0)
- return error(_("submodule git dir '%s' is "
- "inside git dir '%.*s'"),
- git_dir,
- (int)(p - git_dir), git_dir);
- }
- }
-
- return 0;
-}
-
int validate_submodule_path(const char *path)
{
char *p = xstrdup(path);
@@ -2367,9 +2304,6 @@ static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
submodule_name_to_gitdir(&new_gitdir, the_repository, sub->name);
- if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
- die(_("refusing to move '%s' into an existing git dir"),
- real_old_git_dir);
if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
die(_("could not create directory '%s'"), new_gitdir.buf);
real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
@@ -2611,7 +2545,7 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
{
struct strbuf encoded_sub_name = STRBUF_INIT, tmp = STRBUF_INIT;
size_t base_len, encoded_len;
- char *gitdir_path, *key;
+ char *gitdir_path, *key, *p;
long name_max;
/* Allow config override. */
@@ -2655,5 +2589,11 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
die(_("encoded submodule name '%s' is too long (%"PRIuMAX" bytes, limit %"PRIuMAX")"),
encoded_sub_name.buf, (uintmax_t)encoded_len, (uintmax_t)name_max);
+ /* Trigger a BUG if these invariants do not hold */
+ p = buf->buf + buf->len - encoded_len;
+ if (buf->len <= encoded_len || p[-1] != '/' || strcmp(p, encoded_sub_name.buf))
+ BUG("encoded submodule name '%s' is not a suffix of git dir '%s'",
+ encoded_sub_name.buf, buf->buf);
+
strbuf_release(&encoded_sub_name);
}
diff --git a/submodule.h b/submodule.h
index b10e16e6c0..0b7692bc20 100644
--- a/submodule.h
+++ b/submodule.h
@@ -137,11 +137,6 @@ int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name);
-/*
- * Make sure that no submodule's git dir is nested in a sibling submodule's.
- */
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
-
/*
* Make sure that the given submodule path does not follow symlinks.
*/
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 09/10] t7450: move nested gitdir tests to t7425
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (7 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 08/10] submodule: remove validate_submodule_git_dir() Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 10/10] t7425: add gitdir encoding tests Adrian Ratiu
9 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
Now that we are encoding gitdir paths, these tests are not handling
pathological cases anymore, because nested git dirs shouldn't cause
conflicts, so move them from t7450-bad-git-dotfiles.sh to a more
appropriate location where we test mixed gitdir path & encoding use.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/t7425-submodule-mixed-gitdir-paths.sh | 54 ++++++++++++++++++++++++
t/t7450-bad-git-dotfiles.sh | 56 -------------------------
2 files changed, 54 insertions(+), 56 deletions(-)
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
index 31f16d7741..a8c22de070 100755
--- a/t/t7425-submodule-mixed-gitdir-paths.sh
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -3,6 +3,7 @@
test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup: allow file protocol' '
git config --global protocol.file.allow always
@@ -98,4 +99,57 @@ test_expect_success 'fetch mixed submodule changes and verify updates' '
)
'
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of sibling submodules must not be nested' '
+ git clone --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
+test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
+ git clone nested clone_recursive_checkout &&
+ (
+ cd clone_recursive_checkout &&
+
+ git submodule init &&
+ git submodule update thing1 thing2 &&
+
+ # simulate a malicious nested alternate which git should not follow
+ mkdir -p .git/submodules/hippo/hooks/refs &&
+ mkdir -p .git/submodules/hippo/hooks/objects/info &&
+ echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
+ echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
+
+ git checkout -f --recurse-submodules HEAD
+ ) &&
+ verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
+'
+
test_done
diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
index 27254300f8..18624fabc4 100755
--- a/t/t7450-bad-git-dotfiles.sh
+++ b/t/t7450-bad-git-dotfiles.sh
@@ -15,7 +15,6 @@ Such as:
. ./test-lib.sh
. "$TEST_DIRECTORY"/lib-pack.sh
-. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup' '
git config --global protocol.file.allow always
@@ -320,61 +319,6 @@ test_expect_success WINDOWS 'prevent git~1 squatting on Windows' '
fi
'
-# TODO: move these nested gitdir tests to another location in a later commit because
-# they are not pathological cases anymore: by encoding the gitdir paths do not conflict.
-test_expect_success 'setup submodules with nested git dirs' '
- git init nested &&
- test_commit -C nested nested &&
- (
- cd nested &&
- cat >.gitmodules <<-EOF &&
- [submodule "hippo"]
- url = .
- path = thing1
- [submodule "hippo/hooks"]
- url = .
- path = thing2
- EOF
- git clone . thing1 &&
- git clone . thing2 &&
- git add .gitmodules thing1 thing2 &&
- test_tick &&
- git commit -m nested
- )
-'
-
-test_expect_success 'git dirs of sibling submodules must not be nested' '
- git clone --recurse-submodules nested clone_nested &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
-test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
- git clone --recurse-submodules --jobs=2 nested clone_parallel &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
-test_expect_success 'checkout -f --recurse-submodules must corectly handle nested gitdirs' '
- git clone nested clone_recursive_checkout &&
- (
- cd clone_recursive_checkout &&
-
- git submodule init &&
- git submodule update thing1 thing2 &&
-
- # simulate a malicious nested alternate which git should not follow
- mkdir -p .git/submodules/hippo/hooks/refs &&
- mkdir -p .git/submodules/hippo/hooks/objects/info &&
- echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
- echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD &&
-
- git checkout -f --recurse-submodules HEAD
- ) &&
- verify_submodule_gitdir_path clone_nested hippo submodules/hippo &&
- verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
-'
-
test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into different directory' '
test_when_finished "rm -rf sub repo bad-clone" &&
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v2 10/10] t7425: add gitdir encoding tests
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
` (8 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 09/10] t7450: move nested gitdir tests to t7425 Adrian Ratiu
@ 2025-09-08 14:01 ` Adrian Ratiu
9 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 14:01 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Adrian Ratiu
Add some tests to further exercise the gitdir encoding functionality
alongside the existing mixed directory and nested gitdir tests.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
t/t7425-submodule-mixed-gitdir-paths.sh | 47 +++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/t/t7425-submodule-mixed-gitdir-paths.sh b/t/t7425-submodule-mixed-gitdir-paths.sh
index a8c22de070..f467bafaab 100755
--- a/t/t7425-submodule-mixed-gitdir-paths.sh
+++ b/t/t7425-submodule-mixed-gitdir-paths.sh
@@ -152,4 +152,51 @@ test_expect_success 'checkout -f --recurse-submodules must corectly handle neste
verify_submodule_gitdir_path clone_nested hippo/hooks submodules/hippo%2fhooks
'
+test_expect_success 'new style submodule gitdir paths are properly encoded' '
+ (
+ cd main &&
+
+ # add new-style submodule name containing /
+ git submodule add ../new-sub foo/bar &&
+ git commit -m "add foo/bar" &&
+
+ # simulate existing legacy submodule name containing escaping char %
+ git clone --separate-git-dir .git/modules/foo%bar ../legacy-sub foo%bar &&
+ cat >>.gitmodules <<-EOF &&
+ [submodule "foo%bar"]
+ path = foo%bar
+ url = ../legacy-sub
+ EOF
+ git add .gitmodules &&
+ git commit -m "add foo%bar" &&
+
+ # add new style submodule name containing escaping char %
+ git submodule add ../new-sub fooish%bar &&
+ git commit -m "add fooish%bar" &&
+
+ # add a mixed case submdule name
+ git submodule add ../new-sub FooBar &&
+ git commit -m "add FooBar"
+ ) &&
+ verify_submodule_gitdir_path main foo/bar submodules/foo%2fbar &&
+ verify_submodule_gitdir_path main foo%bar modules/foo%bar &&
+ verify_submodule_gitdir_path main fooish%bar submodules/fooish%25bar &&
+ verify_submodule_gitdir_path main FooBar submodules/_foo_bar
+'
+
+test_expect_success 'submodule encoded name exceeds max name limit' '
+ (
+ cd main &&
+
+ # find the system NAME_MAX (fall back to 255 if unknown)
+ name_max=$(getconf NAME_MAX . 2>/dev/null || echo 255) &&
+
+ # each "%" char encodes to "%25" (3 chars), ensure we exceed NAME_MAX
+ count=$((name_max + 10)) &&
+ longname=$(test_seq -f "%%%0.s" 1 $count) &&
+
+ test_must_fail git submodule add ../new-sub "$longname"
+ )
+'
+
test_done
--
2.51.GIT
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH 7/9] submodule: remove validate_submodule_git_dir()
2025-08-16 21:36 ` [PATCH 7/9] submodule: remove validate_submodule_git_dir() Adrian Ratiu
@ 2025-09-08 14:23 ` Phillip Wood
0 siblings, 0 replies; 179+ messages in thread
From: Phillip Wood @ 2025-09-08 14:23 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
Hi Adrian
On 16/08/2025 22:36, Adrian Ratiu wrote:
>
> + /* Trigger a BUG if these invariants do not hold */
> + p = buf->buf + buf->len - encoded_len;
> + if (buf->len <= encoded_len || p[-1] != '/' || strcmp(p, encoded_sub_name.buf))
if buf->len is less than encoded_len then the pointer p is invalid. As a
valid program cannot create an invalid pointer the compiler may assume
that buf->len >= encoded_len. We should check the lengths before
creating the pointer as the original code in
validate_submodule_git_dir() did which looked like
> if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
> strcmp(p, submodule_name))
Thanks
Phillip
> + BUG("encoded submodule name '%s' is not a suffix of git dir '%s'",
> + encoded_sub_name.buf, buf->buf);
> +
> strbuf_release(&encoded_sub_name);
> }
> diff --git a/submodule.h b/submodule.h
> index b10e16e6c0..0b7692bc20 100644
> --- a/submodule.h
> +++ b/submodule.h
> @@ -137,11 +137,6 @@ int submodule_to_gitdir(struct repository *repo,
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name);
>
> -/*
> - * Make sure that no submodule's git dir is nested in a sibling submodule's.
> - */
> -int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
> -
> /*
> * Make sure that the given submodule path does not follow symlinks.
> */
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
` (2 preceding siblings ...)
2025-08-20 21:50 ` Josh Steadmon
@ 2025-09-08 14:23 ` Phillip Wood
2025-09-09 12:02 ` Adrian Ratiu
3 siblings, 1 reply; 179+ messages in thread
From: Phillip Wood @ 2025-09-08 14:23 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Brandon Williams
Hi Adrian
On 16/08/2025 22:36, Adrian Ratiu wrote:
> This adds an ability to override gitdir paths via config files
> (not .gitmodules), such that any encoding scheme can be changed
> and JGit & co don't need to exactly match the default encoding.
Reading the old email thread you linked to in the cover letter, my
understanding is that this was suggested as an alternative to changing
the gitdir for submodules from "modules" to "submodules". Do the later
patches set this key when encoding the gitdir so that JGit can find the
gitdir by reading the config?
Thanks
Phillip
> A new test and a helper are added. The helper will be used by
> further tests exercising gitdir paths & encodings.
>
> Based-on-patch-by: Brandon Williams <bmwill@google.com>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> builtin/submodule--helper.c | 17 +++++++++++++++++
> submodule.c | 11 +++++++++++
> t/lib-verify-submodule-gitdir-path.sh | 15 +++++++++++++++
> t/t7400-submodule-basic.sh | 15 +++++++++++++++
> 4 files changed, 58 insertions(+)
> create mode 100644 t/lib-verify-submodule-gitdir-path.sh
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 7243429c6f..30e40d6c79 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1214,6 +1214,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
> return ret;
> }
>
> +static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
> + struct repository *repo UNUSED)
> +{
> + struct strbuf gitdir = STRBUF_INIT;
> +
> + if (argc != 2)
> + usage(_("git submodule--helper gitdir <name>"));
> +
> + submodule_name_to_gitdir(&gitdir, the_repository, argv[1]);
> +
> + printf("%s\n", gitdir.buf);
> +
> + strbuf_release(&gitdir);
> + return 0;
> +}
> +
> struct sync_cb {
> const char *prefix;
> const char *super_prefix;
> @@ -3597,6 +3613,7 @@ int cmd_submodule__helper(int argc,
> NULL
> };
> struct option options[] = {
> + OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
> OPT_SUBCOMMAND("clone", &fn, module_clone),
> OPT_SUBCOMMAND("add", &fn, module_add),
> OPT_SUBCOMMAND("update", &fn, module_update),
> diff --git a/submodule.c b/submodule.c
> index dbf2244e60..bf78636195 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2611,6 +2611,17 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> * administrators can explicitly set. Nothing has been decided,
> * so for now, just append the name at the end of the path.
> */
> + char *gitdir_path, *key;
> +
> + /* Allow config override. */
> + key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
> + if (!repo_config_get_string(r, key, &gitdir_path)) {
> + strbuf_addstr(buf, gitdir_path);
> + free(key);
> + free(gitdir_path);
> + return;
> + }
> + free(key);
>
> /* Legacy behavior: allow existing paths under modules/<name>. */
> repo_git_path_append(r, buf, "modules/");
> diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
> new file mode 100644
> index 0000000000..fb5cb8eea4
> --- /dev/null
> +++ b/t/lib-verify-submodule-gitdir-path.sh
> @@ -0,0 +1,15 @@
> +# Helper to verify if repo $1 contains a submodule named $2 with gitdir in path $3
> +
> +verify_submodule_gitdir_path() {
> + repo="$1" &&
> + name="$2" &&
> + path="$3" &&
> + (
> + cd "$repo" &&
> + cat >expect <<-EOF &&
> + $(git rev-parse --git-common-dir)/$path
> + EOF
> + git submodule--helper gitdir "$name" >actual &&
> + test_cmp expect actual
> + )
> +}
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 178c386212..f4d4fb8397 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>
> test_expect_success 'setup - enable local submodules' '
> git config --global protocol.file.allow always
> @@ -1505,4 +1506,18 @@ test_expect_success 'submodule add fails when name is reused' '
> )
> '
>
> +test_expect_success 'submodule helper gitdir config overrides' '
> + verify_submodule_gitdir_path test-submodule child submodules/child &&
> + (
> + cd test-submodule &&
> + git config submodule.child.gitdirpath ".git/submodules/custom-child"
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/custom-child &&
> + (
> + cd test-submodule &&
> + git config --unset submodule.child.gitdirpath
> + ) &&
> + verify_submodule_gitdir_path test-submodule child submodules/child
> +'
> +
> test_done
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 2/9] submodule: create new gitdirs under submodules path
2025-08-16 21:36 ` [PATCH 2/9] submodule: create new gitdirs under submodules path Adrian Ratiu
@ 2025-09-08 14:24 ` Phillip Wood
2025-09-08 15:46 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Phillip Wood @ 2025-09-08 14:24 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
Hi Adrian
On 16/08/2025 22:36, Adrian Ratiu wrote:
> This is in preparation for encoding the submodule names to avoid conflicts
> like submodules named foo and foo/bar together with case-insensitive file-
> system handling and other corner cases like reserved filenames on Windows.
>
> Backward compatibility is kept with plain-name modules already existing at
> paths like .git/modules/<name>, however a clear separation between legacy
> (plain) and new (encoded) namespaces is desirable, to avoid situations like
> an existing plain-name module containing the encoding escape character/
>
> Thus we split the new-style (encoded) gitdir name paths to .git/submodules,
> while legacy-style paths remain under .git/modules.
>
> This is just a default directory change with the accompanying test updates,
> in preparation for the actual encoding additions in future commits.
Does this need an extentions.submoduleEncoding (name suggestions
welcome) config key to stop older versions of git trying to read the
repository as they wont be able to locate the gitdir of any submodules
added under .git/submodules?
Thanks
Phillip
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> Documentation/fetch-options.adoc | 2 +-
> Documentation/git-fetch.adoc | 2 +-
> Documentation/git-submodule.adoc | 2 +-
> Documentation/gitsubmodules.adoc | 8 ++--
> setup.c | 2 +-
> submodule.c | 28 +++++++++---
> t/lib-submodule-update.sh | 50 +++++++++++-----------
> t/t0035-safe-bare-repository.sh | 4 +-
> t/t1600-index.sh | 4 +-
> t/t2405-worktree-submodule.sh | 8 ++--
> t/t2501-cwd-empty.sh | 2 +-
> t/t3600-rm.sh | 8 ++--
> t/t5526-fetch-submodules.sh | 2 +-
> t/t5619-clone-local-ambiguous-transport.sh | 4 +-
> t/t6120-describe.sh | 4 +-
> t/t7001-mv.sh | 4 +-
> t/t7400-submodule-basic.sh | 18 ++++----
> t/t7406-submodule-update.sh | 10 ++---
> t/t7407-submodule-foreach.sh | 6 +--
> t/t7408-submodule-reference.sh | 22 +++++-----
> t/t7412-submodule-absorbgitdirs.sh | 22 +++++-----
> t/t7423-submodule-symlinks.sh | 8 ++--
> t/t7450-bad-git-dotfiles.sh | 32 +++++++-------
> t/t7527-builtin-fsmonitor.sh | 4 +-
> 24 files changed, 136 insertions(+), 120 deletions(-)
>
> diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
> index b01372e4b3..f8d3f65009 100644
> --- a/Documentation/fetch-options.adoc
> +++ b/Documentation/fetch-options.adoc
> @@ -210,7 +210,7 @@ ifndef::git-pull[]
> submodule that has commits that are referenced by a newly fetched
> superproject commit but are missing in the local submodule clone. A
> changed submodule can be fetched as long as it is present locally e.g.
> - in `$GIT_DIR/modules/` (see linkgit:gitsubmodules[7]); if the upstream
> + in `$GIT_DIR/submodules/` (see linkgit:gitsubmodules[7]); if the upstream
> adds a new submodule, that submodule cannot be fetched until it is
> cloned e.g. by `git submodule update`.
> +
> diff --git a/Documentation/git-fetch.adoc b/Documentation/git-fetch.adoc
> index 16f5d9d69a..2923a29bef 100644
> --- a/Documentation/git-fetch.adoc
> +++ b/Documentation/git-fetch.adoc
> @@ -304,7 +304,7 @@ include::config/fetch.adoc[]
> BUGS
> ----
> Using --recurse-submodules can only fetch new commits in submodules that are
> -present locally e.g. in `$GIT_DIR/modules/`. If the upstream adds a new
> +present locally e.g. in `$GIT_DIR/submodules/`. If the upstream adds a new
> submodule, that submodule cannot be fetched until it is cloned e.g. by `git
> submodule update`. This is expected to be fixed in a future Git version.
>
> diff --git a/Documentation/git-submodule.adoc b/Documentation/git-submodule.adoc
> index 503c84a200..9389862208 100644
> --- a/Documentation/git-submodule.adoc
> +++ b/Documentation/git-submodule.adoc
> @@ -266,7 +266,7 @@ registered submodules, and sync any nested submodules within.
> absorbgitdirs::
> If a git directory of a submodule is inside the submodule,
> move the git directory of the submodule into its superproject's
> - `$GIT_DIR/modules` path and then connect the git directory and
> + `$GIT_DIR/submodules` path and then connect the git directory and
> its working directory by setting the `core.worktree` and adding
> a .git file pointing to the git directory embedded in the
> superprojects git directory.
> diff --git a/Documentation/gitsubmodules.adoc b/Documentation/gitsubmodules.adoc
> index f7b5a25a0c..061e24f316 100644
> --- a/Documentation/gitsubmodules.adoc
> +++ b/Documentation/gitsubmodules.adoc
> @@ -21,12 +21,12 @@ The submodule has its own history; the repository it is embedded
> in is called a superproject.
>
> On the filesystem, a submodule usually (but not always - see FORMS below)
> -consists of (i) a Git directory located under the `$GIT_DIR/modules/`
> +consists of (i) a Git directory located under the `$GIT_DIR/submodules/`
> directory of its superproject, (ii) a working directory inside the
> superproject's working directory, and a `.git` file at the root of
> the submodule's working directory pointing to (i).
>
> -Assuming the submodule has a Git directory at `$GIT_DIR/modules/foo/`
> +Assuming the submodule has a Git directory at `$GIT_DIR/submodules/foo/`
> and a working directory at `path/to/bar/`, the superproject tracks the
> submodule via a `gitlink` entry in the tree at `path/to/bar` and an entry
> in its `.gitmodules` file (see linkgit:gitmodules[5]) of the form
> @@ -137,7 +137,7 @@ using older versions of Git.
> It is possible to construct these old form repositories manually.
> +
> When deinitialized or deleted (see below), the submodule's Git
> -directory is automatically moved to `$GIT_DIR/modules/<name>/`
> +directory is automatically moved to `$GIT_DIR/submodules/<name>/`
> of the superproject.
>
> * Deinitialized submodule: A `gitlink`, and a `.gitmodules` entry,
> @@ -162,7 +162,7 @@ possible to checkout past commits without requiring fetching
> from another repository.
> +
> To completely remove a submodule, manually delete
> -`$GIT_DIR/modules/<name>/`.
> +`$GIT_DIR/submodules/<name>/`.
>
> ACTIVE SUBMODULES
> -----------------
> diff --git a/setup.c b/setup.c
> index 98ddbf377f..d054dafa6a 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -1416,7 +1416,7 @@ static int is_implicit_bare_repo(const char *path)
> * we are inside $GIT_DIR of a worktree of a non-embedded
> * submodule, whose superproject is not a bare repository.
> */
> - if (strstr(path, "/.git/modules/"))
> + if (strstr(path, "/.git/modules/") || strstr(path, "/.git/submodules/"))
> return 1;
>
> return 0;
> diff --git a/submodule.c b/submodule.c
> index fff3c75570..dbf2244e60 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1278,22 +1278,29 @@ void check_for_new_submodule_commits(struct object_id *oid)
>
> /*
> * Returns 1 if there is at least one submodule gitdir in
> - * $GIT_DIR/modules and 0 otherwise. This follows
> + * $GIT_DIR/(sub)modules and 0 otherwise. This follows
> * submodule_name_to_gitdir(), which looks for submodules in
> - * $GIT_DIR/modules, not $GIT_COMMON_DIR.
> + * $GIT_DIR/(sub)modules, not $GIT_COMMON_DIR.
> *
> - * A submodule can be moved to $GIT_DIR/modules manually by running "git
> - * submodule absorbgitdirs", or it may be initialized there by "git
> - * submodule update".
> + * A submodule can be moved to $GIT_DIR/(sub)modules manually by running
> + * "git submodule absorbgitdirs", or it may be initialized there by
> + * "git submodule update".
> */
> static int repo_has_absorbed_submodules(struct repository *r)
> {
> int ret;
> struct strbuf buf = STRBUF_INIT;
>
> + /* check legacy path */
> repo_git_path_append(r, &buf, "modules/");
> ret = file_exists(buf.buf) && !is_empty_dir(buf.buf);
> + strbuf_reset(&buf);
> +
> + /* new (encoded name) path */
> + repo_git_path_append(r, &buf, "submodules/");
> + ret |= file_exists(buf.buf) && !is_empty_dir(buf.buf);
> strbuf_release(&buf);
> +
> return ret;
> }
>
> @@ -2273,7 +2280,7 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
> *
> * Example: having a submodule named `hippo` and another one named
> * `hippo/hooks` would result in the git directories
> - * `.git/modules/hippo/` and `.git/modules/hippo/hooks/`, respectively,
> + * `.git/submodules/hippo/` and `.git/submodules/hippo/hooks/`, respectively,
> * but the latter directory is already designated to contain the hooks
> * of the former.
> */
> @@ -2604,6 +2611,15 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> * administrators can explicitly set. Nothing has been decided,
> * so for now, just append the name at the end of the path.
> */
> +
> + /* Legacy behavior: allow existing paths under modules/<name>. */
> repo_git_path_append(r, buf, "modules/");
> strbuf_addstr(buf, submodule_name);
> + if (!access(buf->buf, F_OK))
> + return;
> +
> + /* New style (encoded) paths go under submodules/<encoded>. */
> + strbuf_reset(buf);
> + repo_git_path_append(r, buf, "submodules/");
> + strbuf_addstr(buf, submodule_name);
> }
> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
> index 36f767cb74..b6b2be1df5 100644
> --- a/t/lib-submodule-update.sh
> +++ b/t/lib-submodule-update.sh
> @@ -161,7 +161,7 @@ replace_gitfile_with_git_dir () {
> }
>
> # Test that the .git directory in the submodule is unchanged (except for the
> -# core.worktree setting, which appears only in $GIT_DIR/modules/$1/config).
> +# core.worktree setting, which appears only in $GIT_DIR/submodules/$1/config).
> # Call this function before test_submodule_content as the latter might
> # write the index file leading to false positive index differences.
> #
> @@ -170,23 +170,23 @@ replace_gitfile_with_git_dir () {
> test_git_directory_is_unchanged () {
> # does core.worktree point at the right place?
> echo "../../../$1" >expect &&
> - git -C ".git/modules/$1" config core.worktree >actual &&
> + git -C ".git/submodules/$1" config core.worktree >actual &&
> test_cmp expect actual &&
> # remove it temporarily before comparing, as
> # "$1/.git/config" lacks it...
> - git -C ".git/modules/$1" config --unset core.worktree &&
> - diff -r ".git/modules/$1" "$1/.git" &&
> + git -C ".git/submodules/$1" config --unset core.worktree &&
> + diff -r ".git/submodules/$1" "$1/.git" &&
> # ... and then restore.
> - git -C ".git/modules/$1" config core.worktree "../../../$1"
> + git -C ".git/submodules/$1" config core.worktree "../../../$1"
> }
>
> test_git_directory_exists () {
> - test -e ".git/modules/$1" &&
> + test -e ".git/submodules/$1" &&
> if test -f sub1/.git
> then
> # does core.worktree point at the right place?
> echo "../../../$1" >expect &&
> - git -C ".git/modules/$1" config core.worktree >actual &&
> + git -C ".git/submodules/$1" config core.worktree >actual &&
> test_cmp expect actual
> fi
> }
> @@ -225,22 +225,22 @@ reset_work_tree_to () {
> reset_work_tree_to_interested () {
> reset_work_tree_to $1 &&
> # make the submodule git dirs available
> - if ! test -d submodule_update/.git/modules/sub1
> + if ! test -d submodule_update/.git/submodules/sub1
> then
> - mkdir -p submodule_update/.git/modules &&
> - cp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1
> - GIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree
> + mkdir -p submodule_update/.git/submodules &&
> + cp -r submodule_update_repo/.git/submodules/sub1 submodule_update/.git/submodules/sub1
> + GIT_WORK_TREE=. git -C submodule_update/.git/submodules/sub1 config --unset core.worktree
> fi &&
> - if ! test -d submodule_update/.git/modules/sub1/modules/sub2
> + if ! test -d submodule_update/.git/submodules/sub1/submodules/sub2
> then
> - mkdir -p submodule_update/.git/modules/sub1/modules &&
> - cp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2
> + mkdir -p submodule_update/.git/submodules/sub1/submodules &&
> + cp -r submodule_update_repo/.git/submodules/sub1/submodules/sub2 submodule_update/.git/submodules/sub1/submodules/sub2
> # core.worktree is unset for sub2 as it is not checked out
> fi &&
> # indicate we are interested in the submodule:
> git -C submodule_update config submodule.sub1.url "bogus" &&
> # sub1 might not be checked out, so use the git dir
> - git -C submodule_update/.git/modules/sub1 config submodule.sub2.url "bogus"
> + git -C submodule_update/.git/submodules/sub1 config submodule.sub2.url "bogus"
> }
>
> # Test that the superproject contains the content according to commit "$1"
> @@ -742,7 +742,7 @@ test_submodule_recursing_with_args_common () {
> $command remove_sub1 &&
> test_superproject_content origin/remove_sub1 &&
> ! test -e sub1 &&
> - test_must_fail git config -f .git/modules/sub1/config core.worktree
> + test_must_fail git config -f .git/submodules/sub1/config core.worktree
> )
> '
> # ... absorbing a .git directory along the way.
> @@ -753,7 +753,7 @@ test_submodule_recursing_with_args_common () {
> cd submodule_update &&
> git branch -t remove_sub1 origin/remove_sub1 &&
> replace_gitfile_with_git_dir sub1 &&
> - rm -rf .git/modules &&
> + rm -rf .git/submodules &&
> $command remove_sub1 &&
> test_superproject_content origin/remove_sub1 &&
> ! test -e sub1 &&
> @@ -803,8 +803,8 @@ test_submodule_recursing_with_args_common () {
> $command no_submodule &&
> test_superproject_content origin/no_submodule &&
> test_path_is_missing sub1 &&
> - test_must_fail git config -f .git/modules/sub1/config core.worktree &&
> - test_must_fail git config -f .git/modules/sub1/modules/sub2/config core.worktree
> + test_must_fail git config -f .git/submodules/sub1/config core.worktree &&
> + test_must_fail git config -f .git/submodules/sub1/submodules/sub2/config core.worktree
> )
> '
>
> @@ -937,7 +937,7 @@ test_submodule_switch_recursing_with_args () {
> cd submodule_update &&
> git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> replace_gitfile_with_git_dir sub1 &&
> - rm -rf .git/modules &&
> + rm -rf .git/submodules &&
> $command replace_sub1_with_directory &&
> test_superproject_content origin/replace_sub1_with_directory &&
> test_git_directory_exists sub1
> @@ -946,15 +946,15 @@ test_submodule_switch_recursing_with_args () {
>
> # ... and ignored files are ignored
> test_expect_success "$command: replace submodule with a file works ignores ignored files in submodule" '
> - test_when_finished "rm submodule_update/.git/modules/sub1/info/exclude" &&
> + test_when_finished "rm submodule_update/.git/submodules/sub1/info/exclude" &&
> prolog &&
> reset_work_tree_to_interested add_sub1 &&
> (
> cd submodule_update &&
> - rm -rf .git/modules/sub1/info &&
> + rm -rf .git/submodules/sub1/info &&
> git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
> - mkdir .git/modules/sub1/info &&
> - echo ignored >.git/modules/sub1/info/exclude &&
> + mkdir .git/submodules/sub1/info &&
> + echo ignored >.git/submodules/sub1/info/exclude &&
> : >sub1/ignored &&
> $command replace_sub1_with_file &&
> test_superproject_content origin/replace_sub1_with_file &&
> @@ -1034,7 +1034,7 @@ test_submodule_forced_switch_recursing_with_args () {
> cd submodule_update &&
> git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> replace_gitfile_with_git_dir sub1 &&
> - rm -rf .git/modules/sub1 &&
> + rm -rf .git/submodules/sub1 &&
> $command replace_sub1_with_directory &&
> test_superproject_content origin/replace_sub1_with_directory &&
> test_git_directory_exists sub1
> diff --git a/t/t0035-safe-bare-repository.sh b/t/t0035-safe-bare-repository.sh
> index ae7ef092ab..a480ddf8d6 100755
> --- a/t/t0035-safe-bare-repository.sh
> +++ b/t/t0035-safe-bare-repository.sh
> @@ -41,7 +41,7 @@ test_expect_success 'setup an embedded bare repo, secondary worktree and submodu
> submodule add --name subn -- ./bare-repo subd
> ) &&
> test_path_is_dir outer-repo/.git/worktrees/outer-secondary &&
> - test_path_is_dir outer-repo/.git/modules/subn
> + test_path_is_dir outer-repo/.git/submodules/subn
> '
>
> test_expect_success 'safe.bareRepository unset' '
> @@ -100,7 +100,7 @@ test_expect_success 'no trace in $GIT_DIR of secondary worktree' '
> '
>
> test_expect_success 'no trace in $GIT_DIR of a submodule' '
> - expect_accepted_implicit -C outer-repo/.git/modules/subn
> + expect_accepted_implicit -C outer-repo/.git/submodules/subn
> '
>
> test_done
> diff --git a/t/t1600-index.sh b/t/t1600-index.sh
> index 03239e9faa..0e5e8efb20 100755
> --- a/t/t1600-index.sh
> +++ b/t/t1600-index.sh
> @@ -87,10 +87,10 @@ test_expect_success 'index.skipHash config option' '
> git -c protocol.file.allow=always submodule add ./ sub &&
> git config index.skipHash false &&
> git -C sub config index.skipHash true &&
> - rm -f .git/modules/sub/index &&
> + rm -f .git/submodules/sub/index &&
> >sub/file &&
> git -C sub add a &&
> - test_trailing_hash .git/modules/sub/index >hash &&
> + test_trailing_hash .git/submodules/sub/index >hash &&
> test_cmp expect hash &&
> git -C sub fsck
> '
> diff --git a/t/t2405-worktree-submodule.sh b/t/t2405-worktree-submodule.sh
> index 11018f37c7..c18c2efca5 100755
> --- a/t/t2405-worktree-submodule.sh
> +++ b/t/t2405-worktree-submodule.sh
> @@ -62,7 +62,7 @@ test_expect_success 'submodule is checked out after manually adding submodule wo
> test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules in a linked worktree' '
> git -C main worktree add "$base_path/checkout-recurse" --detach &&
> git -C checkout-recurse submodule update --init &&
> - echo "gitdir: ../../main/.git/worktrees/checkout-recurse/modules/sub" >expect-gitfile &&
> + echo "gitdir: ../../main/.git/worktrees/checkout-recurse/submodules/sub" >expect-gitfile &&
> cat checkout-recurse/sub/.git >actual-gitfile &&
> test_cmp expect-gitfile actual-gitfile &&
> git -C main/sub rev-parse HEAD >expect-head-main &&
> @@ -73,7 +73,7 @@ test_expect_success 'checkout --recurse-submodules uses $GIT_DIR for submodules
> test_cmp expect-head-main actual-head-main
> '
>
> -test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config, not in $GIT_COMMON_DIR/modules/<name>/config' '
> +test_expect_success 'core.worktree is removed in $GIT_DIR/submodules/<name>/config, not in $GIT_COMMON_DIR/submodules/<name>/config' '
> echo "../../../sub" >expect-main &&
> git -C main/sub config --get core.worktree >actual-main &&
> test_cmp expect-main actual-main &&
> @@ -81,14 +81,14 @@ test_expect_success 'core.worktree is removed in $GIT_DIR/modules/<name>/config,
> git -C checkout-recurse/sub config --get core.worktree >actual-linked &&
> test_cmp expect-linked actual-linked &&
> git -C checkout-recurse checkout --recurse-submodules first &&
> - test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/modules/sub config --get core.worktree >linked-config &&
> + test_expect_code 1 git -C main/.git/worktrees/checkout-recurse/submodules/sub config --get core.worktree >linked-config &&
> test_must_be_empty linked-config &&
> git -C main/sub config --get core.worktree >actual-main &&
> test_cmp expect-main actual-main
> '
>
> test_expect_success 'unsetting core.worktree does not prevent running commands directly against the submodule repository' '
> - git -C main/.git/worktrees/checkout-recurse/modules/sub log
> + git -C main/.git/worktrees/checkout-recurse/submodules/sub log
> '
>
> test_done
> diff --git a/t/t2501-cwd-empty.sh b/t/t2501-cwd-empty.sh
> index be9140bbaa..bb8751433f 100755
> --- a/t/t2501-cwd-empty.sh
> +++ b/t/t2501-cwd-empty.sh
> @@ -239,7 +239,7 @@ test_submodule_removal () {
> test "$path_status" = dir && test_status=test_must_fail
>
> test_when_finished "git reset --hard HEAD~1" &&
> - test_when_finished "rm -rf .git/modules/my_submodule" &&
> + test_when_finished "rm -rf .git/submodules/my_submodule" &&
>
> git checkout foo/bar/baz &&
>
> diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
> index 1f16e6b522..5b8ed57538 100755
> --- a/t/t3600-rm.sh
> +++ b/t/t3600-rm.sh
> @@ -582,7 +582,7 @@ test_expect_success 'rm of a conflicted populated submodule with a .git director
> (
> cd submod &&
> rm .git &&
> - cp -R ../.git/modules/sub .git &&
> + cp -R ../.git/submodules/sub .git &&
> GIT_WORK_TREE=. git config --unset core.worktree
> ) &&
> test_must_fail git merge conflict2 &&
> @@ -617,9 +617,9 @@ test_expect_success 'rm of a populated submodule with a .git directory migrates
> (
> cd submod &&
> rm .git &&
> - cp -R ../.git/modules/sub .git &&
> + cp -R ../.git/submodules/sub .git &&
> GIT_WORK_TREE=. git config --unset core.worktree &&
> - rm -r ../.git/modules/sub
> + rm -r ../.git/submodules/sub
> ) &&
> git rm submod 2>output.err &&
> test_path_is_missing submod &&
> @@ -709,7 +709,7 @@ test_expect_success "rm absorbs submodule's nested .git directory" '
> (
> cd submod/subsubmod &&
> rm .git &&
> - mv ../../.git/modules/sub/modules/sub .git &&
> + mv ../../.git/submodules/sub/submodules/sub .git &&
> GIT_WORK_TREE=. git config --unset core.worktree
> ) &&
> git rm submod 2>output.err &&
> diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
> index 5e566205ba..b7385bc088 100755
> --- a/t/t5526-fetch-submodules.sh
> +++ b/t/t5526-fetch-submodules.sh
> @@ -1143,7 +1143,7 @@ test_expect_success 'fetch --recurse-submodules updates name-conflicted, unpopul
> head1=$(git -C same-name-1/submodule rev-parse HEAD) &&
> head2=$(git -C same-name-2/submodule rev-parse HEAD) &&
> (
> - cd same-name-downstream/.git/modules/submodule &&
> + cd same-name-downstream/.git/submodules/submodule &&
> # The submodule has core.worktree pointing to the "git
> # rm"-ed directory, overwrite the invalid value. See
> # comment in get_fetch_task_from_changed() for more
> diff --git a/t/t5619-clone-local-ambiguous-transport.sh b/t/t5619-clone-local-ambiguous-transport.sh
> index cce62bf78d..cf2d5e7bfb 100755
> --- a/t/t5619-clone-local-ambiguous-transport.sh
> +++ b/t/t5619-clone-local-ambiguous-transport.sh
> @@ -38,7 +38,7 @@ test_expect_success 'setup' '
> ln -s "$(cd .. && pwd)/sensitive" repo/objects &&
>
> mkdir -p "$HTTPD_URL/dumb" &&
> - ln -s "../../../.git/modules/sub/../../../repo/" "$URI" &&
> + ln -s "../../../.git/submodules/sub/../../../repo/" "$URI" &&
>
> git add . &&
> git commit -m "initial commit"
> @@ -57,7 +57,7 @@ test_expect_success 'ambiguous transport does not lead to arbitrary file-inclusi
> git clone malicious clone &&
> test_must_fail git -C clone submodule update --init 2>err &&
>
> - test_path_is_missing clone/.git/modules/sub/objects/secret &&
> + test_path_is_missing clone/.git/submodules/sub/objects/secret &&
> # We would actually expect "transport .file. not allowed" here,
> # but due to quirks of the URL detection in Git, we mis-parse
> # the absolute path as a bogus URL and die before that step.
> diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
> index 256ccaefb7..56460ae8b5 100755
> --- a/t/t6120-describe.sh
> +++ b/t/t6120-describe.sh
> @@ -357,7 +357,7 @@ test_expect_success 'setup and absorb a submodule' '
> '
>
> test_expect_success 'describe chokes on severely broken submodules' '
> - mv .git/modules/sub1/ .git/modules/sub_moved &&
> + mv .git/submodules/sub1/ .git/submodules/sub_moved &&
> test_must_fail git describe --dirty
> '
>
> @@ -371,7 +371,7 @@ test_expect_success 'describe with --work-tree ignoring a broken submodule' '
> cd "$TEST_DIRECTORY" &&
> git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --broken >"$TRASH_DIRECTORY/out"
> ) &&
> - test_when_finished "mv .git/modules/sub_moved .git/modules/sub1" &&
> + test_when_finished "mv .git/submodules/sub_moved .git/submodules/sub1" &&
> grep broken out
> '
>
> diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
> index 920479e925..89b06ae3c1 100755
> --- a/t/t7001-mv.sh
> +++ b/t/t7001-mv.sh
> @@ -360,7 +360,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and no .gitm
> (
> cd sub &&
> rm -f .git &&
> - cp -R -P -p ../.git/modules/sub .git &&
> + cp -R -P -p ../.git/submodules/sub .git &&
> GIT_WORK_TREE=. git config --unset core.worktree
> ) &&
> mkdir mod &&
> @@ -380,7 +380,7 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
> (
> cd sub &&
> rm -f .git &&
> - cp -R -P -p ../.git/modules/sub .git &&
> + cp -R -P -p ../.git/submodules/sub .git &&
> GIT_WORK_TREE=. git config --unset core.worktree
> ) &&
> mkdir mod &&
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index fd3e7e355e..178c386212 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -163,7 +163,7 @@ test_expect_success 'submodule add' '
> cd addtest &&
> git submodule add -q "$submodurl" submod >actual &&
> test_must_be_empty actual &&
> - echo "gitdir: ../.git/modules/submod" >expect &&
> + echo "gitdir: ../.git/submodules/submod" >expect &&
> test_cmp expect submod/.git &&
> (
> cd submod &&
> @@ -976,21 +976,21 @@ test_expect_success 'submodule add --name allows to replace a submodule with ano
> echo "$submodurl/repo" >expect &&
> git config remote.origin.url >actual &&
> test_cmp expect actual &&
> - echo "gitdir: ../.git/modules/repo" >expect &&
> + echo "gitdir: ../.git/submodules/repo" >expect &&
> test_cmp expect .git
> ) &&
> rm -rf repo &&
> git rm repo &&
> git submodule add -q --name repo_new "$submodurl/bare.git" repo >actual &&
> test_must_be_empty actual &&
> - echo "gitdir: ../.git/modules/submod" >expect &&
> + echo "gitdir: ../.git/submodules/submod" >expect &&
> test_cmp expect submod/.git &&
> (
> cd repo &&
> echo "$submodurl/bare.git" >expect &&
> git config remote.origin.url >actual &&
> test_cmp expect actual &&
> - echo "gitdir: ../.git/modules/repo_new" >expect &&
> + echo "gitdir: ../.git/submodules/repo_new" >expect &&
> test_cmp expect .git
> ) &&
> echo "repo" >expect &&
> @@ -1045,8 +1045,8 @@ test_expect_success 'recursive relative submodules stay relative' '
> (
> cd clone2 &&
> git submodule update --init --recursive &&
> - echo "gitdir: ../.git/modules/sub3" >./sub3/.git_expect &&
> - echo "gitdir: ../../../.git/modules/sub3/modules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
> + echo "gitdir: ../.git/submodules/sub3" >./sub3/.git_expect &&
> + echo "gitdir: ../../../.git/submodules/sub3/submodules/dirdir/subsub" >./sub3/dirdir/subsub/.git_expect
> ) &&
> test_cmp clone2/sub3/.git_expect clone2/sub3/.git &&
> test_cmp clone2/sub3/dirdir/subsub/.git_expect clone2/sub3/dirdir/subsub/.git
> @@ -1108,8 +1108,8 @@ test_expect_success 'submodule deinit should remove the whole submodule section
> '
>
> test_expect_success 'submodule deinit should unset core.worktree' '
> - test_path_is_file .git/modules/example/config &&
> - test_must_fail git config -f .git/modules/example/config core.worktree
> + test_path_is_file .git/submodules/example/config &&
> + test_must_fail git config -f .git/submodules/example/config core.worktree
> '
>
> test_expect_success 'submodule deinit from subdirectory' '
> @@ -1231,7 +1231,7 @@ test_expect_success 'submodule deinit absorbs .git directory if .git is a direct
> (
> cd init &&
> rm .git &&
> - mv ../.git/modules/example .git &&
> + mv ../.git/submodules/example .git &&
> GIT_WORK_TREE=. git config --unset core.worktree
> ) &&
> git submodule deinit init &&
> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
> index 3adab12091..f0c4da1ffa 100755
> --- a/t/t7406-submodule-update.sh
> +++ b/t/t7406-submodule-update.sh
> @@ -864,7 +864,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir' '
> (cd deeper/submodule &&
> git log > ../../expected
> ) &&
> - (cd .git/modules/deeper/submodule &&
> + (cd .git/submodules/deeper/submodule &&
> git log > ../../../../actual
> ) &&
> test_cmp expected actual
> @@ -882,7 +882,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir' '
> (cd deeper/submodule &&
> git log > ../../expected
> ) &&
> - (cd .git/modules/deeper/submodule &&
> + (cd .git/submodules/deeper/submodule &&
> git log > ../../../../actual
> ) &&
> test_cmp expected actual
> @@ -899,7 +899,7 @@ test_expect_success 'submodule add places git-dir in superprojects git-dir recur
> git commit -m "added subsubmodule" &&
> git push origin :
> ) &&
> - (cd .git/modules/deeper/submodule/modules/subsubmodule &&
> + (cd .git/submodules/deeper/submodule/submodules/subsubmodule &&
> git log > ../../../../../actual
> ) &&
> git add deeper/submodule &&
> @@ -949,7 +949,7 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir re
> (cd submodule/subsubmodule &&
> git log > ../../expected
> ) &&
> - (cd .git/modules/submodule/modules/subsubmodule &&
> + (cd .git/submodules/submodule/submodules/subsubmodule &&
> git log > ../../../../../actual
> ) &&
> test_cmp expected actual
> @@ -1298,7 +1298,7 @@ test_expect_success CASE_INSENSITIVE_FS,SYMLINKS \
> git init captain &&
> (
> cd captain &&
> - git submodule add --name x/y "$hook_repo_path" A/modules/x &&
> + git submodule add --name x/y "$hook_repo_path" A/submodules/x &&
> test_tick &&
> git commit -m add-submodule &&
>
> diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
> index 77b6d0040e..75ba826968 100755
> --- a/t/t7407-submodule-foreach.sh
> +++ b/t/t7407-submodule-foreach.sh
> @@ -368,9 +368,9 @@ test_expect_success 'test "update --recursive" with a flag with spaces' '
> git rev-parse --resolve-git-dir nested1/.git &&
> git rev-parse --resolve-git-dir nested1/nested2/.git &&
> git rev-parse --resolve-git-dir nested1/nested2/nested3/.git &&
> - test -f .git/modules/nested1/objects/info/alternates &&
> - test -f .git/modules/nested1/modules/nested2/objects/info/alternates &&
> - test -f .git/modules/nested1/modules/nested2/modules/nested3/objects/info/alternates
> + test -f .git/submodules/nested1/objects/info/alternates &&
> + test -f .git/submodules/nested1/submodules/nested2/objects/info/alternates &&
> + test -f .git/submodules/nested1/submodules/nested2/submodules/nested3/objects/info/alternates
> )
> '
>
> diff --git a/t/t7408-submodule-reference.sh b/t/t7408-submodule-reference.sh
> index f860e7bbf4..25f4aec57e 100755
> --- a/t/t7408-submodule-reference.sh
> +++ b/t/t7408-submodule-reference.sh
> @@ -61,7 +61,7 @@ test_expect_success 'submodule add --reference uses alternates' '
> git commit -m B-super-added &&
> git repack -ad
> ) &&
> - test_alternate_is_used super/.git/modules/sub/objects/info/alternates super/sub
> + test_alternate_is_used super/.git/submodules/sub/objects/info/alternates super/sub
> '
>
> test_expect_success 'submodule add --reference with --dissociate does not use alternates' '
> @@ -71,7 +71,7 @@ test_expect_success 'submodule add --reference with --dissociate does not use al
> git commit -m B-super-added &&
> git repack -ad
> ) &&
> - test_path_is_missing super/.git/modules/sub-dissociate/objects/info/alternates
> + test_path_is_missing super/.git/submodules/sub-dissociate/objects/info/alternates
> '
>
> test_expect_success 'that reference gets used with add' '
> @@ -94,14 +94,14 @@ test_expect_success 'updating superproject keeps alternates' '
> test_when_finished "rm -rf super-clone" &&
> git clone super super-clone &&
> git -C super-clone submodule update --init --reference ../B &&
> - test_alternate_is_used super-clone/.git/modules/sub/objects/info/alternates super-clone/sub
> + test_alternate_is_used super-clone/.git/submodules/sub/objects/info/alternates super-clone/sub
> '
>
> test_expect_success 'updating superproject with --dissociate does not keep alternates' '
> test_when_finished "rm -rf super-clone" &&
> git clone super super-clone &&
> git -C super-clone submodule update --init --reference ../B --dissociate &&
> - test_path_is_missing super-clone/.git/modules/sub/objects/info/alternates
> + test_path_is_missing super-clone/.git/submodules/sub/objects/info/alternates
> '
>
> test_expect_success 'submodules use alternates when cloning a superproject' '
> @@ -112,7 +112,7 @@ test_expect_success 'submodules use alternates when cloning a superproject' '
> # test superproject has alternates setup correctly
> test_alternate_is_used .git/objects/info/alternates . &&
> # test submodule has correct setup
> - test_alternate_is_used .git/modules/sub/objects/info/alternates sub
> + test_alternate_is_used .git/submodules/sub/objects/info/alternates sub
> )
> '
>
> @@ -127,7 +127,7 @@ test_expect_success 'missing submodule alternate fails clone and submodule updat
> # update of the submodule succeeds
> test_must_fail git submodule update --init &&
> # and we have no alternates:
> - test_path_is_missing .git/modules/sub/objects/info/alternates &&
> + test_path_is_missing .git/submodules/sub/objects/info/alternates &&
> test_path_is_missing sub/file1
> )
> '
> @@ -142,7 +142,7 @@ test_expect_success 'ignoring missing submodule alternates passes clone and subm
> # update of the submodule succeeds
> git submodule update --init &&
> # and we have no alternates:
> - test_path_is_missing .git/modules/sub/objects/info/alternates &&
> + test_path_is_missing .git/submodules/sub/objects/info/alternates &&
> test_path_is_file sub/file1
> )
> '
> @@ -176,18 +176,18 @@ test_expect_success 'nested submodule alternate in works and is actually used' '
> # test superproject has alternates setup correctly
> test_alternate_is_used .git/objects/info/alternates . &&
> # immediate submodule has alternate:
> - test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
> + test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
> # nested submodule also has alternate:
> - test_alternate_is_used .git/modules/subwithsub/modules/sub/objects/info/alternates subwithsub/sub
> + test_alternate_is_used .git/submodules/subwithsub/submodules/sub/objects/info/alternates subwithsub/sub
> )
> '
>
> check_that_two_of_three_alternates_are_used() {
> test_alternate_is_used .git/objects/info/alternates . &&
> # immediate submodule has alternate:
> - test_alternate_is_used .git/modules/subwithsub/objects/info/alternates subwithsub &&
> + test_alternate_is_used .git/submodules/subwithsub/objects/info/alternates subwithsub &&
> # but nested submodule has no alternate:
> - test_path_is_missing .git/modules/subwithsub/modules/sub/objects/info/alternates
> + test_path_is_missing .git/submodules/subwithsub/submodules/sub/objects/info/alternates
> }
>
>
> diff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh
> index 0490499573..dbaca9c69f 100755
> --- a/t/t7412-submodule-absorbgitdirs.sh
> +++ b/t/t7412-submodule-absorbgitdirs.sh
> @@ -29,13 +29,13 @@ test_expect_success 'absorb the git dir' '
> cat >expect <<-EOF &&
> Migrating git directory of '\''sub1'\'' from
> '\''$cwd/sub1/.git'\'' to
> - '\''$cwd/.git/modules/sub1'\''
> + '\''$cwd/.git/submodules/sub1'\''
> EOF
> git submodule absorbgitdirs 2>actual &&
> test_cmp expect actual &&
> git fsck &&
> test -f sub1/.git &&
> - test -d .git/modules/sub1 &&
> + test -d .git/submodules/sub1 &&
> git status >actual.1 &&
> git -C sub1 rev-parse HEAD >actual.2 &&
> test_cmp expect.1 actual.1 &&
> @@ -47,7 +47,7 @@ test_expect_success 'absorbing does not fail for deinitialized submodules' '
> git submodule deinit --all &&
> git submodule absorbgitdirs 2>err &&
> test_must_be_empty err &&
> - test -d .git/modules/sub1 &&
> + test -d .git/submodules/sub1 &&
> test -d sub1 &&
> ! test -e sub1/.git
> '
> @@ -68,12 +68,12 @@ test_expect_success 'absorb the git dir in a nested submodule' '
> cat >expect <<-EOF &&
> Migrating git directory of '\''sub1/nested'\'' from
> '\''$cwd/sub1/nested/.git'\'' to
> - '\''$cwd/.git/modules/sub1/modules/nested'\''
> + '\''$cwd/.git/submodules/sub1/submodules/nested'\''
> EOF
> git submodule absorbgitdirs 2>actual &&
> test_cmp expect actual &&
> test -f sub1/nested/.git &&
> - test -d .git/modules/sub1/modules/nested &&
> + test -d .git/submodules/sub1/submodules/nested &&
> git status >actual.1 &&
> git -C sub1/nested rev-parse HEAD >actual.2 &&
> test_cmp expect.1 actual.1 &&
> @@ -84,11 +84,11 @@ test_expect_success 're-setup nested submodule' '
> # un-absorb the direct submodule, to test if the nested submodule
> # is still correct (needs a rewrite of the gitfile only)
> rm -rf sub1/.git &&
> - mv .git/modules/sub1 sub1/.git &&
> + mv .git/submodules/sub1 sub1/.git &&
> GIT_WORK_TREE=. git -C sub1 config --unset core.worktree &&
> # fixup the nested submodule
> - echo "gitdir: ../.git/modules/nested" >sub1/nested/.git &&
> - GIT_WORK_TREE=../../../nested git -C sub1/.git/modules/nested config \
> + echo "gitdir: ../.git/submodules/nested" >sub1/nested/.git &&
> + GIT_WORK_TREE=../../../nested git -C sub1/.git/submodules/nested config \
> core.worktree "../../../nested" &&
> # make sure this re-setup is correct
> git status --ignore-submodules=none &&
> @@ -105,13 +105,13 @@ test_expect_success 'absorb the git dir in a nested submodule' '
> cat >expect <<-EOF &&
> Migrating git directory of '\''sub1'\'' from
> '\''$cwd/sub1/.git'\'' to
> - '\''$cwd/.git/modules/sub1'\''
> + '\''$cwd/.git/submodules/sub1'\''
> EOF
> git submodule absorbgitdirs 2>actual &&
> test_cmp expect actual &&
> test -f sub1/.git &&
> test -f sub1/nested/.git &&
> - test -d .git/modules/sub1/modules/nested &&
> + test -d .git/submodules/sub1/submodules/nested &&
> git status >actual.1 &&
> git -C sub1/nested rev-parse HEAD >actual.2 &&
> test_cmp expect.1 actual.1 &&
> @@ -133,7 +133,7 @@ test_expect_success 'absorb the git dir outside of primary worktree' '
> cat >expect <<-EOF &&
> Migrating git directory of '\''sub2'\'' from
> '\''$cwd/repo-wt/sub2/.git'\'' to
> - '\''$cwd/repo-bare.git/worktrees/repo-wt/modules/sub2'\''
> + '\''$cwd/repo-bare.git/worktrees/repo-wt/submodules/sub2'\''
> EOF
> git -C repo-wt submodule absorbgitdirs 2>actual &&
> test_cmp expect actual
> diff --git a/t/t7423-submodule-symlinks.sh b/t/t7423-submodule-symlinks.sh
> index 3d3c7af3ce..a51235136d 100755
> --- a/t/t7423-submodule-symlinks.sh
> +++ b/t/t7423-submodule-symlinks.sh
> @@ -49,19 +49,19 @@ test_expect_success SYMLINKS 'git restore --recurse-submodules must not be confu
>
> test_expect_success SYMLINKS 'git restore --recurse-submodules must not migrate git dir of symlinked repo' '
> prepare_symlink_to_repo &&
> - rm -rf .git/modules &&
> + rm -rf .git/submodules &&
> test_must_fail git restore --recurse-submodules a/sm &&
> test_path_is_dir a/target/.git &&
> - test_path_is_missing .git/modules/a/sm &&
> + test_path_is_missing .git/submodules/a/sm &&
> test_path_is_missing a/target/submodule_file
> '
>
> test_expect_success SYMLINKS 'git checkout -f --recurse-submodules must not migrate git dir of symlinked repo when removing submodule' '
> prepare_symlink_to_repo &&
> - rm -rf .git/modules &&
> + rm -rf .git/submodules &&
> test_must_fail git checkout -f --recurse-submodules initial &&
> test_path_is_dir a/target/.git &&
> - test_path_is_missing .git/modules/a/sm
> + test_path_is_missing .git/submodules/a/sm
> '
>
> test_done
> diff --git a/t/t7450-bad-git-dotfiles.sh b/t/t7450-bad-git-dotfiles.sh
> index f512eed278..4e2ced3636 100755
> --- a/t/t7450-bad-git-dotfiles.sh
> +++ b/t/t7450-bad-git-dotfiles.sh
> @@ -77,28 +77,28 @@ test_expect_success 'create innocent subrepo' '
>
> test_expect_success 'submodule add refuses invalid names' '
> test_must_fail \
> - git submodule add --name ../../modules/evil "$PWD/innocent" evil
> + git submodule add --name ../../submodules/evil "$PWD/innocent" evil
> '
>
> test_expect_success 'add evil submodule' '
> git submodule add "$PWD/innocent" evil &&
>
> - mkdir modules &&
> - cp -r .git/modules/evil modules &&
> - write_script modules/evil/hooks/post-checkout <<-\EOF &&
> + mkdir submodules &&
> + cp -r .git/submodules/evil submodules &&
> + write_script submodules/evil/hooks/post-checkout <<-\EOF &&
> echo >&2 "RUNNING POST CHECKOUT"
> EOF
>
> git config -f .gitmodules submodule.evil.update checkout &&
> git config -f .gitmodules --rename-section \
> - submodule.evil submodule.../../modules/evil &&
> - git add modules &&
> + submodule.evil submodule.../../submodules/evil &&
> + git add submodules &&
> git commit -am evil
> '
>
> # This step seems like it shouldn't be necessary, since the payload is
> # contained entirely in the evil submodule. But due to the vagaries of the
> -# submodule code, checking out the evil module will fail unless ".git/modules"
> +# submodule code, checking out the evil module will fail unless ".git/submodules"
> # exists. Adding another submodule (with a name that sorts before "evil") is an
> # easy way to make sure this is the case in the victim clone.
> test_expect_success 'add other submodule' '
> @@ -350,8 +350,8 @@ test_expect_success 'submodule git dir nesting detection must work with parallel
> cat err &&
> grep -E "(already exists|is inside git dir|not a git repository)" err &&
> {
> - test_path_is_missing .git/modules/hippo/HEAD ||
> - test_path_is_missing .git/modules/hippo/hooks/HEAD
> + test_path_is_missing .git/submodules/hippo/HEAD ||
> + test_path_is_missing .git/submodules/hippo/hooks/HEAD
> }
> '
>
> @@ -361,10 +361,10 @@ test_expect_success 'checkout -f --recurse-submodules must not use a nested gitd
> cd nested_checkout &&
> git submodule init &&
> git submodule update thing1 &&
> - mkdir -p .git/modules/hippo/hooks/refs &&
> - mkdir -p .git/modules/hippo/hooks/objects/info &&
> - echo "../../../../objects" >.git/modules/hippo/hooks/objects/info/alternates &&
> - echo "ref: refs/heads/master" >.git/modules/hippo/hooks/HEAD
> + mkdir -p .git/submodules/hippo/hooks/refs &&
> + mkdir -p .git/submodules/hippo/hooks/objects/info &&
> + echo "../../../../objects" >.git/submodules/hippo/hooks/objects/info/alternates &&
> + echo "ref: refs/heads/master" >.git/submodules/hippo/hooks/HEAD
> ) &&
> test_must_fail git -C nested_checkout checkout -f --recurse-submodules HEAD 2>err &&
> cat err &&
> @@ -390,13 +390,13 @@ test_expect_success SYMLINKS,!WINDOWS,!MINGW 'submodule must not checkout into d
> git config unset -f repo/.gitmodules submodule.sub.path &&
> printf "\tpath = \"sub\r\"\n" >>repo/.gitmodules &&
>
> - git config unset -f repo/.git/modules/sub/config core.worktree &&
> + git config unset -f repo/.git/submodules/sub/config core.worktree &&
> {
> printf "[core]\n" &&
> printf "\tworktree = \"../../../sub\r\"\n"
> - } >>repo/.git/modules/sub/config &&
> + } >>repo/.git/submodules/sub/config &&
>
> - ln -s .git/modules/sub/hooks repo/sub &&
> + ln -s .git/submodules/sub/hooks repo/sub &&
> git -C repo add -A &&
> git -C repo commit -m submodule &&
>
> diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh
> index 409cd0cd12..ded482fdf2 100755
> --- a/t/t7527-builtin-fsmonitor.sh
> +++ b/t/t7527-builtin-fsmonitor.sh
> @@ -866,7 +866,7 @@ test_expect_success 'submodule always visited' '
> '
>
> # If a submodule has a `sub/.git/` directory (rather than a file
> -# pointing to the super's `.git/modules/sub`) and `core.fsmonitor`
> +# pointing to the super's `.git/submodules/sub`) and `core.fsmonitor`
> # turned on in the submodule and the daemon is not yet started in
> # the submodule, and someone does a `git submodule absorbgitdirs`
> # in the super, Git will recursively invoke `git submodule--helper`
> @@ -895,7 +895,7 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" '
> cat >expect <<-EOF &&
> Migrating git directory of '\''dir_1/dir_2/sub'\'' from
> '\''$cwd/dir_1/dir_2/sub/.git'\'' to
> - '\''$cwd/.git/modules/dir_1/dir_2/sub'\''
> + '\''$cwd/.git/submodules/dir_1/dir_2/sub'\''
> EOF
> GIT_TRACE2_EVENT="$PWD/super-sub.trace" \
> git -C super submodule absorbgitdirs >out 2>actual &&
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 2/9] submodule: create new gitdirs under submodules path
2025-09-08 14:24 ` Phillip Wood
@ 2025-09-08 15:46 ` Adrian Ratiu
2025-09-09 8:53 ` Phillip Wood
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 15:46 UTC (permalink / raw)
To: phillip.wood, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt
On Mon, 08 Sep 2025, Phillip Wood <phillip.wood123@gmail.com>
wrote:
> Hi Adrian
>
Hello Phillip and thanks for the feedback! :)
I just sent v2 at the same time if you want to give that a look,
though the issues you raised are still valid for v2 as well.
> On 16/08/2025 22:36, Adrian Ratiu wrote:
>> This is in preparation for encoding the submodule names to
>> avoid conflicts like submodules named foo and foo/bar together
>> with case-insensitive file- system handling and other corner
>> cases like reserved filenames on Windows. Backward
>> compatibility is kept with plain-name modules already existing
>> at paths like .git/modules/<name>, however a clear separation
>> between legacy (plain) and new (encoded) namespaces is
>> desirable, to avoid situations like an existing plain-name
>> module containing the encoding escape character/ Thus we split
>> the new-style (encoded) gitdir name paths to .git/submodules,
>> while legacy-style paths remain under .git/modules. This is
>> just a default directory change with the accompanying test
>> updates, in preparation for the actual encoding additions in
>> future commits.
>
> Does this need an extentions.submoduleEncoding (name suggestions
> welcome) config key to stop older versions of git trying to read
> the repository as they wont be able to locate the gitdir of any
> submodules added under .git/submodules?
Very good point. I'm a bit unsure we actually need it, likely we
do.
On the one hand, older versions of git can still initialize and
work on submodules under the legacy .git/modules/ path ignoring
the new one...
On the other hand, there is a non-zero risk users will get in
trouble by switching git versions or can lead to
inconsistent/corrupted states, so I'm inclined to say the answer
is yes: better safe than sorry.
So if there are no objections or better ideas, I'll add this in v3.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 07/10] submodule: error out if gitdir name is too long
2025-09-08 14:01 ` [PATCH v2 07/10] submodule: error out if gitdir name is too long Adrian Ratiu
@ 2025-09-08 15:51 ` Jeff King
2025-09-08 17:15 ` Adrian Ratiu
2025-09-30 13:35 ` Kristoffer Haugsbakk
1 sibling, 1 reply; 179+ messages in thread
From: Jeff King @ 2025-09-08 15:51 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Junio C Hamano,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble
On Mon, Sep 08, 2025 at 05:01:14PM +0300, Adrian Ratiu wrote:
> Encoding submodule names increases their name size, so there is an
> increased risk to hit the max filename length in the gitdir path.
> (the likelihood is still rather small, so it's an acceptable risk)
>
> This gitdir file-name-too-long corner case can be be addressed in
> multiple ways, including sharding or trimming, however for now, just
> add the portable logic (suggested by Peff) to detect the corner case
> then error out to avoid comitting to a specific policy (or policies).
Thanks, the compat logic here looks reasonable to me.
As somebody who has not really been looking into or thought about the
topic at all, though, I wondered how necessary pathconf() is here. That
is, I can imagine two alternatives:
- just try to use the path, and we either get an error from
open()/mkdir() or we don't. This would end up with roughly the same
outcome as the current code which calls die(), though it would not
help with eventually fulfilling your TODO.
- set some arbitrary but sane limit (say, 255?). That would make the
behavior consistent across platforms, though it does mean you might
be prevented from using very long submodule names on systems that
could support it.
I dunno. Like I said, this is not a problem I thought a lot about, so
feel free to ignore. Mostly I just notice that we have lived for 20+
years without pathconf, I think mostly by following the philosophy of
the first bullet point above.
-Peff
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 07/10] submodule: error out if gitdir name is too long
2025-09-08 15:51 ` Jeff King
@ 2025-09-08 17:15 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-08 17:15 UTC (permalink / raw)
To: Jeff King
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Junio C Hamano,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble
On Mon, 08 Sep 2025, Jeff King <peff@peff.net> wrote:
> On Mon, Sep 08, 2025 at 05:01:14PM +0300, Adrian Ratiu wrote:
>
>> Encoding submodule names increases their name size, so there is
>> an increased risk to hit the max filename length in the gitdir
>> path. (the likelihood is still rather small, so it's an
>> acceptable risk) This gitdir file-name-too-long corner case
>> can be be addressed in multiple ways, including sharding or
>> trimming, however for now, just add the portable logic
>> (suggested by Peff) to detect the corner case then error out to
>> avoid comitting to a specific policy (or policies).
>
> Thanks, the compat logic here looks reasonable to me.
>
> As somebody who has not really been looking into or thought
> about the topic at all, though, I wondered how necessary
> pathconf() is here. That is, I can imagine two alternatives:
>
> - just try to use the path, and we either get an error from
> open()/mkdir() or we don't. This would end up with roughly
> the same outcome as the current code which calls die(),
> though it would not help with eventually fulfilling your
> TODO.
>
> - set some arbitrary but sane limit (say, 255?). That would
> make the
> behavior consistent across platforms, though it does mean you
> might be prevented from using very long submodule names on
> systems that could support it.
>
> I dunno. Like I said, this is not a problem I thought a lot
> about, so feel free to ignore. Mostly I just notice that we have
> lived for 20+ years without pathconf, I think mostly by
> following the philosophy of the first bullet point above.
>
> -Peff
I can go either (thrice?) way with this. :) No strong opinion.
1. Just let the OS/filesystem default do its own thing: not doing
anything is the easiest and perhaps the best approach sometimes.
2. Benefit of hardcoding: we already hardcode NAME_MAX in
compat/posix.h for platforms which don't have it and fallback to
it. Might as well use it and print a nice error.
3. The compat layer allows us to at least try and detect the
corner-case because we slightly increase the risk of hitting it
(say you have a module name with length >200, encoding it might
push beyond the limit).
Right now it's worth it mostly for the error msg telling the user
why the command fails because a default -ENAMETOOLONG might be
confusing (the plain-text name could be just 200 < NAME_MAX),
while also opening the door to avoid this situation entirely by
addressing the TODO.
Again, I'm fine either way.
Since I split this logic into a separate commit, we could even
drop it now and bring it back later when (if?) we decide to tackle
the TODO (series is big enough already).
Thanks,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 02/10] submodule: create new gitdirs under submodules path
2025-09-08 14:01 ` [PATCH v2 02/10] submodule: create new gitdirs under submodules path Adrian Ratiu
@ 2025-09-09 7:40 ` Patrick Steinhardt
2025-09-09 16:17 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-09-09 7:40 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble
On Mon, Sep 08, 2025 at 05:01:09PM +0300, Adrian Ratiu wrote:
> This is in preparation for encoding the submodule names to avoid conflicts
> like submodules named foo and foo/bar together with case-insensitive file-
> system handling and other corner cases like reserved filenames on Windows.
>
> Backward compatibility is kept with plain-name modules already existing at
> paths like .git/modules/<name>, however a clear separation between legacy
> (plain) and new (encoded) namespaces is desirable, to avoid situations like
> an existing plain-name module containing the encoding escape character/
>
> Thus we split the new-style (encoded) gitdir name paths to .git/submodules,
> while legacy-style paths remain under .git/modules.
>
> This is just a default directory change with the accompanying test updates,
> in preparation for the actual encoding additions in future commits.
One of the questions here is how this move will affect alternate
implementations of Git, like libgit2, JGit or Gitoxide. There's two
angles to this:
- Git needs to handle that those implementations continue to write
submodules into ".git/modules".
- These implementations need to be able to handle the new-style paths.
The first item should work just fine, as we make sure that we handle
both paths. But do the other implementations need any adjustment? I
guess the answer is "yes", so we need to treat this as a backwards
incompatible change as they wouldn't be able to find the submodule
repositories anymore, right?
Ideally, the way that submodules were populated was less fragile. For
example, we could have a "submodule.*.repoPath" config key that gets
populated whenever we clone a submodule. If Git clients knew to use that
field they wouldn't have to second-guess where a previous Git client
stored a specific submodule, but they could just read that path and then
use whatever is stored therein. This would even allow for changes like
using a hash to encode the submodule name.
But to the best of my knowledge such a key does not currently exist,
which is too bad (please correct me if I'm wrong, I'm definitely not an
expert when it comes to submodules).
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 03/10] submodule: add gitdir path config override
2025-09-08 14:01 ` [PATCH v2 03/10] submodule: add gitdir path config override Adrian Ratiu
@ 2025-09-09 7:40 ` Patrick Steinhardt
2025-09-09 17:46 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-09-09 7:40 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Brandon Williams
On Mon, Sep 08, 2025 at 05:01:10PM +0300, Adrian Ratiu wrote:
> This adds an ability to override gitdir paths via config files
> (not .gitmodules), such that any encoding scheme can be changed
> and JGit & co don't need to exactly match the default encoding.
>
> A new test and a helper are added. The helper will be used by
> further tests exercising gitdir paths & encodings.
Aha, so you already do what I'm lamenting about in the preceding commit.
It still raises the question around how to handle this whole migration
now, as alternative implementations don't yet know about this specific
config key. Doing all of this in a single patch series will most likely
result in breakage.
In theory, we should probably first introduce the configuration, then
wait a couple releases for alternative implementations to catch up, and
then switch over. But we still cannot be sure that implementations know
to handle this key alright.
A heavy-handed solution to this would be to introduce a repository
extension. This would ensure that any well-behaved Git client will
refuse to open the repository if it doesn't know about that specific
extension. With the remaining ones we can be sure that they know to
handle the "submodule.*.gitdir" configuration.
As mentioned, extensions are rather heavy-handed. Furthermore, this
whole infra is only really needed under very specific circumstances. So
maybe it could be a viable approach to make this extension opt-in:
- We provide a new configuration "init.useSubmoduleGitdirExtension"
that tells git-init(1) and git-clone(1) to use the new extension for
newly initialized repositories.
- We detect the situation where a submodule cannot be cloned due to a
path conflict. If detected, we print a user-facing hint that tells
them to enable the extension.
Once the user enabled the extension we'll know to use new, encoded
submodule paths from thereon and thus re-initializing the conflicting
submodule should work now.
With all of this I wonder whether we even need a new ".git/submodules"
directory. Couldn't we just continue to create submodules in the old
path and for example create a random suffix as required?
In any case, please stop me if I'm going overboard with my backwards
compatibility concerns :)
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 2/9] submodule: create new gitdirs under submodules path
2025-09-08 15:46 ` Adrian Ratiu
@ 2025-09-09 8:53 ` Phillip Wood
2025-09-09 10:57 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Phillip Wood @ 2025-09-09 8:53 UTC (permalink / raw)
To: Adrian Ratiu, phillip.wood, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
Hi Adrian
On 08/09/2025 16:46, Adrian Ratiu wrote:
> On Mon, 08 Sep 2025, Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Does this need an extentions.submoduleEncoding (name suggestions
>> welcome) config key to stop older versions of git trying to read the
>> repository as they wont be able to locate the gitdir of any
>> submodules added under .git/submodules?
>
> Very good point. I'm a bit unsure we actually need it, likely we do.
>
> On the one hand, older versions of git can still initialize and work on
> submodules under the legacy .git/modules/ path ignoring the new one...
>
> On the other hand, there is a non-zero risk users will get in trouble by
> switching git versions or can lead to inconsistent/corrupted states, so
> I'm inclined to say the answer is yes: better safe than sorry.
If we only needed to convert the submodule name to a gitdir when the
submodule was initialized and all other access went through the .git
file of the submodule in the working tree then I think old clients would
be fine because they'd find the right gitdir by reading the .git file.
I'm not familiar with the submodule code but I don't think that's the
case in which case I agree it would be safer to add an "extestions"
config key.
Thanks
Phillip
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 2/9] submodule: create new gitdirs under submodules path
2025-09-09 8:53 ` Phillip Wood
@ 2025-09-09 10:57 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-09 10:57 UTC (permalink / raw)
To: Phillip Wood, phillip.wood, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt
On Tue, 09 Sep 2025, Phillip Wood <phillip.wood123@gmail.com>
wrote:
> Hi Adrian
>
> On 08/09/2025 16:46, Adrian Ratiu wrote:
>> On Mon, 08 Sep 2025, Phillip Wood <phillip.wood123@gmail.com>
>> wrote:
>>>
>>> Does this need an extentions.submoduleEncoding (name
>>> suggestions welcome) config key to stop older versions of git
>>> trying to read the repository as they wont be able to locate
>>> the gitdir of any submodules added under .git/submodules?
>> Very good point. I'm a bit unsure we actually need it, likely
>> we do. On the one hand, older versions of git can still
>> initialize and work on submodules under the legacy
>> .git/modules/ path ignoring the new one... On the other hand,
>> there is a non-zero risk users will get in trouble by
>> switching git versions or can lead to inconsistent/corrupted
>> states, so I'm inclined to say the answer is yes: better safe
>> than sorry.
>
> If we only needed to convert the submodule name to a gitdir when
> the submodule was initialized and all other access went through
> the .git file of the submodule in the working tree then I think
> old clients would be fine because they'd find the right gitdir
> by reading the .git file. I'm not familiar with the submodule
> code but I don't think that's the case in which case I agree it
> would be safer to add an "extestions" config key.
Yes, your understanding is correct and it goes beyond just the git core
(where we at least have a unified API to compute the gitdir path, so
making the subsequent accesses follow .git file contents would be easy),
it also affects JGit, libgit2 and other implementations, so it's the
most prudent approach to put this behind an extension key.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH 3/9] submodule: add gitdir path config override
2025-09-08 14:23 ` Phillip Wood
@ 2025-09-09 12:02 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-09 12:02 UTC (permalink / raw)
To: phillip.wood, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Stefan Beller,
Patrick Steinhardt, Brandon Williams
On Mon, 08 Sep 2025, Phillip Wood <phillip.wood123@gmail.com>
wrote:
> Hi Adrian
>
> On 16/08/2025 22:36, Adrian Ratiu wrote:
>> This adds an ability to override gitdir paths via config files
>> (not .gitmodules), such that any encoding scheme can be changed
>> and JGit & co don't need to exactly match the default encoding.
>
> Reading the old email thread you linked to in the cover letter,
> my understanding is that this was suggested as an alternative
> to changing the gitdir for submodules from "modules" to
> "submodules". Do the later patches set this key when encoding
> the gitdir so that JGit can find the gitdir by reading the
> config?
I read it more like a complementary / good-to-have feature than a
requirement for getting JGit & co to work or as a path to keep
"legacy" and "new/encoded" modules in a unified dir/namespace. :)
The key is not set automatically, which also avoids unnecessary
config clutter and is mostly useful to change the encoding and
simplify tests.
I see your point though: it's not a necessary requirement for the
git core encoding itself, both due to the separate dir/namespace
and because we'll add an extension in v3, so the other
implementations will have to explicitely add support for it.
Will reword the commit msg to make it more clear in v3. Let me
know if you have more thoughts or any concerns.
Thanks,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 02/10] submodule: create new gitdirs under submodules path
2025-09-09 7:40 ` Patrick Steinhardt
@ 2025-09-09 16:17 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-09 16:17 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble
Hi Patrick and thank you for the feedback! o/
On Tue, 09 Sep 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Mon, Sep 08, 2025 at 05:01:09PM +0300, Adrian Ratiu wrote:
>> This is in preparation for encoding the submodule names to
>> avoid conflicts like submodules named foo and foo/bar together
>> with case-insensitive file- system handling and other corner
>> cases like reserved filenames on Windows. Backward
>> compatibility is kept with plain-name modules already existing
>> at paths like .git/modules/<name>, however a clear separation
>> between legacy (plain) and new (encoded) namespaces is
>> desirable, to avoid situations like an existing plain-name
>> module containing the encoding escape character/ Thus we split
>> the new-style (encoded) gitdir name paths to .git/submodules,
>> while legacy-style paths remain under .git/modules. This is
>> just a default directory change with the accompanying test
>> updates, in preparation for the actual encoding additions in
>> future commits.
>
> One of the questions here is how this move will affect alternate
> implementations of Git, like libgit2, JGit or Gitoxide. There's
> two angles to this:
>
> - Git needs to handle that those implementations continue to
> write
> submodules into ".git/modules".
>
> - These implementations need to be able to handle the
> new-style paths.
>
> The first item should work just fine, as we make sure that we
> handle both paths. But do the other implementations need any
> adjustment? I guess the answer is "yes", so we need to treat
> this as a backwards incompatible change as they wouldn't be able
> to find the submodule repositories anymore, right?
That is correct and also applies to older versions git itself
which do not have this mechanism. Phillip Wood suggested we add an
extension like "extensions.submoduleEncoding" (name suggestions
welcome).
I'll do that in v3 of this series.
>
> Ideally, the way that submodules were populated was less
> fragile. For example, we could have a "submodule.*.repoPath"
> config key that gets populated whenever we clone a submodule. If
> Git clients knew to use that field they wouldn't have to
> second-guess where a previous Git client stored a specific
> submodule, but they could just read that path and then use
> whatever is stored therein. This would even allow for changes
> like using a hash to encode the submodule name.
Slight tangent (I'll respond to your point after this):
Junio asked to please keep the name human-readable and that's why
we use url-encoding which is also widely known and well
understood.
I guess we could add a config to change the name encoding or
hashing mechanism while keeping url-encoding as the
default. Likely in a later series because this one is big enough
now at 10 patches and keeps growing.
One of my Collabora collegues even suggested they would like to
use a pattern like "hash_name" to get the best of both worlds.
>
> But to the best of my knowledge such a key does not currently
> exist, which is too bad (please correct me if I'm wrong, I'm
> definitely not an expert when it comes to submodules).
No, it does not exist. I've added something a little bit similar
with the gitdir path config option in this series, however it is
only used to override default paths computed by git-submodule,
when necessary.
There is also a config clutter problem, if such a key were to be
added by default, since most submodules use default paths.
Phillip had the idea to only compute the path once, during the
initial submodule clone, then reuse it from the .git file inside
the submodule workdir in later actions, however that is not enough
for compatibility with other implementations or older versions.
So yes, to avoid user confusion, multi-implementation
inter-operability problems or risk any repo inconsistency, I'll
make it a breaking change.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 03/10] submodule: add gitdir path config override
2025-09-09 7:40 ` Patrick Steinhardt
@ 2025-09-09 17:46 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-09 17:46 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Brandon Williams
On Tue, 09 Sep 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Mon, Sep 08, 2025 at 05:01:10PM +0300, Adrian Ratiu wrote:
>> This adds an ability to override gitdir paths via config files
>> (not .gitmodules), such that any encoding scheme can be changed
>> and JGit & co don't need to exactly match the default encoding.
>> A new test and a helper are added. The helper will be used by
>> further tests exercising gitdir paths & encodings.
>
> Aha, so you already do what I'm lamenting about in the preceding
> commit.
Not quite. I tried to explain in my previous reply that this only
looks similar and perhaps I should have worded the commit message
differently (I promised Phillip I'll reword it), however it's
intended just to allow simple overrides, without being a unified
way for all implementations to work with gitdirs. :)
> It still raises the question around how to handle this whole
> migration now, as alternative implementations don't yet know
> about this specific config key. Doing all of this in a single
> patch series will most likely result in breakage.
>
> In theory, we should probably first introduce the configuration,
> then wait a couple releases for alternative implementations to
> catch up, and then switch over. But we still cannot be sure that
> implementations know to handle this key alright.
Exactly, they won't. :)
>
> A heavy-handed solution to this would be to introduce a
> repository extension. This would ensure that any well-behaved
> Git client will refuse to open the repository if it doesn't know
> about that specific extension. With the remaining ones we can be
> sure that they know to handle the "submodule.*.gitdir"
> configuration.
>
> As mentioned, extensions are rather heavy-handed. Furthermore,
> this whole infra is only really needed under very specific
> circumstances. So maybe it could be a viable approach to make
> this extension opt-in:
>
> - We provide a new configuration
> "init.useSubmoduleGitdirExtension"
> that tells git-init(1) and git-clone(1) to use the new
> extension for newly initialized repositories.
>
> - We detect the situation where a submodule cannot be cloned
> due to a
> path conflict. If detected, we print a user-facing hint that
> tells them to enable the extension.
This is actually a very good idea, to detect path conflicts and
print a nice message to instruct the user to enable the extension,
instead of the default "error: git dir is inside git dir" or
somesuch confusing msg.
I'll work on it for patch v3 and add some tests.
>
> Once the user enabled the extension we'll know to use new,
> encoded submodule paths from thereon and thus re-initializing
> the conflicting submodule should work now.
>
> With all of this I wonder whether we even need a new
> ".git/submodules" directory. Couldn't we just continue to create
> submodules in the old path and for example create a random
> suffix as required?
Now that we decided to put this logic behind an extension, we
might get away with having a unified directory, yes. It's
something I'll explore for v3. I would really like to avoid having
both "modules" and "submodules" alongside eachother.
>
> In any case, please stop me if I'm going overboard with my
> backwards compatibility concerns :)
You have good ideas and your feedback is much appreciated. Kudos and
thanks!
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-09-10 18:15 ` SZEDER Gábor
2025-09-10 19:30 ` Adrian Ratiu
2025-09-10 20:18 ` Kristoffer Haugsbakk
2025-09-30 13:36 ` Kristoffer Haugsbakk
2 siblings, 1 reply; 179+ messages in thread
From: SZEDER Gábor @ 2025-09-10 18:15 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Brandon Williams
On Mon, Sep 08, 2025 at 05:01:13PM +0300, Adrian Ratiu wrote:
> diff --git a/submodule.c b/submodule.c
> index bf78636195..8e0fd077db 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2588,30 +2593,26 @@ int submodule_to_gitdir(struct repository *repo,
> return ret;
> }
>
> +static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
> +{
> + for (; *src; src++) {
> + unsigned char c = *src;
> + if (c >= 'A' && c <= 'Z') {
> + strbuf_addch(dst, '_');
> + strbuf_addch(dst, c - 'A' + 'a');
> + } else {
> + strbuf_addch(dst, c);
> + }
> + }
> +}
> +
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name)
> {
> - /*
> - * NEEDSWORK: The current way of mapping a submodule's name to
> - * its location in .git/modules/ has problems with some naming
> - * schemes. For example, if a submodule is named "foo" and
> - * another is named "foo/bar" (whether present in the same
> - * superproject commit or not - the problem will arise if both
> - * superproject commits have been checked out at any point in
> - * time), or if two submodule names only have different cases in
> - * a case-insensitive filesystem.
> - *
> - * There are several solutions, including encoding the path in
> - * some way, introducing a submodule.<name>.gitdir config in
> - * .git/config (not .gitmodules) that allows overriding what the
> - * gitdir of a submodule would be (and teach Git, upon noticing
> - * a clash, to automatically determine a non-clashing name and
> - * to write such a config), or introducing a
> - * submodule.<name>.gitdir config in .gitmodules that repo
> - * administrators can explicitly set. Nothing has been decided,
> - * so for now, just append the name at the end of the path.
> - */
> + struct strbuf encoded_sub_name = STRBUF_INIT, tmp = STRBUF_INIT;
> + size_t base_len, encoded_len;
> char *gitdir_path, *key;
> + long name_max;
Some of these new variables are not used or are only written:
submodule.c: In function ‘submodule_name_to_gitdir’:
submodule.c:2615:14: error: unused variable ‘name_max’ [-Werror=unused-variable]
2615 | long name_max;
| ^~~~~~~~
submodule.c:2613:26: error: unused variable ‘encoded_len’ [-Werror=unused-variable]
2613 | size_t base_len, encoded_len;
| ^~~~~~~~~~~
submodule.c:2613:16: error: variable ‘base_len’ set but not used [-Werror=unused-but-set-variable]
2613 | size_t base_len, encoded_len;
| ^~~~~~~~
cc1: all warnings being treated as errors
make: *** [Makefile:2815: submodule.o] Error 1
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts
2025-09-10 18:15 ` SZEDER Gábor
@ 2025-09-10 19:30 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-09-10 19:30 UTC (permalink / raw)
To: SZEDER Gábor
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Brandon Williams
On Wed, 10 Sep 2025, SZEDER Gábor <szeder.dev@gmail.com> wrote:
> On Mon, Sep 08, 2025 at 05:01:13PM +0300, Adrian Ratiu wrote:
>> + struct strbuf encoded_sub_name = STRBUF_INIT, tmp =
>> STRBUF_INIT; + size_t base_len, encoded_len;
>> char *gitdir_path, *key;
>> + long name_max;
>
> Some of these new variables are not used or are only written:
I moved the logic to a new commit but forgot to also move the
variable definitions to the new commit. :)
Will fix this in v3, alongside running more tests & diagnostics on
each commit to ensure they are also bisectable and so on.
Thanks for spotting this!
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
2025-09-10 18:15 ` SZEDER Gábor
@ 2025-09-10 20:18 ` Kristoffer Haugsbakk
2025-09-30 13:36 ` Kristoffer Haugsbakk
2 siblings, 0 replies; 179+ messages in thread
From: Kristoffer Haugsbakk @ 2025-09-10 20:18 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble, Brandon Williams
On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
> Based on previous work by Brandon & all [1].
nit: “& al”/“et al”?
> Based on previous work by Brandon & all [1].
> ...
> Link: https://lore.kernel.org/git/20180807230637.247200-1-bmwill@google.com/ [1]
nit: This practice of appending or I guess prepending[1] the footnote
reference to a Link trailer isn’t commonly done. It’s usually
that regular
[1]: <link>
Thing.
Trailers are almost always pointing to people identities. Not links or
commits or other inanimate things.
† 1: f62dcc7f30d (remote: remove branch->merge_name and fix
branch_release(), 2025-06-23)
>[snip]
--
Kristoffer Haugsbakk
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 08/10] submodule: remove validate_submodule_git_dir()
2025-09-08 14:01 ` [PATCH v2 08/10] submodule: remove validate_submodule_git_dir() Adrian Ratiu
@ 2025-09-30 13:35 ` Kristoffer Haugsbakk
2025-10-03 7:56 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Kristoffer Haugsbakk @ 2025-09-30 13:35 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble
On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
> The validate_submodule_git_dir test is not very useful anymore, after
> submodule names are encoded to resolve gitdir path conflicts.
>
> In other words, the purpouse of gitdir path encoding is precisely to
s/purpouse/purpose/
> avoid such conflicts as this function tries to also prevent.
>
>[snip]
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 07/10] submodule: error out if gitdir name is too long
2025-09-08 14:01 ` [PATCH v2 07/10] submodule: error out if gitdir name is too long Adrian Ratiu
2025-09-08 15:51 ` Jeff King
@ 2025-09-30 13:35 ` Kristoffer Haugsbakk
1 sibling, 0 replies; 179+ messages in thread
From: Kristoffer Haugsbakk @ 2025-09-30 13:35 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble
On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
> Encoding submodule names increases their name size, so there is an
> increased risk to hit the max filename length in the gitdir path.
> (the likelihood is still rather small, so it's an acceptable risk)
>
> This gitdir file-name-too-long corner case can be be addressed in
> multiple ways, including sharding or trimming, however for now, just
> add the portable logic (suggested by Peff) to detect the corner case
> then error out to avoid comitting to a specific policy (or policies).
>
> In the future, instead of throwing an error (which we do now anyway
> without submodule encoding), we could maybe let the user specify via
> configs how to address this case, eg pick trimming or sharding.
s/eg/e.g./
>
> Suggested-by: Jeff King <peff@peff.net>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
2025-09-10 18:15 ` SZEDER Gábor
2025-09-10 20:18 ` Kristoffer Haugsbakk
@ 2025-09-30 13:36 ` Kristoffer Haugsbakk
2 siblings, 0 replies; 179+ messages in thread
From: Kristoffer Haugsbakk @ 2025-09-30 13:36 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble, Brandon Williams
On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
> Based on previous work by Brandon & all [1].
>
> This encodes submodule gitdir names to avoid colisions like nested gitdirs
s/colisions/collisions/
> due to names like "foo" and "foo/bar".
>
> A custom encoding can become unnecesarily complex, while url-encoding is
s/unnecesarily/unnecessarily/
> relatively well-known, however it needs some extending to support case
> insensitive filesystems and quirks like Windows reserving "COM1" names.
> Hence why I opted to encode A as _a, B as _b and so on.
>
>[snip]
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-09-08 14:01 ` [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-09-30 13:37 ` Kristoffer Haugsbakk
0 siblings, 0 replies; 179+ messages in thread
From: Kristoffer Haugsbakk @ 2025-09-30 13:37 UTC (permalink / raw)
To: Adrian Ratiu, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble
On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
> While testing submodule gitdir path encoding, I noticed submodule--helper
> is still using a hardcoded name-based path leading to test failures, so
> convert it to the common helper function introduced by commit ce125d431a
> (submodule: extract path to submodule gitdir func, 2021-09-15) and used
> in other locations accross the source tree.
s/accross/across/
>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v2 08/10] submodule: remove validate_submodule_git_dir()
2025-09-30 13:35 ` Kristoffer Haugsbakk
@ 2025-10-03 7:56 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-03 7:56 UTC (permalink / raw)
To: Kristoffer Haugsbakk, git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, D. Ben Knoble
On Tue, 30 Sep 2025, "Kristoffer Haugsbakk"
<kristofferhaugsbakk@fastmail.com> wrote:
> On Mon, Sep 8, 2025, at 16:01, Adrian Ratiu wrote:
>> The validate_submodule_git_dir test is not very useful anymore,
>> after submodule names are encoded to resolve gitdir path
>> conflicts.
>>
>> In other words, the purpouse of gitdir path encoding is
>> precisely to
>
> s/purpouse/purpose/
Thanks, I'll send v3 with these typos fixed very soon.
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (10 preceding siblings ...)
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (5 more replies)
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
` (3 subsequent siblings)
15 siblings, 6 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Hello everyone,
v3 is much simplified from v2, starting from the design idea that submodule gitdir
name encoding is to be put behind an extensions.submoduleEncoding.
This allowed removal of the modules vs submodules directories split and simplified
our logic quite a lot. Tests have been been squashed in the smaller commits as well.
Many thanks to all who provided feedback, especially Patrick and Phillip who
suggested the extension idea.
This is based on the latest master branch and I've also merged and tested against next.
I pushed the patches to github [1] and also did a CI run [2] which passed (the lone
Win+Meson CI failure seems to be unrelated because it reproduces without the patches).
[1] https://github.com/10ne1/git/tree/dev/aratiu/encoding-v3
[2] https://github.com/10ne1/git/actions/runs/18276914867
Changes between v2 -> v3:
* Put submodule encoding behind an extension (Phillip & Patrick).
* Removed the submodules vs modules directory split (Patrick).
* Undeleted validate_submodule_git_dir() because it still needs to check the default.
* Undeleted tests from t7450-bad-git-dotfiles.sh because they are still required.
* Moved new tests to a new file which enables the extension.
* Moved unused variables to commit which uses them (Szeder Gabor).
* Squashed commits to reduce their number (e.g. tests are together with new logic).
* Fixed a small bug passing the module repo instead of the_repo to the gitdir helper.
* Small commit msg rewording improvements, typos (Kristoffer & Phillip).
Adrian Ratiu (5):
submodule--helper: use submodule_name_to_gitdir in add_submodule
submodule: add gitdir path config override
strbuf: bring back is_rfc3986_unreserved
submodule: encode gitdir paths to avoid conflicts
submodule: error out if gitdir name is too long
Documentation/config/extensions.adoc | 9 ++
Documentation/config/submodule.adoc | 7 ++
Makefile | 5 +
builtin/credential-store.c | 6 -
builtin/submodule--helper.c | 30 ++++-
compat/pathconf.c | 10 ++
compat/posix.h | 8 ++
config.mak.uname | 2 +
meson.build | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
strbuf.c | 6 +
strbuf.h | 2 +
submodule.c | 84 +++++++++----
t/lib-verify-submodule-gitdir-path.sh | 20 ++++
t/meson.build | 1 +
t/t7400-submodule-basic.sh | 9 ++
t/t7425-submodule-encoding.sh | 162 ++++++++++++++++++++++++++
t/t9902-completion.sh | 1 +
20 files changed, 340 insertions(+), 32 deletions(-)
create mode 100644 compat/pathconf.c
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-encoding.sh
--
2.49.1
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 16:37 ` Junio C Hamano
2025-10-06 11:25 ` [PATCH v3 2/5] submodule: add gitdir path config override Adrian Ratiu
` (4 subsequent siblings)
5 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded name-based path leading to test failures, so
convert it to the common helper function introduced by commit ce125d431a
(submodule: extract path to submodule gitdir func, 2021-09-15) and used
in other locations across the source tree.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index fcd73abe53..2873b2780e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3187,13 +3187,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3207,10 +3207,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3219,8 +3220,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3238,7 +3239,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.49.1
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 16:47 ` Junio C Hamano
2025-10-21 8:05 ` Patrick Steinhardt
2025-10-06 11:25 ` [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
` (3 subsequent siblings)
5 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu,
Brandon Williams
This adds the ability to override gitdir paths via config files
(not .gitmodules) such that the encoding scheme (or plain text
name if the encoding extension is disabled) can be changed via
config entries.
These entries are not added by default for all submodules: they
should be used on an as-needed basis.
A new test and a helper are added. The helper will also be used
in further tests exercising gitdir encoding functionality.
Based-on-patch-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/submodule.adoc | 4 ++++
builtin/submodule--helper.c | 17 +++++++++++++++++
submodule.c | 12 ++++++++++++
t/lib-verify-submodule-gitdir-path.sh | 20 ++++++++++++++++++++
t/t7400-submodule-basic.sh | 9 +++++++++
t/t9902-completion.sh | 1 +
6 files changed, 63 insertions(+)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..8f64adfbe3 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,10 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This option sets the gitdir path for submodule <name>, allowing users
+ to override the default path or change the default path name encoding.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2873b2780e..abd20eee53 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1208,6 +1208,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3591,6 +3607,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
diff --git a/submodule.c b/submodule.c
index 35c55155f7..7a2d7cd592 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2604,6 +2604,18 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
* administrators can explicitly set. Nothing has been decided,
* so for now, just append the name at the end of the path.
*/
+ char *gitdir_path, *key;
+
+ /* Allow config override. */
+ key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
+ if (!repo_config_get_string(r, key, &gitdir_path)) {
+ strbuf_addstr(buf, gitdir_path);
+ free(key);
+ free(gitdir_path);
+ return;
+ }
+ free(key);
+
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
}
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..3a83f2d975
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,20 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (e.g. when adding a new submodule), so this only
+# checks the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ cat >expect <<-EOF &&
+ $(git rev-parse --git-common-dir)/$path
+ EOF
+ git submodule--helper gitdir "$name" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index fd3e7e355e..11c84a7bdf 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
test_expect_success 'setup - enable local submodules' '
git config --global protocol.file.allow always
@@ -1505,4 +1506,12 @@ test_expect_success 'submodule add fails when name is reused' '
)
'
+test_expect_success 'submodule helper gitdir config overrides' '
+ verify_submodule_gitdir_path test-submodule child modules/child &&
+ test_config -C test-submodule submodule.child.gitdirpath ".git/modules/custom-child" &&
+ verify_submodule_gitdir_path test-submodule child modules/custom-child &&
+ test_unconfig -C test-submodule submodule.child.gitdirpath &&
+ verify_submodule_gitdir_path test-submodule child modules/child
+'
+
test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 964e1f1569..ffb9c8b522 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.49.1
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 2/5] submodule: add gitdir path config override Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 16:51 ` Junio C Hamano
2025-10-21 8:06 ` Patrick Steinhardt
2025-10-06 11:25 ` [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
` (2 subsequent siblings)
5 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it was the only place using it.
However now we need it to apply URL-encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 6 ------
strbuf.c | 6 ++++++
strbuf.h | 2 ++
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..0acaf1cc82 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -76,12 +76,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/strbuf.c b/strbuf.c
index 6c3851a7f8..e8d84cbb6d 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -817,6 +817,12 @@ void strbuf_addstr_xml_quoted(struct strbuf *buf, const char *s)
}
}
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
static void strbuf_add_urlencode(struct strbuf *sb, const char *s, size_t len,
char_predicate allow_unencoded_fn)
{
diff --git a/strbuf.h b/strbuf.h
index a580ac6084..5139269039 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -640,6 +640,8 @@ static inline void strbuf_complete_line(struct strbuf *sb)
typedef int (*char_predicate)(char ch);
+int is_rfc3986_unreserved(char ch);
+
void strbuf_addstr_urlencode(struct strbuf *sb, const char *name,
char_predicate allow_unencoded_fn);
--
2.49.1
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (2 preceding siblings ...)
2025-10-06 11:25 ` [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 16:57 ` Junio C Hamano
2025-10-06 11:25 ` [PATCH v3 5/5] submodule: error out if gitdir name is too long Adrian Ratiu
2025-10-06 16:21 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Junio C Hamano
5 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
This adds a new submoduleEncoding extension which encodes gitdir names
to avoid collisions due to nested gitdirs or case insensitive filesystems.
A custom encoding can become unnecessarily complex, while url-encoding is
relatively well-known, however it needs some extending to support case
insensitive filesystems, hence why A is encoded as _a, B as _b and so on.
Unfortunately encoding A -> _a (...) is not enough to fix the reserved
Windows file names (e.g. COM1) because worktrees still use names like COM1
even if the gitdirs paths are encoded, so future work is needed to fully
address Windows reserved names.
For now url-encoding is the only option, however in the future we may
add alternatives (other encodings, hashes or even hash_name).
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 9 ++
Documentation/config/submodule.adoc | 3 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 56 ++++++----
t/meson.build | 1 +
t/t7425-submodule-encoding.sh | 146 +++++++++++++++++++++++++++
8 files changed, 204 insertions(+), 20 deletions(-)
create mode 100755 t/t7425-submodule-encoding.sh
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 829f2523fc..c6ab268328 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -73,6 +73,15 @@ relativeWorktrees::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
+submoduleEncoding::
+ If enabled, submodule gitdir paths are encoded to avoid filesystem
+ conflicts due to nested gitdirs or case insensitivity. For now, only
+ url-encoding (rfc3986) is available, with a small addition to encode
+ uppercase to lowercase letters (`A -> _a`, `B -> _b` and so on).
+ Other encoding or hashing methods may be added in the future.
+ Any preexisting non-encoded submodule gitdirs are used as-is, to
+ ease migration and reduce risk of gitdirs not being recognized.
+
worktreeConfig::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 8f64adfbe3..cd7fd5da5b 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -55,6 +55,9 @@ submodule.<name>.active::
submodule.<name>.gitdir::
This option sets the gitdir path for submodule <name>, allowing users
to override the default path or change the default path name encoding.
+ Submodule gitdir encoding is enabled via `extensions.submoduleEncoding`
+ (see linkgit:git-config[1]). This config works both with the extension
+ enabled or disabled.
submodule.active::
A repeated field which contains a pathspec used to match against a
diff --git a/repository.h b/repository.h
index 5808a5d610..7e39b2acf7 100644
--- a/repository.h
+++ b/repository.h
@@ -158,6 +158,7 @@ struct repository {
int repository_format_worktree_config;
int repository_format_relative_worktrees;
int repository_format_precious_objects;
+ int repository_format_submodule_encoding;
/* Indicate if a repository has a different 'commondir' from 'gitdir' */
unsigned different_commondir:1;
diff --git a/setup.c b/setup.c
index 7086741e6c..bf6e815105 100644
--- a/setup.c
+++ b/setup.c
@@ -687,6 +687,9 @@ static enum extension_result handle_extension(const char *var,
} else if (!strcmp(ext, "relativeworktrees")) {
data->relative_worktrees = git_config_bool(var, value);
return EXTENSION_OK;
+ } else if (!strcmp(ext, "submoduleencoding")) {
+ data->submodule_encoding = git_config_bool(var, value);
+ return EXTENSION_OK;
}
return EXTENSION_UNKNOWN;
}
@@ -1865,6 +1868,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
repo_fmt.worktree_config;
the_repository->repository_format_relative_worktrees =
repo_fmt.relative_worktrees;
+ the_repository->repository_format_submodule_encoding =
+ repo_fmt.submodule_encoding;
/* take ownership of repo_fmt.partial_clone */
the_repository->repository_format_partial_clone =
repo_fmt.partial_clone;
@@ -1963,6 +1968,8 @@ void check_repository_format(struct repository_format *fmt)
fmt->ref_storage_format);
the_repository->repository_format_worktree_config =
fmt->worktree_config;
+ the_repository->repository_format_submodule_encoding =
+ fmt->submodule_encoding;
the_repository->repository_format_relative_worktrees =
fmt->relative_worktrees;
the_repository->repository_format_partial_clone =
diff --git a/setup.h b/setup.h
index 8522fa8575..66ec1ceba5 100644
--- a/setup.h
+++ b/setup.h
@@ -130,6 +130,7 @@ struct repository_format {
char *partial_clone; /* value of extensions.partialclone */
int worktree_config;
int relative_worktrees;
+ int submodule_encoding;
int is_bare;
int hash_algo;
int compat_hash_algo;
diff --git a/submodule.c b/submodule.c
index 7a2d7cd592..23b79c9192 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2262,6 +2262,13 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
char *p;
int ret = 0;
+ /*
+ * Skip these checks when extensions.submoduleEncoding is enabled because
+ * it fixes the nesting issues and the suffixes will not match by design.
+ */
+ if (the_repository->repository_format_submodule_encoding)
+ return 0;
+
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
BUG("submodule name '%s' not a suffix of git dir '%s'",
@@ -2581,29 +2588,22 @@ int submodule_to_gitdir(struct repository *repo,
return ret;
}
+static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
+{
+ for (; *src; src++) {
+ unsigned char c = *src;
+ if (c >= 'A' && c <= 'Z') {
+ strbuf_addch(dst, '_');
+ strbuf_addch(dst, c - 'A' + 'a');
+ } else {
+ strbuf_addch(dst, c);
+ }
+ }
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
- /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
- */
char *gitdir_path, *key;
/* Allow config override. */
@@ -2618,4 +2618,20 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+
+ /* Existing legacy non-encoded names are used as-is */
+ if (is_git_directory(buf->buf))
+ return;
+
+ if (the_repository->repository_format_submodule_encoding) {
+ struct strbuf tmp = STRBUF_INIT;
+
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+
+ strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
+ strbuf_addstr_case_encode(buf, tmp.buf);
+
+ strbuf_release(&tmp);
+ }
}
diff --git a/t/meson.build b/t/meson.build
index 11376b9e25..de277227a2 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -882,6 +882,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-encoding.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
new file mode 100755
index 0000000000..4ea385d882
--- /dev/null
+++ b/t/t7425-submodule-encoding.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+
+test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed encoded and non-encoded submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submoduleEncoding true &&
+
+ git submodule add ../new-sub "New Sub" &&
+ test_commit new
+ )
+'
+
+test_expect_success 'verify submodule name is properly encoded' '
+ verify_submodule_gitdir_path main legacy modules/legacy &&
+ verify_submodule_gitdir_path main "New Sub" modules/_new%20_sub
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned-non-encoding &&
+ (
+ cd cloned-non-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/"New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ ) &&
+
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules main cloned-encoding &&
+ (
+ cd cloned-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/_new%20_sub &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'commit and push changes to encoded submodules' '
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ git -C main config receive.denyCurrentBranch updateInstead &&
+ (
+ cd cloned-encoding &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C "New Sub" switch --track -C main origin/main &&
+ test_commit -C "New Sub" second-commit &&
+ git -C "New Sub" push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy "New Sub" &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/_new%20_sub &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
+'
+
+test_done
--
2.49.1
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (3 preceding siblings ...)
2025-10-06 11:25 ` [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-10-06 11:25 ` Adrian Ratiu
2025-10-06 17:06 ` Junio C Hamano
2025-10-21 8:06 ` Patrick Steinhardt
2025-10-06 16:21 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Junio C Hamano
5 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-06 11:25 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Encoding submodule names increases their name size, so there is an
increased risk to hit the max filename length in the gitdir path.
(the likelihood is still rather small, so it's an acceptable risk)
This gitdir file-name-too-long corner case can be be addressed in
multiple ways, including sharding or trimming, however for now, just
add the portable logic (suggested by Peff) to detect the corner case
then error out to avoid committing to a specific policy (or policies).
In the future, instead of throwing an error (which we do now anyway
without submodule encoding), we could maybe let the user specify via
configs how to address this case, e.g. pick trimming or sharding.
At least now we print a nice error instead of the OS defaults which
can be rather cryptic for users.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Makefile | 5 +++++
compat/pathconf.c | 10 ++++++++++
compat/posix.h | 8 ++++++++
config.mak.uname | 2 ++
meson.build | 1 +
submodule.c | 16 ++++++++++++++++
t/t7425-submodule-encoding.sh | 16 ++++++++++++++++
7 files changed, 58 insertions(+)
create mode 100644 compat/pathconf.c
diff --git a/Makefile b/Makefile
index 92fd8d86d8..9f76a67d4b 100644
--- a/Makefile
+++ b/Makefile
@@ -2215,6 +2215,11 @@ ifndef HAVE_PLATFORM_PROCINFO
COMPAT_OBJS += compat/stub/procinfo.o
endif
+ifdef NO_PATHCONF
+ COMPAT_CFLAGS += -DNO_PATHCONF
+ COMPAT_OBJS += compat/pathconf.o
+endif
+
ifdef RUNTIME_PREFIX
ifdef HAVE_BSD_KERN_PROC_SYSCTL
diff --git a/compat/pathconf.c b/compat/pathconf.c
new file mode 100644
index 0000000000..37500cfa0d
--- /dev/null
+++ b/compat/pathconf.c
@@ -0,0 +1,10 @@
+#include "git-compat-util.h"
+
+/*
+ * Minimal stub for platforms without pathconf() (e.g. Windows),
+ * to fall back to NAME_MAX from limits.h or compat/posix.h.
+ */
+long git_pathconf(const char *path UNUSED, int name UNUSED)
+{
+ return -1;
+}
diff --git a/compat/posix.h b/compat/posix.h
index 067a00f33b..aa050fd58c 100644
--- a/compat/posix.h
+++ b/compat/posix.h
@@ -250,6 +250,14 @@ char *gitdirname(char *);
#define NAME_MAX 255
#endif
+#ifdef NO_PATHCONF
+#ifndef _PC_NAME_MAX
+#define _PC_NAME_MAX 1 /* dummy value, only used for git_pathconf */
+#endif
+#define pathconf(a,b) git_pathconf(a,b)
+long git_pathconf(const char *path, int name);
+#endif
+
typedef uintmax_t timestamp_t;
#define PRItime PRIuMAX
#define parse_timestamp strtoumax
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..49ba3de39d 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -473,6 +473,7 @@ ifeq ($(uname_S),Windows)
NEEDS_CRYPTO_WITH_SSL = YesPlease
NO_LIBGEN_H = YesPlease
NO_POLL = YesPlease
+ NO_PATHCONF = YesPlease
NO_SYMLINK_HEAD = YesPlease
NO_IPV6 = YesPlease
NO_SETENV = YesPlease
@@ -688,6 +689,7 @@ ifeq ($(uname_S),MINGW)
NEEDS_CRYPTO_WITH_SSL = YesPlease
NO_LIBGEN_H = YesPlease
NO_POLL = YesPlease
+ NO_PATHCONF = YesPlease
NO_SYMLINK_HEAD = YesPlease
NO_SETENV = YesPlease
NO_STRCASESTR = YesPlease
diff --git a/meson.build b/meson.build
index db1710e229..b65b5e0911 100644
--- a/meson.build
+++ b/meson.build
@@ -1394,6 +1394,7 @@ checkfuncs = {
'initgroups' : [],
'strtoumax' : ['strtoumax.c', 'strtoimax.c'],
'pread' : ['pread.c'],
+ 'pathconf' : ['pathconf.c'],
}
if host_machine.system() == 'windows'
diff --git a/submodule.c b/submodule.c
index 23b79c9192..201e8a8fe5 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2625,13 +2625,29 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
if (the_repository->repository_format_submodule_encoding) {
struct strbuf tmp = STRBUF_INIT;
+ size_t base_len;
+ long name_max;
strbuf_reset(buf);
repo_git_path_append(r, buf, "modules/");
+ base_len = buf->len;
strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
strbuf_addstr_case_encode(buf, tmp.buf);
+ /* Ensure final path length is below NAME_MAX after encoding */
+ name_max = pathconf(buf->buf, _PC_NAME_MAX);
+ if (name_max == -1)
+ name_max = NAME_MAX;
+
+ if (buf->len - base_len > name_max)
+ /*
+ * TODO: make this smarter; instead of erroring out, maybe we could trim or
+ * shard the gitdir names to make them fit under NAME_MAX.
+ */
+ die(_("submodule name %s is too long (%"PRIuMAX" bytes, limit %"PRIuMAX")"),
+ buf->buf, (uintmax_t)buf->len - base_len, (uintmax_t)name_max);
+
strbuf_release(&tmp);
}
}
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
index 4ea385d882..8041781491 100755
--- a/t/t7425-submodule-encoding.sh
+++ b/t/t7425-submodule-encoding.sh
@@ -143,4 +143,20 @@ test_expect_success 'submodule git dir nesting detection must work with parallel
verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
'
+test_expect_success 'submodule encoded name exceeds max name limit' '
+ (
+ cd main &&
+
+ # find the system NAME_MAX (fall back to 255 if unknown)
+ name_max=$(getconf NAME_MAX . 2>/dev/null || echo 255) &&
+
+ # each "%" char encodes to "%25" (3 chars), ensure we exceed NAME_MAX
+ count=$((name_max + 10)) &&
+ longname=$(test_seq -f "%%%0.s" 1 $count | tr -d "\n") &&
+
+ test_must_fail git submodule add ../new-sub "$longname" 2>err &&
+ test_grep "fatal: submodule name .* is too long" err
+ )
+'
+
test_done
--
2.49.1
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (4 preceding siblings ...)
2025-10-06 11:25 ` [PATCH v3 5/5] submodule: error out if gitdir name is too long Adrian Ratiu
@ 2025-10-06 16:21 ` Junio C Hamano
2025-10-07 11:13 ` Adrian Ratiu
5 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 16:21 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> Hello everyone,
>
> v3 is much simplified from v2, starting from the design idea that submodule gitdir
> name encoding is to be put behind an extensions.submoduleEncoding.
This design decision to make it an extension makes a repository with
a new-style submodule incompatible with older Git, which may not matter
all that much unless you use third-party tools that come with their own
version of Git embedded (which by definition can become stale).
If you already have submodules creted under the original scheme,
then add a new submodule that needs this extension, do you enable
this new extension and write the new submodule under encoded name,
and move the existing submodules under their encoded names?
> This allowed removal of the modules vs submodules directories split and simplified
> our logic quite a lot. Tests have been been squashed in the smaller commits as well.
By this statement, I am guessing that the answer is yes? That
would make it consistent. The last thing we want here is the code
that needs to guess which ones are encoded and which ones are not.
Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-10-06 11:25 ` [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-10-06 16:37 ` Junio C Hamano
2025-10-07 9:23 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 16:37 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> While testing submodule gitdir path encoding, I noticed submodule--helper
> is still using a hardcoded name-based path leading to test failures, so
> convert it to the common helper function introduced by commit ce125d431a
> (submodule: extract path to submodule gitdir func, 2021-09-15) and used
> in other locations across the source tree.
OK. To me during my first reading, the above read as if you found
an open coded logic here in add_submodule(), made it into a new
common helper function, and made this part as well as other locations
call that new common helper function. Of course that is not the
case.
Perhaps replacing everything after ", so convert it" with something
simpler like
... to test failures. Call submodule_name_to_gitdir() helper
instead, which was invented exactly for this purpose and
everybody else uses.
might have helped me to avoid such a confusion. I dunno.
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index fcd73abe53..2873b2780e 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -3187,13 +3187,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
>
> static int add_submodule(const struct add_data *add_data)
> {
> - char *submod_gitdir_path;
> struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
> struct string_list reference = STRING_LIST_INIT_NODUP;
> int ret = -1;
>
> /* perhaps the path already exists and is already a git repo, else clone it */
> if (is_directory(add_data->sm_path)) {
> + char *submod_gitdir_path;
This hunk is not related to the theme of the change and not
explained? I think the variable becomes used only within this block
after the patch that loses the use of it on the "else" side, so in
that sense it is not strictly unrelated, but is a fallout of this
change. If we were to mention the change in the log message,
something like "Also narrow the scope of a variable that is no
longer used in the updated code" would suffice.
> @@ -3207,10 +3207,11 @@ static int add_submodule(const struct add_data *add_data)
> free(submod_gitdir_path);
> } else {
> struct child_process cp = CHILD_PROCESS_INIT;
> + struct strbuf submod_gitdir = STRBUF_INIT;
>
> - submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
> + submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
>
> - if (is_directory(submod_gitdir_path)) {
> + if (is_directory(submod_gitdir.buf)) {
> if (!add_data->force) {
> struct strbuf msg = STRBUF_INIT;
> char *die_msg;
So this is the crux of the change, which makes sense. Where do we
release the resource aquired here? Let's see...
> @@ -3219,8 +3220,8 @@ static int add_submodule(const struct add_data *add_data)
> "locally with remote(s):\n"),
> add_data->sm_name);
>
> - append_fetch_remotes(&msg, submod_gitdir_path);
> - free(submod_gitdir_path);
> + append_fetch_remotes(&msg, submod_gitdir.buf);
> + strbuf_release(&submod_gitdir);
... OK, where the variable was freed in the original, so if this is
not the right place to release the resources, then the original was
already buggy.
>
> strbuf_addf(&msg, _("If you want to reuse this local git "
> "directory instead of cloning again from\n"
> @@ -3238,7 +3239,7 @@ static int add_submodule(const struct add_data *add_data)
> "submodule '%s'\n"), add_data->sm_name);
> }
> }
> - free(submod_gitdir_path);
> + strbuf_release(&submod_gitdir);
Ditto.
And the only nonlocal exit other than die() inside this "else"
block appears _after_ this part, so we are OK.
Looks good to me. Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-06 11:25 ` [PATCH v3 2/5] submodule: add gitdir path config override Adrian Ratiu
@ 2025-10-06 16:47 ` Junio C Hamano
2025-10-07 15:41 ` Junio C Hamano
2025-10-21 8:05 ` Patrick Steinhardt
1 sibling, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 16:47 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
[jc: brandon removed from CC list as the address would bounce]
> This adds the ability to override gitdir paths via config files
> (not .gitmodules) such that the encoding scheme (or plain text
> name if the encoding extension is disabled) can be changed via
> config entries.
>
> These entries are not added by default for all submodules: they
> should be used on an as-needed basis.
>
> A new test and a helper are added. The helper will also be used
> in further tests exercising gitdir encoding functionality.
What is the use case of this? The only reasonable use case I can
see is to set this to all the existing submodules when you are
switching the extension on before adding a new submodule, in which
case the old ones will keep using unencoded names, while the new
ones will use encoded ones. But is that a sensible thing to do?
How would we guarantee that existing submodules' vanilla names would
not collide with encoded submodules' names? We haven't seen the
encoded names yet, but I think I saw some mention of URL encoding.
So if I had a submodule whose name is "%41%42%43", set this
configuration because I do not want it treated as URL-encoded, then
enable the extension, and then later add a separate submodule whose
name is "ABC", which may be encoded (remember use of "ABC" here is
only for illustration; replace it with something that do need
encoding if you want a more realistic example) to the same
"%41%42%43".
If that kind of situation is what this new configuration allows, I
do not quite see why it is a good idea to have such a thing.
> Based-on-patch-by: Brandon Williams <bmwill@google.com>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> Documentation/config/submodule.adoc | 4 ++++
> builtin/submodule--helper.c | 17 +++++++++++++++++
> submodule.c | 12 ++++++++++++
> t/lib-verify-submodule-gitdir-path.sh | 20 ++++++++++++++++++++
> t/t7400-submodule-basic.sh | 9 +++++++++
> t/t9902-completion.sh | 1 +
> 6 files changed, 63 insertions(+)
> create mode 100644 t/lib-verify-submodule-gitdir-path.sh
>
> diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
> index 0672d99117..8f64adfbe3 100644
> --- a/Documentation/config/submodule.adoc
> +++ b/Documentation/config/submodule.adoc
> @@ -52,6 +52,10 @@ submodule.<name>.active::
> submodule.active config option. See linkgit:gitsubmodules[7] for
> details.
>
> +submodule.<name>.gitdir::
> + This option sets the gitdir path for submodule <name>, allowing users
> + to override the default path or change the default path name encoding.
> +
> submodule.active::
> A repeated field which contains a pathspec used to match against a
> submodule's path to determine if the submodule is of interest to git
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 2873b2780e..abd20eee53 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1208,6 +1208,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
> return ret;
> }
>
> +static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
> + struct repository *repo)
> +{
> + struct strbuf gitdir = STRBUF_INIT;
> +
> + if (argc != 2)
> + usage(_("git submodule--helper gitdir <name>"));
> +
> + submodule_name_to_gitdir(&gitdir, repo, argv[1]);
> +
> + printf("%s\n", gitdir.buf);
> +
> + strbuf_release(&gitdir);
> + return 0;
> +}
> +
> struct sync_cb {
> const char *prefix;
> const char *super_prefix;
> @@ -3591,6 +3607,7 @@ int cmd_submodule__helper(int argc,
> NULL
> };
> struct option options[] = {
> + OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
> OPT_SUBCOMMAND("clone", &fn, module_clone),
> OPT_SUBCOMMAND("add", &fn, module_add),
> OPT_SUBCOMMAND("update", &fn, module_update),
> diff --git a/submodule.c b/submodule.c
> index 35c55155f7..7a2d7cd592 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2604,6 +2604,18 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> * administrators can explicitly set. Nothing has been decided,
> * so for now, just append the name at the end of the path.
> */
> + char *gitdir_path, *key;
> +
> + /* Allow config override. */
> + key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
> + if (!repo_config_get_string(r, key, &gitdir_path)) {
> + strbuf_addstr(buf, gitdir_path);
> + free(key);
> + free(gitdir_path);
> + return;
> + }
> + free(key);
> +
> repo_git_path_append(r, buf, "modules/");
> strbuf_addstr(buf, submodule_name);
> }
> diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
> new file mode 100644
> index 0000000000..3a83f2d975
> --- /dev/null
> +++ b/t/lib-verify-submodule-gitdir-path.sh
> @@ -0,0 +1,20 @@
> +# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
> +
> +# This does not check filesystem existence. That is done in submodule.c via the
> +# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
> +# might or might not exist (e.g. when adding a new submodule), so this only
> +# checks the expected configuration path, which might be overridden by the user.
> +
> +verify_submodule_gitdir_path() {
> + repo="$1" &&
> + name="$2" &&
> + path="$3" &&
> + (
> + cd "$repo" &&
> + cat >expect <<-EOF &&
> + $(git rev-parse --git-common-dir)/$path
> + EOF
> + git submodule--helper gitdir "$name" >actual &&
> + test_cmp expect actual
> + )
> +}
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index fd3e7e355e..11c84a7bdf 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>
> test_expect_success 'setup - enable local submodules' '
> git config --global protocol.file.allow always
> @@ -1505,4 +1506,12 @@ test_expect_success 'submodule add fails when name is reused' '
> )
> '
>
> +test_expect_success 'submodule helper gitdir config overrides' '
> + verify_submodule_gitdir_path test-submodule child modules/child &&
> + test_config -C test-submodule submodule.child.gitdirpath ".git/modules/custom-child" &&
> + verify_submodule_gitdir_path test-submodule child modules/custom-child &&
> + test_unconfig -C test-submodule submodule.child.gitdirpath &&
> + verify_submodule_gitdir_path test-submodule child modules/child
> +'
> +
> test_done
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 964e1f1569..ffb9c8b522 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
> submodule.sub.fetchRecurseSubmodules Z
> submodule.sub.ignore Z
> submodule.sub.active Z
> + submodule.sub.gitdir Z
> EOF
> '
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved
2025-10-06 11:25 ` [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
@ 2025-10-06 16:51 ` Junio C Hamano
2025-10-06 17:47 ` Junio C Hamano
2025-10-07 9:43 ` Adrian Ratiu
2025-10-21 8:06 ` Patrick Steinhardt
1 sibling, 2 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 16:51 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> is_rfc3986_unreserved() was moved to credential-store.c and was made
> static by f89854362c (credential-store: move related functions to
> credential-store file, 2023-06-06) under a correct assumption, at the
> time, that it was the only place using it.
>
> However now we need it to apply URL-encoding to submodule names when
> constructing gitdir paths, to avoid conflicts, so bring it back.
Why to strbuf, though?
This does not have anything to do with what strbuf does. I could be
possible that strbuf.c had some function that encodes/decodes 3986
in a strbuf and this may have been a useful helper for that feature,
but it is apparent that this helper function is needed by
strbuf.[ch] in today's code, so moving it to strbuf.[ch] makes no
sense to me.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-06 11:25 ` [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
@ 2025-10-06 16:57 ` Junio C Hamano
2025-10-07 14:10 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 16:57 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> A custom encoding can become unnecessarily complex, while url-encoding is
> relatively well-known, however it needs some extending to support case
> insensitive filesystems, hence why A is encoded as _a, B as _b and so on.
Why 'A' cannot be encoded as %41 while encodign 'a' as %61? Are
there case insensitive filesystems that cannot see the difference
between %41 and %61? And they would not collide with COM: and other
anomalies, would they?
> For now url-encoding is the only option, however in the future we may
> add alternatives (other encodings, hashes or even hash_name).
Let's not say "For now".
Choose a single encoding that we can use forever so that we do not
have to upgrade extensions.encodeSubmoduleName with suffixes like
extensions.encodeSubmoduleNamev2, extensions.encodeSubmoduleNamev3,
etc. to cover our earlier mistakes and force renaming on users.
> Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
> Suggested-by: Patrick Steinhardt <ps@pks.im>
> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
> ---
> Documentation/config/extensions.adoc | 9 ++
> Documentation/config/submodule.adoc | 3 +
> repository.h | 1 +
> setup.c | 7 ++
> setup.h | 1 +
> submodule.c | 56 ++++++----
> t/meson.build | 1 +
> t/t7425-submodule-encoding.sh | 146 +++++++++++++++++++++++++++
> 8 files changed, 204 insertions(+), 20 deletions(-)
> create mode 100755 t/t7425-submodule-encoding.sh
>
> diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
> index 829f2523fc..c6ab268328 100644
> --- a/Documentation/config/extensions.adoc
> +++ b/Documentation/config/extensions.adoc
> @@ -73,6 +73,15 @@ relativeWorktrees::
> repaired with either the `--relative-paths` option or with the
> `worktree.useRelativePaths` config set to `true`.
>
> +submoduleEncoding::
> + If enabled, submodule gitdir paths are encoded to avoid filesystem
> + conflicts due to nested gitdirs or case insensitivity. For now, only
> + url-encoding (rfc3986) is available, with a small addition to encode
> + uppercase to lowercase letters (`A -> _a`, `B -> _b` and so on).
> + Other encoding or hashing methods may be added in the future.
> + Any preexisting non-encoded submodule gitdirs are used as-is, to
> + ease migration and reduce risk of gitdirs not being recognized.
> +
> worktreeConfig::
> If enabled, then worktrees will load config settings from the
> `$GIT_DIR/config.worktree` file in addition to the
> diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
> index 8f64adfbe3..cd7fd5da5b 100644
> --- a/Documentation/config/submodule.adoc
> +++ b/Documentation/config/submodule.adoc
> @@ -55,6 +55,9 @@ submodule.<name>.active::
> submodule.<name>.gitdir::
> This option sets the gitdir path for submodule <name>, allowing users
> to override the default path or change the default path name encoding.
> + Submodule gitdir encoding is enabled via `extensions.submoduleEncoding`
> + (see linkgit:git-config[1]). This config works both with the extension
> + enabled or disabled.
>
> submodule.active::
> A repeated field which contains a pathspec used to match against a
> diff --git a/repository.h b/repository.h
> index 5808a5d610..7e39b2acf7 100644
> --- a/repository.h
> +++ b/repository.h
> @@ -158,6 +158,7 @@ struct repository {
> int repository_format_worktree_config;
> int repository_format_relative_worktrees;
> int repository_format_precious_objects;
> + int repository_format_submodule_encoding;
>
> /* Indicate if a repository has a different 'commondir' from 'gitdir' */
> unsigned different_commondir:1;
> diff --git a/setup.c b/setup.c
> index 7086741e6c..bf6e815105 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -687,6 +687,9 @@ static enum extension_result handle_extension(const char *var,
> } else if (!strcmp(ext, "relativeworktrees")) {
> data->relative_worktrees = git_config_bool(var, value);
> return EXTENSION_OK;
> + } else if (!strcmp(ext, "submoduleencoding")) {
> + data->submodule_encoding = git_config_bool(var, value);
> + return EXTENSION_OK;
> }
> return EXTENSION_UNKNOWN;
> }
> @@ -1865,6 +1868,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
> repo_fmt.worktree_config;
> the_repository->repository_format_relative_worktrees =
> repo_fmt.relative_worktrees;
> + the_repository->repository_format_submodule_encoding =
> + repo_fmt.submodule_encoding;
> /* take ownership of repo_fmt.partial_clone */
> the_repository->repository_format_partial_clone =
> repo_fmt.partial_clone;
> @@ -1963,6 +1968,8 @@ void check_repository_format(struct repository_format *fmt)
> fmt->ref_storage_format);
> the_repository->repository_format_worktree_config =
> fmt->worktree_config;
> + the_repository->repository_format_submodule_encoding =
> + fmt->submodule_encoding;
> the_repository->repository_format_relative_worktrees =
> fmt->relative_worktrees;
> the_repository->repository_format_partial_clone =
> diff --git a/setup.h b/setup.h
> index 8522fa8575..66ec1ceba5 100644
> --- a/setup.h
> +++ b/setup.h
> @@ -130,6 +130,7 @@ struct repository_format {
> char *partial_clone; /* value of extensions.partialclone */
> int worktree_config;
> int relative_worktrees;
> + int submodule_encoding;
> int is_bare;
> int hash_algo;
> int compat_hash_algo;
> diff --git a/submodule.c b/submodule.c
> index 7a2d7cd592..23b79c9192 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2262,6 +2262,13 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
> char *p;
> int ret = 0;
>
> + /*
> + * Skip these checks when extensions.submoduleEncoding is enabled because
> + * it fixes the nesting issues and the suffixes will not match by design.
> + */
> + if (the_repository->repository_format_submodule_encoding)
> + return 0;
> +
> if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
> strcmp(p, submodule_name))
> BUG("submodule name '%s' not a suffix of git dir '%s'",
> @@ -2581,29 +2588,22 @@ int submodule_to_gitdir(struct repository *repo,
> return ret;
> }
>
> +static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
> +{
> + for (; *src; src++) {
> + unsigned char c = *src;
> + if (c >= 'A' && c <= 'Z') {
> + strbuf_addch(dst, '_');
> + strbuf_addch(dst, c - 'A' + 'a');
> + } else {
> + strbuf_addch(dst, c);
> + }
> + }
> +}
> +
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name)
> {
> - /*
> - * NEEDSWORK: The current way of mapping a submodule's name to
> - * its location in .git/modules/ has problems with some naming
> - * schemes. For example, if a submodule is named "foo" and
> - * another is named "foo/bar" (whether present in the same
> - * superproject commit or not - the problem will arise if both
> - * superproject commits have been checked out at any point in
> - * time), or if two submodule names only have different cases in
> - * a case-insensitive filesystem.
> - *
> - * There are several solutions, including encoding the path in
> - * some way, introducing a submodule.<name>.gitdir config in
> - * .git/config (not .gitmodules) that allows overriding what the
> - * gitdir of a submodule would be (and teach Git, upon noticing
> - * a clash, to automatically determine a non-clashing name and
> - * to write such a config), or introducing a
> - * submodule.<name>.gitdir config in .gitmodules that repo
> - * administrators can explicitly set. Nothing has been decided,
> - * so for now, just append the name at the end of the path.
> - */
> char *gitdir_path, *key;
>
> /* Allow config override. */
> @@ -2618,4 +2618,20 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
>
> repo_git_path_append(r, buf, "modules/");
> strbuf_addstr(buf, submodule_name);
> +
> + /* Existing legacy non-encoded names are used as-is */
> + if (is_git_directory(buf->buf))
> + return;
> +
> + if (the_repository->repository_format_submodule_encoding) {
> + struct strbuf tmp = STRBUF_INIT;
> +
> + strbuf_reset(buf);
> + repo_git_path_append(r, buf, "modules/");
> +
> + strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
> + strbuf_addstr_case_encode(buf, tmp.buf);
> +
> + strbuf_release(&tmp);
> + }
> }
> diff --git a/t/meson.build b/t/meson.build
> index 11376b9e25..de277227a2 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -882,6 +882,7 @@ integration_tests = [
> 't7422-submodule-output.sh',
> 't7423-submodule-symlinks.sh',
> 't7424-submodule-mixed-ref-formats.sh',
> + 't7425-submodule-encoding.sh',
> 't7450-bad-git-dotfiles.sh',
> 't7500-commit-template-squash-signoff.sh',
> 't7501-commit-basic-functionality.sh',
> diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
> new file mode 100755
> index 0000000000..4ea385d882
> --- /dev/null
> +++ b/t/t7425-submodule-encoding.sh
> @@ -0,0 +1,146 @@
> +#!/bin/sh
> +
> +test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
> +
> +. ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
> +
> +test_expect_success 'setup: allow file protocol' '
> + git config --global protocol.file.allow always
> +'
> +
> +test_expect_success 'create repo with mixed encoded and non-encoded submodules' '
> + git init -b main legacy-sub &&
> + test_commit -C legacy-sub legacy-initial &&
> + legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
> +
> + git init -b main new-sub &&
> + test_commit -C new-sub new-initial &&
> + new_rev=$(git -C new-sub rev-parse HEAD) &&
> +
> + git init -b main main &&
> + (
> + cd main &&
> + git submodule add ../legacy-sub legacy &&
> + test_commit legacy-sub &&
> +
> + git config core.repositoryformatversion 1 &&
> + git config extensions.submoduleEncoding true &&
> +
> + git submodule add ../new-sub "New Sub" &&
> + test_commit new
> + )
> +'
> +
> +test_expect_success 'verify submodule name is properly encoded' '
> + verify_submodule_gitdir_path main legacy modules/legacy &&
> + verify_submodule_gitdir_path main "New Sub" modules/_new%20_sub
> +'
> +
> +test_expect_success 'clone from repo with both legacy and new-style submodules' '
> + git clone --recurse-submodules main cloned-non-encoding &&
> + (
> + cd cloned-non-encoding &&
> +
> + test_path_is_dir .git/modules/legacy &&
> + test_path_is_dir .git/modules/"New Sub" &&
> +
> + git submodule status >list &&
> + test_grep "$legacy_rev legacy" list &&
> + test_grep "$new_rev New Sub" list
> + ) &&
> +
> + git clone -c extensions.submoduleEncoding=true --recurse-submodules main cloned-encoding &&
> + (
> + cd cloned-encoding &&
> +
> + test_path_is_dir .git/modules/legacy &&
> + test_path_is_dir .git/modules/_new%20_sub &&
> +
> + git submodule status >list &&
> + test_grep "$legacy_rev legacy" list &&
> + test_grep "$new_rev New Sub" list
> + )
> +'
> +
> +test_expect_success 'commit and push changes to encoded submodules' '
> + git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
> + git -C new-sub config receive.denyCurrentBranch updateInstead &&
> + git -C main config receive.denyCurrentBranch updateInstead &&
> + (
> + cd cloned-encoding &&
> +
> + git -C legacy switch --track -C main origin/main &&
> + test_commit -C legacy second-commit &&
> + git -C legacy push &&
> +
> + git -C "New Sub" switch --track -C main origin/main &&
> + test_commit -C "New Sub" second-commit &&
> + git -C "New Sub" push &&
> +
> + # Stage and commit submodule changes in superproject
> + git switch --track -C main origin/main &&
> + git add legacy "New Sub" &&
> + git commit -m "update submodules" &&
> +
> + # push superproject commit to main repo
> + git push
> + ) &&
> +
> + # update expected legacy & new submodule checksums
> + legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
> + new_rev=$(git -C new-sub rev-parse HEAD)
> +'
> +
> +test_expect_success 'fetch mixed submodule changes and verify updates' '
> + (
> + cd main &&
> +
> + # only update submodules because superproject was
> + # pushed into at the end of last test
> + git submodule update --init --recursive &&
> +
> + test_path_is_dir .git/modules/legacy &&
> + test_path_is_dir .git/modules/_new%20_sub &&
> +
> + # Verify both submodules are at the expected commits
> + git submodule status >list &&
> + test_grep "$legacy_rev legacy" list &&
> + test_grep "$new_rev New Sub" list
> + )
> +'
> +
> +test_expect_success 'setup submodules with nested git dirs' '
> + git init nested &&
> + test_commit -C nested nested &&
> + (
> + cd nested &&
> + cat >.gitmodules <<-EOF &&
> + [submodule "hippo"]
> + url = .
> + path = thing1
> + [submodule "hippo/hooks"]
> + url = .
> + path = thing2
> + EOF
> + git clone . thing1 &&
> + git clone . thing2 &&
> + git add .gitmodules thing1 thing2 &&
> + test_tick &&
> + git commit -m nested
> + )
> +'
> +
> +test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
> + git clone -c extensions.submoduleEncoding=true --recurse-submodules nested clone_nested &&
> + verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
> + verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks
> +'
> +
> +test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
> + git clone -c extensions.submoduleEncoding=true --recurse-submodules --jobs=2 nested clone_parallel &&
> + verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
> + verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
> +'
> +
> +test_done
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-06 11:25 ` [PATCH v3 5/5] submodule: error out if gitdir name is too long Adrian Ratiu
@ 2025-10-06 17:06 ` Junio C Hamano
2025-10-07 10:17 ` Adrian Ratiu
2025-10-21 8:06 ` Patrick Steinhardt
1 sibling, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 17:06 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> Encoding submodule names increases their name size, so there is an
> increased risk to hit the max filename length in the gitdir path.
> (the likelihood is still rather small, so it's an acceptable risk)
If it is acceptable, can we ignore it?
Just stepping back a bit, how are we keeping track of the mapping
between submodule names vs locations in .git/modules/? Don't we
always go through that mapping and would a half-clever code that
says "heh, that is url encoded and I know how to decode it" and
bypass the mapping a bug?
If we keep track of the mapping ourselves, then the names under
.git/modules/ do not have to be "decodable" by themselves. They can
even be sequence numbers and that would not hit any maximum filename
length before you fill your disk.
No, no I am not suggesting to use sequence numbers; something
remotely readable by humans is better. But my point is that just
like you have to make sure that the encoded name you give to a new
thing does not collide with existing names (you know with "ls
.git/modules/" what names are taken), you can notice your mkdir()
would not error with name-too-long, truncate and twiddle with suffix
to make it unique and retry, without giving a failure to the end
user.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved
2025-10-06 16:51 ` Junio C Hamano
@ 2025-10-06 17:47 ` Junio C Hamano
2025-10-07 9:43 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-06 17:47 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Junio C Hamano <gitster@pobox.com> writes:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> is_rfc3986_unreserved() was moved to credential-store.c and was made
>> static by f89854362c (credential-store: move related functions to
>> credential-store file, 2023-06-06) under a correct assumption, at the
>> time, that it was the only place using it.
>>
>> However now we need it to apply URL-encoding to submodule names when
>> constructing gitdir paths, to avoid conflicts, so bring it back.
>
> Why to strbuf, though?
>
> This does not have anything to do with what strbuf does. I could be
> possible that strbuf.c had some function that encodes/decodes 3986
> in a strbuf and this may have been a useful helper for that feature,
> but it is apparent that this helper function is needed by
"is needed" -> "is not needed", of course. Sorry for a typo.
> strbuf.[ch] in today's code, so moving it to strbuf.[ch] makes no
> sense to me.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-10-06 16:37 ` Junio C Hamano
@ 2025-10-07 9:23 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 9:23 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, 06 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> While testing submodule gitdir path encoding, I noticed
>> submodule--helper is still using a hardcoded name-based path
>> leading to test failures, so convert it to the common helper
>> function introduced by commit ce125d431a (submodule: extract
>> path to submodule gitdir func, 2021-09-15) and used in other
>> locations across the source tree.
>
> OK. To me during my first reading, the above read as if you
> found an open coded logic here in add_submodule(), made it into
> a new common helper function, and made this part as well as
> other locations call that new common helper function. Of course
> that is not the case.
>
> Perhaps replacing everything after ", so convert it" with
> something simpler like
>
> ... to test failures. Call submodule_name_to_gitdir()
> helper instead, which was invented exactly for this purpose
> and everybody else uses.
> might have helped me to avoid such a confusion. I dunno.
Ack, I'll reword as you suggested to make it clearer.
>> diff --git a/builtin/submodule--helper.c
>> b/builtin/submodule--helper.c index fcd73abe53..2873b2780e
>> 100644 --- a/builtin/submodule--helper.c +++
>> b/builtin/submodule--helper.c @@ -3187,13 +3187,13 @@ static
>> void append_fetch_remotes(struct strbuf *msg, const char
>> *git_dir_path)
>> static int add_submodule(const struct add_data *add_data) {
>> - char *submod_gitdir_path;
>> struct module_clone_data clone_data =
>> MODULE_CLONE_DATA_INIT; struct string_list reference =
>> STRING_LIST_INIT_NODUP; int ret = -1; /* perhaps the path
>> already exists and is already a git repo, else clone it */ if
>> (is_directory(add_data->sm_path)) {
>> + char *submod_gitdir_path;
>
> This hunk is not related to the theme of the change and not
> explained? I think the variable becomes used only within this
> block after the patch that loses the use of it on the "else"
> side, so in that sense it is not strictly unrelated, but is a
> fallout of this change. If we were to mention the change in the
> log message, something like "Also narrow the scope of a variable
> that is no longer used in the updated code" would suffice.
Your understanding is correct, yes. I'll add it to the log msg.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved
2025-10-06 16:51 ` Junio C Hamano
2025-10-06 17:47 ` Junio C Hamano
@ 2025-10-07 9:43 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 9:43 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, 06 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> is_rfc3986_unreserved() was moved to credential-store.c and was
>> made static by f89854362c (credential-store: move related
>> functions to credential-store file, 2023-06-06) under a correct
>> assumption, at the time, that it was the only place using it.
>>
>> However now we need it to apply URL-encoding to submodule names
>> when constructing gitdir paths, to avoid conflicts, so bring it
>> back.
>
> Why to strbuf, though?
>
> This does not have anything to do with what strbuf does. I
> could be possible that strbuf.c had some function that
> encodes/decodes 3986 in a strbuf and this may have been a useful
> helper for that feature, but it is apparent that this helper
> function is needed by strbuf.[ch] in today's code, so moving it
> to strbuf.[ch] makes no sense to me.
Agreed. The only reason I moved it back to strbuf is because it
was there in the past, however we can move it to anywhere else.
Perhaps url.[ch] is the best place. Would that be ok?
Other location suggestions are welcome btw. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-06 17:06 ` Junio C Hamano
@ 2025-10-07 10:17 ` Adrian Ratiu
2025-10-07 15:58 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 10:17 UTC (permalink / raw)
To: Junio C Hamano, Jeff King
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Aaron Schrab,
Jonathan Nieder, Patrick Steinhardt, Josh Steadmon, Ben Knoble,
Phillip Wood
On Mon, 06 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> Encoding submodule names increases their name size, so there is
>> an increased risk to hit the max filename length in the gitdir
>> path. (the likelihood is still rather small, so it's an
>> acceptable risk)
>
> If it is acceptable, can we ignore it?
Yes, we can.
Peff actually asked if it's worth pursuing this path. We can
certainly ignore and let the OS fail like it does now, without the
compat wrapper.
The only usefulness of this commit as-is is to to print a nicer
error msg and allow us to eventually address the patch TODO
(sharding/trimming).
Please let me know if you wish to drop this commit in v4. I am
perfectly fine with removing it. :)
>
> Just stepping back a bit, how are we keeping track of the
> mapping between submodule names vs locations in .git/modules/?
> Don't we always go through that mapping and would a half-clever
> code that says "heh, that is url encoded and I know how to
> decode it" and bypass the mapping a bug?
The short answer is no. :) The slightly longer answer is:
Before v3, we used to have a clear filesystem separation:
.git/modules were unencoded
.git/submodules were encoded
After v3, when the extension is enabled, we encode all submodule
names with one exception: if they already exist, at which point
they are treated as unencoded.
>
> If we keep track of the mapping ourselves, then the names under
> .git/modules/ do not have to be "decodable" by themselves. They
> can even be sequence numbers and that would not hit any maximum
> filename length before you fill your disk.
IIRC the consensus was to avoid keeping such a mapping due to
complexity, risk of desynchronizing with the filesystem layout and
so on, so simple rules like the above, or even a simple hash_name
might be better (see below).
>
> No, no I am not suggesting to use sequence numbers; something
> remotely readable by humans is better. But my point is that
> just like you have to make sure that the encoded name you give
> to a new thing does not collide with existing names (you know
> with "ls .git/modules/" what names are taken), you can notice
> your mkdir() would not error with name-too-long, truncate and
> twiddle with suffix to make it unique and retry, without giving
> a failure to the end user.
Everything is on the table and I have no strong opinions. :)
Someone suggested we use something like .git/modules/<hash>_<name>
when the extension is enabled, so we have a simple, unique and
recognizable pattern.
What do you think of that?
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-06 16:21 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Junio C Hamano
@ 2025-10-07 11:13 ` Adrian Ratiu
2025-10-07 15:36 ` Junio C Hamano
2025-10-07 16:21 ` Junio C Hamano
0 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 11:13 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, 06 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> Hello everyone,
>>
>> v3 is much simplified from v2, starting from the design idea
>> that submodule gitdir name encoding is to be put behind an
>> extensions.submoduleEncoding.
>
> This design decision to make it an extension makes a repository
> with a new-style submodule incompatible with older Git, which
> may not matter all that much unless you use third-party tools
> that come with their own version of Git embedded (which by
> definition can become stale).
>
> If you already have submodules creted under the original scheme,
> then add a new submodule that needs this extension, do you
> enable this new extension and write the new submodule under
> encoded name, and move the existing submodules under their
> encoded names?
Excellent observation! We could do that (it's not being done in
v3).
Currently any existing submodule gitdir names are left untouched
and are used as-is (unencoded) after the extension is enabled.
It's been done like this for backwards compatibility, to eliminate
any potential risk of breakage by having to move/update gitdirs,
however maybe we've been overly cautious and we can attempt a
"migration" once the extension is enabled, to move the submodule
gitdirs.
>
>> This allowed removal of the modules vs submodules directories
>> split and simplified our logic quite a lot. Tests have been
>> been squashed in the smaller commits as well.
>
> By this statement, I am guessing that the answer is yes? That
> would make it consistent. The last thing we want here is the
> code that needs to guess which ones are encoded and which ones
> are not.
No, not yet, though you do raise a fair point.
We could do a migration of existing gitdirs to the new encoding to
ensure consistency when the extension is enabled.
This will simplify our logic and assumptions a lot, at the cost of
the initial up-front migration.
Will do this in v4 if nobody has any objections.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-06 16:57 ` Junio C Hamano
@ 2025-10-07 14:10 ` Adrian Ratiu
2025-10-07 17:20 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 14:10 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, 06 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> A custom encoding can become unnecessarily complex, while
>> url-encoding is relatively well-known, however it needs some
>> extending to support case insensitive filesystems, hence why A
>> is encoded as _a, B as _b and so on.
>
> Why 'A' cannot be encoded as %41 while encodign 'a' as %61? Are
> there case insensitive filesystems that cannot see the
> difference between %41 and %61? And they would not collide with
> COM: and other anomalies, would they?
That is correct.
The only reason I chose A -> _a and so on is because it was
suggested in the initial thread from 8+ years ago when people
discussed creating a custom encoding, but we can use anything
else.
I will percent encode the upper case by modifying
is_rfc3986_unreserved().
>> For now url-encoding is the only option, however in the future
>> we may add alternatives (other encodings, hashes or even
>> hash_name).
>
> Let's not say "For now".
>
> Choose a single encoding that we can use forever so that we do
> not have to upgrade extensions.encodeSubmoduleName with suffixes
> like extensions.encodeSubmoduleNamev2,
> extensions.encodeSubmoduleNamev3, etc. to cover our earlier
> mistakes and force renaming on users.
Understood. Will drop that idea.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-07 11:13 ` Adrian Ratiu
@ 2025-10-07 15:36 ` Junio C Hamano
2025-10-07 16:58 ` Adrian Ratiu
2025-10-07 17:27 ` Junio C Hamano
2025-10-07 16:21 ` Junio C Hamano
1 sibling, 2 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 15:36 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> Currently any existing submodule gitdir names are left untouched
> and are used as-is (unencoded) after the extension is enabled.
And in order to make sure that a funny names and paths in existing
submodules that can be misinterpreted as encoded would be registered
with the new submodule.<name>.gitdirpath variable?
That would be a robust way to transition. It also means that you
have a mapping from submodule name to path, and you will have to
make an effort to maintain that mapping and the reality on the
filesystem in sync.
So why not take advantage of the fact that you are making that
effort anyway? It can simplify things quite a bit. Imagine what
would happen if we did this:
- You officially declare that submodule.<name>.gitdirpath is _the_
mapping mechanism, not a mere "override".
- When enabling the extension, you register all submodules that
already has their gitdirs on the filesystem to the mapping
mechanism under their "historical and natural" names.
- When you add a new submodule, you "munge" its name to be used as
a subdirectory name under .git/modules/. You only specify for
the end users the purpose and nature of this munging, perhaps
like
- (purpose) We give each submodule a place in .git/modules
directory of the superproject to store its repository data, but
the names of submodules we find in .gitmodules may not
necessarily be friendly to the filesystem (e.g., "CON:", or
longer than the filesystem allows for a single path component),
or may introduce a subdirectory (e.g., slashes and
backslashes), or two directories whose names only differ in
case may not coexist on your filesystem. That is why we do not
use the name found in .gitmodules as-is.
- (nature) This "munging" would remove problematic bytes or
replace them with safe ones in the name, or truncate overly
long string, or insert sequence numbers to make the result
unique and representable on the filesystem. Hopefully the
result of the munging may still be recognisable as derived from
the original name, but it is *not* designed to be reversible by
itself. The submodule.<name>.gitdirpath is the only and
authoritative place to learn how <name> got munged to produce a
path.
without having to go into the details to avoid tempting users to
write scripts that guess and decode bypassing the mapping.
Hmm?
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-06 16:47 ` Junio C Hamano
@ 2025-10-07 15:41 ` Junio C Hamano
2025-10-21 8:06 ` Patrick Steinhardt
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 15:41 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Junio C Hamano <gitster@pobox.com> writes:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
> [jc: brandon removed from CC list as the address would bounce]
>
>> This adds the ability to override gitdir paths via config files
>> (not .gitmodules) such that the encoding scheme (or plain text
>> name if the encoding extension is disabled) can be changed via
>> config entries.
>>
>> These entries are not added by default for all submodules: they
>> should be used on an as-needed basis.
>>
>> A new test and a helper are added. The helper will also be used
>> in further tests exercising gitdir encoding functionality.
>
> What is the use case of this? The only reasonable use case I can
> see is to set this to all the existing submodules when you are
> switching the extension on before adding a new submodule, in which
> case the old ones will keep using unencoded names, while the new
> ones will use encoded ones.
Two things.
* I no longer mind this setting existing, but I think it should not
be a mere "override", but the authoritative source of truth for
all submodules (see my other response on 0/5).
* The documentation part of this patch says submodule.<name>.gitdir,
but what the code implements is submodule.<name>.gitdirpath.
Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-07 10:17 ` Adrian Ratiu
@ 2025-10-07 15:58 ` Junio C Hamano
0 siblings, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 15:58 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Jeff King, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> IIRC the consensus was to avoid keeping such a mapping due to
> complexity, risk of desynchronizing with the filesystem layout and
> so on, so simple rules like the above, or even a simple hash_name
> might be better (see below).
I do not quite see a merit in the above argument.
You have submodule.<name>.gitdir that you need to make sure your
"risk of desynchronizing" is managed anyway. By making sure
name-to-directory always goes through the single mapping, you reduce
the complexity and concentrate the choice of the path to a single
place, i.e. where it is assigned by munging the name and registered
to the mapping.
> Someone suggested we use something like .git/modules/<hash>_<name>
> when the extension is enabled, so we have a simple, unique and
> recognizable pattern.
>
> What do you think of that?
If there are letters in <name> that you cannot use there (e.g.,
slash, or perhaps <name> is overly long), you have to munge the
<name> part in <hash>_<name> in order to fit the filesystem
constraints. And prefixing random string "<hash>_" would be one way
to make sure two <name>s that happen to collide after munging can
still be differentiated. So I have no problem with prefixing or
suffixing, but I do not think that syntax alone is sufficient to
solve other issues that come from the fact that some names are not
filesystem friendly and can collide with names of other submodules.
Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-07 11:13 ` Adrian Ratiu
2025-10-07 15:36 ` Junio C Hamano
@ 2025-10-07 16:21 ` Junio C Hamano
2025-10-07 17:21 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 16:21 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>> If you already have submodules creted under the original scheme,
>> then add a new submodule that needs this extension, do you
>> enable this new extension and write the new submodule under
>> encoded name, and move the existing submodules under their
>> encoded names?
> ...
> We could do a migration of existing gitdirs to the new encoding to
> ensure consistency when the extension is enabled.
>
> This will simplify our logic and assumptions a lot, at the cost of
> the initial up-front migration.
>
> Will do this in v4 if nobody has any objections.
Let's not.
You have support for submodule.<name>.gitdirpath already, so it is
far safer to use that mechanism to etch-in-stone-fix the existing
submodules and their gitdirs without touching the directories for
migration.
One case you might want to really move directories when migrating is
when two existing submodules' gitdirs are already overlapping, e.g.,
.git/modules/A and .git/modules/A/B are used for submodule A and
submodule A/B. Depending on what "B" is, a project with such a
layout may not be able to upgrade to versions of Git that newly
starts using .git/B directory for a new feature. Introduction of
a new directory or a new file directly underneath $GIT_DIR is rare
but does happen (.git/reftable/ is a relatively recent addition, for
example).
Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-07 15:36 ` Junio C Hamano
@ 2025-10-07 16:58 ` Adrian Ratiu
2025-10-07 17:27 ` Junio C Hamano
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 16:58 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 07 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> Currently any existing submodule gitdir names are left
>> untouched and are used as-is (unencoded) after the extension
>> is enabled.
>
> And in order to make sure that a funny names and paths in
> existing submodules that can be misinterpreted as encoded would
> be registered with the new submodule.<name>.gitdirpath variable?
>
> That would be a robust way to transition. It also means that
> you have a mapping from submodule name to path, and you will
> have to make an effort to maintain that mapping and the reality
> on the filesystem in sync.
>
> So why not take advantage of the fact that you are making that
> effort anyway? It can simplify things quite a bit. Imagine
> what would happen if we did this:
>
> - You officially declare that submodule.<name>.gitdirpath is
> _the_
> mapping mechanism, not a mere "override".
>
> - When enabling the extension, you register all submodules that
> already has their gitdirs on the filesystem to the mapping
> mechanism under their "historical and natural" names.
>
> - When you add a new submodule, you "munge" its name to be used
> as
> a subdirectory name under .git/modules/. You only specify
> for the end users the purpose and nature of this munging,
> perhaps like
>
> - (purpose) We give each submodule a place in .git/modules
> directory of the superproject to store its repository data,
> but the names of submodules we find in .gitmodules may not
> necessarily be friendly to the filesystem (e.g., "CON:", or
> longer than the filesystem allows for a single path
> component), or may introduce a subdirectory (e.g., slashes
> and backslashes), or two directories whose names only
> differ in case may not coexist on your filesystem. That is
> why we do not use the name found in .gitmodules as-is.
>
> - (nature) This "munging" would remove problematic bytes or
> replace them with safe ones in the name, or truncate overly
> long string, or insert sequence numbers to make the result
> unique and representable on the filesystem. Hopefully the
> result of the munging may still be recognisable as derived
> from the original name, but it is *not* designed to be
> reversible by itself. The submodule.<name>.gitdirpath is
> the only and authoritative place to learn how <name> got
> munged to produce a path.
>
> without having to go into the details to avoid tempting users
> to write scripts that guess and decode bypassing the mapping.
>
> Hmm?
Yes, I think that can work. Will do in v4. Thanks!
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-07 14:10 ` Adrian Ratiu
@ 2025-10-07 17:20 ` Junio C Hamano
2025-10-07 17:41 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 17:20 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> I will percent encode the upper case by modifying
> is_rfc3986_unreserved().
Again, do we really need to encode everything? We need to only when
(1) there are other submodules that would collide with us (e.g., by
being only case-different, or by having our name plus a slash as the
prefix of their name, make us overlap with them) or (2) our name is
not filesystem friendly and needs munging. A letter being in an upper
case is not a crime.
>>> For now url-encoding is the only option, however in the future
>>> we may add alternatives (other encodings, hashes or even
>>> hash_name).
>>
>> Let's not say "For now".
>>
>> Choose a single encoding that we can use forever so that we do
>> not have to upgrade extensions.encodeSubmoduleName with suffixes
>> like extensions.encodeSubmoduleNamev2,
>> extensions.encodeSubmoduleNamev3, etc. to cover our earlier
>> mistakes and force renaming on users.
>
> Understood. Will drop that idea.
If we are to consistently use submodule.<name>.gitdirpath as the
authoritative collection of name-path mapping, then the exact
algorithm to derive path from name can be improved over time without
having to worry about an old submodule whose path was computed by
older iteration of the algorithm colliding with a new submodule
whose path is derived by the more modern algorithm. So I do not
mind to make ourselves aware of the possibility that we can tweak
and improve. But as I said elsewhere, I do not think we need to
even say what exactly algorithm is used to the end-users.
Thanks.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-07 16:21 ` Junio C Hamano
@ 2025-10-07 17:21 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 17:21 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 07 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>>> If you already have submodules creted under the original
>>> scheme, then add a new submodule that needs this extension,
>>> do you enable this new extension and write the new submodule
>>> under encoded name, and move the existing submodules under
>>> their encoded names?
>> ... We could do a migration of existing gitdirs to the new
>> encoding to ensure consistency when the extension is enabled.
>>
>> This will simplify our logic and assumptions a lot, at the cost
>> of the initial up-front migration.
>>
>> Will do this in v4 if nobody has any objections.
>
> Let's not.
>
> You have support for submodule.<name>.gitdirpath already, so it
> is far safer to use that mechanism to etch-in-stone-fix the
> existing submodules and their gitdirs without touching the
> directories for migration.
Ack. I'll follow this design path in v4.
I was actually preparing to do the opposite :) i.e. drop the
"override" and do the migration, thanks for clarifying before I
did a re-roll. :)
Will leave this a week or two on the ML in case others want to
review or have more feedback.
>
> One case you might want to really move directories when
> migrating is when two existing submodules' gitdirs are already
> overlapping, e.g., .git/modules/A and .git/modules/A/B are used
> for submodule A and submodule A/B. Depending on what "B" is, a
> project with such a layout may not be able to upgrade to
> versions of Git that newly starts using .git/B directory for a
> new feature. Introduction of a new directory or a new file
> directly underneath $GIT_DIR is rare but does happen
> (.git/reftable/ is a relatively recent addition, for example).
Yes, though I don't think this is a big concern because
submodule.c already has the validate_submodule_git_dir() check
which prevents users from creating overlapping / nested dirs.
When the rare, deliberate clash does happen (like with
.git/reftable) it can be treated like before: do nothing, let the
user fix the repo, or we could add a small one-time migration at
that point in time.
I really like avoiding any kind of automated blanket migration btw,
thank you so much for your feedback Junio, it's really appreciated.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts
2025-10-07 15:36 ` Junio C Hamano
2025-10-07 16:58 ` Adrian Ratiu
@ 2025-10-07 17:27 ` Junio C Hamano
1 sibling, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 17:27 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Junio C Hamano <gitster@pobox.com> writes:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> Currently any existing submodule gitdir names are left untouched
>> and are used as-is (unencoded) after the extension is enabled.
>
> And in order to make sure that a funny names and paths in existing
> submodules that can be misinterpreted as encoded would be registered
> with the new submodule.<name>.gitdirpath variable?
Sorry but this is -ECANNOTPARSE; let me try again.
An existing submodule's gitdir may look like an encoded one (it may
be littered with %xx if urlencode is what you chose), but we do not
want it to be misinterpreted as such. One way to ensure that the
gitdir of the submodule <name> is literally that path is to use
submodule.<name>.gitdirpath to point at the directory, right?
> That would be a robust way to transition.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-07 17:20 ` Junio C Hamano
@ 2025-10-07 17:41 ` Adrian Ratiu
2025-10-07 19:55 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-07 17:41 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 07 Oct 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> I will percent encode the upper case by modifying
>> is_rfc3986_unreserved().
>
> Again, do we really need to encode everything? We need to only
> when (1) there are other submodules that would collide with us
> (e.g., by being only case-different, or by having our name plus
> a slash as the prefix of their name, make us overlap with them)
> or (2) our name is not filesystem friendly and needs munging. A
> letter being in an upper case is not a crime.
Right, I see what you mean now.
The problem becomes how to detect (1) and (2), for example: a
specific name is filesystem-friendly or is case-folding happening
or not?
(ext4 for example can be mounted with casefolding on/off)
This is not a trivial problem, however I think it is solvable. We
can write a few runtime tests to detect things like case-folding,
to decide if encoding is required or not (better ideas welcome).
On the other hand, by always encoding everything we avoid the (1)
(2) detection problems. :)
>
>>>> For now url-encoding is the only option, however in the
>>>> future we may add alternatives (other encodings, hashes or
>>>> even hash_name).
>>> Let's not say "For now". Choose a single encoding that we
>>> can use forever so that we do not have to upgrade
>>> extensions.encodeSubmoduleName with suffixes like
>>> extensions.encodeSubmoduleNamev2,
>>> extensions.encodeSubmoduleNamev3, etc. to cover our earlier
>>> mistakes and force renaming on users.
>>
>> Understood. Will drop that idea.
>
> If we are to consistently use submodule.<name>.gitdirpath as the
> authoritative collection of name-path mapping, then the exact
> algorithm to derive path from name can be improved over time
> without having to worry about an old submodule whose path was
> computed by older iteration of the algorithm colliding with a
> new submodule whose path is derived by the more modern
> algorithm. So I do not mind to make ourselves aware of the
> possibility that we can tweak and improve. But as I said
> elsewhere, I do not think we need to even say what exactly
> algorithm is used to the end-users.
Got it. I'll do the submodule.<name>.gitdirpath centric design in
v4 and will not mention the encoding algorithm to users.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts
2025-10-07 17:41 ` Adrian Ratiu
@ 2025-10-07 19:55 ` Junio C Hamano
0 siblings, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-10-07 19:55 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> The problem becomes how to detect (1) and (2), for example: a
> specific name is filesystem-friendly or is case-folding happening
> or not?
>
> (ext4 for example can be mounted with casefolding on/off)
core.ignorecase is dynamically probed upon reposiory creation, if I
am not mistaken.
But other things like "I have this name, transformed to avoid
slashes and other problematic letters and the result got this long,
would it fit or would I get ENAMETOOLONG?", I think the code may
have to be prepared to try-fail-adjust-retry. We come up with a
proposed "munged" name, try to mkdir() with that name, see that it
fails with ENAMETOOLONG, shorten the munged name and retry. And we
ensure the name we propose is unique among submodule.<name>.gitdirpath
paths in use, then hopefully we find a good name that fits, recognisable
by human, acceptable by the filesystem and unique among submodules.
Even without consulting core.ignorecase, I suspect that the code can
do a similar try-fail-adjust-retry for case-insensitive or NFC/NFD
clashes. We try to mkdir() with a munged name, and if we get
EEXISTS and submodule.<name>.gitdirpath would not have the path we
tried already registered, then we are seeing another path that is
not byte-for-byte identical to ours conflicting, so we can adjust
ours and retry, for example.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-06 11:25 ` [PATCH v3 2/5] submodule: add gitdir path config override Adrian Ratiu
2025-10-06 16:47 ` Junio C Hamano
@ 2025-10-21 8:05 ` Patrick Steinhardt
2025-10-21 11:57 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-10-21 8:05 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Mon, Oct 06, 2025 at 02:25:15PM +0300, Adrian Ratiu wrote:
> diff --git a/submodule.c b/submodule.c
> index 35c55155f7..7a2d7cd592 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2604,6 +2604,18 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> * administrators can explicitly set. Nothing has been decided,
> * so for now, just append the name at the end of the path.
> */
> + char *gitdir_path, *key;
> +
> + /* Allow config override. */
> + key = xstrfmt("submodule.%s.gitdirpath", submodule_name);
> + if (!repo_config_get_string(r, key, &gitdir_path)) {
> + strbuf_addstr(buf, gitdir_path);
> + free(key);
> + free(gitdir_path);
> + return;
> + }
> + free(key);
> +
> repo_git_path_append(r, buf, "modules/");
> strbuf_addstr(buf, submodule_name);
> }
You can use `repo_config_get_string_tmp()` to avoid having to manage the
`gitdir_path` lifetime.
> diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
> new file mode 100644
> index 0000000000..3a83f2d975
> --- /dev/null
> +++ b/t/lib-verify-submodule-gitdir-path.sh
> @@ -0,0 +1,20 @@
> +# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
> +
> +# This does not check filesystem existence. That is done in submodule.c via the
> +# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
> +# might or might not exist (e.g. when adding a new submodule), so this only
> +# checks the expected configuration path, which might be overridden by the user.
> +
> +verify_submodule_gitdir_path() {
> + repo="$1" &&
> + name="$2" &&
> + path="$3" &&
> + (
> + cd "$repo" &&
> + cat >expect <<-EOF &&
> + $(git rev-parse --git-common-dir)/$path
> + EOF
Style nit: we typically don't indent the heredoc body.
> + git submodule--helper gitdir "$name" >actual &&
> + test_cmp expect actual
> + )
> +}
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index fd3e7e355e..11c84a7bdf 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>
> test_expect_success 'setup - enable local submodules' '
> git config --global protocol.file.allow always
> @@ -1505,4 +1506,12 @@ test_expect_success 'submodule add fails when name is reused' '
> )
> '
>
> +test_expect_success 'submodule helper gitdir config overrides' '
> + verify_submodule_gitdir_path test-submodule child modules/child &&
> + test_config -C test-submodule submodule.child.gitdirpath ".git/modules/custom-child" &&
> + verify_submodule_gitdir_path test-submodule child modules/custom-child &&
> + test_unconfig -C test-submodule submodule.child.gitdirpath &&
> + verify_submodule_gitdir_path test-submodule child modules/child
> +'
> +
> test_done
I feel like it's a bit curious that we recognize the configuration even
though "extensions.submodulePath" hasn't been introduced yet. I would
expect that we ignore the config key if that extension is not set, as
the extension otherwise seems to not be doing its job, does it?
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved
2025-10-06 11:25 ` [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
2025-10-06 16:51 ` Junio C Hamano
@ 2025-10-21 8:06 ` Patrick Steinhardt
1 sibling, 0 replies; 179+ messages in thread
From: Patrick Steinhardt @ 2025-10-21 8:06 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, Oct 06, 2025 at 02:25:16PM +0300, Adrian Ratiu wrote:
> diff --git a/strbuf.h b/strbuf.h
> index a580ac6084..5139269039 100644
> --- a/strbuf.h
> +++ b/strbuf.h
> @@ -640,6 +640,8 @@ static inline void strbuf_complete_line(struct strbuf *sb)
>
> typedef int (*char_predicate)(char ch);
>
> +int is_rfc3986_unreserved(char ch);
I think it would help if we had a short comment here explaining what it
does. I doubt that most people immediately go "Ah, RFC3986!". So maybe
explaining in a sentence or two what this is roughly doing would help
them.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-06 11:25 ` [PATCH v3 5/5] submodule: error out if gitdir name is too long Adrian Ratiu
2025-10-06 17:06 ` Junio C Hamano
@ 2025-10-21 8:06 ` Patrick Steinhardt
2025-10-21 13:13 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-10-21 8:06 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, Oct 06, 2025 at 02:25:18PM +0300, Adrian Ratiu wrote:
> diff --git a/compat/posix.h b/compat/posix.h
> index 067a00f33b..aa050fd58c 100644
> --- a/compat/posix.h
> +++ b/compat/posix.h
> @@ -250,6 +250,14 @@ char *gitdirname(char *);
> #define NAME_MAX 255
> #endif
>
> +#ifdef NO_PATHCONF
> +#ifndef _PC_NAME_MAX
> +#define _PC_NAME_MAX 1 /* dummy value, only used for git_pathconf */
Are there platforms that have pathconf(3) but not _PC_NAME_MAX?
> +#endif
> +#define pathconf(a,b) git_pathconf(a,b)
> +long git_pathconf(const char *path, int name);
> +#endif
> +
> typedef uintmax_t timestamp_t;
> #define PRItime PRIuMAX
> #define parse_timestamp strtoumax
Let's adapt this to our coding guidelines to make this easier to parse:
- Nested C preprocessor directives are indented after the hash by one
space per nesting level.
#if FOO
# include <foo.h>
# if BAR
# include <bar.h>
# endif
#endif
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-07 15:41 ` Junio C Hamano
@ 2025-10-21 8:06 ` Patrick Steinhardt
2025-10-21 11:50 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-10-21 8:06 UTC (permalink / raw)
To: Junio C Hamano
Cc: Adrian Ratiu, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, Oct 07, 2025 at 08:41:13AM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> >
> > [jc: brandon removed from CC list as the address would bounce]
> >
> >> This adds the ability to override gitdir paths via config files
> >> (not .gitmodules) such that the encoding scheme (or plain text
> >> name if the encoding extension is disabled) can be changed via
> >> config entries.
> >>
> >> These entries are not added by default for all submodules: they
> >> should be used on an as-needed basis.
> >>
> >> A new test and a helper are added. The helper will also be used
> >> in further tests exercising gitdir encoding functionality.
> >
> > What is the use case of this? The only reasonable use case I can
> > see is to set this to all the existing submodules when you are
> > switching the extension on before adding a new submodule, in which
> > case the old ones will keep using unencoded names, while the new
> > ones will use encoded ones.
>
> Two things.
>
> * I no longer mind this setting existing, but I think it should not
> be a mere "override", but the authoritative source of truth for
> all submodules (see my other response on 0/5).
I think making this the authoritative source of truth for all submodules
in the case where the new extension is enabled does make a ton of sense.
It makes things way easier for us to reason about:
- If the extension is set, we know to always use whatever is in the
gitconfig.
- If any submodule path is missing we know that we are in a broken
repository and can abort accordingly with directions for how to fix
things.
- If we need to enable the extension we can trivially migrate all
existing submodules by just writing their gitdir configuration.
- We can change the exact encoding going forward, as the extension now
only indicates whether or not submodule gitdirs are tracked via the
configuration or encoded "live".
I think especially the last point is a big win, as we are not stuck with
the current encoding schema in case it proves insufficient.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-21 8:06 ` Patrick Steinhardt
@ 2025-10-21 11:50 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-21 11:50 UTC (permalink / raw)
To: Patrick Steinhardt, Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Josh Steadmon, Ben Knoble,
Phillip Wood
On Tue, 21 Oct 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Tue, Oct 07, 2025 at 08:41:13AM -0700, Junio C Hamano wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>> > Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>> >
>> > [jc: brandon removed from CC list as the address would
>> > bounce]
>> >
>> >> This adds the ability to override gitdir paths via config
>> >> files (not .gitmodules) such that the encoding scheme (or
>> >> plain text name if the encoding extension is disabled) can
>> >> be changed via config entries.
>> >>
>> >> These entries are not added by default for all submodules:
>> >> they should be used on an as-needed basis.
>> >>
>> >> A new test and a helper are added. The helper will also be
>> >> used in further tests exercising gitdir encoding
>> >> functionality.
>> >
>> > What is the use case of this? The only reasonable use case I
>> > can see is to set this to all the existing submodules when
>> > you are switching the extension on before adding a new
>> > submodule, in which case the old ones will keep using
>> > unencoded names, while the new ones will use encoded ones.
>> Two things.
>> * I no longer mind this setting existing, but I think it
>> should not
>> be a mere "override", but the authoritative source of truth
>> for all submodules (see my other response on 0/5).
>
> I think making this the authoritative source of truth for all
> submodules in the case where the new extension is enabled does
> make a ton of sense. It makes things way easier for us to
> reason about:
>
> - If the extension is set, we know to always use whatever is
> in the
> gitconfig.
>
> - If any submodule path is missing we know that we are in a
> broken
> repository and can abort accordingly with directions for how
> to fix things.
>
> - If we need to enable the extension we can trivially migrate
> all
> existing submodules by just writing their gitdir
> configuration.
>
> - We can change the exact encoding going forward, as the
> extension now
> only indicates whether or not submodule gitdirs are tracked
> via the configuration or encoded "live".
>
> I think especially the last point is a big win, as we are not
> stuck with the current encoding schema in case it proves
> insufficient.
Agreed, as I also mentioned in replies to Junio, I will follow
this direction in v4. :)
Thanks,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 2/5] submodule: add gitdir path config override
2025-10-21 8:05 ` Patrick Steinhardt
@ 2025-10-21 11:57 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-21 11:57 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Tue, 21 Oct 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Mon, Oct 06, 2025 at 02:25:15PM +0300, Adrian Ratiu wrote:
>> diff --git a/submodule.c b/submodule.c index
>> 35c55155f7..7a2d7cd592 100644 --- a/submodule.c +++
>> b/submodule.c @@ -2604,6 +2604,18 @@ void
>> submodule_name_to_gitdir(struct strbuf *buf, struct repository
>> *r,
>> * administrators can explicitly set. Nothing has been
>> decided, * so for now, just append the name at the end of the
>> path. */
>> + char *gitdir_path, *key; + + /* Allow config
>> override. */ + key = xstrfmt("submodule.%s.gitdirpath",
>> submodule_name); + if (!repo_config_get_string(r, key,
>> &gitdir_path)) { + strbuf_addstr(buf, gitdir_path); +
>> free(key); + free(gitdir_path); + return; +
>> } + free(key); +
>> repo_git_path_append(r, buf, "modules/");
>> strbuf_addstr(buf, submodule_name); }
>
> You can use `repo_config_get_string_tmp()` to avoid having to
> manage the `gitdir_path` lifetime.
Oh, sweet, I wasn't aware of that. Will do, thanks!
>> diff --git a/t/lib-verify-submodule-gitdir-path.sh
>> b/t/lib-verify-submodule-gitdir-path.sh new file mode 100644
>> index 0000000000..3a83f2d975 --- /dev/null +++
>> b/t/lib-verify-submodule-gitdir-path.sh @@ -0,0 +1,20 @@ +#
>> Helper to verify if repo $1 contains a submodule named $2 with
>> gitdir path $3 + +# This does not check filesystem
>> existence. That is done in submodule.c via the +#
>> submodule_name_to_gitdir() API which this helper ends up
>> calling. The gitdirs +# might or might not exist (e.g. when
>> adding a new submodule), so this only +# checks the expected
>> configuration path, which might be overridden by the user. +
>> +verify_submodule_gitdir_path() { + repo="$1" && + name="$2"
>> && + path="$3" && + ( + cd "$repo" && +
>> cat >expect <<-EOF && + $(git rev-parse
>> --git-common-dir)/$path + EOF
>
> Style nit: we typically don't indent the heredoc body.
Ack, will fix in v4.
>> + git submodule--helper gitdir "$name" >actual && +
>> test_cmp expect actual + ) +} diff --git
>> a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index
>> fd3e7e355e..11c84a7bdf 100755 --- a/t/t7400-submodule-basic.sh
>> +++ b/t/t7400-submodule-basic.sh @@ -13,6 +13,7 @@
>> GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
>> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh
>> +. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
>> test_expect_success 'setup - enable local submodules' ' git
>> config --global protocol.file.allow always
>> @@ -1505,4 +1506,12 @@ test_expect_success 'submodule add fails
>> when name is reused' '
>> ) '
>> +test_expect_success 'submodule helper gitdir config overrides'
>> ' + verify_submodule_gitdir_path test-submodule child
>> modules/child && + test_config -C test-submodule
>> submodule.child.gitdirpath ".git/modules/custom-child" && +
>> verify_submodule_gitdir_path test-submodule child
>> modules/custom-child && + test_unconfig -C test-submodule
>> submodule.child.gitdirpath && +
>> verify_submodule_gitdir_path test-submodule child modules/child
>> +' +
>> test_done
>
> I feel like it's a bit curious that we recognize the
> configuration even though "extensions.submodulePath" hasn't been
> introduced yet. I would expect that we ignore the config key if
> that extension is not set, as the extension otherwise seems to
> not be doing its job, does it?
Good point.
In v3 the config wasn't as tied to the extension because it was
just an optional override, however since we'll make it the
centerpiece of the design, this 100% makes sense.
I will reorder the commits & rework this logic in v4 as you and
Junio suggested.
Many thanks,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v3 5/5] submodule: error out if gitdir name is too long
2025-10-21 8:06 ` Patrick Steinhardt
@ 2025-10-21 13:13 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-10-21 13:13 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 21 Oct 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Mon, Oct 06, 2025 at 02:25:18PM +0300, Adrian Ratiu wrote:
>> diff --git a/compat/posix.h b/compat/posix.h index
>> 067a00f33b..aa050fd58c 100644 --- a/compat/posix.h +++
>> b/compat/posix.h @@ -250,6 +250,14 @@ char *gitdirname(char *);
>> #define NAME_MAX 255 #endif
>> +#ifdef NO_PATHCONF +#ifndef _PC_NAME_MAX +#define _PC_NAME_MAX
>> 1 /* dummy value, only used for git_pathconf */
>
> Are there platforms that have pathconf(3) but not _PC_NAME_MAX?
AFAIK no, because they're both part of POSIX and in all known
implementations they are both defined.
>
>> +#endif +#define pathconf(a,b) git_pathconf(a,b) +long
>> git_pathconf(const char *path, int name); +#endif +
>> typedef uintmax_t timestamp_t; #define PRItime PRIuMAX #define
>> parse_timestamp strtoumax
>
> Let's adapt this to our coding guidelines to make this easier to
> parse:
>
> - Nested C preprocessor directives are indented after the hash
> by one
> space per nesting level.
>
> #if FOO # include <foo.h> # if BAR # include <bar.h> #
> endif #endif
Thanks, however I'm inclining towards dropping this commit in v4
as Junio and Peff suggested, since it doesn't add anything other
than the nice error message, so we can let each OS fail with its
own message like they did before this patch.
We could bring it back when/if we decide to implement sharding /
trimming, i.e. to address the TODO in the patch, that way it's more
useful.
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (11 preceding siblings ...)
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
@ 2025-11-07 15:05 ` Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 1/4] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (4 more replies)
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
` (2 subsequent siblings)
15 siblings, 5 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-07 15:05 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Hello everyone,
For those new to this series, we are adding an extension to encode submodule
gitdir paths to avoid filesystem conflicts.
v4 continues with the simplifications started in v3, based on reviewer feedback
(many thanks to everyone, especially to Junio and Patrick).
The biggest design change in v4 is that config submodule.<name>.gitdir becomes
mandatory when extensions.submoduleEncoding is enabled, so instead of being just
an optional override like in previous iterations, in v4 it's the centerpiece of
design, the single point of truth from which submodule gitdir paths are located.
This allows users to not care about the specific encoding being used and also
allows git to improveme the encoding implementation over time: everyone just gets
the submodule gitdirs from the config after the extension is enabled.
Another important change is that we do not encode everything by default when the
extension is enabled: submodule_name_to_gitdir() validates multiple candidates
and picks the best one and encoding only when required.
As always this is based on latest master branch, I've merged into next/seen for
any conflicts, pushed to GitHub [1] and ran the CI with all tests passing [2].
I also added a rangediff between v3 and v4, which might help some reviewers,
though it's rather big because we changed the config/encoding design like I
mentioned above so it might be easier to give it a full review.
Changes between v3 -> v4:
* Replaced bmwill@google.com -> bwilliams.eng@gmail.com (Kristoffer)
* Made the gitdir config the authoritative source of truth for all submodule
git dirs when the extension is enabled (Junio, Patrick)
* Replaced the "encode everything by default" design with a stepwise/retry
validation in submodule_name_to_gitdir() to pick the best canditate (Junio)
* Moved is_rfc3986_unreserved() to url.[ch] instead of strbuf (Junio)
* Fixed a parallel job execution bug I introduced with the extension in v3 (Adrian)
* Improved lib-verify-submodule-gitdir-path.sh to handle relative/abs paths (Adrian)
* Fixed submodule.<name>.gitdir documentation vs code mismatch (Junio)
* Defined the config together with the extension by squashing commits (Patrick)
* Removed the A -> _a, B -> _b custom encoding in favor of a simplified percent
encodin which is only done when case-folding to allow uppercase chars (Junio)
* Dropped patch with pathconf wrapper, as it's not required for basic encoding,
it can be added back if we implement sharding (Junio, Peff, Patrick)
* Used repo_config_get_string_tmp to avoid a char* free call (Patrick)
* Added comment to document is_rfc3986_unreserved() (Patrick)
* Fixed heredoc body indentation (Patrick)
* Fixed trivial doc markup conflict with upstream commit (Adrian)
* Reworded multiple commits for clarity (Junio)
Please let me know if you have any questions or other feedback.
Thank you,
Adrian
[1] https://github.com/10ne1/git/tree/dev/aratiu/encoding-v4
[2] https://github.com/10ne1/git/actions/runs/19170390360
Range-diff against v3:
1: b2e317a8f6 ! 1: 7d34507692 submodule--helper: use submodule_name_to_gitdir in add_submodule
@@ Commit message
submodule--helper: use submodule_name_to_gitdir in add_submodule
While testing submodule gitdir path encoding, I noticed submodule--helper
- is still using a hardcoded name-based path leading to test failures, so
- convert it to the common helper function introduced by commit ce125d431a
- (submodule: extract path to submodule gitdir func, 2021-09-15) and used
- in other locations across the source tree.
+ is still using a hardcoded modules gitdir path leading to test failures.
+
+ Call the submodule_name_to_gitdir() helper instead, which was invented
+ exactly for this purpose and is already used by all the other locations
+ which work on gitdirs.
+
+ Also narrow the scope of the submod_gitdir_path variable which is not
+ used anymore in the updated "else" branch.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
2: 17838ad13f < -: ---------- submodule: add gitdir path config override
3: 4fc7020f26 < -: ---------- strbuf: bring back is_rfc3986_unreserved
-: ---------- > 2: 1e609bdd1a builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
4: dc6d5069ff ! 3: c51fd58669 submodule: encode gitdir paths to avoid conflicts
@@ Metadata
Author: Adrian Ratiu <adrian.ratiu@collabora.com>
## Commit message ##
- submodule: encode gitdir paths to avoid conflicts
+ submodule: add extension to encode gitdir paths
- This adds a new submoduleEncoding extension which encodes gitdir names
- to avoid collisions due to nested gitdirs or case insensitive filesystems.
+ Add a submoduleEncoding extension which fixes filesystem collisions by
+ encoding gitdir paths. At a high level, this implements a mechanism to
+ encode -> validate -> retry until a working gitdir path is found.
- A custom encoding can become unnecessarily complex, while url-encoding is
- relatively well-known, however it needs some extending to support case
- insensitive filesystems, hence why A is encoded as _a, B as _b and so on.
+ Credit goes to Junio for coming up with this design: encoding is only
+ applied when necessary, e.g. uppercase characters are encoded only on
+ case-folding filesystems and only if a real conflict is detected.
- Unfortunately encoding A -> _a (...) is not enough to fix the reserved
- Windows file names (e.g. COM1) because worktrees still use names like COM1
- even if the gitdirs paths are encoded, so future work is needed to fully
- address Windows reserved names.
+ To make this work, we rely on the submodule.<name>.gitdir config as the
+ single source of truth for gitidir paths: the config is always set when
+ the extension is enabled. Users who care about gitdir paths are expected
+ to get/set the config and not the underlying encoding implementation.
- For now url-encoding is the only option, however in the future we may
- add alternatives (other encodings, hashes or even hash_name).
+ This commit adds the basic encoding logic which addresses nested gitdirs.
+ The next commit fixes case-folding, the next commit fixes names longer
+ than NAME_MAX. The idea is the encoding can be improved over time in a
+ way which is transparent to users.
+ Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
+ Based-on-patch-by: Brandon Williams <bwilliams.eng@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
## Documentation/config/extensions.adoc ##
-@@ Documentation/config/extensions.adoc: relativeWorktrees::
+@@ Documentation/config/extensions.adoc: relativeWorktrees:::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
-+submoduleEncoding::
++submoduleEncoding:::
+ If enabled, submodule gitdir paths are encoded to avoid filesystem
-+ conflicts due to nested gitdirs or case insensitivity. For now, only
-+ url-encoding (rfc3986) is available, with a small addition to encode
-+ uppercase to lowercase letters (`A -> _a`, `B -> _b` and so on).
-+ Other encoding or hashing methods may be added in the future.
-+ Any preexisting non-encoded submodule gitdirs are used as-is, to
-+ ease migration and reduce risk of gitdirs not being recognized.
-+
- worktreeConfig::
++ conflicts due to nested gitdirs, case insensitivity or other issues
++ When enabled, the submodule.<name>.gitdir config is always set for
++ all submodulesand is the single point of authority for gitdir paths.
++
+ worktreeConfig:::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
## Documentation/config/submodule.adoc ##
@@ Documentation/config/submodule.adoc: submodule.<name>.active::
- submodule.<name>.gitdir::
- This option sets the gitdir path for submodule <name>, allowing users
- to override the default path or change the default path name encoding.
-+ Submodule gitdir encoding is enabled via `extensions.submoduleEncoding`
-+ (see linkgit:git-config[1]). This config works both with the extension
-+ enabled or disabled.
+ submodule.active config option. See linkgit:gitsubmodules[7] for
+ details.
++submodule.<name>.gitdir::
++ This option sets the gitdir path for submodule <name>, allowing users to
++ override the default path. Only works when `extensions.submoduleEncoding`
++ is enabled, otherwise does nothing. See linkgit:git-config[1] for details.
++
submodule.active::
A repeated field which contains a pathspec used to match against a
+ submodule's path to determine if the submodule is of interest to git
+
+ ## builtin/submodule--helper.c ##
+@@ builtin/submodule--helper.c: static int module_summary(int argc, const char **argv, const char *prefix,
+ return ret;
+ }
+
++static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
++ struct repository *repo)
++{
++ struct strbuf gitdir = STRBUF_INIT;
++
++ if (argc != 2)
++ usage(_("git submodule--helper gitdir <name>"));
++
++ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
++
++ printf("%s\n", gitdir.buf);
++
++ strbuf_release(&gitdir);
++ return 0;
++}
++
+ struct sync_cb {
+ const char *prefix;
+ const char *super_prefix;
+@@ builtin/submodule--helper.c: int cmd_submodule__helper(int argc,
+ NULL
+ };
+ struct option options[] = {
++ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
+ OPT_SUBCOMMAND("clone", &fn, module_clone),
+ OPT_SUBCOMMAND("add", &fn, module_add),
+ OPT_SUBCOMMAND("update", &fn, module_update),
+
+ ## repository.c ##
+@@ repository.c: int repo_init(struct repository *repo,
+ repo->repository_format_worktree_config = format.worktree_config;
+ repo->repository_format_relative_worktrees = format.relative_worktrees;
+ repo->repository_format_precious_objects = format.precious_objects;
++ repo->repository_format_submodule_encoding = format.submodule_encoding;
+
+ /* take ownership of format.partial_clone */
+ repo->repository_format_partial_clone = format.partial_clone;
## repository.h ##
@@ repository.h: struct repository {
@@ setup.h: struct repository_format {
int compat_hash_algo;
## submodule.c ##
-@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
- char *p;
+@@
+ #include "commit-reach.h"
+ #include "read-cache-ll.h"
+ #include "setup.h"
++#include "url.h"
+
+ static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
+ static int initialized_fetch_ref_tips;
+@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
+ return ret;
+ }
+
++/*
++ * Find the last submodule name in the gitdir path (modules can be nested).
++ * Returns a pointer into `path` to the beginning of the name or NULL if not found.
++ */
++static char *find_last_submodule_name(char *git_dir_path)
++{
++ const char *modules_marker = "/modules/";
++ char *p = git_dir_path;
++ char *last = NULL;
++
++ while ((p = strstr(p, modules_marker))) {
++ last = p + strlen(modules_marker);
++ p++;
++ }
++
++ return last;
++}
++
+ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+ {
+ size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
+- char *p;
++ char *p = git_dir + len - suffix_len;
++ bool suffixes_match = !strcmp(p, submodule_name);
int ret = 0;
-+ /*
-+ * Skip these checks when extensions.submoduleEncoding is enabled because
-+ * it fixes the nesting issues and the suffixes will not match by design.
-+ */
-+ if (the_repository->repository_format_submodule_encoding)
+- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
+- strcmp(p, submodule_name))
+- BUG("submodule name '%s' not a suffix of git dir '%s'",
+- submodule_name, git_dir);
+-
+ /*
+ * We prevent the contents of sibling submodules' git directories to
+ * clash.
+@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+ * but the latter directory is already designated to contain the hooks
+ * of the former.
+ */
+- for (; *p; p++) {
++ for (; *p && suffixes_match; p++) {
+ if (is_dir_sep(*p)) {
+ char c = *p;
+
+@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+ }
+ }
+
++ /* tests after this check are only for encoded names, when the extension is enabled */
++ if (!the_repository->repository_format_submodule_encoding)
+ return 0;
+
- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
- strcmp(p, submodule_name))
- BUG("submodule name '%s' not a suffix of git dir '%s'",
++ /* Prevent the use of '/' in names */
++ p = find_last_submodule_name(git_dir);
++ if (p && strchr(p, '/') != NULL)
++ return error("submodule gitdir name '%s' contains unexpected '/'", p);
++
+ return 0;
+ }
+
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
return ret;
}
-+static void strbuf_addstr_case_encode(struct strbuf *dst, const char *src)
++static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
++ const char *submodule_name)
+{
-+ for (; *src; src++) {
-+ unsigned char c = *src;
-+ if (c >= 'A' && c <= 'Z') {
-+ strbuf_addch(dst, '_');
-+ strbuf_addch(dst, c - 'A' + 'a');
-+ } else {
-+ strbuf_addch(dst, c);
-+ }
-+ }
++ char *key;
++
++ if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
++ return -1;
++
++ key = xstrfmt("submodule.%s.gitdir", submodule_name);
++ repo_config_set_gently(the_repository, key, gitdir_path->buf);
++ FREE_AND_NULL(key);
++
++ return 0;
++
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
-- /*
++ const char *gitdir;
++ char *key;
++
++ repo_git_path_append(r, buf, "modules/");
++ strbuf_addstr(buf, submodule_name);
++
++ /* If extensions.submoduleEncoding is disabled, use the plain path set above */
++ if (!r->repository_format_submodule_encoding)
++ return;
++
++ /* Extension is enabled: use the gitdir config if it exists */
++ key = xstrfmt("submodule.%s.gitdir", submodule_name);
++ if (!repo_config_get_string_tmp(r, key, &gitdir)) {
++ strbuf_reset(buf);
++ strbuf_addstr(buf, gitdir);
++ FREE_AND_NULL(key);
++ return;
++ }
++ FREE_AND_NULL(key);
++
+ /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
-- */
- char *gitdir_path, *key;
-
- /* Allow config override. */
-@@ submodule.c: void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
-
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
++ * The gitdir config does not exist, even though the extension is enabled.
++ * Therefore we are in one of the following cases:
+ */
+
-+ /* Existing legacy non-encoded names are used as-is */
-+ if (is_git_directory(buf->buf))
++ /* Case 1: legacy migration of valid plain submodule names */
++ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
-+ if (the_repository->repository_format_submodule_encoding) {
-+ struct strbuf tmp = STRBUF_INIT;
++ /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
++ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+- strbuf_addstr(buf, submodule_name);
++ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
++ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
++ return;
+
-+ strbuf_reset(buf);
-+ repo_git_path_append(r, buf, "modules/");
++ /* Case 3: error out */
++ die(_("Cannot construct a valid gitdir path for submodule '%s': "
++ "please set a unique git config for 'submodule.%s.gitdir'."),
++ submodule_name, submodule_name);
+ }
+
+ ## t/lib-verify-submodule-gitdir-path.sh (new) ##
+@@
++# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
-+ strbuf_addstr_urlencode(&tmp, submodule_name, is_rfc3986_unreserved);
-+ strbuf_addstr_case_encode(buf, tmp.buf);
++# This does not check filesystem existence. That is done in submodule.c via the
++# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
++# might or might not exist (e.g. when adding a new submodule), so this only
++# checks the expected configuration path, which might be overridden by the user.
+
-+ strbuf_release(&tmp);
-+ }
- }
++verify_submodule_gitdir_path() {
++ repo="$1" &&
++ name="$2" &&
++ path="$3" &&
++ (
++ cd "$repo" &&
++ # Compute expected absolute path
++ expected="$(git rev-parse --git-common-dir)/$path" &&
++ expected="$(test-tool path-utils real_path "$expected")" &&
++ # Compute actual absolute path
++ actual="$(git submodule--helper gitdir "$name")" &&
++ actual="$(test-tool path-utils real_path "$actual")" &&
++ echo "$expected" >expect &&
++ echo "$actual" >actual &&
++ test_cmp expect actual
++ )
++}
## t/meson.build ##
@@ t/meson.build: integration_tests = [
@@ t/t7425-submodule-encoding.sh (new)
+
+test_expect_success 'verify submodule name is properly encoded' '
+ verify_submodule_gitdir_path main legacy modules/legacy &&
-+ verify_submodule_gitdir_path main "New Sub" modules/_new%20_sub
++ verify_submodule_gitdir_path main "New Sub" "modules/New Sub"
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
@@ t/t7425-submodule-encoding.sh (new)
+ cd cloned-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
-+ test_path_is_dir .git/modules/_new%20_sub &&
++ test_path_is_dir ".git/modules/New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
@@ t/t7425-submodule-encoding.sh (new)
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
-+ test_path_is_dir .git/modules/_new%20_sub &&
++ test_path_is_dir ".git/modules/New Sub" &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
@@ t/t7425-submodule-encoding.sh (new)
+'
+
+test_done
+
+ ## t/t9902-completion.sh ##
+@@ t/t9902-completion.sh: test_expect_success 'git config set - variable name - __git_compute_second_level
+ submodule.sub.fetchRecurseSubmodules Z
+ submodule.sub.ignore Z
+ submodule.sub.active Z
++ submodule.sub.gitdir Z
+ EOF
+ '
+
5: 775cf131bf < -: ---------- submodule: error out if gitdir name is too long
-: ---------- > 4: 01a5b10d5a submodule: fix case-folding gitdir filesystem colisions
Adrian Ratiu (4):
submodule--helper: use submodule_name_to_gitdir in add_submodule
builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
submodule: add extension to encode gitdir paths
submodule: fix case-folding gitdir filesystem colisions
Documentation/config/extensions.adoc | 6 +
Documentation/config/submodule.adoc | 5 +
builtin/credential-store.c | 7 +-
builtin/submodule--helper.c | 30 ++++-
repository.c | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 155 ++++++++++++++++++++-----
t/lib-verify-submodule-gitdir-path.sh | 24 ++++
t/meson.build | 1 +
t/t7425-submodule-encoding.sh | 161 ++++++++++++++++++++++++++
t/t9902-completion.sh | 1 +
url.c | 23 ++++
url.h | 3 +
15 files changed, 387 insertions(+), 39 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-encoding.sh
--
2.51.0
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v4 1/4] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
@ 2025-11-07 15:05 ` Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 2/4] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
` (3 subsequent siblings)
4 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-07 15:05 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded modules gitdir path leading to test failures.
Call the submodule_name_to_gitdir() helper instead, which was invented
exactly for this purpose and is already used by all the other locations
which work on gitdirs.
Also narrow the scope of the submod_gitdir_path variable which is not
used anymore in the updated "else" branch.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index fcd73abe53..2873b2780e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3187,13 +3187,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3207,10 +3207,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3219,8 +3220,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3238,7 +3239,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v4 2/4] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 1/4] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-11-07 15:05 ` Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 3/4] submodule: add extension to encode gitdir paths Adrian Ratiu
` (2 subsequent siblings)
4 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-07 15:05 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it was the only place using it.
However now we need it to apply URL-encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back as a
public function exposed via url.h, instead of the old helper path
(strbuf), which has nothing to do with 3986 encoding/decoding anymore.
This function will be used by submodule.c in the next commit.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 7 +------
url.c | 11 +++++++++++
url.h | 2 ++
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..bc1453c6b2 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -7,6 +7,7 @@
#include "path.h"
#include "string-list.h"
#include "parse-options.h"
+#include "url.h"
#include "write-or-die.h"
static struct lock_file credential_lock;
@@ -76,12 +77,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/url.c b/url.c
index 282b12495a..0fb1859b28 100644
--- a/url.c
+++ b/url.c
@@ -3,6 +3,17 @@
#include "strbuf.h"
#include "url.h"
+/*
+ * The set of unreserved characters as per STD66 (RFC3986) is
+ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
+ * components without percent-encoding.
+ */
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 2a27c34277..131a262066 100644
--- a/url.h
+++ b/url.h
@@ -21,4 +21,6 @@ char *url_decode_parameter_value(const char **query);
void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
+int is_rfc3986_unreserved(char ch);
+
#endif /* URL_H */
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v4 3/4] submodule: add extension to encode gitdir paths
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 1/4] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 2/4] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
@ 2025-11-07 15:05 ` Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
2025-11-14 23:03 ` [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts Josh Steadmon
4 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-07 15:05 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu,
Brandon Williams
Add a submoduleEncoding extension which fixes filesystem collisions by
encoding gitdir paths. At a high level, this implements a mechanism to
encode -> validate -> retry until a working gitdir path is found.
Credit goes to Junio for coming up with this design: encoding is only
applied when necessary, e.g. uppercase characters are encoded only on
case-folding filesystems and only if a real conflict is detected.
To make this work, we rely on the submodule.<name>.gitdir config as the
single source of truth for gitidir paths: the config is always set when
the extension is enabled. Users who care about gitdir paths are expected
to get/set the config and not the underlying encoding implementation.
This commit adds the basic encoding logic which addresses nested gitdirs.
The next commit fixes case-folding, the next commit fixes names longer
than NAME_MAX. The idea is the encoding can be improved over time in a
way which is transparent to users.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Based-on-patch-by: Brandon Williams <bwilliams.eng@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 6 ++
Documentation/config/submodule.adoc | 5 +
builtin/submodule--helper.c | 17 +++
repository.c | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 110 ++++++++++++++-----
t/lib-verify-submodule-gitdir-path.sh | 24 +++++
t/meson.build | 1 +
t/t7425-submodule-encoding.sh | 146 ++++++++++++++++++++++++++
t/t9902-completion.sh | 1 +
12 files changed, 294 insertions(+), 26 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-encoding.sh
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 532456644b..e33040fff5 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -73,6 +73,12 @@ relativeWorktrees:::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
+submoduleEncoding:::
+ If enabled, submodule gitdir paths are encoded to avoid filesystem
+ conflicts due to nested gitdirs, case insensitivity or other issues
+ When enabled, the submodule.<name>.gitdir config is always set for
+ all submodulesand is the single point of authority for gitdir paths.
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..ddaadc3dc5 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,11 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This option sets the gitdir path for submodule <name>, allowing users to
+ override the default path. Only works when `extensions.submoduleEncoding`
+ is enabled, otherwise does nothing. See linkgit:git-config[1] for details.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2873b2780e..abd20eee53 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1208,6 +1208,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3591,6 +3607,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
diff --git a/repository.c b/repository.c
index 6faf5c7398..26a21c0d71 100644
--- a/repository.c
+++ b/repository.c
@@ -288,6 +288,7 @@ int repo_init(struct repository *repo,
repo->repository_format_worktree_config = format.worktree_config;
repo->repository_format_relative_worktrees = format.relative_worktrees;
repo->repository_format_precious_objects = format.precious_objects;
+ repo->repository_format_submodule_encoding = format.submodule_encoding;
/* take ownership of format.partial_clone */
repo->repository_format_partial_clone = format.partial_clone;
diff --git a/repository.h b/repository.h
index 5808a5d610..7e39b2acf7 100644
--- a/repository.h
+++ b/repository.h
@@ -158,6 +158,7 @@ struct repository {
int repository_format_worktree_config;
int repository_format_relative_worktrees;
int repository_format_precious_objects;
+ int repository_format_submodule_encoding;
/* Indicate if a repository has a different 'commondir' from 'gitdir' */
unsigned different_commondir:1;
diff --git a/setup.c b/setup.c
index 7086741e6c..bf6e815105 100644
--- a/setup.c
+++ b/setup.c
@@ -687,6 +687,9 @@ static enum extension_result handle_extension(const char *var,
} else if (!strcmp(ext, "relativeworktrees")) {
data->relative_worktrees = git_config_bool(var, value);
return EXTENSION_OK;
+ } else if (!strcmp(ext, "submoduleencoding")) {
+ data->submodule_encoding = git_config_bool(var, value);
+ return EXTENSION_OK;
}
return EXTENSION_UNKNOWN;
}
@@ -1865,6 +1868,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
repo_fmt.worktree_config;
the_repository->repository_format_relative_worktrees =
repo_fmt.relative_worktrees;
+ the_repository->repository_format_submodule_encoding =
+ repo_fmt.submodule_encoding;
/* take ownership of repo_fmt.partial_clone */
the_repository->repository_format_partial_clone =
repo_fmt.partial_clone;
@@ -1963,6 +1968,8 @@ void check_repository_format(struct repository_format *fmt)
fmt->ref_storage_format);
the_repository->repository_format_worktree_config =
fmt->worktree_config;
+ the_repository->repository_format_submodule_encoding =
+ fmt->submodule_encoding;
the_repository->repository_format_relative_worktrees =
fmt->relative_worktrees;
the_repository->repository_format_partial_clone =
diff --git a/setup.h b/setup.h
index 8522fa8575..66ec1ceba5 100644
--- a/setup.h
+++ b/setup.h
@@ -130,6 +130,7 @@ struct repository_format {
char *partial_clone; /* value of extensions.partialclone */
int worktree_config;
int relative_worktrees;
+ int submodule_encoding;
int is_bare;
int hash_algo;
int compat_hash_algo;
diff --git a/submodule.c b/submodule.c
index 35c55155f7..ceaff0c1aa 100644
--- a/submodule.c
+++ b/submodule.c
@@ -31,6 +31,7 @@
#include "commit-reach.h"
#include "read-cache-ll.h"
#include "setup.h"
+#include "url.h"
static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
static int initialized_fetch_ref_tips;
@@ -2256,17 +2257,31 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
+/*
+ * Find the last submodule name in the gitdir path (modules can be nested).
+ * Returns a pointer into `path` to the beginning of the name or NULL if not found.
+ */
+static char *find_last_submodule_name(char *git_dir_path)
+{
+ const char *modules_marker = "/modules/";
+ char *p = git_dir_path;
+ char *last = NULL;
+
+ while ((p = strstr(p, modules_marker))) {
+ last = p + strlen(modules_marker);
+ p++;
+ }
+
+ return last;
+}
+
int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
- char *p;
+ char *p = git_dir + len - suffix_len;
+ bool suffixes_match = !strcmp(p, submodule_name);
int ret = 0;
- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
- strcmp(p, submodule_name))
- BUG("submodule name '%s' not a suffix of git dir '%s'",
- submodule_name, git_dir);
-
/*
* We prevent the contents of sibling submodules' git directories to
* clash.
@@ -2277,7 +2292,7 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
* but the latter directory is already designated to contain the hooks
* of the former.
*/
- for (; *p; p++) {
+ for (; *p && suffixes_match; p++) {
if (is_dir_sep(*p)) {
char c = *p;
@@ -2294,6 +2309,15 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
}
}
+ /* tests after this check are only for encoded names, when the extension is enabled */
+ if (!the_repository->repository_format_submodule_encoding)
+ return 0;
+
+ /* Prevent the use of '/' in names */
+ p = find_last_submodule_name(git_dir);
+ if (p && strchr(p, '/') != NULL)
+ return error("submodule gitdir name '%s' contains unexpected '/'", p);
+
return 0;
}
@@ -2581,29 +2605,63 @@ int submodule_to_gitdir(struct repository *repo,
return ret;
}
+static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
+ const char *submodule_name)
+{
+ char *key;
+
+ if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
+ return -1;
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ repo_config_set_gently(the_repository, key, gitdir_path->buf);
+ FREE_AND_NULL(key);
+
+ return 0;
+
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
+ const char *gitdir;
+ char *key;
+
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr(buf, submodule_name);
+
+ /* If extensions.submoduleEncoding is disabled, use the plain path set above */
+ if (!r->repository_format_submodule_encoding)
+ return;
+
+ /* Extension is enabled: use the gitdir config if it exists */
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ if (!repo_config_get_string_tmp(r, key, &gitdir)) {
+ strbuf_reset(buf);
+ strbuf_addstr(buf, gitdir);
+ FREE_AND_NULL(key);
+ return;
+ }
+ FREE_AND_NULL(key);
+
/*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
+ * The gitdir config does not exist, even though the extension is enabled.
+ * Therefore we are in one of the following cases:
*/
+
+ /* Case 1: legacy migration of valid plain submodule names */
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
+ /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ strbuf_reset(buf);
repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
+ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
+ /* Case 3: error out */
+ die(_("Cannot construct a valid gitdir path for submodule '%s': "
+ "please set a unique git config for 'submodule.%s.gitdir'."),
+ submodule_name, submodule_name);
}
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..62794df976
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,24 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (e.g. when adding a new submodule), so this only
+# checks the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ # Compute expected absolute path
+ expected="$(git rev-parse --git-common-dir)/$path" &&
+ expected="$(test-tool path-utils real_path "$expected")" &&
+ # Compute actual absolute path
+ actual="$(git submodule--helper gitdir "$name")" &&
+ actual="$(test-tool path-utils real_path "$actual")" &&
+ echo "$expected" >expect &&
+ echo "$actual" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/meson.build b/t/meson.build
index a5531df415..4187b35aee 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -884,6 +884,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-encoding.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
new file mode 100755
index 0000000000..a42d358f5b
--- /dev/null
+++ b/t/t7425-submodule-encoding.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+
+test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed encoded and non-encoded submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submoduleEncoding true &&
+
+ git submodule add ../new-sub "New Sub" &&
+ test_commit new
+ )
+'
+
+test_expect_success 'verify submodule name is properly encoded' '
+ verify_submodule_gitdir_path main legacy modules/legacy &&
+ verify_submodule_gitdir_path main "New Sub" "modules/New Sub"
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned-non-encoding &&
+ (
+ cd cloned-non-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/"New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ ) &&
+
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules main cloned-encoding &&
+ (
+ cd cloned-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'commit and push changes to encoded submodules' '
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ git -C main config receive.denyCurrentBranch updateInstead &&
+ (
+ cd cloned-encoding &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C "New Sub" switch --track -C main origin/main &&
+ test_commit -C "New Sub" second-commit &&
+ git -C "New Sub" push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy "New Sub" &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 964e1f1569..ffb9c8b522 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
` (2 preceding siblings ...)
2025-11-07 15:05 ` [PATCH v4 3/4] submodule: add extension to encode gitdir paths Adrian Ratiu
@ 2025-11-07 15:05 ` Adrian Ratiu
2025-11-08 18:20 ` Aaron Schrab
2025-11-12 15:28 ` Adrian Ratiu
2025-11-14 23:03 ` [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts Josh Steadmon
4 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-07 15:05 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Add a new check in validate_submodule_git_dir() to detect and
prevent case-folding filesystem colisions. When this new check
is triggered, a stricter casefolding aware URI encoding is used
to percent-encode uppercase characters, e.g. Foo becomes %46oo.
By using this check/retry mechanism the uppercase encoding is
only applied when necessary, so case-sensitive filesystems are
not affected.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 47 +++++++++++++++++++++++++++++++++--
t/t7425-submodule-encoding.sh | 15 +++++++++++
url.c | 12 +++++++++
url.h | 1 +
4 files changed, 73 insertions(+), 2 deletions(-)
diff --git a/submodule.c b/submodule.c
index ceaff0c1aa..ecbffac2c6 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2280,7 +2280,7 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
char *p = git_dir + len - suffix_len;
bool suffixes_match = !strcmp(p, submodule_name);
- int ret = 0;
+ int ret = 0, config_ignorecase = 0;
/*
* We prevent the contents of sibling submodules' git directories to
@@ -2318,6 +2318,42 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
if (p && strchr(p, '/') != NULL)
return error("submodule gitdir name '%s' contains unexpected '/'", p);
+ /* Prevent conflicts on case-folding filesystems */
+ repo_config_get_bool(the_repository, "core.ignorecase", &config_ignorecase);
+ if (ignore_case || config_ignorecase) {
+ char *lower_gitdir = xstrdup(git_dir);
+ char *module_name = find_last_submodule_name(lower_gitdir);
+
+ if (module_name) {
+ for (p = module_name; *p; p++)
+ *p = tolower(*p);
+
+ /*
+ * If lower path is different and already exists, check for collision.
+ * Intentionally double-check to eliminate false-positives.
+ */
+ if (strcmp(lower_gitdir, git_dir) && is_git_directory(lower_gitdir)) {
+ char *canonical = real_pathdup(git_dir, 0);
+ if (canonical) {
+ struct strbuf norm_git_dir = STRBUF_INIT;
+ strbuf_addstr(&norm_git_dir, git_dir);
+ strbuf_normalize_path(&norm_git_dir);
+
+ if (strcmp(canonical, norm_git_dir.buf))
+ ret = error(_("submodule git dir '%s' "
+ "collides with '%s'"),
+ canonical, norm_git_dir.buf);
+
+ strbuf_release(&norm_git_dir);
+ FREE_AND_NULL(canonical);
+ }
+ }
+ }
+
+ FREE_AND_NULL(lower_gitdir);
+ return ret;
+ }
+
return 0;
}
@@ -2653,13 +2689,20 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
if (!validate_and_set_submodule_gitdir(buf, submodule_name))
return;
- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
strbuf_reset(buf);
repo_git_path_append(r, buf, "modules/");
strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
if (!validate_and_set_submodule_gitdir(buf, submodule_name))
return;
+ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr_urlencode(buf, submodule_name, is_casefolding_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
/* Case 3: error out */
die(_("Cannot construct a valid gitdir path for submodule '%s': "
"please set a unique git config for 'submodule.%s.gitdir'."),
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
index a42d358f5b..f92b3e6338 100755
--- a/t/t7425-submodule-encoding.sh
+++ b/t/t7425-submodule-encoding.sh
@@ -143,4 +143,19 @@ test_expect_success 'submodule git dir nesting detection must work with parallel
verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
'
+test_expect_success 'verify case-folding conflict is correctly encoded' '
+ git clone -c extensions.submoduleEncoding=true -c core.ignoreCase=true main cloned-folding &&
+ (
+ cd cloned-folding &&
+
+ git submodule add ../new-sub "folding" &&
+ test_commit lowercase &&
+
+ git submodule add ../new-sub "FoldinG" &&
+ test_commit uppercase
+ ) &&
+ verify_submodule_gitdir_path cloned-folding "folding" "modules/folding" &&
+ verify_submodule_gitdir_path cloned-folding "FoldinG" "modules/%46oldin%47"
+'
+
test_done
diff --git a/url.c b/url.c
index 0fb1859b28..057e6e5c6e 100644
--- a/url.c
+++ b/url.c
@@ -14,6 +14,18 @@ int is_rfc3986_unreserved(char ch)
ch == '-' || ch == '_' || ch == '.' || ch == '~';
}
+/*
+ * This is a variant of is_rfc3986_unreserved() that treats uppercase
+ * letters as "reserved". This forces them to be percent-encoded, allowing
+ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
+ */
+int is_casefolding_rfc3986_unreserved(char c)
+{
+ return (c >= 'a' && c <= 'z') ||
+ (c >= '0' && c <= '9') ||
+ c == '-' || c == '.' || c == '_' || c == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 131a262066..92e3c63514 100644
--- a/url.h
+++ b/url.h
@@ -22,5 +22,6 @@ void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
int is_rfc3986_unreserved(char ch);
+int is_casefolding_rfc3986_unreserved(char c);
#endif /* URL_H */
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-07 15:05 ` [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
@ 2025-11-08 18:20 ` Aaron Schrab
2025-11-10 17:11 ` Adrian Ratiu
2025-11-12 15:28 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Aaron Schrab @ 2025-11-08 18:20 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood
At 17:05 +0200 07 Nov 2025, Adrian Ratiu <adrian.ratiu@collabora.com> wrote:
>Add a new check in validate_submodule_git_dir() to detect and
>prevent case-folding filesystem colisions. When this new check
>is triggered, a stricter casefolding aware URI encoding is used
>to percent-encode uppercase characters, e.g. Foo becomes %46oo.
>
>By using this check/retry mechanism the uppercase encoding is
>only applied when necessary, so case-sensitive filesystems are
>not affected.
What happens if `Foo` is added first and doesn't conflict with anything,
then later a new submodule is added which would naturally get the name
`foo` which would conflict and doesn't have any upper case characters to
encode to avoid the conflict?
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-08 18:20 ` Aaron Schrab
@ 2025-11-10 17:11 ` Adrian Ratiu
2025-11-10 17:31 ` Aaron Schrab
2025-11-10 19:10 ` Junio C Hamano
0 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-10 17:11 UTC (permalink / raw)
To: Aaron Schrab
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood
On Sat, 08 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
> At 17:05 +0200 07 Nov 2025, Adrian Ratiu
> <adrian.ratiu@collabora.com> wrote:
>>Add a new check in validate_submodule_git_dir() to detect and
>>prevent case-folding filesystem colisions. When this new check
>>is triggered, a stricter casefolding aware URI encoding is used
>>to percent-encode uppercase characters, e.g. Foo becomes %46oo.
>>By using this check/retry mechanism the uppercase encoding is
>>only applied when necessary, so case-sensitive filesystems are
>>not affected.
>
> What happens if `Foo` is added first and doesn't conflict with
> anything, then later a new submodule is added which would
> naturally get the name `foo` which would conflict and doesn't
> have any upper case characters to encode to avoid the conflict?
What an excellent question. Thank you!
Right now, in v4, in this case the user adding the second `foo`
module will have to manually set the submodule.foo.gitdir config
to avoid the conflict, because Foo already uses the coliding path.
I can add a test specifically for this edge case to exercise
config overrides on case-folding file-systems conflicts.
OR...
Maybe we could derive a new path automatically (eg foo2 or foo_,
suggestions welcome) and use it if valid. This way, there is no
user intervention.
Do you have any preference?
Thanks again,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 17:11 ` Adrian Ratiu
@ 2025-11-10 17:31 ` Aaron Schrab
2025-11-10 18:27 ` Adrian Ratiu
2025-11-10 19:10 ` Junio C Hamano
1 sibling, 1 reply; 179+ messages in thread
From: Aaron Schrab @ 2025-11-10 17:31 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood
At 19:11 +0200 10 Nov 2025, Adrian Ratiu <adrian.ratiu@collabora.com> wrote:
>On Sat, 08 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
>>What happens if `Foo` is added first and doesn't conflict with
>>anything, then later a new submodule is added which would naturally
>>get the name `foo` which would conflict and doesn't have any upper
>>case characters to encode to avoid the conflict?
>Right now, in v4, in this case the user adding the second `foo` module
>will have to manually set the submodule.foo.gitdir config to avoid the
>conflict, because Foo already uses the coliding path.
I think that description minimizes the impact. I'd think that anyone
with a prior clone (on a case-folding file system) would need to take
that action after pulling the change that added the new submodule.
If action were needed only when running `git submodule add`, I think
that would be fine. But requiring that action in all clones seems a bit
much. Some of those clones may even be managed with automation making it
even more of a problem to add that new configuration.
The action may even be required in new clones, unless the submodule
setup process for new clones sorts entries so that ones with capital
letters come later. Since some common collation rules (thinking mainly
of the `C` locale) will put capital letters first I think that's
unlikely to be the case.
>Maybe we could derive a new path automatically (eg foo2 or foo_,
>suggestions welcome) and use it if valid. This way, there is no user
>intervention.
>
>Do you have any preference?
I certainly don't have a *strong* preference. But, I think `foo2` seems
a bit clearer. Although the implied strategy there for multiple
conflicting names may be too complex for a situation that will likely be
exceedingly rare.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 17:31 ` Aaron Schrab
@ 2025-11-10 18:27 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-10 18:27 UTC (permalink / raw)
To: Aaron Schrab
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood
On Mon, 10 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
> At 19:11 +0200 10 Nov 2025, Adrian Ratiu
> <adrian.ratiu@collabora.com> wrote:
>>On Sat, 08 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
>>>What happens if `Foo` is added first and doesn't conflict with
>>>anything, then later a new submodule is added which would
>>>naturally get the name `foo` which would conflict and doesn't
>>>have any upper case characters to encode to avoid the
>>>conflict?
>
>>Right now, in v4, in this case the user adding the second `foo`
>>module will have to manually set the submodule.foo.gitdir
>>config to avoid the conflict, because Foo already uses the
>>coliding path.
>
> I think that description minimizes the impact. I'd think that
> anyone with a prior clone (on a case-folding file system) would
> need to take that action after pulling the change that added
> the new submodule.
No, because the extension is disabled by default.
>
> If action were needed only when running `git submodule add`, I
> think that would be fine. But requiring that action in all
> clones seems a bit much. Some of those clones may even be
> managed with automation making it even more of a problem to add
> that new configuration.
The extension is opt-in: by default it does nothing. :)
When the extension is enabled, all existing submodules are
automatically added to submodule.<name>.gitdir config without user
intervention.
Enabling the extension guarantees that config exists for all
repositories, unless there is a conflict we cannot solve, like you
pointed above, which is typically only for very rare corner-cases
(that's the intention).
>
> The action may even be required in new clones, unless the
> submodule setup process for new clones sorts entries so that
> ones with capital letters come later. Since some common
> collation rules (thinking mainly of the `C` locale) will put
> capital letters first I think that's unlikely to be the case.
The more I think about this, the more I'm convinced the second
option (see bleow) is the way to go, which is also more in line
with Junio's design to try a new path when validation fails and
repeat.
>>Maybe we could derive a new path automatically (eg foo2 or foo_,
>>suggestions welcome) and use it if valid. This way, there is no
>>user intervention. Do you have any preference?
>
> I certainly don't have a *strong* preference. But, I think
> `foo2` seems a bit clearer. Although the implied strategy there
> for multiple conflicting names may be too complex for a
> situation that will likely be exceedingly rare.
The resolution algorithm is pretty simple:
1. Create a path candidate.
2. Validate if it works: if not, go back to step 1.
We can do this any number of times for all the edge cases we can
detect.
If foo2 also has a conflict, we can just try something else.
If all else fails, we can even hash the submodule name ;)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 17:11 ` Adrian Ratiu
2025-11-10 17:31 ` Aaron Schrab
@ 2025-11-10 19:10 ` Junio C Hamano
2025-11-10 23:01 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-11-10 19:10 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Aaron Schrab, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> On Sat, 08 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
>> At 17:05 +0200 07 Nov 2025, Adrian Ratiu
>> <adrian.ratiu@collabora.com> wrote:
>>>Add a new check in validate_submodule_git_dir() to detect and
>>>prevent case-folding filesystem colisions. When this new check
>>>is triggered, a stricter casefolding aware URI encoding is used
>>>to percent-encode uppercase characters, e.g. Foo becomes %46oo.
>>>By using this check/retry mechanism the uppercase encoding is
>>>only applied when necessary, so case-sensitive filesystems are
>>>not affected.
The .gitdir name munging is a local thing, so it makes sense to do
the casefold mitigation only the filesystem is case folding one,
Your code seems to compare directory names textually, and downcasing
the proposed name for some reason, but I am not sure why we need any
of these complexity. Wouldn't it be the matter of actually trying
to mkdir(2) the name presented (either "foo" or "Foo") and see if
that fails? If it fails (most likely with EEXIST if case folding is
getting in the way, but for any reason), the name is unusable and we
need to "tweak" the name to a usable one at that point by retrying.
Once we find a usable name, we can remember the fact that we already
created a directory for it and reuse that empty directory in the
code where we used to do mkdir(2), no?
> Maybe we could derive a new path automatically (eg foo2 or foo_,
> suggestions welcome) and use it if valid. This way, there is no
> user intervention.
>
> Do you have any preference?
If adding 'foo' and then an attempt to add 'Foo' will automatically
assign a name that does not conflict with 'foo' to the newly added
submodule, then the users would expect the same to happen if the
order to add them are swapped, wouldn't they?
IOW, I do not see why the code wants to treat uppercase and
lowercase letters any differently, and suspect that it might be the
source of additional complication. Also, if there is an existing
module with a funny path "%46oo", you cannot just encode "Foo" into
"%46oo" to avoid crashes with 'foo' and be done anyway, so it feels
like we are inviting more bugs by special casing certain paths (and
not encoding or checking others). Don't we have an issue similar to
"case folding" in macOS wrt UTF-8 canonicalization, too? An
identical Unicode string may be canonicalized in two ways, so in a
presence of a submodule named one way, the other submodule named in
the other canonicalization, while their names may be with different
byte sequences, cannot co-exist in the same directory next to each
other. "Try to mkdir(2) the new name, and see if it succeeds, and
if so use the resulting empty directory" approach would cover that
case with the same mechanism as you need to use for case folding
filesystems, I would imagine.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 19:10 ` Junio C Hamano
@ 2025-11-10 23:01 ` Adrian Ratiu
2025-11-10 23:17 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-10 23:01 UTC (permalink / raw)
To: Junio C Hamano
Cc: Aaron Schrab, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Hi Junio,
On Mon, 10 Nov 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> On Sat, 08 Nov 2025, Aaron Schrab <aaron@schrab.com> wrote:
>>> At 17:05 +0200 07 Nov 2025, Adrian Ratiu
>>> <adrian.ratiu@collabora.com> wrote:
>>>>Add a new check in validate_submodule_git_dir() to detect and
>>>>prevent case-folding filesystem colisions. When this new check
>>>>is triggered, a stricter casefolding aware URI encoding is
>>>>used to percent-encode uppercase characters, e.g. Foo becomes
>>>>%46oo. By using this check/retry mechanism the uppercase
>>>>encoding is only applied when necessary, so case-sensitive
>>>>filesystems are not affected.
>
> The .gitdir name munging is a local thing, so it makes sense to
> do the casefold mitigation only the filesystem is case folding
> one,
That is correct. Case-sensitive filesystems will stop before
reaching the "Case 2.2" attempt, because their gitdirs will pass
validation in the earlier steps.
>
> Your code seems to compare directory names textually, and
> downcasing the proposed name for some reason, but I am not sure
> why we need any of these complexity. Wouldn't it be the matter
> of actually trying to mkdir(2) the name presented (either "foo"
> or "Foo") and see if that fails? If it fails (most likely with
> EEXIST if case folding is getting in the way, but for any
> reason), the name is unusable and we need to "tweak" the name to
> a usable one at that point by retrying. Once we find a usable
> name, we can remember the fact that we already created a
> directory for it and reuse that empty directory in the code
> where we used to do mkdir(2), no?
I tried the mkdir approach. Unfortunately it does not work because
submodule_name_to_gitdir() is called on all submodule paths: both
existing or new, valid or non-valid.
So if a dir already exists, we do not know if there is a
conflict. It might just be a normal valid path verification of an
existing module.
This is why I had to come up with the double-test using to_lower()
to compute a common path, canonicalization and checking for
existence when there is a difference via is_git_directory(). In
this case there is always a conflict and is the only reliable way
I colud think of.
I am very much in favor for finding a simpler check, however we
need something at least as reliable as the current double-test,
which passes all cases I could think of and all CI tests.
>
>> Maybe we could derive a new path automatically (eg foo2 or
>> foo_, suggestions welcome) and use it if valid. This way,
>> there is no user intervention.
>>
>> Do you have any preference?
>
> If adding 'foo' and then an attempt to add 'Foo' will
> automatically assign a name that does not conflict with 'foo' to
> the newly added submodule, then the users would expect the same
> to happen if the order to add them are swapped, wouldn't they?
Yes and it's a valid expectation. :)
I just missed this corner-case which Aaron brought up (many thanks
again Aaron!).
It's a simple fix which I'll do in v5: if validation fails, we
just need to come up with another name and retry.
The probability of conflicts decreases exponentially with each
retry. By the 4-5 retry the probability is so small, at that
point, if we're that desperate, we might just hash the name or use
a random string.
>
> IOW, I do not see why the code wants to treat uppercase and
> lowercase letters any differently, and suspect that it might be
> the source of additional complication. Also, if there is an
> existing module with a funny path "%46oo", you cannot just
> encode "Foo" into "%46oo" to avoid crashes with 'foo' and be
> done anyway, so it feels like we are inviting more bugs by
> special casing certain paths (and not encoding or checking
> others). Don't we have an issue similar to "case folding" in
> macOS wrt UTF-8 canonicalization, too? An identical Unicode
> string may be canonicalized in two ways, so in a presence of a
> submodule named one way, the other submodule named in the other
> canonicalization, while their names may be with different byte
> sequences, cannot co-exist in the same directory next to each
> other. "Try to mkdir(2) the new name, and see if it succeeds,
> and if so use the resulting empty directory" approach would
> cover that case with the same mechanism as you need to use for
> case folding filesystems, I would imagine.
Foo and "%46oo" is another possible conflict, yes, but only when
both "foo" and "%46oo" already exist. The deeper you go in the
retry cycles, the more unlikely conflicts become.
We just need to detect this conflict, come up with another name
and retry. Retry until one sticks.
Again, the probability of conflicts decreases exponentially with
each retry. :) If we're desperate: hash the name or try a random
string before giving up and throwing an error, which is "Case 3"
in the current patch.
(I wrote above why mkdir cannot solve the detection problem, hope
it makes sense).
I think it's also worth mentioning a key idea of this design: we
don't need to detect & fix all corner cases. We can iterate and
improve both the conflict detection and path generation over time,
since they are opaque to users, who rely only on the resulting
submodule.name.gitdir config we publish.
I know it seems like a game of "whack-a-mole", but even if we fix
just half of the possible conflicts, the likeliest ones, we still
are in a much better situation than before. At some point we're
approaching diminishing returns, so extremely rare conflicts might
not even be worth fixing.
I will add a test for this "Foo" + "foo" + "%46oo" in v5 as
well. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 23:01 ` Adrian Ratiu
@ 2025-11-10 23:17 ` Junio C Hamano
2025-11-11 12:41 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-11-10 23:17 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Aaron Schrab, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> I tried the mkdir approach. Unfortunately it does not work because
> submodule_name_to_gitdir() is called on all submodule paths: both
> existing or new, valid or non-valid.
>
> So if a dir already exists, we do not know if there is a
> conflict. It might just be a normal valid path verification of an
> existing module.
Puzzled. Wouldn't that only mean that submodule-name-to-gitdir is a
wrong place to see if the name of a directory you are planning to
use for a newly added submodule is available? IOW, don't you need a
more specific new helper function and perform the check in there?
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-10 23:17 ` Junio C Hamano
@ 2025-11-11 12:41 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-11 12:41 UTC (permalink / raw)
To: Junio C Hamano
Cc: Aaron Schrab, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Mon, 10 Nov 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> I tried the mkdir approach. Unfortunately it does not work
>> because submodule_name_to_gitdir() is called on all submodule
>> paths: both existing or new, valid or non-valid.
>>
>> So if a dir already exists, we do not know if there is a
>> conflict. It might just be a normal valid path verification of
>> an existing module.
>
> Puzzled. Wouldn't that only mean that submodule-name-to-gitdir
> is a wrong place to see if the name of a directory you are
> planning to use for a newly added submodule is available? IOW,
> don't you need a more specific new helper function and perform
> the check in there?
I thought long and hard about this in each version since v1 and
keep coming to the same answer: we need to do it here and can't
avoid it.
The reason is that submodule_name_to_gitdir() is the unified API
where submodule gitdirs get computed consistently and validation
goes hand in hand with computing the valid gitdirs. If you look at
all the places where validate_submodule_git_dir() is currently
called, it's always immediately after submodule_name_to_gitdir(),
on its result.
It even makes sense to remove the validation calls outside
submodule_name_to_gitdir() and do it inside it for each path
attempt, like we do for the encoding cases, so API users don't
have to manually validate the result each time.
This reminds me: we should also validate the gitdir path we get
via the submodule.%s.gitdir config. Will fix this as well in v5.
Another related question I've been pondering is whether to create
two separate validation functions, one for legacy and one for
encoding. It doesn't win us much (currently we return early if
extension/encoding is enabled in the unified function), but we can
do this in v5 as well.
To recap, what I intend to do in v5 (up to now) is:
- Address Aaron's case-fold corner-case of creating Foo then foo.
- Address Junio's case-fold corner-case of creating foo + Foo +
%46oo.
- (Important) Also validate the submodule.%s.gitdir config value.
- Move validation checks inside submodule_name_to_gitdir() to
ensure they run each time, instead of having API users validate
the gitdir result each time. - (Undecided) Split the validation
function in two for the legacy vs encoding/extension cases.
- (Undecided) If all our path attempts fail, try hashing the name
and see if that works before erroring out.
Especially if we implement the last point with hashing, the
chances for conflicts are close to 0.
Here's an interesting idea:
Maybe it's easier to just hash everything when the extension is
enabled since users now always have the submodule.%s.gitdir
config, which is also accesible via the submodule--helper?
Hashing + config would eliminate all this corner-case
complexity. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions
2025-11-07 15:05 ` [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
2025-11-08 18:20 ` Aaron Schrab
@ 2025-11-12 15:28 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-12 15:28 UTC (permalink / raw)
To: Junio C Hamano, Aaron Schrab
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Jonathan Nieder, Patrick Steinhardt, Josh Steadmon, Ben Knoble,
Phillip Wood
On Fri, 07 Nov 2025, Adrian Ratiu <adrian.ratiu@collabora.com>
wrote:
> diff --git a/submodule.c b/submodule.c index
> ceaff0c1aa..ecbffac2c6 100644 --- a/submodule.c +++
> b/submodule.c @@ -2280,7 +2280,7 @@ int
> validate_submodule_git_dir(char *git_dir, const char
> *submodule_name)
> size_t len = strlen(git_dir), suffix_len =
> strlen(submodule_name); char *p = git_dir + len - suffix_len;
> bool suffixes_match = !strcmp(p, submodule_name);
> - int ret = 0; + int ret = 0, config_ignorecase = 0;
> /* * We prevent the contents of sibling submodules' git
> directories to
> @@ -2318,6 +2318,42 @@ int validate_submodule_git_dir(char
> *git_dir, const char *submodule_name)
> if (p && strchr(p, '/') != NULL) return error("submodule
> gitdir name '%s' contains unexpected '/'", p);
> + /* Prevent conflicts on case-folding filesystems */ +
> repo_config_get_bool(the_repository, "core.ignorecase",
> &config_ignorecase); + if (ignore_case ||
> config_ignorecase) { + char *lower_gitdir =
> xstrdup(git_dir); + char *module_name =
> find_last_submodule_name(lower_gitdir); + + if
> (module_name) { + for (p = module_name; *p;
> p++) + *p = tolower(*p); + +
> /* + * If lower path is different and already
> exists, check for collision. + *
> Intentionally double-check to eliminate false-positives. +
> */ + if (strcmp(lower_gitdir, git_dir) &&
> is_git_directory(lower_gitdir)) { +
> char *canonical = real_pathdup(git_dir, 0); +
> if (canonical) { + struct
> strbuf norm_git_dir = STRBUF_INIT; +
> strbuf_addstr(&norm_git_dir, git_dir); +
> strbuf_normalize_path(&norm_git_dir); + +
> if (strcmp(canonical, norm_git_dir.buf)) +
> ret = error(_("submodule git dir '%s' " +
> "collides with '%s'"), +
> canonical, norm_git_dir.buf); + +
> strbuf_release(&norm_git_dir); +
> FREE_AND_NULL(canonical); + } +
> } + } + + FREE_AND_NULL(lower_gitdir); +
> return ret; + } +
> return 0; }
I think I came up with a better case-folding conflict detection
implementation.
In a nutshell, for the next iteration (v5), I intend to do:
DIR *dir = opendir(modules_dir);
...
/* Check for another directory under .git/modules that differs
only in case. */ while ((de = readdir(dir)) != NULL) { if
(!strcmp(de->d_name, ".") || !strcmp(de->d_name, "..")) continue;
if (!strcasecmp(de->d_name, submodule_name) && strcmp(de->d_name,
submodule_name)) { closedir(dir); return error(_("submodule name
'%s' collides with '%s' " "on case-insensitive filesystem"),
submodule_name, de->d_name); } }
We look at existing submodules and we have a conflict if both are
true:
1. Names are equal ignoring the case (case insensitive equality).
2. Names are NOT equal considering the case (case sensitive
inequality).
This is simpler, more robest and passes all my test cases.
Will leave v4 on the ML until next week in case there is more feedback,
then I'll send v5 using this check (I'll also add more tests).
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
` (3 preceding siblings ...)
2025-11-07 15:05 ` [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
@ 2025-11-14 23:03 ` Josh Steadmon
2025-11-17 15:22 ` Adrian Ratiu
4 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-11-14 23:03 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
On 2025.11.07 17:05, Adrian Ratiu wrote:
> Hello everyone,
>
> For those new to this series, we are adding an extension to encode submodule
> gitdir paths to avoid filesystem conflicts.
Disclaimer for the list: Google is funding Adrian's work on this series;
noting this now for transparency.
Hi Adrian, sorry for missing the last couple of versions of this series.
The switch to using an extension may complicate our migration a bit.
Background for the list: Google has been using an early version of this
submodule encoding scheme for years. We have a lot of users'
repositories with this encoding scheme in place on disk, but with no
corresponding extensions.submoduleEncoding config.
I've done some limited testing; the good news is that it looks like
using this series with pre-encoded submodules still works, regardless of
the value of extensions.submoduleEncoding. It would be nice to add some
tests in V5 that we can create some submodules with the extension
enabled, and then disable it later and still work with the encoded
submodules (and then maybe enable it once again).
The first difficulty I see is that there's not a good way to
automatically migrate existing repos to the new extension; we'll have to
ask users to manually set configs on each of their repos. While we are
able to distribute default Git configs for our users,
`core.repositoryFormatValue` and `extensions.*` are obviously special
cases that can't be applied from non-repo-local configs. I don't know
what could be changed in this series to avoid the issue, so I guess I'll
instead just ask the list for ideas for automating this migration. One
idea is to carry a tiny downstream patch to force-enable
`extensions.submoduleEncoding` regardless of the local config, but maybe
someone else has a better idea.
A second issue is that we'd like to be able to set submoduleEncoding for
new repositories, without requiring passing a config on the command
line. Perhaps we could add another config option analogous to
`init.defaultObjectFormat` that we can set in our locally-distributed
config.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts
2025-11-14 23:03 ` [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts Josh Steadmon
@ 2025-11-17 15:22 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-17 15:22 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
Hi Josh and thanks for testing & the feedback.
On Fri, 14 Nov 2025, Josh Steadmon <steadmon@google.com> wrote:
> The switch to using an extension may complicate our migration a
> bit. Background for the list: Google has been using an early
> version of this submodule encoding scheme for years. We have a
> lot of users' repositories with this encoding scheme in place on
> disk, but with no corresponding extensions.submoduleEncoding
> config.
>
> I've done some limited testing; the good news is that it looks
> like using this series with pre-encoded submodules still works,
> regardless of the value of extensions.submoduleEncoding. It
> would be nice to add some tests in V5 that we can create some
> submodules with the extension enabled, and then disable it later
> and still work with the encoded submodules (and then maybe
> enable it once again).
Yes, I'd expect forward-migration (enabling the extension) to be
very smooth. I will add some tests for backwards migration
(disabling the extension) as well in v5.
>
> The first difficulty I see is that there's not a good way to
> automatically migrate existing repos to the new extension; we'll
> have to ask users to manually set configs on each of their
> repos. While we are able to distribute default Git configs for
> our users, `core.repositoryFormatValue` and `extensions.*` are
> obviously special cases that can't be applied from
> non-repo-local configs. I don't know what could be changed in
> this series to avoid the issue, so I guess I'll instead just ask
> the list for ideas for automating this migration. One idea is to
> carry a tiny downstream patch to force-enable
> `extensions.submoduleEncoding` regardless of the local config,
> but maybe someone else has a better idea.
I propose to use a build-time configuration option to force-enable
/ set the config automatically for these types of situations.
This way you just add a flag to your builds and don't need to
carry downstream patches, though it could also be solved with
one-liner downstream patch. :)
>
> A second issue is that we'd like to be able to set
> submoduleEncoding for new repositories, without requiring
> passing a config on the command line. Perhaps we could add
> another config option analogous to `init.defaultObjectFormat`
> that we can set in our locally-distributed config.
This will also be solved by the build-time config option I
proposed above.
I plan to send v5 soon and I'll include these changes, if nobody
objects, along with everything else I've got ready.
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v5 0/7] Encode submodule gitdir names to avoid conflicts
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (12 preceding siblings ...)
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 1/7] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (6 more replies)
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
15 siblings, 7 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Hello everyone,
For those new to the series, we are adding an extension to fix submodule
gitdir path conflicts due to nested gitdirs or case-folding filesystems.
v5 contains code cleanups, more corner-case fixes and a few new commits
briefly described below, after which a range-diff follows.
As always this is based on the latest master, I've checked for conflicts
with the next/seen branches, pushed to GitHub [1] and CI passes [2].
Many thanks to all reviewers!
1: https://github.com/10ne1/git/tree/dev/aratiu/encoding-v5
2: https://github.com/10ne1/git/actions/runs/19515373994
Changes between v4 -> v5:
* Derive some new gitdir names in case the uppercase-extended encoding fails. (Aaron)
* Add more case-folding and extension disabling / cloning test corner-cases. (Aaron, Junio & Josh)
* Run the gitdir validation on the submodules.<name>.gitdir config after reading it. (Adrian)
* New commit: add a build-time option to enable the extension. (Josh)
* New commit: fallback to hashing if all encoding attempts fail. (Adrian)
* New commit: always validate the gitdir in submodule_name_to_git_dir. (Adrian)
* Split encoding validation in a separate function and leave non-enc validation intact. (Adrian)
* Add a CASE_INSENSITIVE_FS prereq for the case-folding tests. This will also significantly
simplify case-folding validation, by not attempting to run on case-sensitive FS. (Adrian)
* Improve case-folding conflict detection by making validation more robust. (Adrian)
* Minor code fixes and improvements (spelling, error checking, indentation). (Adrian)
Range-diff between v4 -> v5:
1: 7d34507692 = 1: 9d5855f3bf submodule--helper: use submodule_name_to_gitdir in add_submodule
2: 1e609bdd1a = 2: 8cfa970a9d builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
-: ---------- > 3: 7bcadf1116 submodule: always validate gitdirs inside submodule_name_to_gitdir
3: c51fd58669 ! 4: 1b5d0b50ef submodule: add extension to encode gitdir paths
@@ Documentation/config/extensions.adoc: relativeWorktrees:::
+submoduleEncoding:::
+ If enabled, submodule gitdir paths are encoded to avoid filesystem
-+ conflicts due to nested gitdirs, case insensitivity or other issues
++ conflicts due to nested gitdirs, case insensitivity or other issues.
+ When enabled, the submodule.<name>.gitdir config is always set for
-+ all submodulesand is the single point of authority for gitdir paths.
++ all submodules and is the single point of authority for gitdir paths.
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
}
+/*
-+ * Find the last submodule name in the gitdir path (modules can be nested).
-+ * Returns a pointer into `path` to the beginning of the name or NULL if not found.
++ * Encoded gitdir validation function used when extensions.submoduleEncoding is enabled.
++ * This does not print errors like the non-encoded version, because encoding is supposed
++ * to mitigate / fix all these.
+ */
-+static char *find_last_submodule_name(char *git_dir_path)
++static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+{
+ const char *modules_marker = "/modules/";
-+ char *p = git_dir_path;
-+ char *last = NULL;
++ char *p = git_dir, *last_submodule_name = NULL;
+
++ if (!the_repository->repository_format_submodule_encoding)
++ BUG("validate_submodule_encoded_git_dir() must be called with "
++ "extensions.submoduleEncoding enabled.");
++
++ /* Find the last submodule name in the gitdir path (modules can be nested). */
+ while ((p = strstr(p, modules_marker))) {
-+ last = p + strlen(modules_marker);
++ last_submodule_name = p + strlen(modules_marker);
+ p++;
+ }
+
-+ return last;
++ /* Prevent the use of '/' in encoded names */
++ if (!last_submodule_name || strchr(last_submodule_name, '/'))
++ return -1;
++
++ return 0;
+}
+
- int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+ static int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
-- char *p;
-+ char *p = git_dir + len - suffix_len;
-+ bool suffixes_match = !strcmp(p, submodule_name);
+ char *p;
int ret = 0;
-- if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
-- strcmp(p, submodule_name))
-- BUG("submodule name '%s' not a suffix of git dir '%s'",
-- submodule_name, git_dir);
--
- /*
- * We prevent the contents of sibling submodules' git directories to
- * clash.
-@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
- * but the latter directory is already designated to contain the hooks
- * of the former.
- */
-- for (; *p; p++) {
-+ for (; *p && suffixes_match; p++) {
- if (is_dir_sep(*p)) {
- char c = *p;
-
-@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
- }
- }
-
-+ /* tests after this check are only for encoded names, when the extension is enabled */
-+ if (!the_repository->repository_format_submodule_encoding)
-+ return 0;
-+
-+ /* Prevent the use of '/' in names */
-+ p = find_last_submodule_name(git_dir);
-+ if (p && strchr(p, '/') != NULL)
-+ return error("submodule gitdir name '%s' contains unexpected '/'", p);
++ if (the_repository->repository_format_submodule_encoding)
++ BUG("validate_submodule_git_dir() must be called with "
++ "extensions.submoduleEncoding disabled.");
+
- return 0;
- }
-
+ if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
+ strcmp(p, submodule_name))
+ BUG("submodule name '%s' not a suffix of git dir '%s'",
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
return ret;
}
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
+{
+ char *key;
+
-+ if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
++ if (validate_submodule_encoded_git_dir(gitdir_path->buf, submodule_name))
+ return -1;
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
+ FREE_AND_NULL(key);
+
+ return 0;
-+
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
+ strbuf_addstr(buf, submodule_name);
+
+ /* If extensions.submoduleEncoding is disabled, use the plain path set above */
-+ if (!r->repository_format_submodule_encoding)
-+ return;
++ if (!r->repository_format_submodule_encoding) {
++ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
++ die(_("refusing to create/use '%s' in another submodule's "
++ "git dir"), buf->buf);
++
++ return; /* plain gitdir is valid for use */
++ }
+
+ /* Extension is enabled: use the gitdir config if it exists */
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
+ strbuf_reset(buf);
+ strbuf_addstr(buf, gitdir);
+ FREE_AND_NULL(key);
++
++ /* validate because users might have modified the config */
++ if (validate_submodule_encoded_git_dir(buf->buf, submodule_name))
++ die(_("Invalid 'submodule.%s.gitdir' config: '%s' please check "
++ "if it is unique or conflicts with another module"),
++ submodule_name, gitdir);
++
+ return;
+ }
+ FREE_AND_NULL(key);
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
+ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
-+
-+ /* Case 3: error out */
+
+- if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+- die(_("refusing to create/use '%s' in another submodule's "
+- "git dir"), buf->buf);
++ /* Case 3: Nothing worked: error out */
+ die(_("Cannot construct a valid gitdir path for submodule '%s': "
+ "please set a unique git config for 'submodule.%s.gitdir'."),
+ submodule_name, submodule_name);
@@ t/t7425-submodule-encoding.sh (new)
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
++ # trigger the "die_path_inside_submodule" check
++ test_must_fail git submodule add ../new-sub "legacy/nested" &&
++
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submoduleEncoding true &&
+
+ git submodule add ../new-sub "New Sub" &&
-+ test_commit new
++ test_commit new &&
++
++ # retrigger the "die_path_inside_submodule" check with encoding
++ test_must_fail git submodule add ../new-sub "New Sub/nested2"
+ )
+'
+
@@ t/t7425-submodule-encoding.sh (new)
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
+'
+
++test_expect_success 'disabling extensions.submoduleEncoding prevents nested submodules' '
++ (
++ cd clone_nested &&
++ # disable extension and verify failure
++ git config extensions.submoduleEncoding false &&
++ test_must_fail git submodule add ./thing2 hippo/foobar &&
++ # re-enable extension and verify it works
++ git config extensions.submoduleEncoding true &&
++ git submodule add ./thing2 hippo/foobar
++ )
++'
++
+test_done
## t/t9902-completion.sh ##
4: 01a5b10d5a < -: ---------- submodule: fix case-folding gitdir filesystem colisions
-: ---------- > 5: 2bf1c116a2 submodule: fix case-folding gitdir filesystem colisions
-: ---------- > 6: b607d7ca39 submodule: use hashed name for gitdir
-: ---------- > 7: 9b4890cfd2 meson/Makefile: allow setting submodule encoding at build time
Adrian Ratiu (7):
submodule--helper: use submodule_name_to_gitdir in add_submodule
builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
submodule: always validate gitdirs inside submodule_name_to_gitdir
submodule: add extension to encode gitdir paths
submodule: fix case-folding gitdir filesystem colisions
submodule: use hashed name for gitdir
meson/Makefile: allow setting submodule encoding at build time
Documentation/config/extensions.adoc | 8 +
Documentation/config/submodule.adoc | 5 +
Makefile | 5 +
builtin/credential-store.c | 7 +-
builtin/submodule--helper.c | 51 +++--
configure.ac | 23 +++
meson.build | 4 +
meson_options.txt | 2 +
repository.c | 1 +
repository.h | 1 +
setup.c | 15 ++
setup.h | 1 +
submodule.c | 234 ++++++++++++++++++-----
submodule.h | 5 -
t/lib-verify-submodule-gitdir-path.sh | 24 +++
t/meson.build | 1 +
t/t7425-submodule-encoding.sh | 258 ++++++++++++++++++++++++++
t/t9902-completion.sh | 1 +
url.c | 23 +++
url.h | 3 +
20 files changed, 591 insertions(+), 81 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-encoding.sh
--
2.51.0
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v5 1/7] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
` (5 subsequent siblings)
6 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded modules gitdir path leading to test failures.
Call the submodule_name_to_gitdir() helper instead, which was invented
exactly for this purpose and is already used by all the other locations
which work on gitdirs.
Also narrow the scope of the submod_gitdir_path variable which is not
used anymore in the updated "else" branch.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index fcd73abe53..2873b2780e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3187,13 +3187,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3207,10 +3207,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3219,8 +3220,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3238,7 +3239,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 1/7] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-12-05 12:16 ` Patrick Steinhardt
2025-11-19 21:10 ` [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
` (4 subsequent siblings)
6 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it was the only place using it.
However now we need it to apply URL-encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back as a
public function exposed via url.h, instead of the old helper path
(strbuf), which has nothing to do with 3986 encoding/decoding anymore.
This function will be used by submodule.c in the next commit.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 7 +------
url.c | 11 +++++++++++
url.h | 2 ++
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..bc1453c6b2 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -7,6 +7,7 @@
#include "path.h"
#include "string-list.h"
#include "parse-options.h"
+#include "url.h"
#include "write-or-die.h"
static struct lock_file credential_lock;
@@ -76,12 +77,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/url.c b/url.c
index 282b12495a..0fb1859b28 100644
--- a/url.c
+++ b/url.c
@@ -3,6 +3,17 @@
#include "strbuf.h"
#include "url.h"
+/*
+ * The set of unreserved characters as per STD66 (RFC3986) is
+ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
+ * components without percent-encoding.
+ */
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 2a27c34277..131a262066 100644
--- a/url.h
+++ b/url.h
@@ -21,4 +21,6 @@ char *url_decode_parameter_value(const char **query);
void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
+int is_rfc3986_unreserved(char ch);
+
#endif /* URL_H */
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 1/7] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-12-05 12:17 ` Patrick Steinhardt
2025-11-19 21:10 ` [PATCH v5 4/7] submodule: add extension to encode gitdir paths Adrian Ratiu
` (3 subsequent siblings)
6 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Move the ad-hoc validation checks sprinkled across the source tree,
after calling submodule_name_to_gitdir() into the function proper,
which now always validates the gitdir before returning it.
This also makes parallel operations a bit safer due to checking and
erroring out each time the unified API detects a problem instead of
having one extra hardcoded validation check in submodule--helper.c.
It simplifies the API usage as well since users who don't have to
validate the submodule_name_to_gitdir() result themselves anymore
and reduces the risks of API users forgetting to validate.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 21 ---------------------
submodule.c | 30 ++++++------------------------
submodule.h | 5 -----
3 files changed, 6 insertions(+), 50 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2873b2780e..9914ca0786 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1703,10 +1703,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
clone_data_path = to_free = xstrfmt("%s/%s", repo_get_work_tree(the_repository),
clone_data->path);
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
-
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
@@ -1780,23 +1776,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
free(path);
}
- /*
- * We already performed this check at the beginning of this function,
- * before cloning the objects. This tries to detect racy behavior e.g.
- * in parallel clones, where another process could easily have made the
- * gitdir nested _after_ it was created.
- *
- * To prevent further harm coming from this unintentionally-nested
- * gitdir, let's disable it by deleting the `HEAD` file.
- */
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
- char *head = xstrfmt("%s/HEAD", sm_gitdir);
- unlink(head);
- free(head);
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
- }
-
connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
p = repo_submodule_path(the_repository, clone_data_path, "config");
diff --git a/submodule.c b/submodule.c
index 35c55155f7..8ef028f26b 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2153,30 +2153,11 @@ int submodule_move_head(const char *path, const char *super_prefix,
if (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {
if (old_head) {
- if (!submodule_uses_gitfile(path))
- absorb_git_dir_into_superproject(path,
- super_prefix);
- else {
- char *dotgit = xstrfmt("%s/.git", path);
- char *git_dir = xstrdup(read_gitfile(dotgit));
-
- free(dotgit);
- if (validate_submodule_git_dir(git_dir,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in "
- "another submodule's git dir"),
- git_dir);
- free(git_dir);
- }
+ absorb_git_dir_into_superproject(path, super_prefix);
} else {
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
- if (validate_submodule_git_dir(gitdir.buf,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in another "
- "submodule's git dir"),
- gitdir.buf);
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
@@ -2256,7 +2237,7 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+static int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
char *p;
@@ -2355,9 +2336,6 @@ static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
submodule_name_to_gitdir(&new_gitdir, the_repository, sub->name);
- if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
- die(_("refusing to move '%s' into an existing git dir"),
- real_old_git_dir);
if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
die(_("could not create directory '%s'"), new_gitdir.buf);
real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
@@ -2606,4 +2584,8 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
*/
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+
+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
}
diff --git a/submodule.h b/submodule.h
index b10e16e6c0..0b7692bc20 100644
--- a/submodule.h
+++ b/submodule.h
@@ -137,11 +137,6 @@ int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name);
-/*
- * Make sure that no submodule's git dir is nested in a sibling submodule's.
- */
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
-
/*
* Make sure that the given submodule path does not follow symlinks.
*/
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
` (2 preceding siblings ...)
2025-11-19 21:10 ` [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-12-05 12:19 ` Patrick Steinhardt
2025-11-19 21:10 ` [PATCH v5 5/7] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
` (2 subsequent siblings)
6 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu,
Brandon Williams
Add a submoduleEncoding extension which fixes filesystem collisions by
encoding gitdir paths. At a high level, this implements a mechanism to
encode -> validate -> retry until a working gitdir path is found.
Credit goes to Junio for coming up with this design: encoding is only
applied when necessary, e.g. uppercase characters are encoded only on
case-folding filesystems and only if a real conflict is detected.
To make this work, we rely on the submodule.<name>.gitdir config as the
single source of truth for gitidir paths: the config is always set when
the extension is enabled. Users who care about gitdir paths are expected
to get/set the config and not the underlying encoding implementation.
This commit adds the basic encoding logic which addresses nested gitdirs.
The next commit fixes case-folding, the next commit fixes names longer
than NAME_MAX. The idea is the encoding can be improved over time in a
way which is transparent to users.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Based-on-patch-by: Brandon Williams <bwilliams.eng@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 6 +
Documentation/config/submodule.adoc | 5 +
builtin/submodule--helper.c | 17 +++
repository.c | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 117 ++++++++++++++----
t/lib-verify-submodule-gitdir-path.sh | 24 ++++
t/meson.build | 1 +
t/t7425-submodule-encoding.sh | 164 ++++++++++++++++++++++++++
t/t9902-completion.sh | 1 +
12 files changed, 323 insertions(+), 22 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-encoding.sh
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 532456644b..4861d01894 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -73,6 +73,12 @@ relativeWorktrees:::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
+submoduleEncoding:::
+ If enabled, submodule gitdir paths are encoded to avoid filesystem
+ conflicts due to nested gitdirs, case insensitivity or other issues.
+ When enabled, the submodule.<name>.gitdir config is always set for
+ all submodules and is the single point of authority for gitdir paths.
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..ddaadc3dc5 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,11 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This option sets the gitdir path for submodule <name>, allowing users to
+ override the default path. Only works when `extensions.submoduleEncoding`
+ is enabled, otherwise does nothing. See linkgit:git-config[1] for details.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 9914ca0786..72440121a8 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1208,6 +1208,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3570,6 +3586,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
diff --git a/repository.c b/repository.c
index 6faf5c7398..26a21c0d71 100644
--- a/repository.c
+++ b/repository.c
@@ -288,6 +288,7 @@ int repo_init(struct repository *repo,
repo->repository_format_worktree_config = format.worktree_config;
repo->repository_format_relative_worktrees = format.relative_worktrees;
repo->repository_format_precious_objects = format.precious_objects;
+ repo->repository_format_submodule_encoding = format.submodule_encoding;
/* take ownership of format.partial_clone */
repo->repository_format_partial_clone = format.partial_clone;
diff --git a/repository.h b/repository.h
index 5808a5d610..7e39b2acf7 100644
--- a/repository.h
+++ b/repository.h
@@ -158,6 +158,7 @@ struct repository {
int repository_format_worktree_config;
int repository_format_relative_worktrees;
int repository_format_precious_objects;
+ int repository_format_submodule_encoding;
/* Indicate if a repository has a different 'commondir' from 'gitdir' */
unsigned different_commondir:1;
diff --git a/setup.c b/setup.c
index 7086741e6c..bf6e815105 100644
--- a/setup.c
+++ b/setup.c
@@ -687,6 +687,9 @@ static enum extension_result handle_extension(const char *var,
} else if (!strcmp(ext, "relativeworktrees")) {
data->relative_worktrees = git_config_bool(var, value);
return EXTENSION_OK;
+ } else if (!strcmp(ext, "submoduleencoding")) {
+ data->submodule_encoding = git_config_bool(var, value);
+ return EXTENSION_OK;
}
return EXTENSION_UNKNOWN;
}
@@ -1865,6 +1868,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
repo_fmt.worktree_config;
the_repository->repository_format_relative_worktrees =
repo_fmt.relative_worktrees;
+ the_repository->repository_format_submodule_encoding =
+ repo_fmt.submodule_encoding;
/* take ownership of repo_fmt.partial_clone */
the_repository->repository_format_partial_clone =
repo_fmt.partial_clone;
@@ -1963,6 +1968,8 @@ void check_repository_format(struct repository_format *fmt)
fmt->ref_storage_format);
the_repository->repository_format_worktree_config =
fmt->worktree_config;
+ the_repository->repository_format_submodule_encoding =
+ fmt->submodule_encoding;
the_repository->repository_format_relative_worktrees =
fmt->relative_worktrees;
the_repository->repository_format_partial_clone =
diff --git a/setup.h b/setup.h
index 8522fa8575..66ec1ceba5 100644
--- a/setup.h
+++ b/setup.h
@@ -130,6 +130,7 @@ struct repository_format {
char *partial_clone; /* value of extensions.partialclone */
int worktree_config;
int relative_worktrees;
+ int submodule_encoding;
int is_bare;
int hash_algo;
int compat_hash_algo;
diff --git a/submodule.c b/submodule.c
index 8ef028f26b..07cb4694cf 100644
--- a/submodule.c
+++ b/submodule.c
@@ -31,6 +31,7 @@
#include "commit-reach.h"
#include "read-cache-ll.h"
#include "setup.h"
+#include "url.h"
static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
static int initialized_fetch_ref_tips;
@@ -2237,12 +2238,43 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
+/*
+ * Encoded gitdir validation function used when extensions.submoduleEncoding is enabled.
+ * This does not print errors like the non-encoded version, because encoding is supposed
+ * to mitigate / fix all these.
+ */
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+{
+ const char *modules_marker = "/modules/";
+ char *p = git_dir, *last_submodule_name = NULL;
+
+ if (!the_repository->repository_format_submodule_encoding)
+ BUG("validate_submodule_encoded_git_dir() must be called with "
+ "extensions.submoduleEncoding enabled.");
+
+ /* Find the last submodule name in the gitdir path (modules can be nested). */
+ while ((p = strstr(p, modules_marker))) {
+ last_submodule_name = p + strlen(modules_marker);
+ p++;
+ }
+
+ /* Prevent the use of '/' in encoded names */
+ if (!last_submodule_name || strchr(last_submodule_name, '/'))
+ return -1;
+
+ return 0;
+}
+
static int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
char *p;
int ret = 0;
+ if (the_repository->repository_format_submodule_encoding)
+ BUG("validate_submodule_git_dir() must be called with "
+ "extensions.submoduleEncoding disabled.");
+
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
BUG("submodule name '%s' not a suffix of git dir '%s'",
@@ -2559,33 +2591,74 @@ int submodule_to_gitdir(struct repository *repo,
return ret;
}
+static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
+ const char *submodule_name)
+{
+ char *key;
+
+ if (validate_submodule_encoded_git_dir(gitdir_path->buf, submodule_name))
+ return -1;
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ repo_config_set_gently(the_repository, key, gitdir_path->buf);
+ FREE_AND_NULL(key);
+
+ return 0;
+}
+
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
+ const char *gitdir;
+ char *key;
+
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr(buf, submodule_name);
+
+ /* If extensions.submoduleEncoding is disabled, use the plain path set above */
+ if (!r->repository_format_submodule_encoding) {
+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
+
+ return; /* plain gitdir is valid for use */
+ }
+
+ /* Extension is enabled: use the gitdir config if it exists */
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ if (!repo_config_get_string_tmp(r, key, &gitdir)) {
+ strbuf_reset(buf);
+ strbuf_addstr(buf, gitdir);
+ FREE_AND_NULL(key);
+
+ /* validate because users might have modified the config */
+ if (validate_submodule_encoded_git_dir(buf->buf, submodule_name))
+ die(_("Invalid 'submodule.%s.gitdir' config: '%s' please check "
+ "if it is unique or conflicts with another module"),
+ submodule_name, gitdir);
+
+ return;
+ }
+ FREE_AND_NULL(key);
+
/*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
+ * The gitdir config does not exist, even though the extension is enabled.
+ * Therefore we are in one of the following cases:
*/
+
+ /* Case 1: legacy migration of valid plain submodule names */
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
+ /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ strbuf_reset(buf);
repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
+ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
- if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), buf->buf);
+ /* Case 3: Nothing worked: error out */
+ die(_("Cannot construct a valid gitdir path for submodule '%s': "
+ "please set a unique git config for 'submodule.%s.gitdir'."),
+ submodule_name, submodule_name);
}
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..62794df976
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,24 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (e.g. when adding a new submodule), so this only
+# checks the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ # Compute expected absolute path
+ expected="$(git rev-parse --git-common-dir)/$path" &&
+ expected="$(test-tool path-utils real_path "$expected")" &&
+ # Compute actual absolute path
+ actual="$(git submodule--helper gitdir "$name")" &&
+ actual="$(test-tool path-utils real_path "$actual")" &&
+ echo "$expected" >expect &&
+ echo "$actual" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/meson.build b/t/meson.build
index a5531df415..4187b35aee 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -884,6 +884,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-encoding.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
new file mode 100755
index 0000000000..f877887549
--- /dev/null
+++ b/t/t7425-submodule-encoding.sh
@@ -0,0 +1,164 @@
+#!/bin/sh
+
+test_description='submodules handle mixed legacy and new (encoded) style gitdir paths'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed encoded and non-encoded submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ # trigger the "die_path_inside_submodule" check
+ test_must_fail git submodule add ../new-sub "legacy/nested" &&
+
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submoduleEncoding true &&
+
+ git submodule add ../new-sub "New Sub" &&
+ test_commit new &&
+
+ # retrigger the "die_path_inside_submodule" check with encoding
+ test_must_fail git submodule add ../new-sub "New Sub/nested2"
+ )
+'
+
+test_expect_success 'verify submodule name is properly encoded' '
+ verify_submodule_gitdir_path main legacy modules/legacy &&
+ verify_submodule_gitdir_path main "New Sub" "modules/New Sub"
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned-non-encoding &&
+ (
+ cd cloned-non-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/"New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ ) &&
+
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules main cloned-encoding &&
+ (
+ cd cloned-encoding &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'commit and push changes to encoded submodules' '
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ git -C main config receive.denyCurrentBranch updateInstead &&
+ (
+ cd cloned-encoding &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C "New Sub" switch --track -C main origin/main &&
+ test_commit -C "New Sub" second-commit &&
+ git -C "New Sub" push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy "New Sub" &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules nested clone_nested &&
+ verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone -c extensions.submoduleEncoding=true --recurse-submodules --jobs=2 nested clone_parallel &&
+ verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks
+'
+
+test_expect_success 'disabling extensions.submoduleEncoding prevents nested submodules' '
+ (
+ cd clone_nested &&
+ # disable extension and verify failure
+ git config extensions.submoduleEncoding false &&
+ test_must_fail git submodule add ./thing2 hippo/foobar &&
+ # re-enable extension and verify it works
+ git config extensions.submoduleEncoding true &&
+ git submodule add ./thing2 hippo/foobar
+ )
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 964e1f1569..ffb9c8b522 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 5/7] submodule: fix case-folding gitdir filesystem colisions
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
` (3 preceding siblings ...)
2025-11-19 21:10 ` [PATCH v5 4/7] submodule: add extension to encode gitdir paths Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 6/7] submodule: use hashed name for gitdir Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time Adrian Ratiu
6 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Add a new check when extension.submoduleEncoding is enabled to
detect and prevent case-folding filesystem colisions. When this
new check is triggered, a stricter casefolding aware URI encoding
is used to percent-encode uppercase characters.
By using this check/retry mechanism the uppercase encoding is
only applied when necessary, so case-sensitive filesystems are
not affected.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 79 ++++++++++++++++++++++++++++++++++-
t/t7425-submodule-encoding.sh | 35 ++++++++++++++++
url.c | 12 ++++++
url.h | 1 +
4 files changed, 125 insertions(+), 2 deletions(-)
diff --git a/submodule.c b/submodule.c
index 07cb4694cf..b3f74f7e3c 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2238,15 +2238,58 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
+static int check_casefolding_conflict(const char *git_dir,
+ const char *submodule_name,
+ const bool suffixes_match)
+{
+ char *p, *modules_dir = xstrdup(git_dir);
+ struct dirent *de;
+ DIR *dir = NULL;
+ int ret = 0;
+
+ if ((p = find_last_dir_sep(modules_dir)))
+ *p = '\0';
+
+ /* No conflict is possible if modules_dir doesn't exist (first clone) */
+ if (!is_directory(modules_dir))
+ goto cleanup;
+
+ dir = opendir(modules_dir);
+ if (!dir) {
+ ret = -1;
+ goto cleanup;
+ }
+
+ /* Check for another directory under .git/modules that differs only in case. */
+ while ((de = readdir(dir))) {
+ if (!strcmp(de->d_name, ".") || !strcmp(de->d_name, ".."))
+ continue;
+
+ if ((suffixes_match || is_git_directory(git_dir)) &&
+ !strcasecmp(de->d_name, submodule_name) &&
+ strcmp(de->d_name, submodule_name)) {
+ ret = -1; /* collision found */
+ break;
+ }
+ }
+
+cleanup:
+ if (dir)
+ closedir(dir);
+ FREE_AND_NULL(modules_dir);
+ return ret;
+}
+
/*
* Encoded gitdir validation function used when extensions.submoduleEncoding is enabled.
* This does not print errors like the non-encoded version, because encoding is supposed
* to mitigate / fix all these.
*/
-static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name)
{
const char *modules_marker = "/modules/";
char *p = git_dir, *last_submodule_name = NULL;
+ int config_ignorecase = 0;
if (!the_repository->repository_format_submodule_encoding)
BUG("validate_submodule_encoded_git_dir() must be called with "
@@ -2262,6 +2305,14 @@ static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodu
if (!last_submodule_name || strchr(last_submodule_name, '/'))
return -1;
+ /* Prevent conflicts on case-folding filesystems */
+ repo_config_get_bool(the_repository, "core.ignorecase", &config_ignorecase);
+ if (ignore_case || config_ignorecase) {
+ bool suffixes_match = !strcmp(last_submodule_name, submodule_name);
+ return check_casefolding_conflict(git_dir, submodule_name,
+ suffixes_match);
+ }
+
return 0;
}
@@ -2650,13 +2701,37 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
if (!validate_and_set_submodule_gitdir(buf, submodule_name))
return;
- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
strbuf_reset(buf);
repo_git_path_append(r, buf, "modules/");
strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
if (!validate_and_set_submodule_gitdir(buf, submodule_name))
return;
+ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr_urlencode(buf, submodule_name, is_casefolding_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
+ /* Case 2.3: Try some derived gitdir names, see if one sticks */
+ for (char c = '0'; c <= '9'; c++) {
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
+ strbuf_addch(buf, c);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr_urlencode(buf, submodule_name, is_casefolding_rfc3986_unreserved);
+ strbuf_addch(buf, c);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+ }
+
/* Case 3: Nothing worked: error out */
die(_("Cannot construct a valid gitdir path for submodule '%s': "
"please set a unique git config for 'submodule.%s.gitdir'."),
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
index f877887549..093238939a 100755
--- a/t/t7425-submodule-encoding.sh
+++ b/t/t7425-submodule-encoding.sh
@@ -161,4 +161,39 @@ test_expect_success 'disabling extensions.submoduleEncoding prevents nested subm
)
'
+test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are correctly encoded' '
+ git clone -c extensions.submoduleEncoding=true main cloned-folding &&
+ (
+ cd cloned-folding &&
+
+ # conflict: the "folding" gitdir will already be taken
+ git submodule add ../new-sub "folding" &&
+ test_commit lowercase &&
+ git submodule add ../new-sub "FoldinG" &&
+ test_commit uppercase &&
+
+ # conflict: the "foo" gitdir will already be taken
+ git submodule add ../new-sub "FOO" &&
+ test_commit uppercase-foo &&
+ git submodule add ../new-sub "foo" &&
+ test_commit lowercase-foo &&
+
+ # create a multi conflict between foobar, fooBar and foo%42ar
+ # the "foo" gitdir will already be taken
+ git submodule add ../new-sub "foobar" &&
+ test_commit lowercase-foobar &&
+ git submodule add ../new-sub "foo%42ar" &&
+ test_commit encoded-foo%42ar &&
+ git submodule add ../new-sub "fooBar" &&
+ test_commit mixed-fooBar
+ ) &&
+ verify_submodule_gitdir_path cloned-folding "folding" "modules/folding" &&
+ verify_submodule_gitdir_path cloned-folding "FoldinG" "modules/%46oldin%47" &&
+ verify_submodule_gitdir_path cloned-folding "FOO" "modules/FOO" &&
+ verify_submodule_gitdir_path cloned-folding "foo" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-folding "foobar" "modules/foobar" &&
+ verify_submodule_gitdir_path cloned-folding "foo%42ar" "modules/foo%42ar" &&
+ verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
+'
+
test_done
diff --git a/url.c b/url.c
index 0fb1859b28..057e6e5c6e 100644
--- a/url.c
+++ b/url.c
@@ -14,6 +14,18 @@ int is_rfc3986_unreserved(char ch)
ch == '-' || ch == '_' || ch == '.' || ch == '~';
}
+/*
+ * This is a variant of is_rfc3986_unreserved() that treats uppercase
+ * letters as "reserved". This forces them to be percent-encoded, allowing
+ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
+ */
+int is_casefolding_rfc3986_unreserved(char c)
+{
+ return (c >= 'a' && c <= 'z') ||
+ (c >= '0' && c <= '9') ||
+ c == '-' || c == '.' || c == '_' || c == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 131a262066..92e3c63514 100644
--- a/url.h
+++ b/url.h
@@ -22,5 +22,6 @@ void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
int is_rfc3986_unreserved(char ch);
+int is_casefolding_rfc3986_unreserved(char c);
#endif /* URL_H */
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 6/7] submodule: use hashed name for gitdir
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
` (4 preceding siblings ...)
2025-11-19 21:10 ` [PATCH v5 5/7] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time Adrian Ratiu
6 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
If none of the previous steps work and we reach case 2.4, try to
hash the submodule name and see if that can be a valid gitdir
before giving up and throwing an error.
This is a "last resort" type of measure to avoid conflicts since
it loses the gitdir human readability. Itis not such a big deal
because users are now supposed to use the submodule.<name>.gitdir
config as the single source of truth for gitdir paths.
This logic will be reached in very rare cases, as can be seen in
the test we added.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 20 +++++++++++-
t/t7425-submodule-encoding.sh | 59 +++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+), 1 deletion(-)
diff --git a/submodule.c b/submodule.c
index b3f74f7e3c..2c0df96d55 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2660,8 +2660,12 @@ static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
+ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
+ char hex_name_hash[GIT_MAX_HEXSZ + 1];
+ struct git_hash_ctx ctx;
const char *gitdir;
- char *key;
+ char *key, header[128];
+ int header_len;
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
@@ -2732,6 +2736,20 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
return;
}
+ /* Case 2.4: If all the above failed, try a hash of the name as a last resort */
+ header_len = snprintf(header, sizeof(header), "blob %zu", strlen(submodule_name));
+ the_hash_algo->init_fn(&ctx);
+ the_hash_algo->update_fn(&ctx, header, header_len);
+ the_hash_algo->update_fn(&ctx, "\0", 1);
+ the_hash_algo->update_fn(&ctx, submodule_name, strlen(submodule_name));
+ the_hash_algo->final_fn(raw_name_hash, &ctx);
+ hash_to_hex_algop_r(hex_name_hash, raw_name_hash, the_hash_algo);
+ strbuf_reset(buf);
+ repo_git_path_append(r, buf, "modules/");
+ strbuf_addstr(buf, hex_name_hash);
+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
+ return;
+
/* Case 3: Nothing worked: error out */
die(_("Cannot construct a valid gitdir path for submodule '%s': "
"please set a unique git config for 'submodule.%s.gitdir'."),
diff --git a/t/t7425-submodule-encoding.sh b/t/t7425-submodule-encoding.sh
index 093238939a..10703b34c8 100755
--- a/t/t7425-submodule-encoding.sh
+++ b/t/t7425-submodule-encoding.sh
@@ -196,4 +196,63 @@ test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are corre
verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
'
+test_expect_success CASE_INSENSITIVE_FS 'verify hashing conflict resolution as a last resort' '
+ git clone -c extensions.submoduleEncoding=true main cloned-hash &&
+ (
+ cd cloned-hash &&
+
+ # conflict: add all submodule conflicting variants until we reach the
+ # final hashing conflict resolution for submodule "foo"
+ git submodule add ../new-sub "foo" &&
+ git submodule add ../new-sub "foo0" &&
+ git submodule add ../new-sub "foo1" &&
+ git submodule add ../new-sub "foo2" &&
+ git submodule add ../new-sub "foo3" &&
+ git submodule add ../new-sub "foo4" &&
+ git submodule add ../new-sub "foo5" &&
+ git submodule add ../new-sub "foo6" &&
+ git submodule add ../new-sub "foo7" &&
+ git submodule add ../new-sub "foo8" &&
+ git submodule add ../new-sub "foo9" &&
+ git submodule add ../new-sub "%46oo" &&
+ git submodule add ../new-sub "%46oo0" &&
+ git submodule add ../new-sub "%46oo1" &&
+ git submodule add ../new-sub "%46oo2" &&
+ git submodule add ../new-sub "%46oo3" &&
+ git submodule add ../new-sub "%46oo4" &&
+ git submodule add ../new-sub "%46oo5" &&
+ git submodule add ../new-sub "%46oo6" &&
+ git submodule add ../new-sub "%46oo7" &&
+ git submodule add ../new-sub "%46oo8" &&
+ git submodule add ../new-sub "%46oo9" &&
+ test_commit add-foo-variants &&
+ git submodule add ../new-sub "Foo" &&
+ test_commit add-uppercase-foo
+ ) &&
+ verify_submodule_gitdir_path cloned-hash "foo" "modules/foo" &&
+ verify_submodule_gitdir_path cloned-hash "foo0" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-hash "foo1" "modules/foo1" &&
+ verify_submodule_gitdir_path cloned-hash "foo2" "modules/foo2" &&
+ verify_submodule_gitdir_path cloned-hash "foo3" "modules/foo3" &&
+ verify_submodule_gitdir_path cloned-hash "foo4" "modules/foo4" &&
+ verify_submodule_gitdir_path cloned-hash "foo5" "modules/foo5" &&
+ verify_submodule_gitdir_path cloned-hash "foo6" "modules/foo6" &&
+ verify_submodule_gitdir_path cloned-hash "foo7" "modules/foo7" &&
+ verify_submodule_gitdir_path cloned-hash "foo8" "modules/foo8" &&
+ verify_submodule_gitdir_path cloned-hash "foo9" "modules/foo9" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo" "modules/%46oo" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo0" "modules/%46oo0" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo1" "modules/%46oo1" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo2" "modules/%46oo2" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo3" "modules/%46oo3" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo4" "modules/%46oo4" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo5" "modules/%46oo5" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo6" "modules/%46oo6" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo7" "modules/%46oo7" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo8" "modules/%46oo8" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo9" "modules/%46oo9" &&
+ hash=$(printf "Foo" | git hash-object --stdin) &&
+ verify_submodule_gitdir_path cloned-hash "Foo" "modules/${hash}"
+'
+
test_done
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
` (5 preceding siblings ...)
2025-11-19 21:10 ` [PATCH v5 6/7] submodule: use hashed name for gitdir Adrian Ratiu
@ 2025-11-19 21:10 ` Adrian Ratiu
2025-12-05 12:19 ` Patrick Steinhardt
6 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-11-19 21:10 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Some users find it difficult to distribute repo config changes for
enabling extensions.submoduleEncoding, or to enable it by passing
the config via cmdline, so we add a build-time option which can
enable the extension for convenience.
It is still disabled by default and the build-time default is
overridden by the repo-specific configs.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 2 ++
Makefile | 5 +++++
configure.ac | 23 +++++++++++++++++++++++
meson.build | 4 ++++
meson_options.txt | 2 ++
setup.c | 8 ++++++++
6 files changed, 44 insertions(+)
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 4861d01894..436f7fb52e 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -78,6 +78,8 @@ submoduleEncoding:::
conflicts due to nested gitdirs, case insensitivity or other issues.
When enabled, the submodule.<name>.gitdir config is always set for
all submodules and is the single point of authority for gitdir paths.
+ Can also be enabled via the submodule-encoding build option. The repo
+ config takes precedence over the build-time default.
worktreeConfig:::
If enabled, then worktrees will load config settings from the
diff --git a/Makefile b/Makefile
index 7e0f77e298..4c7bf75c68 100644
--- a/Makefile
+++ b/Makefile
@@ -2344,6 +2344,11 @@ ifdef INCLUDE_LIBGIT_RS
BASIC_CFLAGS += -fvisibility=hidden
endif
+ifdef SUBMODULE_ENCODING_BY_DEFAULT
+ # Set submoduleEncoding extension default specified at build time
+ BASIC_CFLAGS += -DSUBMODULE_ENCODING_BY_DEFAULT=$(SUBMODULE_ENCODING_BY_DEFAULT)
+endif
+
ifeq ($(TCLTK_PATH),)
NO_TCLTK = NoThanks
endif
diff --git a/configure.ac b/configure.ac
index cfb50112bf..202b1e309b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -229,6 +229,29 @@ AC_ARG_ENABLE([cssmin],
GIT_CONF_SUBST([CSSMIN])
])
+# Define option to enable the submodule encoding extension by default
+AC_ARG_ENABLE([submodule-encoding],
+ [AS_HELP_STRING([--enable-submodule-encoding],
+ [Enable the submoduleEncoding extension by default at build time.]
+ [--disable-submodule-encoding will keep the current default (disabled).])],
+[
+case "$enableval" in
+ yes) SUBMODULE_ENCODING_BY_DEFAULT=1
+ AC_MSG_NOTICE([Submodule encoding will be enabled by default.])
+ ;;
+ no) SUBMODULE_ENCODING_BY_DEFAULT=0
+ AC_MSG_NOTICE([Submodule encoding will not be enabled by default.])
+ ;;
+ *) AC_MSG_ERROR([--enable-submodule-encoding takes yes or no.])
+ ;;
+esac
+],
+[
+ SUBMODULE_ENCODING_BY_DEFAULT=0
+])
+
+GIT_CONF_SUBST([SUBMODULE_ENCODING_BY_DEFAULT])
+
## Site configuration (override autodetection)
## --with-PACKAGE[=ARG] and --without-PACKAGE
AC_MSG_NOTICE([CHECKS for site configuration])
diff --git a/meson.build b/meson.build
index 1f95a06edb..992d433efd 100644
--- a/meson.build
+++ b/meson.build
@@ -945,6 +945,10 @@ else
build_options_config.set('NO_PERL_CPAN_FALLBACKS', '')
endif
+submodule_encoding_by_default = get_option('submodule-encoding').to_int()
+libgit_c_args += '-DSUBMODULE_ENCODING_BY_DEFAULT=' + submodule_encoding_by_default.to_string()
+build_options_config.set('SUBMODULE_ENCODING_BY_DEFAULT', submodule_encoding_by_default)
+
zlib_backend = get_option('zlib_backend')
if zlib_backend in ['auto', 'zlib-ng']
zlib_ng = dependency('zlib-ng', required: zlib_backend == 'zlib-ng')
diff --git a/meson_options.txt b/meson_options.txt
index e0be260ae1..d9d569a166 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -21,6 +21,8 @@ option('runtime_prefix', type: 'boolean', value: false,
description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
option('sane_tool_path', type: 'array', value: [],
description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
+option('submodule-encoding', type: 'boolean', value: false,
+ description: 'Enable submoduleEncoding extension by default at build time.')
# Build information compiled into Git and other parts like documentation.
option('build_date', type: 'string', value: '',
diff --git a/setup.c b/setup.c
index bf6e815105..de2d4e7238 100644
--- a/setup.c
+++ b/setup.c
@@ -1747,6 +1747,14 @@ const char *setup_git_directory_gently(int *nongit_ok)
*/
repo_config_clear(the_repository);
+ /*
+ * Set build-time default for submodule encoding.
+ * This can be overridden by the repository's config.
+ */
+#ifdef SUBMODULE_ENCODING_BY_DEFAULT
+ repo_fmt.submodule_encoding = SUBMODULE_ENCODING_BY_DEFAULT;
+#endif
+
/*
* Let's assume that we are in a git repository.
* If it turns out later that we are somewhere else, the value will be
--
2.51.0
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-11-19 21:10 ` [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
@ 2025-12-05 12:16 ` Patrick Steinhardt
2025-12-05 17:25 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-05 12:16 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Wed, Nov 19, 2025 at 11:10:25PM +0200, Adrian Ratiu wrote:
> is_rfc3986_unreserved() was moved to credential-store.c and was made
> static by f89854362c (credential-store: move related functions to
> credential-store file, 2023-06-06) under a correct assumption, at the
> time, that it was the only place using it.
>
> However now we need it to apply URL-encoding to submodule names when
> constructing gitdir paths, to avoid conflicts, so bring it back as a
> public function exposed via url.h, instead of the old helper path
> (strbuf), which has nothing to do with 3986 encoding/decoding anymore.
>
> This function will be used by submodule.c in the next commit.
Nit: this statement isn't true anymore :)
> diff --git a/url.c b/url.c
> index 282b12495a..0fb1859b28 100644
> --- a/url.c
> +++ b/url.c
> @@ -3,6 +3,17 @@
> #include "strbuf.h"
> #include "url.h"
>
> +/*
> + * The set of unreserved characters as per STD66 (RFC3986) is
> + * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
> + * components without percent-encoding.
> + */
> +int is_rfc3986_unreserved(char ch)
> +{
> + return isalnum(ch) ||
> + ch == '-' || ch == '_' || ch == '.' || ch == '~';
> +}
> +
> int is_urlschemechar(int first_flag, int ch)
> {
> /*
Nit: the function documentation should rather live in the header file.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-11-19 21:10 ` [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
@ 2025-12-05 12:17 ` Patrick Steinhardt
2025-12-05 18:17 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-05 12:17 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Wed, Nov 19, 2025 at 11:10:26PM +0200, Adrian Ratiu wrote:
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 2873b2780e..9914ca0786 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1780,23 +1776,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
> free(path);
> }
>
> - /*
> - * We already performed this check at the beginning of this function,
> - * before cloning the objects. This tries to detect racy behavior e.g.
> - * in parallel clones, where another process could easily have made the
> - * gitdir nested _after_ it was created.
> - *
> - * To prevent further harm coming from this unintentionally-nested
> - * gitdir, let's disable it by deleting the `HEAD` file.
> - */
> - if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
> - char *head = xstrfmt("%s/HEAD", sm_gitdir);
> - unlink(head);
> - free(head);
> - die(_("refusing to create/use '%s' in another submodule's "
> - "git dir"), sm_gitdir);
> - }
> -
> connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
>
> p = repo_submodule_path(the_repository, clone_data_path, "config");
Hm. This one is a bit puzzling to me. This seems to explicitly be a
check about a TOCTOU-style race, where a concurrent process might have
created the parent repository after our initial validation of the path.
We don't call `submodule_name_to_gitdir()` inbetween those two calls
though, so why is this not a concern anymore with the unified API?
> diff --git a/submodule.c b/submodule.c
> index 35c55155f7..8ef028f26b 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2153,30 +2153,11 @@ int submodule_move_head(const char *path, const char *super_prefix,
>
> if (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {
> if (old_head) {
> - if (!submodule_uses_gitfile(path))
> - absorb_git_dir_into_superproject(path,
> - super_prefix);
> - else {
> - char *dotgit = xstrfmt("%s/.git", path);
> - char *git_dir = xstrdup(read_gitfile(dotgit));
> -
> - free(dotgit);
> - if (validate_submodule_git_dir(git_dir,
> - sub->name) < 0)
> - die(_("refusing to create/use '%s' in "
> - "another submodule's git dir"),
> - git_dir);
> - free(git_dir);
> - }
> + absorb_git_dir_into_superproject(path, super_prefix);
> } else {
> struct strbuf gitdir = STRBUF_INIT;
> submodule_name_to_gitdir(&gitdir, the_repository,
> sub->name);
> - if (validate_submodule_git_dir(gitdir.buf,
> - sub->name) < 0)
> - die(_("refusing to create/use '%s' in another "
> - "submodule's git dir"),
> - gitdir.buf);
> connect_work_tree_and_git_dir(path, gitdir.buf, 0);
> strbuf_release(&gitdir);
The second case here makes sense to me, as we do call
`submodule_name_to_gitdir()`. But in the first branch of the condition
we retrieve the path directly, so we're not guarded by the validation
anymore, are we?
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-11-19 21:10 ` [PATCH v5 4/7] submodule: add extension to encode gitdir paths Adrian Ratiu
@ 2025-12-05 12:19 ` Patrick Steinhardt
2025-12-05 19:30 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-05 12:19 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Wed, Nov 19, 2025 at 11:10:27PM +0200, Adrian Ratiu wrote:
> Add a submoduleEncoding extension which fixes filesystem collisions by
> encoding gitdir paths. At a high level, this implements a mechanism to
> encode -> validate -> retry until a working gitdir path is found.
>
> Credit goes to Junio for coming up with this design: encoding is only
> applied when necessary, e.g. uppercase characters are encoded only on
> case-folding filesystems and only if a real conflict is detected.
>
> To make this work, we rely on the submodule.<name>.gitdir config as the
> single source of truth for gitidir paths: the config is always set when
s/gitidir/gitdir/
> diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
> index 532456644b..4861d01894 100644
> --- a/Documentation/config/extensions.adoc
> +++ b/Documentation/config/extensions.adoc
> @@ -73,6 +73,12 @@ relativeWorktrees:::
> repaired with either the `--relative-paths` option or with the
> `worktree.useRelativePaths` config set to `true`.
>
> +submoduleEncoding:::
> + If enabled, submodule gitdir paths are encoded to avoid filesystem
> + conflicts due to nested gitdirs, case insensitivity or other issues.
> + When enabled, the submodule.<name>.gitdir config is always set for
> + all submodules and is the single point of authority for gitdir paths.
> +
> worktreeConfig:::
> If enabled, then worktrees will load config settings from the
> `$GIT_DIR/config.worktree` file in addition to the
I think the fact that the submodule gitdir paths are encoded now is
secondary to this repository extension. The more important fact is that
this changes the source of truth where the submodule gitdir path is
actually derived from: before it was derived on the fly, whereas now it
is persisted in the gitconfig.
It follows that because the source of truth is now a persistent entry in
the configuration, other implementations can read it without having to
understand how exactly the value was computed in the first place. So an
implementation may arbitrarily change the algorithm it uses to derive
that path from now on, and it doesn't necessarily have to encode
anything.
So I'd propose to rename the extension and rephrase its description
accordingly. It could for example be called something along the lines of
"submodulePathConfig".
> diff --git a/submodule.c b/submodule.c
> index 8ef028f26b..07cb4694cf 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2559,33 +2591,74 @@ int submodule_to_gitdir(struct repository *repo,
> return ret;
> }
>
> +static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
> + const char *submodule_name)
> +{
> + char *key;
> +
> + if (validate_submodule_encoded_git_dir(gitdir_path->buf, submodule_name))
> + return -1;
> +
> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
> + repo_config_set_gently(the_repository, key, gitdir_path->buf);
> + FREE_AND_NULL(key);
I think a simple call to `free()` should be sufficient here. There is no
risk of it being used afterwards.
> + return 0;
> +}
> +
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name)
> {
> + const char *gitdir;
> + char *key;
> +
> + repo_git_path_append(r, buf, "modules/");
> + strbuf_addstr(buf, submodule_name);
> +
> + /* If extensions.submoduleEncoding is disabled, use the plain path set above */
> + if (!r->repository_format_submodule_encoding) {
> + if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
> + die(_("refusing to create/use '%s' in another submodule's "
> + "git dir"), buf->buf);
> +
> + return; /* plain gitdir is valid for use */
> + }
> +
> + /* Extension is enabled: use the gitdir config if it exists */
> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
> + if (!repo_config_get_string_tmp(r, key, &gitdir)) {
> + strbuf_reset(buf);
> + strbuf_addstr(buf, gitdir);
> + FREE_AND_NULL(key);
> +
> + /* validate because users might have modified the config */
> + if (validate_submodule_encoded_git_dir(buf->buf, submodule_name))
> + die(_("Invalid 'submodule.%s.gitdir' config: '%s' please check "
> + "if it is unique or conflicts with another module"),
Nit: error messages start with a lower-case character.
> + submodule_name, gitdir);
> +
> + return;
> + }
> + FREE_AND_NULL(key);
> +
> /*
> - * NEEDSWORK: The current way of mapping a submodule's name to
> - * its location in .git/modules/ has problems with some naming
> - * schemes. For example, if a submodule is named "foo" and
> - * another is named "foo/bar" (whether present in the same
> - * superproject commit or not - the problem will arise if both
> - * superproject commits have been checked out at any point in
> - * time), or if two submodule names only have different cases in
> - * a case-insensitive filesystem.
> - *
> - * There are several solutions, including encoding the path in
> - * some way, introducing a submodule.<name>.gitdir config in
> - * .git/config (not .gitmodules) that allows overriding what the
> - * gitdir of a submodule would be (and teach Git, upon noticing
> - * a clash, to automatically determine a non-clashing name and
> - * to write such a config), or introducing a
> - * submodule.<name>.gitdir config in .gitmodules that repo
> - * administrators can explicitly set. Nothing has been decided,
> - * so for now, just append the name at the end of the path.
> + * The gitdir config does not exist, even though the extension is enabled.
> + * Therefore we are in one of the following cases:
> */
> +
> + /* Case 1: legacy migration of valid plain submodule names */
> + if (!validate_and_set_submodule_gitdir(buf, submodule_name))
> + return;
> +
> + /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
> + strbuf_reset(buf);
> repo_git_path_append(r, buf, "modules/");
> - strbuf_addstr(buf, submodule_name);
> + strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
> + if (!validate_and_set_submodule_gitdir(buf, submodule_name))
> + return;
>
> - if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
> - die(_("refusing to create/use '%s' in another submodule's "
> - "git dir"), buf->buf);
> + /* Case 3: Nothing worked: error out */
> + die(_("Cannot construct a valid gitdir path for submodule '%s': "
> + "please set a unique git config for 'submodule.%s.gitdir'."),
> + submodule_name, submodule_name);
It feels somewhat fragile to me that we unconditionally handle these
cases and try to find old submodule directories. If the extension is
enabled I'd expect that the submodule configuration is the _only_ source
of truth.
May I propose that we instead always error out in case the submodule
configuration does not exist? In the best case we'd then give the user a
nice error message that tells them how to run the migration manually.
(Coming back from reading subsequent patches) Maybe what's putting me
off is that this function is seemingly used for two things:
1. To derive the submodule path in case we know it should already
exist.
2. To compute the submodule path so we can end up writing it into the
"submodule.*.gitdir" variable.
I think we should tell these two cases apart. In the first case I expect
that we never fall back to a computed name, but bail out in case the
configuration key does not exist. And in the second case it of course
makes sense to compute the actual path that we want to store in the
configuration.
Thanks!
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-11-19 21:10 ` [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time Adrian Ratiu
@ 2025-12-05 12:19 ` Patrick Steinhardt
2025-12-05 19:42 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-05 12:19 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
> Some users find it difficult to distribute repo config changes for
> enabling extensions.submoduleEncoding, or to enable it by passing
> the config via cmdline, so we add a build-time option which can
> enable the extension for convenience.
Wouldn't it be more sensible to make this a runtime configuration key
that users can configure in their gitconfig?
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-12-05 12:16 ` Patrick Steinhardt
@ 2025-12-05 17:25 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-05 17:25 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Wed, Nov 19, 2025 at 11:10:25PM +0200, Adrian Ratiu wrote:
>> is_rfc3986_unreserved() was moved to credential-store.c and was made
>> static by f89854362c (credential-store: move related functions to
>> credential-store file, 2023-06-06) under a correct assumption, at the
>> time, that it was the only place using it.
>>
>> However now we need it to apply URL-encoding to submodule names when
>> constructing gitdir paths, to avoid conflicts, so bring it back as a
>> public function exposed via url.h, instead of the old helper path
>> (strbuf), which has nothing to do with 3986 encoding/decoding anymore.
>>
>> This function will be used by submodule.c in the next commit.
>
> Nit: this statement isn't true anymore :)
>
Indeed, thanks for catching these.
I will fix both nits in v6.
>> diff --git a/url.c b/url.c
>> index 282b12495a..0fb1859b28 100644
>> --- a/url.c
>> +++ b/url.c
>> @@ -3,6 +3,17 @@
>> #include "strbuf.h"
>> #include "url.h"
>>
>> +/*
>> + * The set of unreserved characters as per STD66 (RFC3986) is
>> + * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
>> + * components without percent-encoding.
>> + */
>> +int is_rfc3986_unreserved(char ch)
>> +{
>> + return isalnum(ch) ||
>> + ch == '-' || ch == '_' || ch == '.' || ch == '~';
>> +}
>> +
>> int is_urlschemechar(int first_flag, int ch)
>> {
>> /*
>
> Nit: the function documentation should rather live in the header file.
>
> Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-12-05 12:17 ` Patrick Steinhardt
@ 2025-12-05 18:17 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-05 18:17 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Wed, Nov 19, 2025 at 11:10:26PM +0200, Adrian Ratiu wrote:
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index 2873b2780e..9914ca0786 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -1780,23 +1776,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
>> free(path);
>> }
>>
>> - /*
>> - * We already performed this check at the beginning of this function,
>> - * before cloning the objects. This tries to detect racy behavior e.g.
>> - * in parallel clones, where another process could easily have made the
>> - * gitdir nested _after_ it was created.
>> - *
>> - * To prevent further harm coming from this unintentionally-nested
>> - * gitdir, let's disable it by deleting the `HEAD` file.
>> - */
>> - if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
>> - char *head = xstrfmt("%s/HEAD", sm_gitdir);
>> - unlink(head);
>> - free(head);
>> - die(_("refusing to create/use '%s' in another submodule's "
>> - "git dir"), sm_gitdir);
>> - }
>> -
>> connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
>>
>> p = repo_submodule_path(the_repository, clone_data_path, "config");
>
> Hm. This one is a bit puzzling to me. This seems to explicitly be a
> check about a TOCTOU-style race, where a concurrent process might have
> created the parent repository after our initial validation of the path.
> We don't call `submodule_name_to_gitdir()` inbetween those two calls
> though, so why is this not a concern anymore with the unified API?
Excellent catch.
It still is a concern and this specific check needs to be added back.
My aim with this patch is to:
1. Avoid redundant validation calls for submodule_name_to_gitdir().
2. Avoid the risk of callers forgetting to validate.
3. Ensure gitdir paths provided by users via configs are always valid,
so I added the validation directly into submodule_name_to_gitdir().
Nothing prevents us from running the validation as many times as we
need and, in this case, it's perfectly justified to run it twice.
I'll add it back in v6.
>> diff --git a/submodule.c b/submodule.c
>> index 35c55155f7..8ef028f26b 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -2153,30 +2153,11 @@ int submodule_move_head(const char *path, const char *super_prefix,
>>
>> if (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {
>> if (old_head) {
>> - if (!submodule_uses_gitfile(path))
>> - absorb_git_dir_into_superproject(path,
>> - super_prefix);
>> - else {
>> - char *dotgit = xstrfmt("%s/.git", path);
>> - char *git_dir = xstrdup(read_gitfile(dotgit));
>> -
>> - free(dotgit);
>> - if (validate_submodule_git_dir(git_dir,
>> - sub->name) < 0)
>> - die(_("refusing to create/use '%s' in "
>> - "another submodule's git dir"),
>> - git_dir);
>> - free(git_dir);
>> - }
>> + absorb_git_dir_into_superproject(path, super_prefix);
>> } else {
>> struct strbuf gitdir = STRBUF_INIT;
>> submodule_name_to_gitdir(&gitdir, the_repository,
>> sub->name);
>> - if (validate_submodule_git_dir(gitdir.buf,
>> - sub->name) < 0)
>> - die(_("refusing to create/use '%s' in another "
>> - "submodule's git dir"),
>> - gitdir.buf);
>> connect_work_tree_and_git_dir(path, gitdir.buf, 0);
>> strbuf_release(&gitdir);
>
> The second case here makes sense to me, as we do call
> `submodule_name_to_gitdir()`. But in the first branch of the condition
> we retrieve the path directly, so we're not guarded by the validation
> anymore, are we?
The !submodule_uses_gitfile(path) case never validated and relied on the
validation done by absorb_git_dir_into_superproject() which calls both
validate_submodule_path() and submodule_name_to_gitdir() and has
additional checks.
So in essence the validation moved to absorb_git_dir_into_superproject().
If you prefer I can try to refactor absorb_git_dir_into_superproject() a
bit to make it clearer, otherwise I can just add back the validation
here as well... we can validate as many times as we like. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-05 12:19 ` Patrick Steinhardt
@ 2025-12-05 19:30 ` Adrian Ratiu
2025-12-05 22:47 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-05 19:30 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Wed, Nov 19, 2025 at 11:10:27PM +0200, Adrian Ratiu wrote:
>> Add a submoduleEncoding extension which fixes filesystem collisions by
>> encoding gitdir paths. At a high level, this implements a mechanism to
>> encode -> validate -> retry until a working gitdir path is found.
>>
>> Credit goes to Junio for coming up with this design: encoding is only
>> applied when necessary, e.g. uppercase characters are encoded only on
>> case-folding filesystems and only if a real conflict is detected.
>>
>> To make this work, we rely on the submodule.<name>.gitdir config as the
>> single source of truth for gitidir paths: the config is always set when
>
> s/gitidir/gitdir/
Ack, will fix.
>
>> diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
>> index 532456644b..4861d01894 100644
>> --- a/Documentation/config/extensions.adoc
>> +++ b/Documentation/config/extensions.adoc
>> @@ -73,6 +73,12 @@ relativeWorktrees:::
>> repaired with either the `--relative-paths` option or with the
>> `worktree.useRelativePaths` config set to `true`.
>>
>> +submoduleEncoding:::
>> + If enabled, submodule gitdir paths are encoded to avoid filesystem
>> + conflicts due to nested gitdirs, case insensitivity or other issues.
>> + When enabled, the submodule.<name>.gitdir config is always set for
>> + all submodules and is the single point of authority for gitdir paths.
>> +
>> worktreeConfig:::
>> If enabled, then worktrees will load config settings from the
>> `$GIT_DIR/config.worktree` file in addition to the
>
> I think the fact that the submodule gitdir paths are encoded now is
> secondary to this repository extension. The more important fact is that
> this changes the source of truth where the submodule gitdir path is
> actually derived from: before it was derived on the fly, whereas now it
> is persisted in the gitconfig.
>
> It follows that because the source of truth is now a persistent entry in
> the configuration, other implementations can read it without having to
> understand how exactly the value was computed in the first place. So an
> implementation may arbitrarily change the algorithm it uses to derive
> that path from now on, and it doesn't necessarily have to encode
> anything.
>
> So I'd propose to rename the extension and rephrase its description
> accordingly. It could for example be called something along the lines of
> "submodulePathConfig".
I think this is a very reasonable suggestion, thanks!
If nobody has objections or better suggestions, I will rename the
extension to "submodulePathConfig" and reword the description as you
suggested.
>
>> diff --git a/submodule.c b/submodule.c
>> index 8ef028f26b..07cb4694cf 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -2559,33 +2591,74 @@ int submodule_to_gitdir(struct repository *repo,
>> return ret;
>> }
>>
>> +static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
>> + const char *submodule_name)
>> +{
>> + char *key;
>> +
>> + if (validate_submodule_encoded_git_dir(gitdir_path->buf, submodule_name))
>> + return -1;
>> +
>> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
>> + repo_config_set_gently(the_repository, key, gitdir_path->buf);
>> + FREE_AND_NULL(key);
>
> I think a simple call to `free()` should be sufficient here. There is no
> risk of it being used afterwards.
Ack, will fix.
>
>> + return 0;
>> +}
>> +
>> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
>> const char *submodule_name)
>> {
>> + const char *gitdir;
>> + char *key;
>> +
>> + repo_git_path_append(r, buf, "modules/");
>> + strbuf_addstr(buf, submodule_name);
>> +
>> + /* If extensions.submoduleEncoding is disabled, use the plain path set above */
>> + if (!r->repository_format_submodule_encoding) {
>> + if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
>> + die(_("refusing to create/use '%s' in another submodule's "
>> + "git dir"), buf->buf);
>> +
>> + return; /* plain gitdir is valid for use */
>> + }
>> +
>> + /* Extension is enabled: use the gitdir config if it exists */
>> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
>> + if (!repo_config_get_string_tmp(r, key, &gitdir)) {
>> + strbuf_reset(buf);
>> + strbuf_addstr(buf, gitdir);
>> + FREE_AND_NULL(key);
>> +
>> + /* validate because users might have modified the config */
>> + if (validate_submodule_encoded_git_dir(buf->buf, submodule_name))
>> + die(_("Invalid 'submodule.%s.gitdir' config: '%s' please check "
>> + "if it is unique or conflicts with another module"),
>
> Nit: error messages start with a lower-case character.
Ack, will fix.
>
>> + submodule_name, gitdir);
>> +
>> + return;
>> + }
>> + FREE_AND_NULL(key);
>> +
>> /*
>> - * NEEDSWORK: The current way of mapping a submodule's name to
>> - * its location in .git/modules/ has problems with some naming
>> - * schemes. For example, if a submodule is named "foo" and
>> - * another is named "foo/bar" (whether present in the same
>> - * superproject commit or not - the problem will arise if both
>> - * superproject commits have been checked out at any point in
>> - * time), or if two submodule names only have different cases in
>> - * a case-insensitive filesystem.
>> - *
>> - * There are several solutions, including encoding the path in
>> - * some way, introducing a submodule.<name>.gitdir config in
>> - * .git/config (not .gitmodules) that allows overriding what the
>> - * gitdir of a submodule would be (and teach Git, upon noticing
>> - * a clash, to automatically determine a non-clashing name and
>> - * to write such a config), or introducing a
>> - * submodule.<name>.gitdir config in .gitmodules that repo
>> - * administrators can explicitly set. Nothing has been decided,
>> - * so for now, just append the name at the end of the path.
>> + * The gitdir config does not exist, even though the extension is enabled.
>> + * Therefore we are in one of the following cases:
>> */
>> +
>> + /* Case 1: legacy migration of valid plain submodule names */
>> + if (!validate_and_set_submodule_gitdir(buf, submodule_name))
>> + return;
>> +
>> + /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
>> + strbuf_reset(buf);
>> repo_git_path_append(r, buf, "modules/");
>> - strbuf_addstr(buf, submodule_name);
>> + strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
>> + if (!validate_and_set_submodule_gitdir(buf, submodule_name))
>> + return;
>>
>> - if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
>> - die(_("refusing to create/use '%s' in another submodule's "
>> - "git dir"), buf->buf);
>> + /* Case 3: Nothing worked: error out */
>> + die(_("Cannot construct a valid gitdir path for submodule '%s': "
>> + "please set a unique git config for 'submodule.%s.gitdir'."),
>> + submodule_name, submodule_name);
>
> It feels somewhat fragile to me that we unconditionally handle these
> cases and try to find old submodule directories. If the extension is
> enabled I'd expect that the submodule configuration is the _only_ source
> of truth.
>
> May I propose that we instead always error out in case the submodule
> configuration does not exist? In the best case we'd then give the user a
> nice error message that tells them how to run the migration manually.
Junio told me to not do any kind of manual migration and just attempt
new names until one works and then use it consistently.
That's why the "submodule.%s.gitdir" path is always used if set and
has precedence (no new names are attempted). :)
>
> (Coming back from reading subsequent patches) Maybe what's putting me
> off is that this function is seemingly used for two things:
>
> 1. To derive the submodule path in case we know it should already
> exist.
>
> 2. To compute the submodule path so we can end up writing it into the
> "submodule.*.gitdir" variable.
>
> I think we should tell these two cases apart. In the first case I expect
> that we never fall back to a computed name, but bail out in case the
> configuration key does not exist. And in the second case it of course
> makes sense to compute the actual path that we want to store in the
> configuration.
I think I understand where you're coming from.
Even before my patches, the unmodified submodule_name_to_gitdir() is
used for both new (non-existing) and old (existing) submodules.
It has no way of knowing whether a submodule exists, whether it should
exist, or whether a new name is required for a new clone, which will
eventually exist in the future.
If I also understood your suggestion, you just need an additional check
to verify if the path pointed to by "submodule.%s.gitdir" is an existing
gitdir and error out if not?
Or did I misunderstood your suggestion and you mean to bail out if the
config key is missing entirely for any submodule when the extension is
enabled?
That would imply a manual migration by the user which is something both
Aaron and Junio asked me to avoid and Josh also said they want to avoid
setting any kind of config keys (or distributing configs), so that's why
I also added the compile-time extension option, to ease the transition,
together with the "retry-on-fallback" approach for setting the config.
I am in favor of implementing the split you suggested, however how do we
automatically figure out if a name's gitdir **should** exist if you mean
the latter not the former? :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-05 12:19 ` Patrick Steinhardt
@ 2025-12-05 19:42 ` Adrian Ratiu
2025-12-05 22:52 ` Junio C Hamano
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-05 19:42 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
>> Some users find it difficult to distribute repo config changes for
>> enabling extensions.submoduleEncoding, or to enable it by passing
>> the config via cmdline, so we add a build-time option which can
>> enable the extension for convenience.
>
> Wouldn't it be more sensible to make this a runtime configuration key
> that users can configure in their gitconfig?
The request I got from a combination of feedback from Junio, Aaron and
Josh is to avoid any kind of required user intervention or manual
migration, to find ways to automate the transition as much as possible.
If possible without even having to change or distribute configs or set
cmdline parameters to enable the new extension (that is why I added this
off-by-default build option in v5).
We could add a runtime gitconfig key in addition to the build and repo
config options, I see no issue with that.
I have no horse in this race btw, just trying to make everyone happy. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-05 19:30 ` Adrian Ratiu
@ 2025-12-05 22:47 ` Junio C Hamano
2025-12-06 11:59 ` Patrick Steinhardt
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-12-05 22:47 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Patrick Steinhardt, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>> It feels somewhat fragile to me that we unconditionally handle these
>> cases and try to find old submodule directories. If the extension is
>> enabled I'd expect that the submodule configuration is the _only_ source
>> of truth.
>>
>> May I propose that we instead always error out in case the submodule
>> configuration does not exist? In the best case we'd then give the user a
>> nice error message that tells them how to run the migration manually.
>
> Junio told me to not do any kind of manual migration and just attempt
> new names until one works and then use it consistently.
Indeed, but I do not think that has much relevance to Patrick's
comment. What I meant say was
- With extension, we know that the repository will use the
configuration item as the sole source of truth. Unlike the "we
now store submodule dirs to munged paths" design that we saw long
ago, which would have required us to sometimes move the existing
directories to match the munging scheme, we do not have to
manually migrate the existing directories. Instead, we can just
record the pathnames they already use.
- And with extension, when we add a new submodule, we would need to
give them an entry in the configuration that does not conflict
with those used by the existing submodules. As Patrick mentions,
this name does not have to be done with any reversible mapping,
and third-party software and Git reimplementations can always
refer to the configuration to learn the path without knowing how
the path is chosen themselves.
- The above two would ensure that the configuration would always
exist once a submodule enters our system (and "git submodule init"
is done to cause its gitdir created).
So, unless I am missing some corner case that still exists while
bootstrapping a new style repository with extension, Patrick's
"error when configuration is missing" sounds quite sensible to me.
I am officially still on vacation, so I'd stop here for now ;-).
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-05 19:42 ` Adrian Ratiu
@ 2025-12-05 22:52 ` Junio C Hamano
2025-12-06 12:02 ` Patrick Steinhardt
2025-12-08 9:42 ` Adrian Ratiu
0 siblings, 2 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-12-05 22:52 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Patrick Steinhardt, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
>> On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
>>> Some users find it difficult to distribute repo config changes for
>>> enabling extensions.submoduleEncoding, or to enable it by passing
>>> the config via cmdline, so we add a build-time option which can
>>> enable the extension for convenience.
>>
>> Wouldn't it be more sensible to make this a runtime configuration key
>> that users can configure in their gitconfig?
>
> The request I got from a combination of feedback from Junio, Aaron and
> Josh is to avoid any kind of required user intervention or manual
> migration, to find ways to automate the transition as much as possible.
How would that lead to build-time behaviour change, though?
Users in managed environments like $CORP can rely on /etc/gitconfig
or equivalents managed by their corp-eng, so I am having a hard time
imagining why we need anything more than an configuration variable
looked at runtime.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-05 22:47 ` Junio C Hamano
@ 2025-12-06 11:59 ` Patrick Steinhardt
2025-12-06 16:38 ` Junio C Hamano
2025-12-08 9:10 ` Adrian Ratiu
0 siblings, 2 replies; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-06 11:59 UTC (permalink / raw)
To: Junio C Hamano
Cc: Adrian Ratiu, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Sat, Dec 06, 2025 at 07:47:56AM +0900, Junio C Hamano wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
> >> It feels somewhat fragile to me that we unconditionally handle these
> >> cases and try to find old submodule directories. If the extension is
> >> enabled I'd expect that the submodule configuration is the _only_ source
> >> of truth.
> >>
> >> May I propose that we instead always error out in case the submodule
> >> configuration does not exist? In the best case we'd then give the user a
> >> nice error message that tells them how to run the migration manually.
> >
> > Junio told me to not do any kind of manual migration and just attempt
> > new names until one works and then use it consistently.
>
> Indeed, but I do not think that has much relevance to Patrick's
> comment. What I meant say was
>
> - With extension, we know that the repository will use the
> configuration item as the sole source of truth. Unlike the "we
> now store submodule dirs to munged paths" design that we saw long
> ago, which would have required us to sometimes move the existing
> directories to match the munging scheme, we do not have to
> manually migrate the existing directories. Instead, we can just
> record the pathnames they already use.
>
> - And with extension, when we add a new submodule, we would need to
> give them an entry in the configuration that does not conflict
> with those used by the existing submodules. As Patrick mentions,
> this name does not have to be done with any reversible mapping,
> and third-party software and Git reimplementations can always
> refer to the configuration to learn the path without knowing how
> the path is chosen themselves.
>
> - The above two would ensure that the configuration would always
> exist once a submodule enters our system (and "git submodule init"
> is done to cause its gitdir created).
>
> So, unless I am missing some corner case that still exists while
> bootstrapping a new style repository with extension, Patrick's
> "error when configuration is missing" sounds quite sensible to me.
>
> I am officially still on vacation, so I'd stop here for now ;-).
I guess the one edge case is when somebody manually turns on the
extension after they have already initialized submodules. In that case
the gitdir paths would of course not exist in the gitconfig. But I think
blindly falling back to a source of truth different than the
configuration is wrong, as it would mean that the we have "a single
source of truth unless we don't". It would kind of defeat the whole
purpose of the extension in my opinion, as implementations cannot rely
on it.
Maybe the right approach would be to tell users to never manually enable
the extension and instead to provide a command that both:
- Persists the submodule gitdirs for any populated submodules in the
gitconfig.
- Enables the repsitory extension.
If we had that then we could count on the submodule gitdirs to exist in
the gitconfig, and if they don't we would die with an error message that
indicates that the repository is broken, maybe even with a hint for the
user on how to fix it.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-05 22:52 ` Junio C Hamano
@ 2025-12-06 12:02 ` Patrick Steinhardt
2025-12-06 16:48 ` Junio C Hamano
2025-12-08 9:23 ` Adrian Ratiu
2025-12-08 9:42 ` Adrian Ratiu
1 sibling, 2 replies; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-06 12:02 UTC (permalink / raw)
To: Junio C Hamano
Cc: Adrian Ratiu, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sat, Dec 06, 2025 at 07:52:13AM +0900, Junio C Hamano wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
> > On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> >> On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
> >>> Some users find it difficult to distribute repo config changes for
> >>> enabling extensions.submoduleEncoding, or to enable it by passing
> >>> the config via cmdline, so we add a build-time option which can
> >>> enable the extension for convenience.
> >>
> >> Wouldn't it be more sensible to make this a runtime configuration key
> >> that users can configure in their gitconfig?
> >
> > The request I got from a combination of feedback from Junio, Aaron and
> > Josh is to avoid any kind of required user intervention or manual
> > migration, to find ways to automate the transition as much as possible.
>
> How would that lead to build-time behaviour change, though?
>
> Users in managed environments like $CORP can rely on /etc/gitconfig
> or equivalents managed by their corp-eng, so I am having a hard time
> imagining why we need anything more than an configuration variable
> looked at runtime.
I guess you could kind of have both: make it a runtime configuration
key, but have its default depend on a build configuration. You could for
example auto-enable it in case `WITH_BREAKING_CHANGES` is enabled. But I
myself am not sure whether the latter would really be all that important
in the first place.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-06 11:59 ` Patrick Steinhardt
@ 2025-12-06 16:38 ` Junio C Hamano
2025-12-08 9:01 ` Adrian Ratiu
2025-12-08 9:10 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-12-06 16:38 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: Adrian Ratiu, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
Patrick Steinhardt <ps@pks.im> writes:
> Maybe the right approach would be to tell users to never manually enable
> the extension and instead to provide a command that both:
>
> - Persists the submodule gitdirs for any populated submodules in the
> gitconfig.
>
> - Enables the repsitory extension.
>
> If we had that then we could count on the submodule gitdirs to exist in
> the gitconfig, and if they don't we would die with an error message that
> indicates that the repository is broken, maybe even with a hint for the
> user on how to fix it.
I personally like the simplicity of this approach.
I haven't however thought about operational complexity, if one has
an existing user base that have been using a custom pathname munging
code that needs to be migrated to the new scheme.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-06 12:02 ` Patrick Steinhardt
@ 2025-12-06 16:48 ` Junio C Hamano
2025-12-08 9:23 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-12-06 16:48 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: Adrian Ratiu, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
Patrick Steinhardt <ps@pks.im> writes:
>> How would that lead to build-time behaviour change, though?
>>
>> Users in managed environments like $CORP can rely on /etc/gitconfig
>> or equivalents managed by their corp-eng, so I am having a hard time
>> imagining why we need anything more than an configuration variable
>> looked at runtime.
>
> I guess you could kind of have both: make it a runtime configuration
> key, but have its default depend on a build configuration. You could for
> example auto-enable it in case `WITH_BREAKING_CHANGES` is enabled. But I
> myself am not sure whether the latter would really be all that important
> in the first place.
Me neither ;-)
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-06 16:38 ` Junio C Hamano
@ 2025-12-08 9:01 ` Adrian Ratiu
2025-12-08 11:46 ` Patrick Steinhardt
0 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-08 9:01 UTC (permalink / raw)
To: Junio C Hamano, Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Josh Steadmon, Ben Knoble,
Phillip Wood, Brandon Williams
On Sun, 07 Dec 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
>> Maybe the right approach would be to tell users to never manually enable
>> the extension and instead to provide a command that both:
>>
>> - Persists the submodule gitdirs for any populated submodules in the
>> gitconfig.
>>
>> - Enables the repsitory extension.
>>
>> If we had that then we could count on the submodule gitdirs to exist in
>> the gitconfig, and if they don't we would die with an error message that
>> indicates that the repository is broken, maybe even with a hint for the
>> user on how to fix it.
>
> I personally like the simplicity of this approach.
>
> I haven't however thought about operational complexity, if one has
> an existing user base that have been using a custom pathname munging
> code that needs to be migrated to the new scheme.
The good news is that for all the real-world use cases I'm aware of,
migration to the new scheme should just work.
The problem we're discussing here (automatic fallback vs requiring the
gitdir to always exist + using a migration tool) does add a bit of
operational complexity, however it might be manageable.
I have to see how this works in practice, especially with the users
setting the config (likely I'll add an error to block that path) and the
build-config to enable the extension by default (will likely run the
migration tool automatically on existing repositories if enabled via the
build-config).
I will attempt this approach in v6.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-06 11:59 ` Patrick Steinhardt
2025-12-06 16:38 ` Junio C Hamano
@ 2025-12-08 9:10 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-08 9:10 UTC (permalink / raw)
To: Patrick Steinhardt, Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Josh Steadmon, Ben Knoble,
Phillip Wood, Brandon Williams
On Sat, 06 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Sat, Dec 06, 2025 at 07:47:56AM +0900, Junio C Hamano wrote:
>> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>>
>> >> It feels somewhat fragile to me that we unconditionally handle these
>> >> cases and try to find old submodule directories. If the extension is
>> >> enabled I'd expect that the submodule configuration is the _only_ source
>> >> of truth.
>> >>
>> >> May I propose that we instead always error out in case the submodule
>> >> configuration does not exist? In the best case we'd then give the user a
>> >> nice error message that tells them how to run the migration manually.
>> >
>> > Junio told me to not do any kind of manual migration and just attempt
>> > new names until one works and then use it consistently.
>>
>> Indeed, but I do not think that has much relevance to Patrick's
>> comment. What I meant say was
>>
>> - With extension, we know that the repository will use the
>> configuration item as the sole source of truth. Unlike the "we
>> now store submodule dirs to munged paths" design that we saw long
>> ago, which would have required us to sometimes move the existing
>> directories to match the munging scheme, we do not have to
>> manually migrate the existing directories. Instead, we can just
>> record the pathnames they already use.
>>
>> - And with extension, when we add a new submodule, we would need to
>> give them an entry in the configuration that does not conflict
>> with those used by the existing submodules. As Patrick mentions,
>> this name does not have to be done with any reversible mapping,
>> and third-party software and Git reimplementations can always
>> refer to the configuration to learn the path without knowing how
>> the path is chosen themselves.
>>
>> - The above two would ensure that the configuration would always
>> exist once a submodule enters our system (and "git submodule init"
>> is done to cause its gitdir created).
>>
>> So, unless I am missing some corner case that still exists while
>> bootstrapping a new style repository with extension, Patrick's
>> "error when configuration is missing" sounds quite sensible to me.
>>
>> I am officially still on vacation, so I'd stop here for now ;-).
>
> I guess the one edge case is when somebody manually turns on the
> extension after they have already initialized submodules. In that case
> the gitdir paths would of course not exist in the gitconfig. But I think
> blindly falling back to a source of truth different than the
> configuration is wrong, as it would mean that the we have "a single
> source of truth unless we don't". It would kind of defeat the whole
> purpose of the extension in my opinion, as implementations cannot rely
> on it.
Haven't thought about it that way and indeed it makes a lot of sense.
100% agreed.
>
> Maybe the right approach would be to tell users to never manually enable
> the extension and instead to provide a command that both:
>
> - Persists the submodule gitdirs for any populated submodules in the
> gitconfig.
>
> - Enables the repsitory extension.
>
> If we had that then we could count on the submodule gitdirs to exist in
> the gitconfig, and if they don't we would die with an error message that
> indicates that the repository is broken, maybe even with a hint for the
> user on how to fix it.
Thank you Patrick for clarifying this, really appreciate it.
I will implement this design in v6.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-06 12:02 ` Patrick Steinhardt
2025-12-06 16:48 ` Junio C Hamano
@ 2025-12-08 9:23 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-08 9:23 UTC (permalink / raw)
To: Patrick Steinhardt, Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Ben Knoble, Phillip Wood,
Josh Steadmon
On Sat, 06 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Sat, Dec 06, 2025 at 07:52:13AM +0900, Junio C Hamano wrote:
>> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>>
>> > On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
>> >> On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
>> >>> Some users find it difficult to distribute repo config changes for
>> >>> enabling extensions.submoduleEncoding, or to enable it by passing
>> >>> the config via cmdline, so we add a build-time option which can
>> >>> enable the extension for convenience.
>> >>
>> >> Wouldn't it be more sensible to make this a runtime configuration key
>> >> that users can configure in their gitconfig?
>> >
>> > The request I got from a combination of feedback from Junio, Aaron and
>> > Josh is to avoid any kind of required user intervention or manual
>> > migration, to find ways to automate the transition as much as possible.
>>
>> How would that lead to build-time behaviour change, though?
>>
>> Users in managed environments like $CORP can rely on /etc/gitconfig
>> or equivalents managed by their corp-eng, so I am having a hard time
>> imagining why we need anything more than an configuration variable
>> looked at runtime.
>
> I guess you could kind of have both: make it a runtime configuration
> key, but have its default depend on a build configuration. You could for
> example auto-enable it in case `WITH_BREAKING_CHANGES` is enabled. But I
> myself am not sure whether the latter would really be all that important
> in the first place.
I'll have to revisit this after I implement the migration command design
you suggested for v6, to figure out exactly how to best implement this.
Thank you for this suggestion as well, likely I'll do something very
similar if it ends up being necessary.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time
2025-12-05 22:52 ` Junio C Hamano
2025-12-06 12:02 ` Patrick Steinhardt
@ 2025-12-08 9:42 ` Adrian Ratiu
1 sibling, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-08 9:42 UTC (permalink / raw)
To: Junio C Hamano, Josh Steadmon, Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Ben Knoble, Phillip Wood
On Sat, 06 Dec 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
>> On Fri, 05 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
>>> On Wed, Nov 19, 2025 at 11:10:30PM +0200, Adrian Ratiu wrote:
>>>> Some users find it difficult to distribute repo config changes for
>>>> enabling extensions.submoduleEncoding, or to enable it by passing
>>>> the config via cmdline, so we add a build-time option which can
>>>> enable the extension for convenience.
>>>
>>> Wouldn't it be more sensible to make this a runtime configuration key
>>> that users can configure in their gitconfig?
>>
>> The request I got from a combination of feedback from Junio, Aaron and
>> Josh is to avoid any kind of required user intervention or manual
>> migration, to find ways to automate the transition as much as possible.
>
> How would that lead to build-time behaviour change, though?
>
> Users in managed environments like $CORP can rely on /etc/gitconfig
> or equivalents managed by their corp-eng, so I am having a hard time
> imagining why we need anything more than an configuration variable
> looked at runtime.
Please see Josh's message:
https://public-inbox.org/git/20250816213642.3517822-1-adrian.ratiu@collabora.com/T/#m7d0d75126c81bef7d3619e53da6fa0cd69426570
A config variable looked up at runtime would solve most cases
highlighted there, except for the force-enable
`extensions.submoduleEncoding` regardless of the local config.
That is why I added this option :) though we do not have a local config
in v5 because I saw no reason for it at the time.
For v6 I will likely implement Patrick's suggestion to introduce a
config variable and just set its default at build-time which seems like
the cleanest way to do it (and automatically run the v6 migration
command instead of the current automatic fallback).
That would allow a smooth automatic transition which will address Josh's
requirements.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-08 9:01 ` Adrian Ratiu
@ 2025-12-08 11:46 ` Patrick Steinhardt
2025-12-08 15:48 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-08 11:46 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Junio C Hamano, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Mon, Dec 08, 2025 at 11:01:33AM +0200, Adrian Ratiu wrote:
> On Sun, 07 Dec 2025, Junio C Hamano <gitster@pobox.com> wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> >
> >> Maybe the right approach would be to tell users to never manually enable
> >> the extension and instead to provide a command that both:
> >>
> >> - Persists the submodule gitdirs for any populated submodules in the
> >> gitconfig.
> >>
> >> - Enables the repsitory extension.
> >>
> >> If we had that then we could count on the submodule gitdirs to exist in
> >> the gitconfig, and if they don't we would die with an error message that
> >> indicates that the repository is broken, maybe even with a hint for the
> >> user on how to fix it.
> >
> > I personally like the simplicity of this approach.
> >
> > I haven't however thought about operational complexity, if one has
> > an existing user base that have been using a custom pathname munging
> > code that needs to be migrated to the new scheme.
>
> The good news is that for all the real-world use cases I'm aware of,
> migration to the new scheme should just work.
>
> The problem we're discussing here (automatic fallback vs requiring the
> gitdir to always exist + using a migration tool) does add a bit of
> operational complexity, however it might be manageable.
>
> I have to see how this works in practice, especially with the users
> setting the config (likely I'll add an error to block that path) and the
> build-config to enable the extension by default (will likely run the
> migration tool automatically on existing repositories if enabled via the
> build-config).
>
> I will attempt this approach in v6.
One suggestion: it might be sensible to move auto-migration into a
subsequent patch series. That way we can focus on the general approach
of the new extension at first, and the potentially-bigger discussion
around whether or not to auto-migrate users can then be had separately.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v5 4/7] submodule: add extension to encode gitdir paths
2025-12-08 11:46 ` Patrick Steinhardt
@ 2025-12-08 15:48 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-08 15:48 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: Junio C Hamano, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood, Brandon Williams
On Mon, 08 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Mon, Dec 08, 2025 at 11:01:33AM +0200, Adrian Ratiu wrote:
>> On Sun, 07 Dec 2025, Junio C Hamano <gitster@pobox.com> wrote:
>> > Patrick Steinhardt <ps@pks.im> writes:
>> >
>> >> Maybe the right approach would be to tell users to never manually enable
>> >> the extension and instead to provide a command that both:
>> >>
>> >> - Persists the submodule gitdirs for any populated submodules in the
>> >> gitconfig.
>> >>
>> >> - Enables the repsitory extension.
>> >>
>> >> If we had that then we could count on the submodule gitdirs to exist in
>> >> the gitconfig, and if they don't we would die with an error message that
>> >> indicates that the repository is broken, maybe even with a hint for the
>> >> user on how to fix it.
>> >
>> > I personally like the simplicity of this approach.
>> >
>> > I haven't however thought about operational complexity, if one has
>> > an existing user base that have been using a custom pathname munging
>> > code that needs to be migrated to the new scheme.
>>
>> The good news is that for all the real-world use cases I'm aware of,
>> migration to the new scheme should just work.
>>
>> The problem we're discussing here (automatic fallback vs requiring the
>> gitdir to always exist + using a migration tool) does add a bit of
>> operational complexity, however it might be manageable.
>>
>> I have to see how this works in practice, especially with the users
>> setting the config (likely I'll add an error to block that path) and the
>> build-config to enable the extension by default (will likely run the
>> migration tool automatically on existing repositories if enabled via the
>> build-config).
>>
>> I will attempt this approach in v6.
>
> One suggestion: it might be sensible to move auto-migration into a
> subsequent patch series. That way we can focus on the general approach
> of the new extension at first, and the potentially-bigger discussion
> around whether or not to auto-migrate users can then be had separately.
Hm, yes, I started implementing the feedback and the closer I get to v6,
the more it feels like we should split this series in two or even three
independent patches/series:
1. The base gitdir path extension and config infra, renamed as you
suggested, to be independent of the actual encoding.
2. The encoding.
3. The migration.
Let's see where this takes us. :)
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (13 preceding siblings ...)
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (11 more replies)
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
15 siblings, 12 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Hello everyone,
For those new to the series, we're implementing a submodule gitdir
extension which allows us to have a unified way to determine gitdirs
and do things like encode submodule paths to avoid FS conflicts.
I debated with myself whether to split this into 2 separate series,
because after the latest redesign, our logic is cleanly split in 2:
Patches 1-6 implement the basic mechanisms of the new extension.
Patches 7-10 add submodule encoding on top.
Considering that all patches depend upon another and I'd be breaking
review continuity and range-diffs, I opted to send v6 as one series.
If anyone feels strongly about this, we could split in 2 series or
even land the patches 1-6 and 7-10 separately.
As always, this is based on the latest master branch, I've checkd
for conflicts with next/seen, pushed to Github [1] and succesfully
ran the CI [2].
1: https://github.com/10ne1/git/tree/dev/aratiu/encoding-v6
2: https://github.com/10ne1/git/actions/runs/20188634517
Changes in v6:
* Renamed the extension from submoduleEncoding to submodulePathConfig (Patrick)
* Decoupled the extension from the encoding conflict resolution (Patrick)
* Simplified submodule_name_to_gitdir() to error out if gitdir config is missing (Patrick)
* Not doing automated gitdir fallback migration anymore (Patrick)
* Added a global config (not repo config) for enabling the extension (Patrick)
* Added a migration command to submodule--helper (Patrick, Junio)
* Removed the build-time configuration option (Adrian, Patrick, Junio)
* Fixed a parallel clone race I introduced while moving gitdir validation (Peff)
Was able to reproduce this race when using stress-limit > 30 like the following:
make && (cd t && ./t7450-bad-git-dotfiles.sh --stress-limit=50)
* Added back an extra validation in submodule_move_head() for logic consistency (Patrick)
* Fixed minor nits, moved function comments to header files and so on (Patrick & Adrian)
v5 -> v6 range-diff:
1: 9d5855f3bf = 1: 9d5855f3bf submodule--helper: use submodule_name_to_gitdir in add_submodule
3: 7bcadf1116 ! 2: df2d7703f1 submodule: always validate gitdirs inside submodule_name_to_gitdir
@@ Commit message
after calling submodule_name_to_gitdir() into the function proper,
which now always validates the gitdir before returning it.
- This also makes parallel operations a bit safer due to checking and
- erroring out each time the unified API detects a problem instead of
- having one extra hardcoded validation check in submodule--helper.c.
+ This simplifies the API and helps to:
+ 1. Avoid redundant validation calls after submodule_name_to_gitdir().
+ 2. Avoid the risk of callers forgetting to validate.
+ 3. Ensure gitdir paths provided by users via configs are always valid
+ (config gitdir paths are added in a subsequent commit).
- It simplifies the API usage as well since users who don't have to
- validate the submodule_name_to_gitdir() result themselves anymore
- and reduces the risks of API users forgetting to validate.
+ The validation function can still be called as many times as needed
+ outside submodule_name_to_gitdir(), for example we keep two calls
+ which are still required, to avoid parallel clone races by re-running
+ the validation in builtin/submodule-helper.c.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
@@ builtin/submodule--helper.c: static int clone_submodule(const struct module_clon
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
-@@ builtin/submodule--helper.c: static int clone_submodule(const struct module_clone_data *clone_data,
- free(path);
- }
-
-- /*
-- * We already performed this check at the beginning of this function,
-- * before cloning the objects. This tries to detect racy behavior e.g.
-- * in parallel clones, where another process could easily have made the
-- * gitdir nested _after_ it was created.
-- *
-- * To prevent further harm coming from this unintentionally-nested
-- * gitdir, let's disable it by deleting the `HEAD` file.
-- */
-- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0) {
-- char *head = xstrfmt("%s/HEAD", sm_gitdir);
-- unlink(head);
-- free(head);
-- die(_("refusing to create/use '%s' in another submodule's "
-- "git dir"), sm_gitdir);
-- }
--
- connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
-
- p = repo_submodule_path(the_repository, clone_data_path, "config");
## submodule.c ##
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
-
- if (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {
- if (old_head) {
-- if (!submodule_uses_gitfile(path))
-- absorb_git_dir_into_superproject(path,
-- super_prefix);
-- else {
-- char *dotgit = xstrfmt("%s/.git", path);
-- char *git_dir = xstrdup(read_gitfile(dotgit));
--
-- free(dotgit);
-- if (validate_submodule_git_dir(git_dir,
-- sub->name) < 0)
-- die(_("refusing to create/use '%s' in "
-- "another submodule's git dir"),
-- git_dir);
-- free(git_dir);
-- }
-+ absorb_git_dir_into_superproject(path, super_prefix);
- } else {
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
-@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
- return ret;
- }
-
--int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
-+static int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
- {
- size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
- char *p;
@@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
@@ submodule.c: void submodule_name_to_gitdir(struct strbuf *buf, struct repository
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
}
-
- ## submodule.h ##
-@@ submodule.h: int submodule_to_gitdir(struct repository *repo,
- void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
- const char *submodule_name);
-
--/*
-- * Make sure that no submodule's git dir is nested in a sibling submodule's.
-- */
--int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
--
- /*
- * Make sure that the given submodule path does not follow symlinks.
- */
-: ---------- > 3: fb5e6aec6a builtin/submodule--helper: add gitdir command
-: ---------- > 4: d3f5cf47bd submodule: introduce extensions.submodulePathConfig
-: ---------- > 5: 131cf17a80 submodule: allow runtime enabling extensions.submodulePathConfig
-: ---------- > 6: 529c8f0618 submodule--helper: add gitdir migration command
2: 8cfa970a9d ! 7: 66a7973f39 builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
@@ Commit message
public function exposed via url.h, instead of the old helper path
(strbuf), which has nothing to do with 3986 encoding/decoding anymore.
- This function will be used by submodule.c in the next commit.
+ This function will be used in subsequent commits which do the encoding.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
@@ url.c
#include "strbuf.h"
#include "url.h"
-+/*
-+ * The set of unreserved characters as per STD66 (RFC3986) is
-+ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
-+ * components without percent-encoding.
-+ */
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
@@ url.h: char *url_decode_parameter_value(const char **query);
void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
++/*
++ * The set of unreserved characters as per STD66 (RFC3986) is
++ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
++ * components without percent-encoding.
++ */
+int is_rfc3986_unreserved(char ch);
+
#endif /* URL_H */
4: 1b5d0b50ef < -: ---------- submodule: add extension to encode gitdir paths
-: ---------- > 8: d8789b3d8f submodule--helper: fix filesystem collisions by encoding gitdir paths
5: 2bf1c116a2 ! 9: 426c610fe1 submodule: fix case-folding gitdir filesystem colisions
@@ Metadata
Author: Adrian Ratiu <adrian.ratiu@collabora.com>
## Commit message ##
- submodule: fix case-folding gitdir filesystem colisions
+ submodule: fix case-folding gitdir filesystem collisions
- Add a new check when extension.submoduleEncoding is enabled to
+ Add a new check when extension.submodulePathConfig is enabled, to
detect and prevent case-folding filesystem colisions. When this
new check is triggered, a stricter casefolding aware URI encoding
is used to percent-encode uppercase characters.
@@ Commit message
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
+ ## builtin/submodule--helper.c ##
+@@ builtin/submodule--helper.c: static void create_default_gitdir_config(const char *submodule_name)
+ return;
+ }
+
+- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
++ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
+@@ builtin/submodule--helper.c: static void create_default_gitdir_config(const char *submodule_name)
+ return;
+ }
+
++ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
++ strbuf_reset(&gitdir_path);
++ repo_git_path_append(the_repository, &gitdir_path, "modules/");
++ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
++ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
++ return;
++
++ /* Case 2.3: Try some derived gitdir names, see if one sticks */
++ for (char c = '0'; c <= '9'; c++) {
++ strbuf_reset(&gitdir_path);
++ repo_git_path_append(the_repository, &gitdir_path, "modules/");
++ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
++ strbuf_addch(&gitdir_path, c);
++ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
++ return;
++
++ strbuf_reset(&gitdir_path);
++ repo_git_path_append(the_repository, &gitdir_path, "modules/");
++ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
++ strbuf_addch(&gitdir_path, c);
++ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
++ return;
++ }
++
+ /* Case 3: nothing worked, error out */
+ die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
+ "Please ensure it is set, for example by running something like: "
+
## submodule.c ##
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
return ret;
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
+cleanup:
+ if (dir)
+ closedir(dir);
-+ FREE_AND_NULL(modules_dir);
++ free(modules_dir);
+ return ret;
+}
+
/*
- * Encoded gitdir validation function used when extensions.submoduleEncoding is enabled.
+ * Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
* This does not print errors like the non-encoded version, because encoding is supposed
* to mitigate / fix all these.
*/
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
char *p = git_dir, *last_submodule_name = NULL;
+ int config_ignorecase = 0;
- if (!the_repository->repository_format_submodule_encoding)
+ if (!the_repository->repository_format_submodule_path_cfg)
BUG("validate_submodule_encoded_git_dir() must be called with "
@@ submodule.c: static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodu
if (!last_submodule_name || strchr(last_submodule_name, '/'))
@@ submodule.c: static int validate_submodule_encoded_git_dir(char *git_dir, const
return 0;
}
-@@ submodule.c: void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
- if (!validate_and_set_submodule_gitdir(buf, submodule_name))
- return;
-
-- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
-+ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
- strbuf_reset(buf);
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
- if (!validate_and_set_submodule_gitdir(buf, submodule_name))
- return;
-
-+ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
-+ strbuf_reset(buf);
-+ repo_git_path_append(r, buf, "modules/");
-+ strbuf_addstr_urlencode(buf, submodule_name, is_casefolding_rfc3986_unreserved);
-+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
-+ return;
-+
-+ /* Case 2.3: Try some derived gitdir names, see if one sticks */
-+ for (char c = '0'; c <= '9'; c++) {
-+ strbuf_reset(buf);
-+ repo_git_path_append(r, buf, "modules/");
-+ strbuf_addstr_urlencode(buf, submodule_name, is_rfc3986_unreserved);
-+ strbuf_addch(buf, c);
-+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
-+ return;
-+
-+ strbuf_reset(buf);
-+ repo_git_path_append(r, buf, "modules/");
-+ strbuf_addstr_urlencode(buf, submodule_name, is_casefolding_rfc3986_unreserved);
-+ strbuf_addch(buf, c);
-+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
-+ return;
-+ }
-+
- /* Case 3: Nothing worked: error out */
- die(_("Cannot construct a valid gitdir path for submodule '%s': "
- "please set a unique git config for 'submodule.%s.gitdir'."),
- ## t/t7425-submodule-encoding.sh ##
-@@ t/t7425-submodule-encoding.sh: test_expect_success 'disabling extensions.submoduleEncoding prevents nested subm
+ ## t/t7425-submodule-gitdir-path-extension.sh ##
+@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success 'disabling extensions.submodulePathConfig prevents nested su
)
'
+test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are correctly encoded' '
-+ git clone -c extensions.submoduleEncoding=true main cloned-folding &&
++ git clone -c extensions.submodulePathConfig=true main cloned-folding &&
+ (
+ cd cloned-folding &&
+
@@ url.c: int is_rfc3986_unreserved(char ch)
ch == '-' || ch == '_' || ch == '.' || ch == '~';
}
-+/*
-+ * This is a variant of is_rfc3986_unreserved() that treats uppercase
-+ * letters as "reserved". This forces them to be percent-encoded, allowing
-+ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
-+ */
+int is_casefolding_rfc3986_unreserved(char c)
+{
+ return (c >= 'a' && c <= 'z') ||
@@ url.c: int is_rfc3986_unreserved(char ch)
/*
## url.h ##
-@@ url.h: void end_url_with_slash(struct strbuf *buf, const char *url);
- void str_end_url_with_slash(const char *url, char **dest);
-
+@@ url.h: void str_end_url_with_slash(const char *url, char **dest);
+ */
int is_rfc3986_unreserved(char ch);
-+int is_casefolding_rfc3986_unreserved(char c);
++/*
++ * This is a variant of is_rfc3986_unreserved() that treats uppercase
++ * letters as "reserved". This forces them to be percent-encoded, allowing
++ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
++ */
++int is_casefolding_rfc3986_unreserved(char c);
++
#endif /* URL_H */
6: b607d7ca39 ! 10: 80b1c1fb59 submodule: use hashed name for gitdir
@@ Metadata
Author: Adrian Ratiu <adrian.ratiu@collabora.com>
## Commit message ##
- submodule: use hashed name for gitdir
+ submodule: hash the submodule name for the gitdir path
- If none of the previous steps work and we reach case 2.4, try to
- hash the submodule name and see if that can be a valid gitdir
- before giving up and throwing an error.
+ If none of the previous plain-text / encoding / derivation steps work
+ and case 2.4 is reached, then try a hash of the submodule name to see
+ if that can be a valid gitdir before giving up and throwing an error.
- This is a "last resort" type of measure to avoid conflicts since
- it loses the gitdir human readability. Itis not such a big deal
- because users are now supposed to use the submodule.<name>.gitdir
- config as the single source of truth for gitdir paths.
-
- This logic will be reached in very rare cases, as can be seen in
- the test we added.
+ This is a "last resort" type of measure to avoid conflicts since it
+ loses the human readability of the gitdir path. This logic will be
+ reached in rare cases, as can be seen in the test we added.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
- ## submodule.c ##
-@@ submodule.c: static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
- void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
- const char *submodule_name)
+ ## builtin/submodule--helper.c ##
+@@ builtin/submodule--helper.c: static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
+ static void create_default_gitdir_config(const char *submodule_name)
{
-+ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
-+ char hex_name_hash[GIT_MAX_HEXSZ + 1];
+ struct strbuf gitdir_path = STRBUF_INIT;
+ struct git_hash_ctx ctx;
- const char *gitdir;
-- char *key;
-+ char *key, header[128];
++ char hex_name_hash[GIT_MAX_HEXSZ + 1], header[128];
++ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
+ int header_len;
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
-@@ submodule.c: void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
+ /* The config is set only when extensions.submodulePathConfig is enabled */
+ if (!the_repository->repository_format_submodule_path_cfg &&
+@@ builtin/submodule--helper.c: static void create_default_gitdir_config(const char *submodule_name)
return;
}
@@ submodule.c: void submodule_name_to_gitdir(struct strbuf *buf, struct repository
+ the_hash_algo->update_fn(&ctx, submodule_name, strlen(submodule_name));
+ the_hash_algo->final_fn(raw_name_hash, &ctx);
+ hash_to_hex_algop_r(hex_name_hash, raw_name_hash, the_hash_algo);
-+ strbuf_reset(buf);
-+ repo_git_path_append(r, buf, "modules/");
-+ strbuf_addstr(buf, hex_name_hash);
-+ if (!validate_and_set_submodule_gitdir(buf, submodule_name))
++ strbuf_reset(&gitdir_path);
++ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", hex_name_hash);
++ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
++ strbuf_release(&gitdir_path);
+ return;
++ }
+
- /* Case 3: Nothing worked: error out */
- die(_("Cannot construct a valid gitdir path for submodule '%s': "
- "please set a unique git config for 'submodule.%s.gitdir'."),
+ /* Case 3: nothing worked, error out */
+ die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
+ "Please ensure it is set, for example by running something like: "
- ## t/t7425-submodule-encoding.sh ##
-@@ t/t7425-submodule-encoding.sh: test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are corre
+ ## t/t7425-submodule-gitdir-path-extension.sh ##
+@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are corre
verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
'
+test_expect_success CASE_INSENSITIVE_FS 'verify hashing conflict resolution as a last resort' '
-+ git clone -c extensions.submoduleEncoding=true main cloned-hash &&
++ git clone -c extensions.submodulePathConfig=true main cloned-hash &&
+ (
+ cd cloned-hash &&
+
7: 9b4890cfd2 < -: ---------- meson/Makefile: allow setting submodule encoding at build time
Adrian Ratiu (10):
submodule--helper: use submodule_name_to_gitdir in add_submodule
submodule: always validate gitdirs inside submodule_name_to_gitdir
builtin/submodule--helper: add gitdir command
submodule: introduce extensions.submodulePathConfig
submodule: allow runtime enabling extensions.submodulePathConfig
submodule--helper: add gitdir migration command
builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
submodule--helper: fix filesystem collisions by encoding gitdir paths
submodule: fix case-folding gitdir filesystem collisions
submodule: hash the submodule name for the gitdir path
Documentation/config/extensions.adoc | 12 +
Documentation/config/submodule.adoc | 7 +
builtin/credential-store.c | 7 +-
builtin/submodule--helper.c | 204 +++++++++++-
repository.c | 1 +
repository.h | 1 +
setup.c | 7 +
setup.h | 1 +
submodule.c | 160 +++++++--
submodule.h | 2 +
t/lib-verify-submodule-gitdir-path.sh | 24 ++
t/meson.build | 1 +
t/t7425-submodule-gitdir-path-extension.sh | 369 +++++++++++++++++++++
t/t9902-completion.sh | 1 +
url.c | 13 +
url.h | 14 +
16 files changed, 776 insertions(+), 48 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-gitdir-path-extension.sh
--
2.51.2
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v6 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
` (10 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded modules gitdir path leading to test failures.
Call the submodule_name_to_gitdir() helper instead, which was invented
exactly for this purpose and is already used by all the other locations
which work on gitdirs.
Also narrow the scope of the submod_gitdir_path variable which is not
used anymore in the updated "else" branch.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 35f6cf735e..13b5e4ed68 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3183,13 +3183,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3203,10 +3203,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3215,8 +3216,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3234,7 +3235,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-13 8:08 ` [PATCH v6 03/10] builtin/submodule--helper: add gitdir command Adrian Ratiu
` (9 subsequent siblings)
11 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Move the ad-hoc validation checks sprinkled across the source tree,
after calling submodule_name_to_gitdir() into the function proper,
which now always validates the gitdir before returning it.
This simplifies the API and helps to:
1. Avoid redundant validation calls after submodule_name_to_gitdir().
2. Avoid the risk of callers forgetting to validate.
3. Ensure gitdir paths provided by users via configs are always valid
(config gitdir paths are added in a subsequent commit).
The validation function can still be called as many times as needed
outside submodule_name_to_gitdir(), for example we keep two calls
which are still required, to avoid parallel clone races by re-running
the validation in builtin/submodule-helper.c.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 4 ----
submodule.c | 12 ++++--------
2 files changed, 4 insertions(+), 12 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 13b5e4ed68..f1fc098614 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1699,10 +1699,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
clone_data_path = to_free = xstrfmt("%s/%s", repo_get_work_tree(the_repository),
clone_data->path);
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
-
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
diff --git a/submodule.c b/submodule.c
index 40a5c6fb9d..f645372a18 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2166,11 +2166,6 @@ int submodule_move_head(const char *path, const char *super_prefix,
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
- if (validate_submodule_git_dir(gitdir.buf,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in another "
- "submodule's git dir"),
- gitdir.buf);
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
@@ -2349,9 +2344,6 @@ static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
submodule_name_to_gitdir(&new_gitdir, the_repository, sub->name);
- if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
- die(_("refusing to move '%s' into an existing git dir"),
- real_old_git_dir);
if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
die(_("could not create directory '%s'"), new_gitdir.buf);
real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
@@ -2600,4 +2592,8 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
*/
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+
+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
}
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 03/10] builtin/submodule--helper: add gitdir command
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
` (8 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu,
Brandon Williams
This exposes the gitdir name computed by submodule_name_to_gitdir()
internally, to make it easier for users and tests to interact with it.
Next commit will add a gitdir configuration, so this helper can also be
used to easily query that config or validate any gitdir path the user
sets (submodule_name_to_git_dir now runs the validation logic, since
our previous commit).
Based-on-patch-by: Brandon Williams <bwilliams.eng@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f1fc098614..3bc139ff9c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1204,6 +1204,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3583,6 +3599,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (2 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 03/10] builtin/submodule--helper: add gitdir command Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 23:22 ` Josh Steadmon
2025-12-13 8:08 ` [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
` (7 subsequent siblings)
11 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
The idea of this extension is to abstract away the submodule gitdir
path implementation: everyone is expected to use the config and not
worry about how the path is computed internally, either in git or
other implementations.
With this extension enabled, the submodule.<name>.gitdir repo config
becomes the single source of truth for all submodule gitdir paths.
The submodule.<name>.gitdir config is added automatically for all new
submodules when this extension is enabled.
Git will throw an error if the extension is enabled and a config is
missing, advising users how to migrate. Migration is manual for now.
E.g. to add a missing config entry for an existing "foo" module:
git config submodule.foo.gitdir .git/modules/foo
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 8 ++
Documentation/config/submodule.adoc | 7 ++
builtin/submodule--helper.c | 51 ++++++++
repository.c | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 59 +++++----
t/lib-verify-submodule-gitdir-path.sh | 24 ++++
t/meson.build | 1 +
t/t7425-submodule-gitdir-path-extension.sh | 138 +++++++++++++++++++++
t/t9902-completion.sh | 1 +
12 files changed, 274 insertions(+), 25 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-gitdir-path-extension.sh
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 532456644b..6ce1dcc98b 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -73,6 +73,14 @@ relativeWorktrees:::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
+submodulePathConfig:::
+ If enabled, the submodule.<name>.gitdir config is the single source of
+ truth for submodule gitdir paths and is always set for new submodules.
+ Git will error if a module does not have submodule.<name>.gitdir set.
+ Existing pre-extension submodules need to be migrated by adding the
+ missing config entries. This is done manually for now, e.g. for each
+ submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..4cf7424cda 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,13 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This sets the gitdir path for submodule <name>. It only works when
+ `extensions.submodulePathConfig` is enabled, otherwise it does nothing.
+ When the extension is enabled, this config is the single source of truth
+ for submodule gitdir paths and git will throw an error if it is missing.
+ See linkgit:git-config[1] for details.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 3bc139ff9c..699ac32004 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -435,6 +435,52 @@ struct init_cb {
};
#define INIT_CB_INIT { 0 }
+static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
+ const char *submodule_name)
+{
+ const char *value;
+ char *key;
+
+ if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
+ return -1;
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+
+ /* Nothing to do if the config already exists. */
+ if (!repo_config_get_string_tmp(the_repository, key, &value)) {
+ free(key);
+ return 0;
+ }
+
+ if (repo_config_set_gently(the_repository, key, gitdir_path->buf)) {
+ free(key);
+ return -1;
+ }
+
+ free(key);
+ return 0;
+}
+
+static void create_default_gitdir_config(const char *submodule_name)
+{
+ struct strbuf gitdir_path = STRBUF_INIT;
+
+ /* The config is set only when extensions.submodulePathConfig is enabled */
+ if (!the_repository->repository_format_submodule_path_cfg)
+ return;
+
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
+ die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
+ "Please ensure it is set, for example by running something like: "
+ "'git config submodule.%s.gitdir .git/modules/%s'"),
+ submodule_name, submodule_name, submodule_name);
+}
+
static void init_submodule(const char *path, const char *prefix,
const char *super_prefix,
unsigned int flags)
@@ -511,6 +557,9 @@ static void init_submodule(const char *path, const char *prefix,
if (repo_config_set_gently(the_repository, sb.buf, upd))
die(_("Failed to register update mode for submodule path '%s'"), displaypath);
}
+
+ create_default_gitdir_config(sub->name);
+
strbuf_release(&sb);
free(displaypath);
free(url);
@@ -3574,6 +3623,8 @@ static int module_add(int argc, const char **argv, const char *prefix,
add_data.progress = !!progress;
add_data.dissociate = !!dissociate;
+ create_default_gitdir_config(add_data.sm_name);
+
if (add_submodule(&add_data))
goto cleanup;
configure_added_submodule(&add_data);
diff --git a/repository.c b/repository.c
index 863f24411b..6cb1247f4b 100644
--- a/repository.c
+++ b/repository.c
@@ -281,6 +281,7 @@ int repo_init(struct repository *repo,
repo->repository_format_worktree_config = format.worktree_config;
repo->repository_format_relative_worktrees = format.relative_worktrees;
repo->repository_format_precious_objects = format.precious_objects;
+ repo->repository_format_submodule_path_cfg = format.submodule_path_cfg;
/* take ownership of format.partial_clone */
repo->repository_format_partial_clone = format.partial_clone;
diff --git a/repository.h b/repository.h
index 6063c4b846..7141237f97 100644
--- a/repository.h
+++ b/repository.h
@@ -165,6 +165,7 @@ struct repository {
int repository_format_worktree_config;
int repository_format_relative_worktrees;
int repository_format_precious_objects;
+ int repository_format_submodule_path_cfg;
/* Indicate if a repository has a different 'commondir' from 'gitdir' */
unsigned different_commondir:1;
diff --git a/setup.c b/setup.c
index 3a6a048620..428427d689 100644
--- a/setup.c
+++ b/setup.c
@@ -686,6 +686,9 @@ static enum extension_result handle_extension(const char *var,
} else if (!strcmp(ext, "relativeworktrees")) {
data->relative_worktrees = git_config_bool(var, value);
return EXTENSION_OK;
+ } else if (!strcmp(ext, "submodulepathconfig")) {
+ data->submodule_path_cfg = git_config_bool(var, value);
+ return EXTENSION_OK;
}
return EXTENSION_UNKNOWN;
}
@@ -1947,6 +1950,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
repo_fmt.worktree_config;
the_repository->repository_format_relative_worktrees =
repo_fmt.relative_worktrees;
+ the_repository->repository_format_submodule_path_cfg =
+ repo_fmt.submodule_path_cfg;
/* take ownership of repo_fmt.partial_clone */
the_repository->repository_format_partial_clone =
repo_fmt.partial_clone;
@@ -2045,6 +2050,8 @@ void check_repository_format(struct repository_format *fmt)
fmt->ref_storage_format);
the_repository->repository_format_worktree_config =
fmt->worktree_config;
+ the_repository->repository_format_submodule_path_cfg =
+ fmt->submodule_path_cfg;
the_repository->repository_format_relative_worktrees =
fmt->relative_worktrees;
the_repository->repository_format_partial_clone =
diff --git a/setup.h b/setup.h
index d55dcc6608..0738dec244 100644
--- a/setup.h
+++ b/setup.h
@@ -167,6 +167,7 @@ struct repository_format {
char *partial_clone; /* value of extensions.partialclone */
int worktree_config;
int relative_worktrees;
+ int submodule_path_cfg;
int is_bare;
int hash_algo;
int compat_hash_algo;
diff --git a/submodule.c b/submodule.c
index f645372a18..85ca7ea0fb 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2570,30 +2570,39 @@ int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
- /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
- */
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
+ const char *gitdir;
+ char *key;
+ int ret;
+
+ /* If extensions.submodulePathConfig is disabled, continue to use the plain path */
+ if (!r->repository_format_submodule_path_cfg) {
+ repo_git_path_append(r, buf, "modules/%s", submodule_name);
+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
+
+ return; /* plain gitdir is valid for use */
+ }
+
+ /* Extension is enabled: use the gitdir config if it exists */
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ ret = repo_config_get_string_tmp(r, key, &gitdir);
+ FREE_AND_NULL(key);
+
+ if (!ret) {
+ strbuf_addstr(buf, gitdir);
+
+ /* validate because users might have modified the config */
+ if (validate_submodule_git_dir(buf->buf, submodule_name))
+ die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
+ "if it is unique or conflicts with another module"),
+ submodule_name, gitdir);
+
+ return; /* gitdir from config is valid for use */
+ }
- if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), buf->buf);
+ die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
+ "Please ensure it is set, for example by running something like: "
+ "'git config submodule.%s.gitdir .git/modules/%s'"),
+ submodule_name, submodule_name, submodule_name, submodule_name);
}
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..62794df976
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,24 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (e.g. when adding a new submodule), so this only
+# checks the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ # Compute expected absolute path
+ expected="$(git rev-parse --git-common-dir)/$path" &&
+ expected="$(test-tool path-utils real_path "$expected")" &&
+ # Compute actual absolute path
+ actual="$(git submodule--helper gitdir "$name")" &&
+ actual="$(test-tool path-utils real_path "$actual")" &&
+ echo "$expected" >expect &&
+ echo "$actual" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/meson.build b/t/meson.build
index d3d0be2822..c101faca31 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -886,6 +886,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-gitdir-path-extension.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
new file mode 100755
index 0000000000..5d52a289f8
--- /dev/null
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -0,0 +1,138 @@
+#!/bin/sh
+
+test_description='submodulePathConfig extension works as expected'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed extension submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ # trigger the "die_path_inside_submodule" check
+ test_must_fail git submodule add ../new-sub "legacy/nested" &&
+
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submodulePathConfig true &&
+
+ git submodule add ../new-sub "New Sub" &&
+ test_commit new &&
+
+ # retrigger the "die_path_inside_submodule" check with encoding
+ test_must_fail git submodule add ../new-sub "New Sub/nested2"
+ )
+'
+
+test_expect_success 'verify new submodule gitdir config' '
+ git -C main config submodule."New Sub".gitdir > actual &&
+ echo ".git/modules/New Sub" > expect &&
+ test_cmp expect actual &&
+ verify_submodule_gitdir_path main "New Sub" "modules/New Sub"
+'
+
+test_expect_success 'manual add and verify legacy submodule gitdir config' '
+ # the legacy module should not contain a gitdir config, because it
+ # was added before the extension was enabled. Add and test it.
+ test_must_fail git -C main config submodule.legacy.gitdir &&
+ git -C main config submodule.legacy.gitdir .git/modules/legacy &&
+ git -C main config submodule.legacy.gitdir > actual &&
+ echo ".git/modules/legacy" > expect &&
+ test_cmp expect actual &&
+ verify_submodule_gitdir_path main "legacy" "modules/legacy"
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned-non-extension &&
+ (
+ cd cloned-non-extension &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/"New Sub" &&
+
+ test_must_fail git config submodule.legacy.gitdir &&
+ test_must_fail git config submodule."New Sub".gitdir &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ ) &&
+
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules main cloned-extension &&
+ (
+ cd cloned-extension &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ git config submodule.legacy.gitdir &&
+ git config submodule."New Sub".gitdir &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'commit and push changes to encoded submodules' '
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ git -C main config receive.denyCurrentBranch updateInstead &&
+ (
+ cd cloned-extension &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C "New Sub" switch --track -C main origin/main &&
+ test_commit -C "New Sub" second-commit &&
+ git -C "New Sub" push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy "New Sub" &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 964e1f1569..ffb9c8b522 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (3 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-13 8:08 ` [PATCH v6 06/10] submodule--helper: add gitdir migration command Adrian Ratiu
` (6 subsequent siblings)
11 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
This adds the ability to enable the new extension via a runtime
config to avoid having to enable it in each repo configuration.
Suggested-by: Patrick Steinhardt <ps@pks.im>
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 2 +
builtin/submodule--helper.c | 8 +++-
submodule.c | 4 +-
submodule.h | 2 +
t/t7425-submodule-gitdir-path-extension.sh | 46 ++++++++++++++++++++++
5 files changed, 59 insertions(+), 3 deletions(-)
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 6ce1dcc98b..929e4e1bf1 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -77,6 +77,8 @@ submodulePathConfig:::
If enabled, the submodule.<name>.gitdir config is the single source of
truth for submodule gitdir paths and is always set for new submodules.
Git will error if a module does not have submodule.<name>.gitdir set.
+ This extension can also be enabled as a global runtime config, with
+ the local repository config having precedence (overwrites it).
Existing pre-extension submodules need to be migrated by adding the
missing config entries. This is done manually for now, e.g. for each
submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 699ac32004..2b5b4f575b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -466,7 +466,8 @@ static void create_default_gitdir_config(const char *submodule_name)
struct strbuf gitdir_path = STRBUF_INIT;
/* The config is set only when extensions.submodulePathConfig is enabled */
- if (!the_repository->repository_format_submodule_path_cfg)
+ if (!the_repository->repository_format_submodule_path_cfg &&
+ !submodule_path_config_enabled)
return;
repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
@@ -3483,7 +3484,7 @@ static void die_on_repo_without_commits(const char *path)
static int module_add(int argc, const char **argv, const char *prefix,
struct repository *repo UNUSED)
{
- int force = 0, quiet = 0, progress = 0, dissociate = 0;
+ int force = 0, quiet = 0, progress = 0, dissociate = 0, path_cfg_ext = 0;
struct add_data add_data = ADD_DATA_INIT;
const char *ref_storage_format = NULL;
char *to_free = NULL;
@@ -3517,6 +3518,9 @@ static int module_add(int argc, const char **argv, const char *prefix,
argc = parse_options(argc, argv, prefix, options, usage, 0);
+ if (!repo_config_get_bool(the_repository, "extensions.submodulepathconfig", &path_cfg_ext))
+ submodule_path_config_enabled = path_cfg_ext;
+
if (!is_writing_gitmodules_ok())
die(_("please make sure that the .gitmodules file is in the working tree"));
diff --git a/submodule.c b/submodule.c
index 85ca7ea0fb..5752909999 100644
--- a/submodule.c
+++ b/submodule.c
@@ -32,6 +32,8 @@
#include "read-cache-ll.h"
#include "setup.h"
+int submodule_path_config_enabled;
+
static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
static int initialized_fetch_ref_tips;
static struct oid_array ref_tips_before_fetch;
@@ -2575,7 +2577,7 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
int ret;
/* If extensions.submodulePathConfig is disabled, continue to use the plain path */
- if (!r->repository_format_submodule_path_cfg) {
+ if (!r->repository_format_submodule_path_cfg && !submodule_path_config_enabled) {
repo_git_path_append(r, buf, "modules/%s", submodule_name);
if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
die(_("refusing to create/use '%s' in another submodule's "
diff --git a/submodule.h b/submodule.h
index b10e16e6c0..c15630bf26 100644
--- a/submodule.h
+++ b/submodule.h
@@ -172,4 +172,6 @@ void absorb_git_dir_into_superproject(const char *path,
*/
int get_superproject_working_tree(struct strbuf *buf);
+extern int submodule_path_config_enabled;
+
#endif
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 5d52a289f8..2f198bff82 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -135,4 +135,50 @@ test_expect_success 'fetch mixed submodule changes and verify updates' '
)
'
+test_expect_success 'runtime config extensions.submodulePathConfig on new repo' '
+ git config --global extensions.submodulePathConfig true &&
+ git init -b main runtime-test-new-repo &&
+ (
+ cd runtime-test-new-repo &&
+
+ git init -b main sub &&
+ test_commit -C sub sub-initial &&
+
+ git submodule add ./sub sub &&
+
+ # Verify that the gitdir config was created correctly
+ git config submodule.sub.gitdir > actual &&
+ echo ".git/modules/sub" > expect &&
+ test_cmp expect actual
+ )
+'
+
+test_expect_success 'runtime config extensions.submodulePathConfig on existing repo' '
+ # create a repo with the extension disabled then enable it
+ git config --global extensions.submodulePathConfig false &&
+ git init -b main runtime-test-existing-repo &&
+ (
+ cd runtime-test-existing-repo &&
+
+ git init -b main sub &&
+ test_commit -C sub sub-initial &&
+
+ git submodule add ./sub sub &&
+
+ # gitdir should not exist for this repo: it must be migrated
+ test_must_fail git config submodule.sub.gitdir
+ ) &&
+ git config --global extensions.submodulePathConfig true &&
+ (
+ cd runtime-test-existing-repo &&
+
+ git submodule add ./sub sub2 &&
+
+ # gitdir should exist after enabling the global config
+ git config submodule.sub2.gitdir > actual &&
+ echo ".git/modules/sub2" > expect &&
+ test_cmp expect actual
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 06/10] submodule--helper: add gitdir migration command
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (4 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-13 8:08 ` [PATCH v6 07/10] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
` (5 subsequent siblings)
11 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Manually running
"git config submodule.<name>.gitdir .git/modules/<name>"
for each submodule can be impractical, so add a migration command to
submodule--helper to automatically create configs for all submodules
as required by extensions.submodulePathConfig.
The command calls create_default_gitdir_config() which validates the
gitdir paths before adding the configs.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 6 ++-
builtin/submodule--helper.c | 58 ++++++++++++++++++++++
t/t7425-submodule-gitdir-path-extension.sh | 34 +++++++++++++
3 files changed, 96 insertions(+), 2 deletions(-)
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 929e4e1bf1..63c8727c3b 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -80,8 +80,10 @@ submodulePathConfig:::
This extension can also be enabled as a global runtime config, with
the local repository config having precedence (overwrites it).
Existing pre-extension submodules need to be migrated by adding the
- missing config entries. This is done manually for now, e.g. for each
- submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
+ missing config entries. This can be done manually, e.g. for each
+ submodule: "git config submodule.<name>.gitdir .git/modules/<name>",
+ or via the "git submodule--helper migrate-gitdir-configs" command
+ which iterates over all submodules and attempts to migrate them.
worktreeConfig:::
If enabled, then worktrees will load config settings from the
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2b5b4f575b..458dc863df 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1270,6 +1270,63 @@ static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
return 0;
}
+static int module_migrate(int argc UNUSED, const char **argv UNUSED,
+ const char *prefix UNUSED, struct repository *repo)
+{
+ struct strbuf module_dir = STRBUF_INIT;
+ DIR *dir;
+ struct dirent *de;
+
+ if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
+ die(_("could not set core.repositoryformatversion to 1. "
+ "Please enable it for migration to work, for example: "
+ "git config core.repositoryformatversion 1"));
+
+ if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
+ die(_("could not enable submodulePathConfig extension. It is required "
+ "for migration to work. Please enable it in the root repo: "
+ "git config extensions.submodulePathConfig true"));
+
+ repo->repository_format_submodule_path_cfg = 1;
+
+ repo_git_path_append(repo, &module_dir, "modules/");
+
+ dir = opendir(module_dir.buf);
+ if (!dir)
+ die(_("could not open '%s'"), module_dir.buf);
+
+ while ((de = readdir(dir))) {
+ struct strbuf gitdir_path = STRBUF_INIT;
+ char *key;
+ const char *value;
+
+ if (is_dot_or_dotdot(de->d_name))
+ continue;
+
+ strbuf_addf(&gitdir_path, "%s/%s", module_dir.buf, de->d_name);
+ if (!is_git_directory(gitdir_path.buf)) {
+ strbuf_release(&gitdir_path);
+ continue;
+ }
+ strbuf_release(&gitdir_path);
+
+ key = xstrfmt("submodule.%s.gitdir", de->d_name);
+ if (!repo_config_get_string_tmp(repo, key, &value)) {
+ /* Already has a gitdir config, nothing to do. */
+ free(key);
+ continue;
+ }
+ free(key);
+
+ create_default_gitdir_config(de->d_name);
+ }
+
+ closedir(dir);
+ strbuf_release(&module_dir);
+
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3654,6 +3711,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("migrate-gitdir-configs", &fn, module_migrate),
OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 2f198bff82..b7f0e8cdf4 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -181,4 +181,38 @@ test_expect_success 'runtime config extensions.submodulePathConfig on existing r
)
'
+test_expect_success 'submodule--helper migrates legacy modules' '
+ git init sm-repo-1 &&
+ test_commit -C sm-repo-1 initial-1 &&
+ git init sm-repo-2 &&
+ test_commit -C sm-repo-2 initial-2 &&
+
+ # ensure the global config is disabled so we can actually test migration
+ git config --global extensions.submodulePathConfig false &&
+
+ git init -b main migrate-test &&
+ (
+ cd migrate-test &&
+
+ git submodule add ../sm-repo-1 sub1 &&
+ git submodule add ../sm-repo-2 sub2 &&
+ test_commit add-submodules &&
+
+ # gitdir configs should not exist
+ test_must_fail git config submodule.sub1.gitdir &&
+ test_must_fail git config submodule.sub2.gitdir &&
+
+ git submodule--helper migrate-gitdir-configs &&
+
+ # gitdir configs must exist after migration
+ git config submodule.sub1.gitdir >actual &&
+ echo ".git/modules/sub1" >expect &&
+ test_cmp expect actual &&
+
+ git config submodule.sub2.gitdir >actual &&
+ echo ".git/modules/sub2" >expect &&
+ test_cmp expect actual
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 07/10] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (5 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 06/10] submodule--helper: add gitdir migration command Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 08/10] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
` (4 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it was the only place using it.
However now we need it to apply URL-encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back as a
public function exposed via url.h, instead of the old helper path
(strbuf), which has nothing to do with 3986 encoding/decoding anymore.
This function will be used in subsequent commits which do the encoding.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 7 +------
url.c | 6 ++++++
url.h | 7 +++++++
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..bc1453c6b2 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -7,6 +7,7 @@
#include "path.h"
#include "string-list.h"
#include "parse-options.h"
+#include "url.h"
#include "write-or-die.h"
static struct lock_file credential_lock;
@@ -76,12 +77,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/url.c b/url.c
index 282b12495a..adc289229c 100644
--- a/url.c
+++ b/url.c
@@ -3,6 +3,12 @@
#include "strbuf.h"
#include "url.h"
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 2a27c34277..e644c3c809 100644
--- a/url.h
+++ b/url.h
@@ -21,4 +21,11 @@ char *url_decode_parameter_value(const char **query);
void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
+/*
+ * The set of unreserved characters as per STD66 (RFC3986) is
+ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
+ * components without percent-encoding.
+ */
+int is_rfc3986_unreserved(char ch);
+
#endif /* URL_H */
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 08/10] submodule--helper: fix filesystem collisions by encoding gitdir paths
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (6 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 07/10] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 09/10] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
` (3 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Fix nested filesystem collisions by url-encoding gitdir paths stored
in submodule.%s.gitdir, when extensions.submodulePathConfig is enabled.
Credit goes to Junio and Patrick for coming up with this design: the
encoding is only applied when necessary, to newly added submodules.
Existing modules don't need the encoding because git already errors
out when detecting nested gitdirs before this patch.
This commit adds the basic url-encoding and some tests. Next commits
extend the encode -> validate -> retry loop to fix more conflicts.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 12 +++++
submodule.c | 42 +++++++++++++++-
t/t7425-submodule-gitdir-path-extension.sh | 57 ++++++++++++++++++++++
3 files changed, 110 insertions(+), 1 deletion(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 458dc863df..2a8d3a9d92 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -34,6 +34,7 @@
#include "list-objects-filter-options.h"
#include "wildmatch.h"
#include "strbuf.h"
+#include "url.h"
#define OPT_QUIET (1 << 0)
#define OPT_CACHED (1 << 1)
@@ -470,12 +471,23 @@ static void create_default_gitdir_config(const char *submodule_name)
!submodule_path_config_enabled)
return;
+ /* Case 1: try the plain module name */
repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
strbuf_release(&gitdir_path);
return;
}
+ /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
+ /* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
"'git config submodule.%s.gitdir .git/modules/%s'"),
diff --git a/submodule.c b/submodule.c
index 5752909999..bf6db1f2b9 100644
--- a/submodule.c
+++ b/submodule.c
@@ -31,6 +31,7 @@
#include "commit-reach.h"
#include "read-cache-ll.h"
#include "setup.h"
+#include "url.h"
int submodule_path_config_enabled;
@@ -2247,12 +2248,43 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+/*
+ * Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
+ * This does not print errors like the non-encoded version, because encoding is supposed
+ * to mitigate / fix all these.
+ */
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+{
+ const char *modules_marker = "/modules/";
+ char *p = git_dir, *last_submodule_name = NULL;
+
+ if (!the_repository->repository_format_submodule_path_cfg)
+ BUG("validate_submodule_encoded_git_dir() must be called with "
+ "extensions.submodulePathConfig enabled.");
+
+ /* Find the last submodule name in the gitdir path (modules can be nested). */
+ while ((p = strstr(p, modules_marker))) {
+ last_submodule_name = p + strlen(modules_marker);
+ p++;
+ }
+
+ /* Prevent the use of '/' in encoded names */
+ if (!last_submodule_name || strchr(last_submodule_name, '/'))
+ return -1;
+
+ return 0;
+}
+
+static int validate_submodule_legacy_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
char *p;
int ret = 0;
+ if (the_repository->repository_format_submodule_path_cfg)
+ BUG("validate_submodule_git_dir() must be called with "
+ "extensions.submodulePathConfig disabled.");
+
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
BUG("submodule name '%s' not a suffix of git dir '%s'",
@@ -2288,6 +2320,14 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
return 0;
}
+int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+{
+ if (!the_repository->repository_format_submodule_path_cfg)
+ return validate_submodule_legacy_git_dir(git_dir, submodule_name);
+
+ return validate_submodule_encoded_git_dir(git_dir, submodule_name);
+}
+
int validate_submodule_path(const char *path)
{
char *p = xstrdup(path);
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index b7f0e8cdf4..7c4f87b79e 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -215,4 +215,61 @@ test_expect_success 'submodule--helper migrates legacy modules' '
)
'
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules nested clone_nested &&
+
+ verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
+ git -C clone_nested config submodule.hippo.gitdir > actual &&
+ test_grep "\.git/modules/hippo$" actual &&
+
+ verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks &&
+ git -C clone_nested config submodule.hippo/hooks.gitdir > actual &&
+ test_grep "\.git/modules/hippo%2fhooks$" actual
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules --jobs=2 nested clone_parallel &&
+
+ verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
+ git -C clone_nested config submodule.hippo.gitdir > actual &&
+ test_grep "\.git/modules/hippo$" actual &&
+
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks &&
+ git -C clone_nested config submodule.hippo/hooks.gitdir > actual &&
+ test_grep "\.git/modules/hippo%2fhooks$" actual
+'
+
+test_expect_success 'disabling extensions.submodulePathConfig prevents nested submodules' '
+ (
+ cd clone_nested &&
+ # disable extension and verify failure
+ git config extensions.submodulePathConfig false &&
+ test_must_fail git submodule add ./thing2 hippo/foobar &&
+ # re-enable extension and verify it works
+ git config extensions.submodulePathConfig true &&
+ git submodule add ./thing2 hippo/foobar
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 09/10] submodule: fix case-folding gitdir filesystem collisions
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (7 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 08/10] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 10/10] submodule: hash the submodule name for the gitdir path Adrian Ratiu
` (2 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Add a new check when extension.submodulePathConfig is enabled, to
detect and prevent case-folding filesystem colisions. When this
new check is triggered, a stricter casefolding aware URI encoding
is used to percent-encode uppercase characters.
By using this check/retry mechanism the uppercase encoding is
only applied when necessary, so case-sensitive filesystems are
not affected.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 26 ++++++++++-
submodule.c | 53 +++++++++++++++++++++-
t/t7425-submodule-gitdir-path-extension.sh | 35 ++++++++++++++
url.c | 7 +++
url.h | 7 +++
5 files changed, 126 insertions(+), 2 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2a8d3a9d92..5851d49b28 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -478,7 +478,7 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
strbuf_reset(&gitdir_path);
repo_git_path_append(the_repository, &gitdir_path, "modules/");
strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
@@ -487,6 +487,30 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
+ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+
+ /* Case 2.3: Try some derived gitdir names, see if one sticks */
+ for (char c = '0'; c <= '9'; c++) {
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
+ strbuf_addch(&gitdir_path, c);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
+ strbuf_addch(&gitdir_path, c);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+ }
+
/* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
diff --git a/submodule.c b/submodule.c
index bf6db1f2b9..ae131a56b2 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2248,15 +2248,58 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
+static int check_casefolding_conflict(const char *git_dir,
+ const char *submodule_name,
+ const bool suffixes_match)
+{
+ char *p, *modules_dir = xstrdup(git_dir);
+ struct dirent *de;
+ DIR *dir = NULL;
+ int ret = 0;
+
+ if ((p = find_last_dir_sep(modules_dir)))
+ *p = '\0';
+
+ /* No conflict is possible if modules_dir doesn't exist (first clone) */
+ if (!is_directory(modules_dir))
+ goto cleanup;
+
+ dir = opendir(modules_dir);
+ if (!dir) {
+ ret = -1;
+ goto cleanup;
+ }
+
+ /* Check for another directory under .git/modules that differs only in case. */
+ while ((de = readdir(dir))) {
+ if (!strcmp(de->d_name, ".") || !strcmp(de->d_name, ".."))
+ continue;
+
+ if ((suffixes_match || is_git_directory(git_dir)) &&
+ !strcasecmp(de->d_name, submodule_name) &&
+ strcmp(de->d_name, submodule_name)) {
+ ret = -1; /* collision found */
+ break;
+ }
+ }
+
+cleanup:
+ if (dir)
+ closedir(dir);
+ free(modules_dir);
+ return ret;
+}
+
/*
* Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
* This does not print errors like the non-encoded version, because encoding is supposed
* to mitigate / fix all these.
*/
-static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name)
{
const char *modules_marker = "/modules/";
char *p = git_dir, *last_submodule_name = NULL;
+ int config_ignorecase = 0;
if (!the_repository->repository_format_submodule_path_cfg)
BUG("validate_submodule_encoded_git_dir() must be called with "
@@ -2272,6 +2315,14 @@ static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodu
if (!last_submodule_name || strchr(last_submodule_name, '/'))
return -1;
+ /* Prevent conflicts on case-folding filesystems */
+ repo_config_get_bool(the_repository, "core.ignorecase", &config_ignorecase);
+ if (ignore_case || config_ignorecase) {
+ bool suffixes_match = !strcmp(last_submodule_name, submodule_name);
+ return check_casefolding_conflict(git_dir, submodule_name,
+ suffixes_match);
+ }
+
return 0;
}
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 7c4f87b79e..a3c6c0c6b9 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -272,4 +272,39 @@ test_expect_success 'disabling extensions.submodulePathConfig prevents nested su
)
'
+test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are correctly encoded' '
+ git clone -c extensions.submodulePathConfig=true main cloned-folding &&
+ (
+ cd cloned-folding &&
+
+ # conflict: the "folding" gitdir will already be taken
+ git submodule add ../new-sub "folding" &&
+ test_commit lowercase &&
+ git submodule add ../new-sub "FoldinG" &&
+ test_commit uppercase &&
+
+ # conflict: the "foo" gitdir will already be taken
+ git submodule add ../new-sub "FOO" &&
+ test_commit uppercase-foo &&
+ git submodule add ../new-sub "foo" &&
+ test_commit lowercase-foo &&
+
+ # create a multi conflict between foobar, fooBar and foo%42ar
+ # the "foo" gitdir will already be taken
+ git submodule add ../new-sub "foobar" &&
+ test_commit lowercase-foobar &&
+ git submodule add ../new-sub "foo%42ar" &&
+ test_commit encoded-foo%42ar &&
+ git submodule add ../new-sub "fooBar" &&
+ test_commit mixed-fooBar
+ ) &&
+ verify_submodule_gitdir_path cloned-folding "folding" "modules/folding" &&
+ verify_submodule_gitdir_path cloned-folding "FoldinG" "modules/%46oldin%47" &&
+ verify_submodule_gitdir_path cloned-folding "FOO" "modules/FOO" &&
+ verify_submodule_gitdir_path cloned-folding "foo" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-folding "foobar" "modules/foobar" &&
+ verify_submodule_gitdir_path cloned-folding "foo%42ar" "modules/foo%42ar" &&
+ verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
+'
+
test_done
diff --git a/url.c b/url.c
index adc289229c..3ca5987e90 100644
--- a/url.c
+++ b/url.c
@@ -9,6 +9,13 @@ int is_rfc3986_unreserved(char ch)
ch == '-' || ch == '_' || ch == '.' || ch == '~';
}
+int is_casefolding_rfc3986_unreserved(char c)
+{
+ return (c >= 'a' && c <= 'z') ||
+ (c >= '0' && c <= '9') ||
+ c == '-' || c == '.' || c == '_' || c == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index e644c3c809..cd9140e994 100644
--- a/url.h
+++ b/url.h
@@ -28,4 +28,11 @@ void str_end_url_with_slash(const char *url, char **dest);
*/
int is_rfc3986_unreserved(char ch);
+/*
+ * This is a variant of is_rfc3986_unreserved() that treats uppercase
+ * letters as "reserved". This forces them to be percent-encoded, allowing
+ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
+ */
+int is_casefolding_rfc3986_unreserved(char c);
+
#endif /* URL_H */
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v6 10/10] submodule: hash the submodule name for the gitdir path
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (8 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 09/10] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
@ 2025-12-13 8:08 ` Adrian Ratiu
2025-12-13 14:03 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Ben Knoble
2025-12-16 23:20 ` Josh Steadmon
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-13 8:08 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
If none of the previous plain-text / encoding / derivation steps work
and case 2.4 is reached, then try a hash of the submodule name to see
if that can be a valid gitdir before giving up and throwing an error.
This is a "last resort" type of measure to avoid conflicts since it
loses the human readability of the gitdir path. This logic will be
reached in rare cases, as can be seen in the test we added.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 19 +++++++
t/t7425-submodule-gitdir-path-extension.sh | 59 ++++++++++++++++++++++
2 files changed, 78 insertions(+)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 5851d49b28..3cbacadbe8 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -465,6 +465,10 @@ static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
static void create_default_gitdir_config(const char *submodule_name)
{
struct strbuf gitdir_path = STRBUF_INIT;
+ struct git_hash_ctx ctx;
+ char hex_name_hash[GIT_MAX_HEXSZ + 1], header[128];
+ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
+ int header_len;
/* The config is set only when extensions.submodulePathConfig is enabled */
if (!the_repository->repository_format_submodule_path_cfg &&
@@ -511,6 +515,21 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
+ /* Case 2.4: If all the above failed, try a hash of the name as a last resort */
+ header_len = snprintf(header, sizeof(header), "blob %zu", strlen(submodule_name));
+ the_hash_algo->init_fn(&ctx);
+ the_hash_algo->update_fn(&ctx, header, header_len);
+ the_hash_algo->update_fn(&ctx, "\0", 1);
+ the_hash_algo->update_fn(&ctx, submodule_name, strlen(submodule_name));
+ the_hash_algo->final_fn(raw_name_hash, &ctx);
+ hash_to_hex_algop_r(hex_name_hash, raw_name_hash, the_hash_algo);
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", hex_name_hash);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
/* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index a3c6c0c6b9..1599d28587 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -307,4 +307,63 @@ test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are corre
verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
'
+test_expect_success CASE_INSENSITIVE_FS 'verify hashing conflict resolution as a last resort' '
+ git clone -c extensions.submodulePathConfig=true main cloned-hash &&
+ (
+ cd cloned-hash &&
+
+ # conflict: add all submodule conflicting variants until we reach the
+ # final hashing conflict resolution for submodule "foo"
+ git submodule add ../new-sub "foo" &&
+ git submodule add ../new-sub "foo0" &&
+ git submodule add ../new-sub "foo1" &&
+ git submodule add ../new-sub "foo2" &&
+ git submodule add ../new-sub "foo3" &&
+ git submodule add ../new-sub "foo4" &&
+ git submodule add ../new-sub "foo5" &&
+ git submodule add ../new-sub "foo6" &&
+ git submodule add ../new-sub "foo7" &&
+ git submodule add ../new-sub "foo8" &&
+ git submodule add ../new-sub "foo9" &&
+ git submodule add ../new-sub "%46oo" &&
+ git submodule add ../new-sub "%46oo0" &&
+ git submodule add ../new-sub "%46oo1" &&
+ git submodule add ../new-sub "%46oo2" &&
+ git submodule add ../new-sub "%46oo3" &&
+ git submodule add ../new-sub "%46oo4" &&
+ git submodule add ../new-sub "%46oo5" &&
+ git submodule add ../new-sub "%46oo6" &&
+ git submodule add ../new-sub "%46oo7" &&
+ git submodule add ../new-sub "%46oo8" &&
+ git submodule add ../new-sub "%46oo9" &&
+ test_commit add-foo-variants &&
+ git submodule add ../new-sub "Foo" &&
+ test_commit add-uppercase-foo
+ ) &&
+ verify_submodule_gitdir_path cloned-hash "foo" "modules/foo" &&
+ verify_submodule_gitdir_path cloned-hash "foo0" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-hash "foo1" "modules/foo1" &&
+ verify_submodule_gitdir_path cloned-hash "foo2" "modules/foo2" &&
+ verify_submodule_gitdir_path cloned-hash "foo3" "modules/foo3" &&
+ verify_submodule_gitdir_path cloned-hash "foo4" "modules/foo4" &&
+ verify_submodule_gitdir_path cloned-hash "foo5" "modules/foo5" &&
+ verify_submodule_gitdir_path cloned-hash "foo6" "modules/foo6" &&
+ verify_submodule_gitdir_path cloned-hash "foo7" "modules/foo7" &&
+ verify_submodule_gitdir_path cloned-hash "foo8" "modules/foo8" &&
+ verify_submodule_gitdir_path cloned-hash "foo9" "modules/foo9" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo" "modules/%46oo" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo0" "modules/%46oo0" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo1" "modules/%46oo1" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo2" "modules/%46oo2" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo3" "modules/%46oo3" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo4" "modules/%46oo4" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo5" "modules/%46oo5" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo6" "modules/%46oo6" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo7" "modules/%46oo7" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo8" "modules/%46oo8" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo9" "modules/%46oo9" &&
+ hash=$(printf "Foo" | git hash-object --stdin) &&
+ verify_submodule_gitdir_path cloned-hash "Foo" "modules/${hash}"
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (9 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 10/10] submodule: hash the submodule name for the gitdir path Adrian Ratiu
@ 2025-12-13 14:03 ` Ben Knoble
2025-12-15 16:28 ` Adrian Ratiu
2025-12-16 23:20 ` Josh Steadmon
11 siblings, 1 reply; 179+ messages in thread
From: Ben Knoble @ 2025-12-13 14:03 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Phillip Wood
> Le 13 déc. 2025 à 03:09, Adrian Ratiu <adrian.ratiu@collabora.com> a écrit :
>
> Hello everyone,
>
> For those new to the series, we're implementing a submodule gitdir
> extension which allows us to have a unified way to determine gitdirs
> and do things like encode submodule paths to avoid FS conflicts.
Hi there, I admit I haven’t followed this series closely. I use submodules quite a bit but haven’t yet peered into the depths of the implementation.
I read over the documentation changes in this series, and it’s not clear to me how or why I would use this new feature (I don’t mean there’s no benefit! Just that I’m having a hard time parsing it out.). By “how” I mean: I can see how to set config and run the migrator; what does that unlock for me to now go and do?
Does one of the previous cover letters explain how this is useful to submodule users? If so which, and perhaps the docs could also contain a “here’s when/why you might want this extension enabled and what it allows you to do”?
Or maybe this is meant to be not too user-facing, in which case I’m curious who would turn this on and why still :)
Again, I am mostly curious, so please don’t read this as an attempt to hold the series hostage! :)
Best,
Ben
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-13 14:03 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Ben Knoble
@ 2025-12-15 16:28 ` Adrian Ratiu
2025-12-16 0:53 ` Junio C Hamano
2025-12-18 3:43 ` Ben Knoble
0 siblings, 2 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-15 16:28 UTC (permalink / raw)
To: Ben Knoble
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Phillip Wood
On Sat, 13 Dec 2025, Ben Knoble <ben.knoble@gmail.com> wrote:
>> Le 13 déc. 2025 à 03:09, Adrian Ratiu <adrian.ratiu@collabora.com> a écrit :
>>
>> Hello everyone,
>>
>> For those new to the series, we're implementing a submodule gitdir
>> extension which allows us to have a unified way to determine gitdirs
>> and do things like encode submodule paths to avoid FS conflicts.
>
> Hi there, I admit I haven’t followed this series closely. I use submodules quite a bit but haven’t yet peered into the depths of the implementation.
>
> I read over the documentation changes in this series, and it’s not clear to me how or why I would use this new feature (I don’t mean there’s no benefit! Just that I’m having a hard time parsing it out.). By “how” I mean: I can see how to set config and run the migrator; what does that unlock for me to now go and do?
>
> Does one of the previous cover letters explain how this is useful to submodule users? If so which, and perhaps the docs could also contain a “here’s when/why you might want this extension enabled and what it allows you to do”?
>
> Or maybe this is meant to be not too user-facing, in which case I’m curious who would turn this on and why still :)
>
> Again, I am mostly curious, so please don’t read this as an attempt to hold the series hostage! :)
It's perfectly ok to ask, no problem. :)
This series is for the minority of users who either:
1. Encounter errors like the following in submodule.c:
die(_("refusing to create/use '%s' in another submodule's "...)
These errors can happen due to a number of factors, like
case-insensitive filesystems or submodule layouts.
2. Need to specify non-standard gitdir repository paths, different from
the currently hardcoded .git/modules/<plain-name> location.
With this series, the gitdir config becomes the unified way to
set/get the gitdir paths, so you can move them around as needed.
It also helps other git implementations who don't need to exactly
match git's behaviour: the config becomes the standard interface.
If you are not in one of the two above cases, then there is no reason to
enable this and it won't affect you.
Hope this is clear, maybe we could spell it out better in the
documentation (suggestions welcome btw) or even tell users in the error
messages to enable this extension.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-15 16:28 ` Adrian Ratiu
@ 2025-12-16 0:53 ` Junio C Hamano
2025-12-18 3:43 ` Ben Knoble
1 sibling, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-12-16 0:53 UTC (permalink / raw)
To: Adrian Ratiu
Cc: Ben Knoble, git, Emily Shaffer, Rodrigo Damazio Bovendorp,
Jeff King, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> 1. Encounter errors like the following in submodule.c:
> die(_("refusing to create/use '%s' in another submodule's "...)
>
> These errors can happen due to a number of factors, like
> case-insensitive filesystems or submodule layouts.
>
> 2. Need to specify non-standard gitdir repository paths, different from
> the currently hardcoded .git/modules/<plain-name> location.
Unlike 1. that hints where the need might come from (e.g., the
mention of case insensitivity), 2. has no hint on why one may want
to use "non-standard gitdir", which is better than nothing but
probably still not helpful enough.
Perhaps giving a concrete example or two in the documentation may
help? "Imagine you have submodule X at path P on the master branch,
and then you want to add another submodule Y at path Q on a separate
branch that does not have the submodule X yet. If path P and path Q
overlaps THIS WAY, then THIS AND THAT BAD THINGS HAPPEN. This feature
lets users work this around by DOING THIS AND THAT".
> If you are not in one of the two above cases, then there is no reason to
> enable this and it won't affect you.
>
> Hope this is clear, maybe we could spell it out better in the
> documentation (suggestions welcome btw) or even tell users in the error
> messages to enable this extension.
Absolutlely. Developers answering questions only here will waste
the brain cycles spent while coming up with the answer, so please do
document what audiences the feature is meant to help.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-12-13 8:08 ` [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
@ 2025-12-16 9:09 ` Patrick Steinhardt
0 siblings, 0 replies; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-16 9:09 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sat, Dec 13, 2025 at 10:08:08AM +0200, Adrian Ratiu wrote:
> Move the ad-hoc validation checks sprinkled across the source tree,
> after calling submodule_name_to_gitdir() into the function proper,
> which now always validates the gitdir before returning it.
>
> This simplifies the API and helps to:
> 1. Avoid redundant validation calls after submodule_name_to_gitdir().
> 2. Avoid the risk of callers forgetting to validate.
> 3. Ensure gitdir paths provided by users via configs are always valid
> (config gitdir paths are added in a subsequent commit).
>
> The validation function can still be called as many times as needed
> outside submodule_name_to_gitdir(), for example we keep two calls
> which are still required, to avoid parallel clone races by re-running
> the validation in builtin/submodule-helper.c.
Yup, this looks obviously correct now as the sites where we remove calls
to `validate_submodule_git_dir()` are all sites where we call
`submodule_name_to_gitdir()` immediately before.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig
2025-12-13 8:08 ` [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 9:45 ` Adrian Ratiu
2025-12-16 23:22 ` Josh Steadmon
1 sibling, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-16 9:09 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sat, Dec 13, 2025 at 10:08:10AM +0200, Adrian Ratiu wrote:
> diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
> index 532456644b..6ce1dcc98b 100644
> --- a/Documentation/config/extensions.adoc
> +++ b/Documentation/config/extensions.adoc
> @@ -73,6 +73,14 @@ relativeWorktrees:::
> repaired with either the `--relative-paths` option or with the
> `worktree.useRelativePaths` config set to `true`.
>
> +submodulePathConfig:::
> + If enabled, the submodule.<name>.gitdir config is the single source of
> + truth for submodule gitdir paths and is always set for new submodules.
> + Git will error if a module does not have submodule.<name>.gitdir set.
> + Existing pre-extension submodules need to be migrated by adding the
> + missing config entries. This is done manually for now, e.g. for each
> + submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
> +
> worktreeConfig:::
> If enabled, then worktrees will load config settings from the
> `$GIT_DIR/config.worktree` file in addition to the
Yup, makes sense.
> diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
> index 0672d99117..4cf7424cda 100644
> --- a/Documentation/config/submodule.adoc
> +++ b/Documentation/config/submodule.adoc
> @@ -52,6 +52,13 @@ submodule.<name>.active::
> submodule.active config option. See linkgit:gitsubmodules[7] for
> details.
>
> +submodule.<name>.gitdir::
> + This sets the gitdir path for submodule <name>. It only works when
> + `extensions.submodulePathConfig` is enabled, otherwise it does nothing.
This reads a tiny bit awkward. How about: "This configuration is only
respected when..."?
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 3bc139ff9c..699ac32004 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -435,6 +435,52 @@ struct init_cb {
> };
> #define INIT_CB_INIT { 0 }
>
> +static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
> + const char *submodule_name)
> +{
> + const char *value;
> + char *key;
> +
> + if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
> + return -1;
> +
> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
> +
> + /* Nothing to do if the config already exists. */
Nit: additional leading space.
> +static void create_default_gitdir_config(const char *submodule_name)
> +{
> + struct strbuf gitdir_path = STRBUF_INIT;
> +
> + /* The config is set only when extensions.submodulePathConfig is enabled */
> + if (!the_repository->repository_format_submodule_path_cfg)
> + return;
> +
> + repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
> + if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
> + strbuf_release(&gitdir_path);
> + return;
> + }
> +
> + die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
> + "Please ensure it is set, for example by running something like: "
> + "'git config submodule.%s.gitdir .git/modules/%s'"),
> + submodule_name, submodule_name, submodule_name);
> +}
> +
> static void init_submodule(const char *path, const char *prefix,
> const char *super_prefix,
> unsigned int flags)
Okay, here we populate the configuration if and only if the repository
extension is enabled. Makes sense.
> diff --git a/submodule.c b/submodule.c
> index f645372a18..85ca7ea0fb 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2570,30 +2570,39 @@ int submodule_to_gitdir(struct repository *repo,
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name)
> {
> - /*
> - * NEEDSWORK: The current way of mapping a submodule's name to
> - * its location in .git/modules/ has problems with some naming
> - * schemes. For example, if a submodule is named "foo" and
> - * another is named "foo/bar" (whether present in the same
> - * superproject commit or not - the problem will arise if both
> - * superproject commits have been checked out at any point in
> - * time), or if two submodule names only have different cases in
> - * a case-insensitive filesystem.
> - *
> - * There are several solutions, including encoding the path in
> - * some way, introducing a submodule.<name>.gitdir config in
> - * .git/config (not .gitmodules) that allows overriding what the
> - * gitdir of a submodule would be (and teach Git, upon noticing
> - * a clash, to automatically determine a non-clashing name and
> - * to write such a config), or introducing a
> - * submodule.<name>.gitdir config in .gitmodules that repo
> - * administrators can explicitly set. Nothing has been decided,
> - * so for now, just append the name at the end of the path.
> - */
> - repo_git_path_append(r, buf, "modules/");
> - strbuf_addstr(buf, submodule_name);
> + const char *gitdir;
> + char *key;
> + int ret;
> +
> + /* If extensions.submodulePathConfig is disabled, continue to use the plain path */
> + if (!r->repository_format_submodule_path_cfg) {
> + repo_git_path_append(r, buf, "modules/%s", submodule_name);
> + if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
> + die(_("refusing to create/use '%s' in another submodule's "
> + "git dir"), buf->buf);
> +
> + return; /* plain gitdir is valid for use */
> + }
> +
> + /* Extension is enabled: use the gitdir config if it exists */
> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
> + ret = repo_config_get_string_tmp(r, key, &gitdir);
> + FREE_AND_NULL(key);
> +
> + if (!ret) {
> + strbuf_addstr(buf, gitdir);
> +
> + /* validate because users might have modified the config */
> + if (validate_submodule_git_dir(buf->buf, submodule_name))
> + die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
> + "if it is unique or conflicts with another module"),
> + submodule_name, gitdir);
> +
> + return; /* gitdir from config is valid for use */
> + }
>
> - if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
> - die(_("refusing to create/use '%s' in another submodule's "
> - "git dir"), buf->buf);
> + die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
> + "Please ensure it is set, for example by running something like: "
> + "'git config submodule.%s.gitdir .git/modules/%s'"),
> + submodule_name, submodule_name, submodule_name, submodule_name);
I think the logic would flow a bit more naturally if we didn't have all
the early returns. Something like the following untested and uncompiled
code:
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
if (!r->repository_format_submodule_path_cfg) {
/*
* If extensions.submodulePathConfig is disabled,
* continue to use the plain path.
*/
repo_git_path_append(r, buf, "modules/%s", submodule_name);
} else {
const char *gitdir;
char *key;
/* Otherwise, if the extension is enabled, we use the gitdir config. */
key = xstrfmt("submodule.%s.gitdir", submodule_name);
if (repo_config_get_string_tmp(r, key, &gitdir)) {
die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
"Please ensure it is set, for example by running something like: "
"'git config submodule.%s.gitdir .git/modules/%s'"),
submodule_name, submodule_name, submodule_name, submodule_name);
}
strbuf_addstr(buf, gitdir);
FREE_AND_NULL(key);
}
if (validate_submodule_git_dir(buf->buf, submodule_name)) {
die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
"if it is unique or conflicts with another module"),
submodule_name, gitdir);
}
}
I think it would make sense to also hint at the extension in the error
message here so that users know _why_ we expect the key to be set.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig
2025-12-13 8:08 ` [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 10:01 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-16 9:09 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sat, Dec 13, 2025 at 10:08:11AM +0200, Adrian Ratiu wrote:
> This adds the ability to enable the new extension via a runtime
> config to avoid having to enable it in each repo configuration.
I think this doesn't quite match what Junio and I suggested.
As far as I understand, this new runtime option will default-enable the
repository extension in _all_ repositories. But this isn't really
something that we should be doing, as it means that Git may now respect
the gitconfig even though the extension isn't enabled. Other
implementations of Git that don't understand the global configuration
will thus start to misbehave in that case.
The second issue is that the option will cause all existing repositories
that have submodules to be broken, as we don't have the "gitdir" config
key yet.
My suggestion was thus to have a global configuration that causes both
git-init(1) and git-clone(1) to automatically set the repository
extension for _new_ repositories. This allows developers not worry about
this in the future anymore, but it means that they will have to migrate
existing repositories, at least if they care about the extension. Which
I think is a fair tradeoff.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 06/10] submodule--helper: add gitdir migration command
2025-12-13 8:08 ` [PATCH v6 06/10] submodule--helper: add gitdir migration command Adrian Ratiu
@ 2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 10:17 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Patrick Steinhardt @ 2025-12-16 9:09 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sat, Dec 13, 2025 at 10:08:12AM +0200, Adrian Ratiu wrote:
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 2b5b4f575b..458dc863df 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1270,6 +1270,63 @@ static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
> return 0;
> }
>
> +static int module_migrate(int argc UNUSED, const char **argv UNUSED,
> + const char *prefix UNUSED, struct repository *repo)
> +{
> + struct strbuf module_dir = STRBUF_INIT;
> + DIR *dir;
> + struct dirent *de;
> +
> + if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
> + die(_("could not set core.repositoryformatversion to 1. "
> + "Please enable it for migration to work, for example: "
> + "git config core.repositoryformatversion 1"));
> +
> + if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
> + die(_("could not enable submodulePathConfig extension. It is required "
> + "for migration to work. Please enable it in the root repo: "
> + "git config extensions.submodulePathConfig true"));
I think we should reverse the ordering here and first create the
"gitdir" config entries before we enable the extension itself. This
ensures that a halfway-aborted migration will still leave us in a good
state, as the config entries will be ignored until the extension is
enabled.
Patrick
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig
2025-12-16 9:09 ` Patrick Steinhardt
@ 2025-12-16 9:45 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-16 9:45 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 16 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Sat, Dec 13, 2025 at 10:08:10AM +0200, Adrian Ratiu wrote:
>> diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
>> index 0672d99117..4cf7424cda 100644
>> --- a/Documentation/config/submodule.adoc
>> +++ b/Documentation/config/submodule.adoc
>> @@ -52,6 +52,13 @@ submodule.<name>.active::
>> submodule.active config option. See linkgit:gitsubmodules[7] for
>> details.
>>
>> +submodule.<name>.gitdir::
>> + This sets the gitdir path for submodule <name>. It only works when
>> + `extensions.submodulePathConfig` is enabled, otherwise it does nothing.
>
> This reads a tiny bit awkward. How about: "This configuration is only
> respected when..."?
Ack, will fix on the next reroll.
>> diff --git a/submodule.c b/submodule.c
>> index f645372a18..85ca7ea0fb 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -2570,30 +2570,39 @@ int submodule_to_gitdir(struct repository *repo,
>> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
>> const char *submodule_name)
>> {
>> - /*
>> - * NEEDSWORK: The current way of mapping a submodule's name to
>> - * its location in .git/modules/ has problems with some naming
>> - * schemes. For example, if a submodule is named "foo" and
>> - * another is named "foo/bar" (whether present in the same
>> - * superproject commit or not - the problem will arise if both
>> - * superproject commits have been checked out at any point in
>> - * time), or if two submodule names only have different cases in
>> - * a case-insensitive filesystem.
>> - *
>> - * There are several solutions, including encoding the path in
>> - * some way, introducing a submodule.<name>.gitdir config in
>> - * .git/config (not .gitmodules) that allows overriding what the
>> - * gitdir of a submodule would be (and teach Git, upon noticing
>> - * a clash, to automatically determine a non-clashing name and
>> - * to write such a config), or introducing a
>> - * submodule.<name>.gitdir config in .gitmodules that repo
>> - * administrators can explicitly set. Nothing has been decided,
>> - * so for now, just append the name at the end of the path.
>> - */
>> - repo_git_path_append(r, buf, "modules/");
>> - strbuf_addstr(buf, submodule_name);
>> + const char *gitdir;
>> + char *key;
>> + int ret;
>> +
>> + /* If extensions.submodulePathConfig is disabled, continue to use the plain path */
>> + if (!r->repository_format_submodule_path_cfg) {
>> + repo_git_path_append(r, buf, "modules/%s", submodule_name);
>> + if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
>> + die(_("refusing to create/use '%s' in another submodule's "
>> + "git dir"), buf->buf);
>> +
>> + return; /* plain gitdir is valid for use */
>> + }
>> +
>> + /* Extension is enabled: use the gitdir config if it exists */
>> + key = xstrfmt("submodule.%s.gitdir", submodule_name);
>> + ret = repo_config_get_string_tmp(r, key, &gitdir);
>> + FREE_AND_NULL(key);
>> +
>> + if (!ret) {
>> + strbuf_addstr(buf, gitdir);
>> +
>> + /* validate because users might have modified the config */
>> + if (validate_submodule_git_dir(buf->buf, submodule_name))
>> + die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
>> + "if it is unique or conflicts with another module"),
>> + submodule_name, gitdir);
>> +
>> + return; /* gitdir from config is valid for use */
>> + }
>>
>> - if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
>> - die(_("refusing to create/use '%s' in another submodule's "
>> - "git dir"), buf->buf);
>> + die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
>> + "Please ensure it is set, for example by running something like: "
>> + "'git config submodule.%s.gitdir .git/modules/%s'"),
>> + submodule_name, submodule_name, submodule_name, submodule_name);
>
> I think the logic would flow a bit more naturally if we didn't have all
> the early returns. Something like the following untested and uncompiled
> code:
>
> void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
> const char *submodule_name)
> {
> if (!r->repository_format_submodule_path_cfg) {
> /*
> * If extensions.submodulePathConfig is disabled,
> * continue to use the plain path.
> */
> repo_git_path_append(r, buf, "modules/%s", submodule_name);
> } else {
> const char *gitdir;
> char *key;
>
> /* Otherwise, if the extension is enabled, we use the gitdir config. */
> key = xstrfmt("submodule.%s.gitdir", submodule_name);
>
> if (repo_config_get_string_tmp(r, key, &gitdir)) {
> die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
> "Please ensure it is set, for example by running something like: "
> "'git config submodule.%s.gitdir .git/modules/%s'"),
> submodule_name, submodule_name, submodule_name, submodule_name);
> }
>
> strbuf_addstr(buf, gitdir);
> FREE_AND_NULL(key);
> }
>
> if (validate_submodule_git_dir(buf->buf, submodule_name)) {
> die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
> "if it is unique or conflicts with another module"),
> submodule_name, gitdir);
> }
> }
>
> I think it would make sense to also hint at the extension in the error
> message here so that users know _why_ we expect the key to be set.
Ack, will fix in the next reroll.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig
2025-12-16 9:09 ` Patrick Steinhardt
@ 2025-12-16 10:01 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-16 10:01 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 16 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Sat, Dec 13, 2025 at 10:08:11AM +0200, Adrian Ratiu wrote:
>> This adds the ability to enable the new extension via a runtime
>> config to avoid having to enable it in each repo configuration.
>
> I think this doesn't quite match what Junio and I suggested.
>
> As far as I understand, this new runtime option will default-enable the
> repository extension in _all_ repositories. But this isn't really
> something that we should be doing, as it means that Git may now respect
> the gitconfig even though the extension isn't enabled. Other
> implementations of Git that don't understand the global configuration
> will thus start to misbehave in that case.
>
> The second issue is that the option will cause all existing repositories
> that have submodules to be broken, as we don't have the "gitdir" config
> key yet.
>
> My suggestion was thus to have a global configuration that causes both
> git-init(1) and git-clone(1) to automatically set the repository
> extension for _new_ repositories. This allows developers not worry about
> this in the future anymore, but it means that they will have to migrate
> existing repositories, at least if they care about the extension. Which
> I think is a fair tradeoff.
Right, I misunderstood what you asked for. Will fix.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 06/10] submodule--helper: add gitdir migration command
2025-12-16 9:09 ` Patrick Steinhardt
@ 2025-12-16 10:17 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-16 10:17 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Josh Steadmon,
Ben Knoble, Phillip Wood
On Tue, 16 Dec 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Sat, Dec 13, 2025 at 10:08:12AM +0200, Adrian Ratiu wrote:
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index 2b5b4f575b..458dc863df 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -1270,6 +1270,63 @@ static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
>> return 0;
>> }
>>
>> +static int module_migrate(int argc UNUSED, const char **argv UNUSED,
>> + const char *prefix UNUSED, struct repository *repo)
>> +{
>> + struct strbuf module_dir = STRBUF_INIT;
>> + DIR *dir;
>> + struct dirent *de;
>> +
>> + if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
>> + die(_("could not set core.repositoryformatversion to 1. "
>> + "Please enable it for migration to work, for example: "
>> + "git config core.repositoryformatversion 1"));
>> +
>> + if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
>> + die(_("could not enable submodulePathConfig extension. It is required "
>> + "for migration to work. Please enable it in the root repo: "
>> + "git config extensions.submodulePathConfig true"));
>
> I think we should reverse the ordering here and first create the
> "gitdir" config entries before we enable the extension itself. This
> ensures that a halfway-aborted migration will still leave us in a good
> state, as the config entries will be ignored until the extension is
> enabled.
Good point. Will do.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
` (10 preceding siblings ...)
2025-12-13 14:03 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Ben Knoble
@ 2025-12-16 23:20 ` Josh Steadmon
2025-12-17 8:17 ` Adrian Ratiu
11 siblings, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-12-16 23:20 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
On 2025.12.13 10:08, Adrian Ratiu wrote:
> Hello everyone,
>
> For those new to the series, we're implementing a submodule gitdir
> extension which allows us to have a unified way to determine gitdirs
> and do things like encode submodule paths to avoid FS conflicts.
I'm afraid I've gotten a bit lost with this series. IIUC, we no longer
try to encode submodule gitdirs by default, instead we do this only if
we detect a conflict with an existing gitdir. However, in all of my
local testing, I have been unable to produce a conflict that triggers
this encoding. Instead, everything hits the error:
"fatal: A git directory for 'nested%2fsub' is found locally with remote(s):"
from `builtin/submodule--helper.c:3389`.
This happens regardless of the setting of
`extensions.submodulepathconfig` in either the repo's local config, or
my user config.
My testing setup has been as follows:
* Using our locally-built Git with our downstream patch of [1] included:
* create a repo "sub"
* create a repo "super"
* In "super":
* mkdir nested
* git submodule add ../sub nested/sub
* Verify that the submodule's gitdir is .git/modules/nested%2fsub
* Using a build of git from upstream `next` plus this series:
* git config set --global extensions.submodulepathconfig true
* git clone --recurse-submodules super super2
* create a repo "nested%2fsub"
* In "super2":
* git submodule add ../nested%2fsub
At this point I'd expect the collision detection / encoding to take
effect, but instead I get the error listed above.
[1] https://lore.kernel.org/git/20180807230637.247200-1-bmwill@google.com/
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig
2025-12-13 8:08 ` [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
@ 2025-12-16 23:22 ` Josh Steadmon
2025-12-17 7:30 ` Adrian Ratiu
1 sibling, 1 reply; 179+ messages in thread
From: Josh Steadmon @ 2025-12-16 23:22 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
On 2025.12.13 10:08, Adrian Ratiu wrote:
> The idea of this extension is to abstract away the submodule gitdir
> path implementation: everyone is expected to use the config and not
> worry about how the path is computed internally, either in git or
> other implementations.
>
> With this extension enabled, the submodule.<name>.gitdir repo config
> becomes the single source of truth for all submodule gitdir paths.
>
> The submodule.<name>.gitdir config is added automatically for all new
> submodules when this extension is enabled.
>
> Git will throw an error if the extension is enabled and a config is
> missing, advising users how to migrate. Migration is manual for now.
This part doesn't seem accurate in my testing. When cloning a project
with `--recurse-submodules` and with the extension enabled globally, the
resulting .git/config does not include gitdir configs for any of the
cloned submodules, yet no error occurs.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig
2025-12-16 23:22 ` Josh Steadmon
@ 2025-12-17 7:30 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-17 7:30 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
On Tue, 16 Dec 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.12.13 10:08, Adrian Ratiu wrote:
>> The idea of this extension is to abstract away the submodule gitdir
>> path implementation: everyone is expected to use the config and not
>> worry about how the path is computed internally, either in git or
>> other implementations.
>>
>> With this extension enabled, the submodule.<name>.gitdir repo config
>> becomes the single source of truth for all submodule gitdir paths.
>>
>> The submodule.<name>.gitdir config is added automatically for all new
>> submodules when this extension is enabled.
>>
>> Git will throw an error if the extension is enabled and a config is
>> missing, advising users how to migrate. Migration is manual for now.
>
> This part doesn't seem accurate in my testing. When cloning a project
> with `--recurse-submodules` and with the extension enabled globally, the
> resulting .git/config does not include gitdir configs for any of the
> cloned submodules, yet no error occurs.
Patrick already pointed out in the other patch that I misunderstood how
the global config is supposed to work.
What you point out here is a side-effect of that. :)
I will fix the global config in v7. indeed it should throw an error.
I will also add four test combinations for cloning w/o
--recurse-submodules and global config on/off.
I also missed these cases in the tests I added.
Many thanks,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-16 23:20 ` Josh Steadmon
@ 2025-12-17 8:17 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-17 8:17 UTC (permalink / raw)
To: Josh Steadmon
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Ben Knoble, Phillip Wood
On Tue, 16 Dec 2025, Josh Steadmon <steadmon@google.com> wrote:
> On 2025.12.13 10:08, Adrian Ratiu wrote:
>> Hello everyone,
>>
>> For those new to the series, we're implementing a submodule gitdir
>> extension which allows us to have a unified way to determine gitdirs
>> and do things like encode submodule paths to avoid FS conflicts.
>
> I'm afraid I've gotten a bit lost with this series. IIUC, we no longer
> try to encode submodule gitdirs by default, instead we do this only if
> we detect a conflict with an existing gitdir. However, in all of my
> local testing, I have been unable to produce a conflict that triggers
> this encoding. Instead, everything hits the error:
> "fatal: A git directory for 'nested%2fsub' is found locally with remote(s):"
> from `builtin/submodule--helper.c:3389`.
>
> This happens regardless of the setting of
> `extensions.submodulepathconfig` in either the repo's local config, or
> my user config.
>
> My testing setup has been as follows:
> * Using our locally-built Git with our downstream patch of [1] included:
> * create a repo "sub"
> * create a repo "super"
> * In "super":
> * mkdir nested
> * git submodule add ../sub nested/sub
> * Verify that the submodule's gitdir is .git/modules/nested%2fsub
> * Using a build of git from upstream `next` plus this series:
> * git config set --global extensions.submodulepathconfig true
> * git clone --recurse-submodules super super2
> * create a repo "nested%2fsub"
> * In "super2":
> * git submodule add ../nested%2fsub
>
> At this point I'd expect the collision detection / encoding to take
> effect, but instead I get the error listed above.
This is a good test case, thanks, I will add something very similar to
t7425-submodule-gitdir-path-extension.sh in v7.
Please wait for v7, where I will fix the global config, then you can
retry the steps above.
Just one important thing to look out for:
Existing submodules will _not_ get their submodule.<name>.gitdir
automatically created when enabling the extension, so you will get
errors like "gitdir does not exist" after enabling the extension.
Those are expected and I assume you already got errors like those.
The gitdirs can be created manually, for example:
git config submodule.<name>.gitdir .git/modules/<name>
Or they can all be created automatically:
git submodule--helper migrate-gitdir-configs
When running the migration command in a repo containing submodules, it
will also enable the extension in the super repo config, in addition to
creating all the submodule gitdirs configs, so running just that single
command should be enough.
Hope this all makes sense and happy to answer any questions,
Adrian
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding
2025-12-15 16:28 ` Adrian Ratiu
2025-12-16 0:53 ` Junio C Hamano
@ 2025-12-18 3:43 ` Ben Knoble
1 sibling, 0 replies; 179+ messages in thread
From: Ben Knoble @ 2025-12-18 3:43 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Phillip Wood
> Le 15 déc. 2025 à 11:29, Adrian Ratiu <adrian.ratiu@collabora.com> a écrit :
>
> On Sat, 13 Dec 2025, Ben Knoble <ben.knoble@gmail.com> wrote:
>>>> Le 13 déc. 2025 à 03:09, Adrian Ratiu <adrian.ratiu@collabora.com> a écrit :
>>>
>>> Hello everyone,
>>>
>>> For those new to the series, we're implementing a submodule gitdir
>>> extension which allows us to have a unified way to determine gitdirs
>>> and do things like encode submodule paths to avoid FS conflicts.
>>
>> Hi there, I admit I haven’t followed this series closely. I use submodules quite a bit but haven’t yet peered into the depths of the implementation.
>>
>> I read over the documentation changes in this series, and it’s not clear to me how or why I would use this new feature (I don’t mean there’s no benefit! Just that I’m having a hard time parsing it out.). By “how” I mean: I can see how to set config and run the migrator; what does that unlock for me to now go and do?
>>
>> Does one of the previous cover letters explain how this is useful to submodule users? If so which, and perhaps the docs could also contain a “here’s when/why you might want this extension enabled and what it allows you to do”?
>>
>> Or maybe this is meant to be not too user-facing, in which case I’m curious who would turn this on and why still :)
>>
>> Again, I am mostly curious, so please don’t read this as an attempt to hold the series hostage! :)
>
> It's perfectly ok to ask, no problem. :)
>
> This series is for the minority of users who either:
>
> 1. Encounter errors like the following in submodule.c:
> die(_("refusing to create/use '%s' in another submodule's "...)
>
> These errors can happen due to a number of factors, like
> case-insensitive filesystems or submodule layouts.
>
> 2. Need to specify non-standard gitdir repository paths, different from
> the currently hardcoded .git/modules/<plain-name> location.
>
> With this series, the gitdir config becomes the unified way to
> set/get the gitdir paths, so you can move them around as needed.
> It also helps other git implementations who don't need to exactly
> match git's behaviour: the config becomes the standard interface.
>
> If you are not in one of the two above cases, then there is no reason to
> enable this and it won't affect you.
>
> Hope this is clear, maybe we could spell it out better in the
> documentation (suggestions welcome btw) or even tell users in the error
> messages to enable this extension.
Thanks! That helped, and I am not in either case ;) I agree with Junio’s downthread points about docs.
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v7 00/11] Add submodulePathConfig extension and gitdir encoding
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
` (14 preceding siblings ...)
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 01/11] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
` (11 more replies)
15 siblings, 12 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Hello everyone,
For those new to the series, we're implementing a submodule gitdir
extension which allows us to have a unified way to determine gitdirs
and do things like encode submodule paths to avoid FS conflicts.
v7 addresses all feedback received in v6, containing just iterative
improvements to code, tests and documentation without any big design
changes this time.
Patches 1-6 implement the basic mechanisms of the new extension.
Patches 7-11 improve filesystem conflict detection and resolution.
As always, this is based on the latest master branch, I've checkd
for conflicts with next/seen, pushed to Github [1] and succesfully
ran the CI [2].
1: https://github.com/10ne1/git/tree/dev/aratiu/encoding-v7
2: https://github.com/10ne1/git/actions/runs/20384003122
Changes in v7:
* Simplified submodule_name_to_gitdir() code structure. Logically it still is
the same like in v6, just easier to read. (Patrick)
* Reworked the global config to enable the extension only for new repos
during clone and init operations (Patrick, Junio)
* Improved validation of existing submodules with encoded names (Josh, Emily)
* Added more tests for init, clone w/o --recurse-submodules and a conflict (Josh)
* Migration command creates gitdir configs before enabling the extension (Patrick)
* Reworded and reformatted the extension doc (Junio, Ben)
* Reworded submodule.<name>.gitdir documentation for clarity (Patrick)
* Added references to extensions.submodulePathConfig in error msgs (Junio, Patrick)
* Minor whitespace fixes and others nits (Patrick)
Range-diff v6 -> v7:
1: a6024a7569 = 1: a6024a7569 submodule--helper: use submodule_name_to_gitdir in add_submodule
2: 59058180eb = 2: 59058180eb submodule: always validate gitdirs inside submodule_name_to_gitdir
3: b685f4f9ff = 3: b685f4f9ff builtin/submodule--helper: add gitdir command
4: 0247c28cbc ! 4: a5d6db00ac submodule: introduce extensions.submodulePathConfig
@@ Documentation/config/extensions.adoc: relativeWorktrees:::
`worktree.useRelativePaths` config set to `true`.
+submodulePathConfig:::
-+ If enabled, the submodule.<name>.gitdir config is the single source of
-+ truth for submodule gitdir paths and is always set for new submodules.
-+ Git will error if a module does not have submodule.<name>.gitdir set.
-+ Existing pre-extension submodules need to be migrated by adding the
-+ missing config entries. This is done manually for now, e.g. for each
-+ submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
++ This extension is for the minority of users who:
+++
++--
++* Encounter errors like `refusing to create ... in another submodule's git dir`
++ due to a number of reasons, like case-insensitive filesystem conflicts when
++ creating modules named `foo` and `Foo`.
++* Require more flexible submodule layouts, for example due to nested names like
++ `foo`, `foo/bar` and `foo/baz` not supported by the default gitdir mechanism
++ which uses `.git/modules/<plain-name>` locations, causing further conflicts.
++--
+++
++When `extensions.submodulePathConfig` is enabled, the `submodule.<name>.gitdir`
++config becomes the single source of truth for all submodule gitdir paths and is
++automatically set for all new submodules both during clone and init operations.
+++
++Git will error out if a module does not have a corresponding
++`submodule.<name>.gitdir` set.
+++
++Existing (pre-extension) submodules need to be migrated by adding the missing
++config entries. This is done manually for now, e.g. for each submodule:
++`git config submodule.<name>.gitdir .git/modules/<name>`.
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
@@ Documentation/config/submodule.adoc: submodule.<name>.active::
details.
+submodule.<name>.gitdir::
-+ This sets the gitdir path for submodule <name>. It only works when
-+ `extensions.submodulePathConfig` is enabled, otherwise it does nothing.
-+ When the extension is enabled, this config is the single source of truth
-+ for submodule gitdir paths and git will throw an error if it is missing.
++ This sets the gitdir path for submodule <name>. This configuration is
++ respected when `extensions.submodulePathConfig` is enabled, otherwise it
++ has no effect. When enabled, this config becomes the single source of
++ truth for submodule gitdir paths and git will error if it is missing.
+ See linkgit:git-config[1] for details.
+
submodule.active::
@@ builtin/submodule--helper.c: struct init_cb {
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+
-+ /* Nothing to do if the config already exists. */
++ /* Nothing to do if the config already exists. */
+ if (!repo_config_get_string_tmp(the_repository, key, &value)) {
+ free(key);
+ return 0;
@@ builtin/submodule--helper.c: struct init_cb {
+{
+ struct strbuf gitdir_path = STRBUF_INIT;
+
-+ /* The config is set only when extensions.submodulePathConfig is enabled */
-+ if (!the_repository->repository_format_submodule_path_cfg)
-+ return;
-+
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
@@ builtin/submodule--helper.c: static void init_submodule(const char *path, const
die(_("Failed to register update mode for submodule path '%s'"), displaypath);
}
+
-+ create_default_gitdir_config(sub->name);
++ if (the_repository->repository_format_submodule_path_cfg)
++ create_default_gitdir_config(sub->name);
+
strbuf_release(&sb);
free(displaypath);
free(url);
+@@ builtin/submodule--helper.c: static int clone_submodule(const struct module_clone_data *clone_data,
+ char *head = xstrfmt("%s/HEAD", sm_gitdir);
+ unlink(head);
+ free(head);
+- die(_("refusing to create/use '%s' in another submodule's "
+- "git dir"), sm_gitdir);
++ die(_("refusing to create/use '%s' in another submodule's git dir. "
++ "Enabling extensions.submodulePathConfig should fix this."),
++ sm_gitdir);
+ }
+
+ connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
@@ builtin/submodule--helper.c: static int module_add(int argc, const char **argv, const char *prefix,
add_data.progress = !!progress;
add_data.dissociate = !!dissociate;
-+ create_default_gitdir_config(add_data.sm_name);
++ if (the_repository->repository_format_submodule_path_cfg)
++ create_default_gitdir_config(add_data.sm_name);
+
if (add_submodule(&add_data))
goto cleanup;
@@ setup.h: struct repository_format {
int compat_hash_algo;
## submodule.c ##
+@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
+ if (validate_submodule_git_dir(git_dir,
+ sub->name) < 0)
+ die(_("refusing to create/use '%s' in "
+- "another submodule's git dir"),
+- git_dir);
++ "another submodule's git dir. "
++ "Enabling extensions.submodulePathConfig "
++ "should fix this."), git_dir);
+ free(git_dir);
+ }
+ } else {
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
@@ submodule.c: int submodule_to_gitdir(struct repository *repo,
- */
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
-+ const char *gitdir;
-+ char *key;
-+ int ret;
-+
-+ /* If extensions.submodulePathConfig is disabled, continue to use the plain path */
+ if (!r->repository_format_submodule_path_cfg) {
++ /*
++ * If extensions.submodulePathConfig is disabled,
++ * continue to use the plain path.
++ */
+ repo_git_path_append(r, buf, "modules/%s", submodule_name);
-+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
-+ die(_("refusing to create/use '%s' in another submodule's "
-+ "git dir"), buf->buf);
++ } else {
++ const char *gitdir;
++ char *key;
++ int ret;
++
++ /* Otherwise the extension is enabled, so use the gitdir config. */
++ key = xstrfmt("submodule.%s.gitdir", submodule_name);
++ ret = repo_config_get_string_tmp(r, key, &gitdir);
++ FREE_AND_NULL(key);
++
++ if (ret)
++ die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
++ "Please ensure it is set, for example by running something like: "
++ "'git config submodule.%s.gitdir .git/modules/%s'. For details "
++ "see the extensions.submodulePathConfig documentation."),
++ submodule_name, submodule_name, submodule_name, submodule_name);
+
-+ return; /* plain gitdir is valid for use */
-+ }
-+
-+ /* Extension is enabled: use the gitdir config if it exists */
-+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
-+ ret = repo_config_get_string_tmp(r, key, &gitdir);
-+ FREE_AND_NULL(key);
-+
-+ if (!ret) {
+ strbuf_addstr(buf, gitdir);
-+
-+ /* validate because users might have modified the config */
-+ if (validate_submodule_git_dir(buf->buf, submodule_name))
-+ die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
-+ "if it is unique or conflicts with another module"),
-+ submodule_name, gitdir);
-+
-+ return; /* gitdir from config is valid for use */
+ }
- if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), buf->buf);
-+ die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
-+ "Please ensure it is set, for example by running something like: "
-+ "'git config submodule.%s.gitdir .git/modules/%s'"),
-+ submodule_name, submodule_name, submodule_name, submodule_name);
++ /* validate because users might have modified the config */
++ if (validate_submodule_git_dir(buf->buf, submodule_name))
++ die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
++ "if it is unique or conflicts with another module"),
++ submodule_name, buf->buf);
}
## t/lib-verify-submodule-gitdir-path.sh (new) ##
5: 486dd2c45c < -: ---------- submodule: allow runtime enabling extensions.submodulePathConfig
-: ---------- > 5: 6ee88ee00a submodule: allow runtime enabling extensions.submodulePathConfig
6: 9d5e050701 ! 6: 7f93a5cee0 submodule--helper: add gitdir migration command
@@ Commit message
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
## Documentation/config/extensions.adoc ##
-@@ Documentation/config/extensions.adoc: submodulePathConfig:::
- This extension can also be enabled as a global runtime config, with
- the local repository config having precedence (overwrites it).
- Existing pre-extension submodules need to be migrated by adding the
-- missing config entries. This is done manually for now, e.g. for each
-- submodule: "git config submodule.<name>.gitdir .git/modules/<name>".
-+ missing config entries. This can be done manually, e.g. for each
-+ submodule: "git config submodule.<name>.gitdir .git/modules/<name>",
-+ or via the "git submodule--helper migrate-gitdir-configs" command
-+ which iterates over all submodules and attempts to migrate them.
-
- worktreeConfig:::
- If enabled, then worktrees will load config settings from the
+@@ Documentation/config/extensions.adoc: Git will error out if a module does not have a corresponding
+ `submodule.<name>.gitdir` set.
+ +
+ Existing (pre-extension) submodules need to be migrated by adding the missing
+-config entries. This is done manually for now, e.g. for each submodule:
+-`git config submodule.<name>.gitdir .git/modules/<name>`.
++config entries. This can be done manually, e.g. for each submodule:
++`git config submodule.<name>.gitdir .git/modules/<name>`, or via the
++`git submodule--helper migrate-gitdir-configs` command which iterates over all
++submodules and attempts to migrate them.
+ +
+ The extension can be enabled automatically for new repositories by setting
+ `init.autoSetupSubmodulePathConfig` to `true`, for example by running
## builtin/submodule--helper.c ##
@@ builtin/submodule--helper.c: static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
@@ builtin/submodule--helper.c: static int module_gitdir(int argc, const char **arg
+ DIR *dir;
+ struct dirent *de;
+
-+ if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
-+ die(_("could not set core.repositoryformatversion to 1. "
-+ "Please enable it for migration to work, for example: "
-+ "git config core.repositoryformatversion 1"));
-+
-+ if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
-+ die(_("could not enable submodulePathConfig extension. It is required "
-+ "for migration to work. Please enable it in the root repo: "
-+ "git config extensions.submodulePathConfig true"));
-+
-+ repo->repository_format_submodule_path_cfg = 1;
-+
+ repo_git_path_append(repo, &module_dir, "modules/");
+
+ dir = opendir(module_dir.buf);
@@ builtin/submodule--helper.c: static int module_gitdir(int argc, const char **arg
+ closedir(dir);
+ strbuf_release(&module_dir);
+
++ if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
++ die(_("could not set core.repositoryformatversion to 1. "
++ "Please enable it for migration to work, for example: "
++ "git config core.repositoryformatversion 1"));
++
++ if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
++ die(_("could not enable submodulePathConfig extension. It is required "
++ "for migration to work. Please enable it in the root repo: "
++ "git config extensions.submodulePathConfig true"));
++
++ repo->repository_format_submodule_path_cfg = 1;
++
+ return 0;
+}
+
@@ builtin/submodule--helper.c: int cmd_submodule__helper(int argc,
OPT_SUBCOMMAND("add", &fn, module_add),
## t/t7425-submodule-gitdir-path-extension.sh ##
-@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success 'runtime config extensions.submodulePathConfig on existing r
- )
+@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success '`git clone --recurse-submodules` respects init.autoSetupSub
+ git config --global --unset init.autoSetupSubmodulePathConfig
'
+test_expect_success 'submodule--helper migrates legacy modules' '
-+ git init sm-repo-1 &&
-+ test_commit -C sm-repo-1 initial-1 &&
-+ git init sm-repo-2 &&
-+ test_commit -C sm-repo-2 initial-2 &&
-+
-+ # ensure the global config is disabled so we can actually test migration
-+ git config --global extensions.submodulePathConfig false &&
-+
-+ git init -b main migrate-test &&
+ (
-+ cd migrate-test &&
-+
-+ git submodule add ../sm-repo-1 sub1 &&
-+ git submodule add ../sm-repo-2 sub2 &&
-+ test_commit add-submodules &&
++ cd upstream &&
+
-+ # gitdir configs should not exist
++ # previous submodules exist and were not migrated yet
+ test_must_fail git config submodule.sub1.gitdir &&
+ test_must_fail git config submodule.sub2.gitdir &&
++ test_path_is_dir .git/modules/sub1 &&
++ test_path_is_dir .git/modules/sub2 &&
+
++ # run migration
+ git submodule--helper migrate-gitdir-configs &&
+
-+ # gitdir configs must exist after migration
++ # test that migration worked
+ git config submodule.sub1.gitdir >actual &&
+ echo ".git/modules/sub1" >expect &&
+ test_cmp expect actual &&
-+
+ git config submodule.sub2.gitdir >actual &&
+ echo ".git/modules/sub2" >expect &&
++ test_cmp expect actual &&
++
++ # repository extension is enabled after migration
++ git config extensions.submodulePathConfig > actual &&
++ echo "true" > expect &&
+ test_cmp expect actual
+ )
+'
++
++test_expect_success '`git clone --recurse-submodules` works after migration' '
++ test_when_finished "rm -rf repo-clone-recursive" &&
++
++ # test with extension disabled after the upstream repo was migrated
++ git clone --recurse-submodules upstream repo-clone-recursive &&
++ (
++ cd repo-clone-recursive &&
++
++ # init.autoSetupSubmodulePathConfig was disabled before clone, so
++ # the repo extension config should also be off, the migration ignored
++ test_must_fail git config extensions.submodulePathConfig &&
++
++ # modules should look like there was no migration done
++ test_must_fail git config submodule.sub1.gitdir &&
++ test_must_fail git config submodule.sub2.gitdir &&
++ test_path_is_dir .git/modules/sub1 &&
++ test_path_is_dir .git/modules/sub2
++ ) &&
++ rm -rf repo-clone-recursive &&
++
++ # enable the extension, then retry the clone
++ git config --global init.autoSetupSubmodulePathConfig true &&
++ git clone --recurse-submodules upstream repo-clone-recursive &&
++ (
++ cd repo-clone-recursive &&
++
++ # repository extension is enabled
++ git config extensions.submodulePathConfig > actual &&
++ echo "true" > expect &&
++ test_cmp expect actual &&
++
++ # gitdir configs exist for submodules
++ git config submodule.sub1.gitdir &&
++ git config submodule.sub2.gitdir &&
++ test_path_is_dir .git/modules/sub1 &&
++ test_path_is_dir .git/modules/sub2
++ )
++'
+
test_done
7: a164370edc = 7: 2e55521bbf builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
8: 94f785817a ! 8: 7fd920aa70 submodule--helper: fix filesystem collisions by encoding gitdir paths
@@ builtin/submodule--helper.c
#define OPT_QUIET (1 << 0)
#define OPT_CACHED (1 << 1)
@@ builtin/submodule--helper.c: static void create_default_gitdir_config(const char *submodule_name)
- !submodule_path_config_enabled)
- return;
+ {
+ struct strbuf gitdir_path = STRBUF_INIT;
+ /* Case 1: try the plain module name */
repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
@@ submodule.c
#include "setup.h"
+#include "url.h"
- int submodule_path_config_enabled;
-
+ static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
+ static int initialized_fetch_ref_tips;
@@ submodule.c: int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
@@ submodule.c: int validate_submodule_git_dir(char *git_dir, const char *submodule
char *p = xstrdup(path);
## t/t7425-submodule-gitdir-path-extension.sh ##
-@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success 'submodule--helper migrates legacy modules' '
+@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success '`git clone --recurse-submodules` works after migration' '
)
'
@@ t/t7425-submodule-gitdir-path-extension.sh: test_expect_success 'submodule--help
+ (
+ cd clone_nested &&
+ # disable extension and verify failure
-+ git config extensions.submodulePathConfig false &&
++ git config --replace-all extensions.submodulePathConfig false &&
+ test_must_fail git submodule add ./thing2 hippo/foobar &&
+ # re-enable extension and verify it works
-+ git config extensions.submodulePathConfig true &&
++ git config --replace-all extensions.submodulePathConfig true &&
+ git submodule add ./thing2 hippo/foobar
+ )
+'
9: 0061979221 = 9: e21656aad4 submodule: fix case-folding gitdir filesystem collisions
10: 6d71434f39 ! 10: 9e3da1c585 submodule: hash the submodule name for the gitdir path
@@ builtin/submodule--helper.c: static int validate_and_set_submodule_gitdir(struct
+ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
+ int header_len;
- /* The config is set only when extensions.submodulePathConfig is enabled */
- if (!the_repository->repository_format_submodule_path_cfg &&
+ /* Case 1: try the plain module name */
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
@@ builtin/submodule--helper.c: static void create_default_gitdir_config(const char *submodule_name)
return;
}
-: ---------- > 11: 56bca88799 submodule: detect conflicts with existing gitdir configs
Adrian Ratiu (11):
submodule--helper: use submodule_name_to_gitdir in add_submodule
submodule: always validate gitdirs inside submodule_name_to_gitdir
builtin/submodule--helper: add gitdir command
submodule: introduce extensions.submodulePathConfig
submodule: allow runtime enabling extensions.submodulePathConfig
submodule--helper: add gitdir migration command
builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
submodule--helper: fix filesystem collisions by encoding gitdir paths
submodule: fix case-folding gitdir filesystem collisions
submodule: hash the submodule name for the gitdir path
submodule: detect conflicts with existing gitdir configs
Documentation/config/extensions.adoc | 29 ++
Documentation/config/init.adoc | 6 +
Documentation/config/submodule.adoc | 7 +
builtin/credential-store.c | 7 +-
builtin/submodule--helper.c | 201 +++++++-
repository.c | 1 +
repository.h | 1 +
setup.c | 19 +-
setup.h | 1 +
submodule.c | 219 +++++++--
t/lib-verify-submodule-gitdir-path.sh | 24 +
t/meson.build | 1 +
t/t7425-submodule-gitdir-path-extension.sh | 508 +++++++++++++++++++++
t/t9902-completion.sh | 1 +
url.c | 13 +
url.h | 14 +
16 files changed, 1000 insertions(+), 52 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-gitdir-path-extension.sh
--
2.51.2
^ permalink raw reply [flat|nested] 179+ messages in thread
* [PATCH v7 01/11] submodule--helper: use submodule_name_to_gitdir in add_submodule
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 02/11] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
` (10 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
While testing submodule gitdir path encoding, I noticed submodule--helper
is still using a hardcoded modules gitdir path leading to test failures.
Call the submodule_name_to_gitdir() helper instead, which was invented
exactly for this purpose and is already used by all the other locations
which work on gitdirs.
Also narrow the scope of the submod_gitdir_path variable which is not
used anymore in the updated "else" branch.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 35f6cf735e..13b5e4ed68 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -3183,13 +3183,13 @@ static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path)
static int add_submodule(const struct add_data *add_data)
{
- char *submod_gitdir_path;
struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT;
struct string_list reference = STRING_LIST_INIT_NODUP;
int ret = -1;
/* perhaps the path already exists and is already a git repo, else clone it */
if (is_directory(add_data->sm_path)) {
+ char *submod_gitdir_path;
struct strbuf sm_path = STRBUF_INIT;
strbuf_addstr(&sm_path, add_data->sm_path);
submod_gitdir_path = xstrfmt("%s/.git", add_data->sm_path);
@@ -3203,10 +3203,11 @@ static int add_submodule(const struct add_data *add_data)
free(submod_gitdir_path);
} else {
struct child_process cp = CHILD_PROCESS_INIT;
+ struct strbuf submod_gitdir = STRBUF_INIT;
- submod_gitdir_path = xstrfmt(".git/modules/%s", add_data->sm_name);
+ submodule_name_to_gitdir(&submod_gitdir, the_repository, add_data->sm_name);
- if (is_directory(submod_gitdir_path)) {
+ if (is_directory(submod_gitdir.buf)) {
if (!add_data->force) {
struct strbuf msg = STRBUF_INIT;
char *die_msg;
@@ -3215,8 +3216,8 @@ static int add_submodule(const struct add_data *add_data)
"locally with remote(s):\n"),
add_data->sm_name);
- append_fetch_remotes(&msg, submod_gitdir_path);
- free(submod_gitdir_path);
+ append_fetch_remotes(&msg, submod_gitdir.buf);
+ strbuf_release(&submod_gitdir);
strbuf_addf(&msg, _("If you want to reuse this local git "
"directory instead of cloning again from\n"
@@ -3234,7 +3235,7 @@ static int add_submodule(const struct add_data *add_data)
"submodule '%s'\n"), add_data->sm_name);
}
}
- free(submod_gitdir_path);
+ strbuf_release(&submod_gitdir);
clone_data.prefix = add_data->prefix;
clone_data.path = add_data->sm_path;
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 02/11] submodule: always validate gitdirs inside submodule_name_to_gitdir
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 01/11] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 03/11] builtin/submodule--helper: add gitdir command Adrian Ratiu
` (9 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Move the ad-hoc validation checks sprinkled across the source tree,
after calling submodule_name_to_gitdir() into the function proper,
which now always validates the gitdir before returning it.
This simplifies the API and helps to:
1. Avoid redundant validation calls after submodule_name_to_gitdir().
2. Avoid the risk of callers forgetting to validate.
3. Ensure gitdir paths provided by users via configs are always valid
(config gitdir paths are added in a subsequent commit).
The validation function can still be called as many times as needed
outside submodule_name_to_gitdir(), for example we keep two calls
which are still required, to avoid parallel clone races by re-running
the validation in builtin/submodule-helper.c.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 4 ----
submodule.c | 12 ++++--------
2 files changed, 4 insertions(+), 12 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 13b5e4ed68..f1fc098614 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1699,10 +1699,6 @@ static int clone_submodule(const struct module_clone_data *clone_data,
clone_data_path = to_free = xstrfmt("%s/%s", repo_get_work_tree(the_repository),
clone_data->path);
- if (validate_submodule_git_dir(sm_gitdir, clone_data->name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
-
if (!file_exists(sm_gitdir)) {
if (clone_data->require_init && !stat(clone_data_path, &st) &&
!is_empty_dir(clone_data_path))
diff --git a/submodule.c b/submodule.c
index 40a5c6fb9d..f645372a18 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2166,11 +2166,6 @@ int submodule_move_head(const char *path, const char *super_prefix,
struct strbuf gitdir = STRBUF_INIT;
submodule_name_to_gitdir(&gitdir, the_repository,
sub->name);
- if (validate_submodule_git_dir(gitdir.buf,
- sub->name) < 0)
- die(_("refusing to create/use '%s' in another "
- "submodule's git dir"),
- gitdir.buf);
connect_work_tree_and_git_dir(path, gitdir.buf, 0);
strbuf_release(&gitdir);
@@ -2349,9 +2344,6 @@ static void relocate_single_git_dir_into_superproject(const char *path,
die(_("could not lookup name for submodule '%s'"), path);
submodule_name_to_gitdir(&new_gitdir, the_repository, sub->name);
- if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
- die(_("refusing to move '%s' into an existing git dir"),
- real_old_git_dir);
if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
die(_("could not create directory '%s'"), new_gitdir.buf);
real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
@@ -2600,4 +2592,8 @@ void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
*/
repo_git_path_append(r, buf, "modules/");
strbuf_addstr(buf, submodule_name);
+
+ if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
+ die(_("refusing to create/use '%s' in another submodule's "
+ "git dir"), buf->buf);
}
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 03/11] builtin/submodule--helper: add gitdir command
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 01/11] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 02/11] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
` (8 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu,
Brandon Williams
This exposes the gitdir name computed by submodule_name_to_gitdir()
internally, to make it easier for users and tests to interact with it.
Next commit will add a gitdir configuration, so this helper can also be
used to easily query that config or validate any gitdir path the user
sets (submodule_name_to_git_dir now runs the validation logic, since
our previous commit).
Based-on-patch-by: Brandon Williams <bwilliams.eng@gmail.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f1fc098614..3bc139ff9c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1204,6 +1204,22 @@ static int module_summary(int argc, const char **argv, const char *prefix,
return ret;
}
+static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
+ struct repository *repo)
+{
+ struct strbuf gitdir = STRBUF_INIT;
+
+ if (argc != 2)
+ usage(_("git submodule--helper gitdir <name>"));
+
+ submodule_name_to_gitdir(&gitdir, repo, argv[1]);
+
+ printf("%s\n", gitdir.buf);
+
+ strbuf_release(&gitdir);
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3583,6 +3599,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
OPT_SUBCOMMAND("update", &fn, module_update),
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (2 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 03/11] builtin/submodule--helper: add gitdir command Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-21 3:27 ` Junio C Hamano
2025-12-20 10:15 ` [PATCH v7 05/11] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
` (7 subsequent siblings)
11 siblings, 1 reply; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
The idea of this extension is to abstract away the submodule gitdir
path implementation: everyone is expected to use the config and not
worry about how the path is computed internally, either in git or
other implementations.
With this extension enabled, the submodule.<name>.gitdir repo config
becomes the single source of truth for all submodule gitdir paths.
The submodule.<name>.gitdir config is added automatically for all new
submodules when this extension is enabled.
Git will throw an error if the extension is enabled and a config is
missing, advising users how to migrate. Migration is manual for now.
E.g. to add a missing config entry for an existing "foo" module:
git config submodule.foo.gitdir .git/modules/foo
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 23 ++++
Documentation/config/submodule.adoc | 7 ++
builtin/submodule--helper.c | 54 +++++++-
repository.c | 1 +
repository.h | 1 +
setup.c | 7 ++
setup.h | 1 +
submodule.c | 60 +++++----
t/lib-verify-submodule-gitdir-path.sh | 24 ++++
t/meson.build | 1 +
t/t7425-submodule-gitdir-path-extension.sh | 138 +++++++++++++++++++++
t/t9902-completion.sh | 1 +
12 files changed, 289 insertions(+), 29 deletions(-)
create mode 100644 t/lib-verify-submodule-gitdir-path.sh
create mode 100755 t/t7425-submodule-gitdir-path-extension.sh
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 532456644b..e15b93f2fb 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -73,6 +73,29 @@ relativeWorktrees:::
repaired with either the `--relative-paths` option or with the
`worktree.useRelativePaths` config set to `true`.
+submodulePathConfig:::
+ This extension is for the minority of users who:
++
+--
+* Encounter errors like `refusing to create ... in another submodule's git dir`
+ due to a number of reasons, like case-insensitive filesystem conflicts when
+ creating modules named `foo` and `Foo`.
+* Require more flexible submodule layouts, for example due to nested names like
+ `foo`, `foo/bar` and `foo/baz` not supported by the default gitdir mechanism
+ which uses `.git/modules/<plain-name>` locations, causing further conflicts.
+--
++
+When `extensions.submodulePathConfig` is enabled, the `submodule.<name>.gitdir`
+config becomes the single source of truth for all submodule gitdir paths and is
+automatically set for all new submodules both during clone and init operations.
++
+Git will error out if a module does not have a corresponding
+`submodule.<name>.gitdir` set.
++
+Existing (pre-extension) submodules need to be migrated by adding the missing
+config entries. This is done manually for now, e.g. for each submodule:
+`git config submodule.<name>.gitdir .git/modules/<name>`.
+
worktreeConfig:::
If enabled, then worktrees will load config settings from the
`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/config/submodule.adoc b/Documentation/config/submodule.adoc
index 0672d99117..9c260a69f6 100644
--- a/Documentation/config/submodule.adoc
+++ b/Documentation/config/submodule.adoc
@@ -52,6 +52,13 @@ submodule.<name>.active::
submodule.active config option. See linkgit:gitsubmodules[7] for
details.
+submodule.<name>.gitdir::
+ This sets the gitdir path for submodule <name>. This configuration is
+ respected when `extensions.submodulePathConfig` is enabled, otherwise it
+ has no effect. When enabled, this config becomes the single source of
+ truth for submodule gitdir paths and git will error if it is missing.
+ See linkgit:git-config[1] for details.
+
submodule.active::
A repeated field which contains a pathspec used to match against a
submodule's path to determine if the submodule is of interest to git
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 3bc139ff9c..f8cae345a5 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -435,6 +435,48 @@ struct init_cb {
};
#define INIT_CB_INIT { 0 }
+static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
+ const char *submodule_name)
+{
+ const char *value;
+ char *key;
+
+ if (validate_submodule_git_dir(gitdir_path->buf, submodule_name))
+ return -1;
+
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+
+ /* Nothing to do if the config already exists. */
+ if (!repo_config_get_string_tmp(the_repository, key, &value)) {
+ free(key);
+ return 0;
+ }
+
+ if (repo_config_set_gently(the_repository, key, gitdir_path->buf)) {
+ free(key);
+ return -1;
+ }
+
+ free(key);
+ return 0;
+}
+
+static void create_default_gitdir_config(const char *submodule_name)
+{
+ struct strbuf gitdir_path = STRBUF_INIT;
+
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
+ die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
+ "Please ensure it is set, for example by running something like: "
+ "'git config submodule.%s.gitdir .git/modules/%s'"),
+ submodule_name, submodule_name, submodule_name);
+}
+
static void init_submodule(const char *path, const char *prefix,
const char *super_prefix,
unsigned int flags)
@@ -511,6 +553,10 @@ static void init_submodule(const char *path, const char *prefix,
if (repo_config_set_gently(the_repository, sb.buf, upd))
die(_("Failed to register update mode for submodule path '%s'"), displaypath);
}
+
+ if (the_repository->repository_format_submodule_path_cfg)
+ create_default_gitdir_config(sub->name);
+
strbuf_release(&sb);
free(displaypath);
free(url);
@@ -1801,8 +1847,9 @@ static int clone_submodule(const struct module_clone_data *clone_data,
char *head = xstrfmt("%s/HEAD", sm_gitdir);
unlink(head);
free(head);
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), sm_gitdir);
+ die(_("refusing to create/use '%s' in another submodule's git dir. "
+ "Enabling extensions.submodulePathConfig should fix this."),
+ sm_gitdir);
}
connect_work_tree_and_git_dir(clone_data_path, sm_gitdir, 0);
@@ -3574,6 +3621,9 @@ static int module_add(int argc, const char **argv, const char *prefix,
add_data.progress = !!progress;
add_data.dissociate = !!dissociate;
+ if (the_repository->repository_format_submodule_path_cfg)
+ create_default_gitdir_config(add_data.sm_name);
+
if (add_submodule(&add_data))
goto cleanup;
configure_added_submodule(&add_data);
diff --git a/repository.c b/repository.c
index 863f24411b..6cb1247f4b 100644
--- a/repository.c
+++ b/repository.c
@@ -281,6 +281,7 @@ int repo_init(struct repository *repo,
repo->repository_format_worktree_config = format.worktree_config;
repo->repository_format_relative_worktrees = format.relative_worktrees;
repo->repository_format_precious_objects = format.precious_objects;
+ repo->repository_format_submodule_path_cfg = format.submodule_path_cfg;
/* take ownership of format.partial_clone */
repo->repository_format_partial_clone = format.partial_clone;
diff --git a/repository.h b/repository.h
index 6063c4b846..7141237f97 100644
--- a/repository.h
+++ b/repository.h
@@ -165,6 +165,7 @@ struct repository {
int repository_format_worktree_config;
int repository_format_relative_worktrees;
int repository_format_precious_objects;
+ int repository_format_submodule_path_cfg;
/* Indicate if a repository has a different 'commondir' from 'gitdir' */
unsigned different_commondir:1;
diff --git a/setup.c b/setup.c
index 3a6a048620..428427d689 100644
--- a/setup.c
+++ b/setup.c
@@ -686,6 +686,9 @@ static enum extension_result handle_extension(const char *var,
} else if (!strcmp(ext, "relativeworktrees")) {
data->relative_worktrees = git_config_bool(var, value);
return EXTENSION_OK;
+ } else if (!strcmp(ext, "submodulepathconfig")) {
+ data->submodule_path_cfg = git_config_bool(var, value);
+ return EXTENSION_OK;
}
return EXTENSION_UNKNOWN;
}
@@ -1947,6 +1950,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
repo_fmt.worktree_config;
the_repository->repository_format_relative_worktrees =
repo_fmt.relative_worktrees;
+ the_repository->repository_format_submodule_path_cfg =
+ repo_fmt.submodule_path_cfg;
/* take ownership of repo_fmt.partial_clone */
the_repository->repository_format_partial_clone =
repo_fmt.partial_clone;
@@ -2045,6 +2050,8 @@ void check_repository_format(struct repository_format *fmt)
fmt->ref_storage_format);
the_repository->repository_format_worktree_config =
fmt->worktree_config;
+ the_repository->repository_format_submodule_path_cfg =
+ fmt->submodule_path_cfg;
the_repository->repository_format_relative_worktrees =
fmt->relative_worktrees;
the_repository->repository_format_partial_clone =
diff --git a/setup.h b/setup.h
index d55dcc6608..0738dec244 100644
--- a/setup.h
+++ b/setup.h
@@ -167,6 +167,7 @@ struct repository_format {
char *partial_clone; /* value of extensions.partialclone */
int worktree_config;
int relative_worktrees;
+ int submodule_path_cfg;
int is_bare;
int hash_algo;
int compat_hash_algo;
diff --git a/submodule.c b/submodule.c
index f645372a18..e3692009cd 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2158,8 +2158,9 @@ int submodule_move_head(const char *path, const char *super_prefix,
if (validate_submodule_git_dir(git_dir,
sub->name) < 0)
die(_("refusing to create/use '%s' in "
- "another submodule's git dir"),
- git_dir);
+ "another submodule's git dir. "
+ "Enabling extensions.submodulePathConfig "
+ "should fix this."), git_dir);
free(git_dir);
}
} else {
@@ -2570,30 +2571,35 @@ int submodule_to_gitdir(struct repository *repo,
void submodule_name_to_gitdir(struct strbuf *buf, struct repository *r,
const char *submodule_name)
{
- /*
- * NEEDSWORK: The current way of mapping a submodule's name to
- * its location in .git/modules/ has problems with some naming
- * schemes. For example, if a submodule is named "foo" and
- * another is named "foo/bar" (whether present in the same
- * superproject commit or not - the problem will arise if both
- * superproject commits have been checked out at any point in
- * time), or if two submodule names only have different cases in
- * a case-insensitive filesystem.
- *
- * There are several solutions, including encoding the path in
- * some way, introducing a submodule.<name>.gitdir config in
- * .git/config (not .gitmodules) that allows overriding what the
- * gitdir of a submodule would be (and teach Git, upon noticing
- * a clash, to automatically determine a non-clashing name and
- * to write such a config), or introducing a
- * submodule.<name>.gitdir config in .gitmodules that repo
- * administrators can explicitly set. Nothing has been decided,
- * so for now, just append the name at the end of the path.
- */
- repo_git_path_append(r, buf, "modules/");
- strbuf_addstr(buf, submodule_name);
+ if (!r->repository_format_submodule_path_cfg) {
+ /*
+ * If extensions.submodulePathConfig is disabled,
+ * continue to use the plain path.
+ */
+ repo_git_path_append(r, buf, "modules/%s", submodule_name);
+ } else {
+ const char *gitdir;
+ char *key;
+ int ret;
+
+ /* Otherwise the extension is enabled, so use the gitdir config. */
+ key = xstrfmt("submodule.%s.gitdir", submodule_name);
+ ret = repo_config_get_string_tmp(r, key, &gitdir);
+ FREE_AND_NULL(key);
+
+ if (ret)
+ die(_("the 'submodule.%s.gitdir' config does not exist for module '%s'. "
+ "Please ensure it is set, for example by running something like: "
+ "'git config submodule.%s.gitdir .git/modules/%s'. For details "
+ "see the extensions.submodulePathConfig documentation."),
+ submodule_name, submodule_name, submodule_name, submodule_name);
+
+ strbuf_addstr(buf, gitdir);
+ }
- if (validate_submodule_git_dir(buf->buf, submodule_name) < 0)
- die(_("refusing to create/use '%s' in another submodule's "
- "git dir"), buf->buf);
+ /* validate because users might have modified the config */
+ if (validate_submodule_git_dir(buf->buf, submodule_name))
+ die(_("invalid 'submodule.%s.gitdir' config: '%s' please check "
+ "if it is unique or conflicts with another module"),
+ submodule_name, buf->buf);
}
diff --git a/t/lib-verify-submodule-gitdir-path.sh b/t/lib-verify-submodule-gitdir-path.sh
new file mode 100644
index 0000000000..62794df976
--- /dev/null
+++ b/t/lib-verify-submodule-gitdir-path.sh
@@ -0,0 +1,24 @@
+# Helper to verify if repo $1 contains a submodule named $2 with gitdir path $3
+
+# This does not check filesystem existence. That is done in submodule.c via the
+# submodule_name_to_gitdir() API which this helper ends up calling. The gitdirs
+# might or might not exist (e.g. when adding a new submodule), so this only
+# checks the expected configuration path, which might be overridden by the user.
+
+verify_submodule_gitdir_path() {
+ repo="$1" &&
+ name="$2" &&
+ path="$3" &&
+ (
+ cd "$repo" &&
+ # Compute expected absolute path
+ expected="$(git rev-parse --git-common-dir)/$path" &&
+ expected="$(test-tool path-utils real_path "$expected")" &&
+ # Compute actual absolute path
+ actual="$(git submodule--helper gitdir "$name")" &&
+ actual="$(test-tool path-utils real_path "$actual")" &&
+ echo "$expected" >expect &&
+ echo "$actual" >actual &&
+ test_cmp expect actual
+ )
+}
diff --git a/t/meson.build b/t/meson.build
index 459c52a489..0b1de3251a 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -887,6 +887,7 @@ integration_tests = [
't7422-submodule-output.sh',
't7423-submodule-symlinks.sh',
't7424-submodule-mixed-ref-formats.sh',
+ 't7425-submodule-gitdir-path-extension.sh',
't7450-bad-git-dotfiles.sh',
't7500-commit-template-squash-signoff.sh',
't7501-commit-basic-functionality.sh',
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
new file mode 100755
index 0000000000..5d52a289f8
--- /dev/null
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -0,0 +1,138 @@
+#!/bin/sh
+
+test_description='submodulePathConfig extension works as expected'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-verify-submodule-gitdir-path.sh
+
+test_expect_success 'setup: allow file protocol' '
+ git config --global protocol.file.allow always
+'
+
+test_expect_success 'create repo with mixed extension submodules' '
+ git init -b main legacy-sub &&
+ test_commit -C legacy-sub legacy-initial &&
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+
+ git init -b main new-sub &&
+ test_commit -C new-sub new-initial &&
+ new_rev=$(git -C new-sub rev-parse HEAD) &&
+
+ git init -b main main &&
+ (
+ cd main &&
+ git submodule add ../legacy-sub legacy &&
+ test_commit legacy-sub &&
+
+ # trigger the "die_path_inside_submodule" check
+ test_must_fail git submodule add ../new-sub "legacy/nested" &&
+
+ git config core.repositoryformatversion 1 &&
+ git config extensions.submodulePathConfig true &&
+
+ git submodule add ../new-sub "New Sub" &&
+ test_commit new &&
+
+ # retrigger the "die_path_inside_submodule" check with encoding
+ test_must_fail git submodule add ../new-sub "New Sub/nested2"
+ )
+'
+
+test_expect_success 'verify new submodule gitdir config' '
+ git -C main config submodule."New Sub".gitdir > actual &&
+ echo ".git/modules/New Sub" > expect &&
+ test_cmp expect actual &&
+ verify_submodule_gitdir_path main "New Sub" "modules/New Sub"
+'
+
+test_expect_success 'manual add and verify legacy submodule gitdir config' '
+ # the legacy module should not contain a gitdir config, because it
+ # was added before the extension was enabled. Add and test it.
+ test_must_fail git -C main config submodule.legacy.gitdir &&
+ git -C main config submodule.legacy.gitdir .git/modules/legacy &&
+ git -C main config submodule.legacy.gitdir > actual &&
+ echo ".git/modules/legacy" > expect &&
+ test_cmp expect actual &&
+ verify_submodule_gitdir_path main "legacy" "modules/legacy"
+'
+
+test_expect_success 'clone from repo with both legacy and new-style submodules' '
+ git clone --recurse-submodules main cloned-non-extension &&
+ (
+ cd cloned-non-extension &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir .git/modules/"New Sub" &&
+
+ test_must_fail git config submodule.legacy.gitdir &&
+ test_must_fail git config submodule."New Sub".gitdir &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ ) &&
+
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules main cloned-extension &&
+ (
+ cd cloned-extension &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ git config submodule.legacy.gitdir &&
+ git config submodule."New Sub".gitdir &&
+
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_expect_success 'commit and push changes to encoded submodules' '
+ git -C legacy-sub config receive.denyCurrentBranch updateInstead &&
+ git -C new-sub config receive.denyCurrentBranch updateInstead &&
+ git -C main config receive.denyCurrentBranch updateInstead &&
+ (
+ cd cloned-extension &&
+
+ git -C legacy switch --track -C main origin/main &&
+ test_commit -C legacy second-commit &&
+ git -C legacy push &&
+
+ git -C "New Sub" switch --track -C main origin/main &&
+ test_commit -C "New Sub" second-commit &&
+ git -C "New Sub" push &&
+
+ # Stage and commit submodule changes in superproject
+ git switch --track -C main origin/main &&
+ git add legacy "New Sub" &&
+ git commit -m "update submodules" &&
+
+ # push superproject commit to main repo
+ git push
+ ) &&
+
+ # update expected legacy & new submodule checksums
+ legacy_rev=$(git -C legacy-sub rev-parse HEAD) &&
+ new_rev=$(git -C new-sub rev-parse HEAD)
+'
+
+test_expect_success 'fetch mixed submodule changes and verify updates' '
+ (
+ cd main &&
+
+ # only update submodules because superproject was
+ # pushed into at the end of last test
+ git submodule update --init --recursive &&
+
+ test_path_is_dir .git/modules/legacy &&
+ test_path_is_dir ".git/modules/New Sub" &&
+
+ # Verify both submodules are at the expected commits
+ git submodule status >list &&
+ test_grep "$legacy_rev legacy" list &&
+ test_grep "$new_rev New Sub" list
+ )
+'
+
+test_done
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 964e1f1569..ffb9c8b522 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -3053,6 +3053,7 @@ test_expect_success 'git config set - variable name - __git_compute_second_level
submodule.sub.fetchRecurseSubmodules Z
submodule.sub.ignore Z
submodule.sub.active Z
+ submodule.sub.gitdir Z
EOF
'
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 05/11] submodule: allow runtime enabling extensions.submodulePathConfig
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (3 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 06/11] submodule--helper: add gitdir migration command Adrian Ratiu
` (6 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Add a new config `init.autoSetupSubmodulePathConfig` which allows
enabling `extensions.submodulePathConfig` for new submodules by
default (those created via git init or clone).
Important: setting init.autoSetupSubmodulePathConfig = true does
not globally enable `extensions.submodulePathConfig`. Existing
repositories will still have the extension disabled and will
require migration (for example via git submodule--helper command
added in the next commit).
Suggested-by: Patrick Steinhardt <ps@pks.im>
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 4 +
Documentation/config/init.adoc | 6 +
setup.c | 12 +-
t/t7425-submodule-gitdir-path-extension.sh | 125 +++++++++++++++++++++
4 files changed, 146 insertions(+), 1 deletion(-)
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index e15b93f2fb..0968ac3d5c 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -95,6 +95,10 @@ Git will error out if a module does not have a corresponding
Existing (pre-extension) submodules need to be migrated by adding the missing
config entries. This is done manually for now, e.g. for each submodule:
`git config submodule.<name>.gitdir .git/modules/<name>`.
++
+The extension can be enabled automatically for new repositories by setting
+`init.autoSetupSubmodulePathConfig` to `true`, for example by running
+`git config --global init.autoSetupSubmodulePathConfig true`.
worktreeConfig:::
If enabled, then worktrees will load config settings from the
diff --git a/Documentation/config/init.adoc b/Documentation/config/init.adoc
index e45b2a8121..293a2ddbdf 100644
--- a/Documentation/config/init.adoc
+++ b/Documentation/config/init.adoc
@@ -18,3 +18,9 @@ endif::[]
See `--ref-format=` in linkgit:git-init[1]. Both the command line
option and the `GIT_DEFAULT_REF_FORMAT` environment variable take
precedence over this config.
+
+init.autoSetupSubmodulePathConfig::
+ A boolean that specifies if `git init` and `git clone` should
+ automatically set `extensions.submodulePathConfig` to `true`. This
+ allows all new repositories to automatically use the submodule path
+ extension. Defaults to `false` when unset.
diff --git a/setup.c b/setup.c
index 428427d689..3e05fe7c58 100644
--- a/setup.c
+++ b/setup.c
@@ -2661,7 +2661,7 @@ int init_db(const char *git_dir, const char *real_git_dir,
const char *initial_branch,
int init_shared_repository, unsigned int flags)
{
- int reinit;
+ int reinit, auto_setup_submodule_path_config = 0;
int exist_ok = flags & INIT_DB_EXIST_OK;
char *original_git_dir = real_pathdup(git_dir, 1);
struct repository_format repo_fmt = REPOSITORY_FORMAT_INIT;
@@ -2712,6 +2712,16 @@ int init_db(const char *git_dir, const char *real_git_dir,
initial_branch, flags & INIT_DB_QUIET);
create_object_directory();
+ repo_config_get_bool(the_repository, "init.autoSetupSubmodulePathConfig",
+ &auto_setup_submodule_path_config);
+ if (auto_setup_submodule_path_config) {
+ int version = 0;
+ repo_config_get_int(the_repository, "core.repositoryformatversion", &version);
+ if (version < 1)
+ repo_config_set(the_repository, "core.repositoryformatversion", "1");
+ repo_config_set(the_repository, "extensions.submodulepathconfig", "true");
+ }
+
if (repo_settings_get_shared_repository(the_repository)) {
char buf[10];
/* We do not spell "group" and such, so that
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 5d52a289f8..06ee1ff86b 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -135,4 +135,129 @@ test_expect_success 'fetch mixed submodule changes and verify updates' '
)
'
+test_expect_success '`git init` respects init.autoSetupSubmodulePathConfig' '
+ git config --global init.autoSetupSubmodulePathConfig true &&
+ git init repo-init &&
+ git -C repo-init config extensions.submodulePathConfig > actual &&
+ echo true > expect &&
+ test_cmp expect actual &&
+ # create a submodule and check gitdir
+ (
+ cd repo-init &&
+ git init -b main sub &&
+ test_commit -C sub sub-initial &&
+ git submodule add ./sub sub &&
+ git config submodule.sub.gitdir > actual &&
+ echo ".git/modules/sub" > expect &&
+ test_cmp expect actual
+ ) &&
+ git config --global --unset init.autoSetupSubmodulePathConfig
+'
+
+test_expect_success '`git init` does not set extension by default' '
+ git init upstream &&
+ test_commit -C upstream initial &&
+ test_must_fail git -C upstream config extensions.submodulePathConfig &&
+ # create a pair of submodules and check gitdir is not created
+ git init -b main sub &&
+ test_commit -C sub sub-initial &&
+ (
+ cd upstream &&
+ git submodule add ../sub sub1 &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_must_fail git config submodule.sub1.gitdir &&
+ git submodule add ../sub sub2 &&
+ test_path_is_dir .git/modules/sub2 &&
+ test_must_fail git config submodule.sub2.gitdir &&
+ git commit -m "Add submodules"
+ )
+'
+
+test_expect_success '`git clone` does not set extension by default' '
+ test_when_finished "rm -rf repo-clone-no-ext" &&
+ git clone upstream repo-clone-no-ext &&
+ (
+ cd repo-clone-no-ext &&
+
+ test_must_fail git config extensions.submodulePathConfig &&
+ test_path_is_missing .git/modules/sub1 &&
+ test_path_is_missing .git/modules/sub2 &&
+
+ # create a submodule and check gitdir is not created
+ git submodule add ../sub sub3 &&
+ test_must_fail git config submodule.sub3.gitdir
+ )
+'
+
+test_expect_success '`git clone --recurse-submodules` does not set extension by default' '
+ test_when_finished "rm -rf repo-clone-no-ext" &&
+ git clone --recurse-submodules upstream repo-clone-no-ext &&
+ (
+ cd repo-clone-no-ext &&
+
+ # verify that that submodules do not have gitdir set
+ test_must_fail git config extensions.submodulePathConfig &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_must_fail git config submodule.sub1.gitdir &&
+ test_path_is_dir .git/modules/sub2 &&
+ test_must_fail git config submodule.sub2.gitdir &&
+
+ # create another submodule and check that gitdir is not created
+ git submodule add ../sub sub3 &&
+ test_path_is_dir .git/modules/sub3 &&
+ test_must_fail git config submodule.sub3.gitdir
+ )
+
+'
+
+test_expect_success '`git clone` respects init.autoSetupSubmodulePathConfig' '
+ test_when_finished "rm -rf repo-clone" &&
+ git config --global init.autoSetupSubmodulePathConfig true &&
+ git clone upstream repo-clone &&
+ (
+ cd repo-clone &&
+
+ # verify new repo extension is inherited from global config
+ git config extensions.submodulePathConfig > actual &&
+ echo true > expect &&
+ test_cmp expect actual &&
+
+ # new submodule has a gitdir config
+ git submodule add ../sub sub &&
+ test_path_is_dir .git/modules/sub &&
+ git config submodule.sub.gitdir > actual &&
+ echo ".git/modules/sub" > expect &&
+ test_cmp expect actual
+ ) &&
+ git config --global --unset init.autoSetupSubmodulePathConfig
+'
+
+test_expect_success '`git clone --recurse-submodules` respects init.autoSetupSubmodulePathConfig' '
+ test_when_finished "rm -rf repo-clone-recursive" &&
+ git config --global init.autoSetupSubmodulePathConfig true &&
+ git clone --recurse-submodules upstream repo-clone-recursive &&
+ (
+ cd repo-clone-recursive &&
+
+ # verify new repo extension is inherited from global config
+ git config extensions.submodulePathConfig > actual &&
+ echo true > expect &&
+ test_cmp expect actual &&
+
+ # previous submodules should exist
+ git config submodule.sub1.gitdir &&
+ git config submodule.sub2.gitdir &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_path_is_dir .git/modules/sub2 &&
+
+ # create another submodule and check that gitdir is created
+ git submodule add ../sub new-sub &&
+ test_path_is_dir .git/modules/new-sub &&
+ git config submodule.new-sub.gitdir > actual &&
+ echo ".git/modules/new-sub" > expect &&
+ test_cmp expect actual
+ ) &&
+ git config --global --unset init.autoSetupSubmodulePathConfig
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 06/11] submodule--helper: add gitdir migration command
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (4 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 05/11] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 07/11] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
` (5 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Manually running
"git config submodule.<name>.gitdir .git/modules/<name>"
for each submodule can be impractical, so add a migration command to
submodule--helper to automatically create configs for all submodules
as required by extensions.submodulePathConfig.
The command calls create_default_gitdir_config() which validates the
gitdir paths before adding the configs.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
Documentation/config/extensions.adoc | 6 +-
builtin/submodule--helper.c | 58 +++++++++++++++++++
t/t7425-submodule-gitdir-path-extension.sh | 67 ++++++++++++++++++++++
3 files changed, 129 insertions(+), 2 deletions(-)
diff --git a/Documentation/config/extensions.adoc b/Documentation/config/extensions.adoc
index 0968ac3d5c..62dc112874 100644
--- a/Documentation/config/extensions.adoc
+++ b/Documentation/config/extensions.adoc
@@ -93,8 +93,10 @@ Git will error out if a module does not have a corresponding
`submodule.<name>.gitdir` set.
+
Existing (pre-extension) submodules need to be migrated by adding the missing
-config entries. This is done manually for now, e.g. for each submodule:
-`git config submodule.<name>.gitdir .git/modules/<name>`.
+config entries. This can be done manually, e.g. for each submodule:
+`git config submodule.<name>.gitdir .git/modules/<name>`, or via the
+`git submodule--helper migrate-gitdir-configs` command which iterates over all
+submodules and attempts to migrate them.
+
The extension can be enabled automatically for new repositories by setting
`init.autoSetupSubmodulePathConfig` to `true`, for example by running
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f8cae345a5..5a6436f18f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1266,6 +1266,63 @@ static int module_gitdir(int argc, const char **argv, const char *prefix UNUSED,
return 0;
}
+static int module_migrate(int argc UNUSED, const char **argv UNUSED,
+ const char *prefix UNUSED, struct repository *repo)
+{
+ struct strbuf module_dir = STRBUF_INIT;
+ DIR *dir;
+ struct dirent *de;
+
+ repo_git_path_append(repo, &module_dir, "modules/");
+
+ dir = opendir(module_dir.buf);
+ if (!dir)
+ die(_("could not open '%s'"), module_dir.buf);
+
+ while ((de = readdir(dir))) {
+ struct strbuf gitdir_path = STRBUF_INIT;
+ char *key;
+ const char *value;
+
+ if (is_dot_or_dotdot(de->d_name))
+ continue;
+
+ strbuf_addf(&gitdir_path, "%s/%s", module_dir.buf, de->d_name);
+ if (!is_git_directory(gitdir_path.buf)) {
+ strbuf_release(&gitdir_path);
+ continue;
+ }
+ strbuf_release(&gitdir_path);
+
+ key = xstrfmt("submodule.%s.gitdir", de->d_name);
+ if (!repo_config_get_string_tmp(repo, key, &value)) {
+ /* Already has a gitdir config, nothing to do. */
+ free(key);
+ continue;
+ }
+ free(key);
+
+ create_default_gitdir_config(de->d_name);
+ }
+
+ closedir(dir);
+ strbuf_release(&module_dir);
+
+ if (repo_config_set_gently(repo, "core.repositoryformatversion", "1"))
+ die(_("could not set core.repositoryformatversion to 1. "
+ "Please enable it for migration to work, for example: "
+ "git config core.repositoryformatversion 1"));
+
+ if (repo_config_set_gently(repo, "extensions.submodulePathConfig", "true"))
+ die(_("could not enable submodulePathConfig extension. It is required "
+ "for migration to work. Please enable it in the root repo: "
+ "git config extensions.submodulePathConfig true"));
+
+ repo->repository_format_submodule_path_cfg = 1;
+
+ return 0;
+}
+
struct sync_cb {
const char *prefix;
const char *super_prefix;
@@ -3649,6 +3706,7 @@ int cmd_submodule__helper(int argc,
NULL
};
struct option options[] = {
+ OPT_SUBCOMMAND("migrate-gitdir-configs", &fn, module_migrate),
OPT_SUBCOMMAND("gitdir", &fn, module_gitdir),
OPT_SUBCOMMAND("clone", &fn, module_clone),
OPT_SUBCOMMAND("add", &fn, module_add),
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 06ee1ff86b..6ca9f13a59 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -260,4 +260,71 @@ test_expect_success '`git clone --recurse-submodules` respects init.autoSetupSub
git config --global --unset init.autoSetupSubmodulePathConfig
'
+test_expect_success 'submodule--helper migrates legacy modules' '
+ (
+ cd upstream &&
+
+ # previous submodules exist and were not migrated yet
+ test_must_fail git config submodule.sub1.gitdir &&
+ test_must_fail git config submodule.sub2.gitdir &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_path_is_dir .git/modules/sub2 &&
+
+ # run migration
+ git submodule--helper migrate-gitdir-configs &&
+
+ # test that migration worked
+ git config submodule.sub1.gitdir >actual &&
+ echo ".git/modules/sub1" >expect &&
+ test_cmp expect actual &&
+ git config submodule.sub2.gitdir >actual &&
+ echo ".git/modules/sub2" >expect &&
+ test_cmp expect actual &&
+
+ # repository extension is enabled after migration
+ git config extensions.submodulePathConfig > actual &&
+ echo "true" > expect &&
+ test_cmp expect actual
+ )
+'
+
+test_expect_success '`git clone --recurse-submodules` works after migration' '
+ test_when_finished "rm -rf repo-clone-recursive" &&
+
+ # test with extension disabled after the upstream repo was migrated
+ git clone --recurse-submodules upstream repo-clone-recursive &&
+ (
+ cd repo-clone-recursive &&
+
+ # init.autoSetupSubmodulePathConfig was disabled before clone, so
+ # the repo extension config should also be off, the migration ignored
+ test_must_fail git config extensions.submodulePathConfig &&
+
+ # modules should look like there was no migration done
+ test_must_fail git config submodule.sub1.gitdir &&
+ test_must_fail git config submodule.sub2.gitdir &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_path_is_dir .git/modules/sub2
+ ) &&
+ rm -rf repo-clone-recursive &&
+
+ # enable the extension, then retry the clone
+ git config --global init.autoSetupSubmodulePathConfig true &&
+ git clone --recurse-submodules upstream repo-clone-recursive &&
+ (
+ cd repo-clone-recursive &&
+
+ # repository extension is enabled
+ git config extensions.submodulePathConfig > actual &&
+ echo "true" > expect &&
+ test_cmp expect actual &&
+
+ # gitdir configs exist for submodules
+ git config submodule.sub1.gitdir &&
+ git config submodule.sub2.gitdir &&
+ test_path_is_dir .git/modules/sub1 &&
+ test_path_is_dir .git/modules/sub2
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 07/11] builtin/credential-store: move is_rfc3986_unreserved to url.[ch]
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (5 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 06/11] submodule--helper: add gitdir migration command Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 08/11] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
` (4 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
is_rfc3986_unreserved() was moved to credential-store.c and was made
static by f89854362c (credential-store: move related functions to
credential-store file, 2023-06-06) under a correct assumption, at the
time, that it was the only place using it.
However now we need it to apply URL-encoding to submodule names when
constructing gitdir paths, to avoid conflicts, so bring it back as a
public function exposed via url.h, instead of the old helper path
(strbuf), which has nothing to do with 3986 encoding/decoding anymore.
This function will be used in subsequent commits which do the encoding.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/credential-store.c | 7 +------
url.c | 6 ++++++
url.h | 7 +++++++
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index b74e06cc93..bc1453c6b2 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -7,6 +7,7 @@
#include "path.h"
#include "string-list.h"
#include "parse-options.h"
+#include "url.h"
#include "write-or-die.h"
static struct lock_file credential_lock;
@@ -76,12 +77,6 @@ static void rewrite_credential_file(const char *fn, struct credential *c,
die_errno("unable to write credential store");
}
-static int is_rfc3986_unreserved(char ch)
-{
- return isalnum(ch) ||
- ch == '-' || ch == '_' || ch == '.' || ch == '~';
-}
-
static int is_rfc3986_reserved_or_unreserved(char ch)
{
if (is_rfc3986_unreserved(ch))
diff --git a/url.c b/url.c
index 282b12495a..adc289229c 100644
--- a/url.c
+++ b/url.c
@@ -3,6 +3,12 @@
#include "strbuf.h"
#include "url.h"
+int is_rfc3986_unreserved(char ch)
+{
+ return isalnum(ch) ||
+ ch == '-' || ch == '_' || ch == '.' || ch == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index 2a27c34277..e644c3c809 100644
--- a/url.h
+++ b/url.h
@@ -21,4 +21,11 @@ char *url_decode_parameter_value(const char **query);
void end_url_with_slash(struct strbuf *buf, const char *url);
void str_end_url_with_slash(const char *url, char **dest);
+/*
+ * The set of unreserved characters as per STD66 (RFC3986) is
+ * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
+ * components without percent-encoding.
+ */
+int is_rfc3986_unreserved(char ch);
+
#endif /* URL_H */
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 08/11] submodule--helper: fix filesystem collisions by encoding gitdir paths
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (6 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 07/11] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 09/11] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
` (3 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Fix nested filesystem collisions by url-encoding gitdir paths stored
in submodule.%s.gitdir, when extensions.submodulePathConfig is enabled.
Credit goes to Junio and Patrick for coming up with this design: the
encoding is only applied when necessary, to newly added submodules.
Existing modules don't need the encoding because git already errors
out when detecting nested gitdirs before this patch.
This commit adds the basic url-encoding and some tests. Next commits
extend the encode -> validate -> retry loop to fix more conflicts.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 12 +++++
submodule.c | 42 +++++++++++++++-
t/t7425-submodule-gitdir-path-extension.sh | 57 ++++++++++++++++++++++
3 files changed, 110 insertions(+), 1 deletion(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 5a6436f18f..81ba95d11c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -34,6 +34,7 @@
#include "list-objects-filter-options.h"
#include "wildmatch.h"
#include "strbuf.h"
+#include "url.h"
#define OPT_QUIET (1 << 0)
#define OPT_CACHED (1 << 1)
@@ -465,12 +466,23 @@ static void create_default_gitdir_config(const char *submodule_name)
{
struct strbuf gitdir_path = STRBUF_INIT;
+ /* Case 1: try the plain module name */
repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
strbuf_release(&gitdir_path);
return;
}
+ /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
+ /* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
"'git config submodule.%s.gitdir .git/modules/%s'"),
diff --git a/submodule.c b/submodule.c
index e3692009cd..cf32872ec7 100644
--- a/submodule.c
+++ b/submodule.c
@@ -31,6 +31,7 @@
#include "commit-reach.h"
#include "read-cache-ll.h"
#include "setup.h"
+#include "url.h"
static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
static int initialized_fetch_ref_tips;
@@ -2246,12 +2247,43 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
-int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+/*
+ * Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
+ * This does not print errors like the non-encoded version, because encoding is supposed
+ * to mitigate / fix all these.
+ */
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+{
+ const char *modules_marker = "/modules/";
+ char *p = git_dir, *last_submodule_name = NULL;
+
+ if (!the_repository->repository_format_submodule_path_cfg)
+ BUG("validate_submodule_encoded_git_dir() must be called with "
+ "extensions.submodulePathConfig enabled.");
+
+ /* Find the last submodule name in the gitdir path (modules can be nested). */
+ while ((p = strstr(p, modules_marker))) {
+ last_submodule_name = p + strlen(modules_marker);
+ p++;
+ }
+
+ /* Prevent the use of '/' in encoded names */
+ if (!last_submodule_name || strchr(last_submodule_name, '/'))
+ return -1;
+
+ return 0;
+}
+
+static int validate_submodule_legacy_git_dir(char *git_dir, const char *submodule_name)
{
size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
char *p;
int ret = 0;
+ if (the_repository->repository_format_submodule_path_cfg)
+ BUG("validate_submodule_git_dir() must be called with "
+ "extensions.submodulePathConfig disabled.");
+
if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
strcmp(p, submodule_name))
BUG("submodule name '%s' not a suffix of git dir '%s'",
@@ -2287,6 +2319,14 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
return 0;
}
+int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+{
+ if (!the_repository->repository_format_submodule_path_cfg)
+ return validate_submodule_legacy_git_dir(git_dir, submodule_name);
+
+ return validate_submodule_encoded_git_dir(git_dir, submodule_name);
+}
+
int validate_submodule_path(const char *path)
{
char *p = xstrdup(path);
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 6ca9f13a59..dbe18f2925 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -327,4 +327,61 @@ test_expect_success '`git clone --recurse-submodules` works after migration' '
)
'
+test_expect_success 'setup submodules with nested git dirs' '
+ git init nested &&
+ test_commit -C nested nested &&
+ (
+ cd nested &&
+ cat >.gitmodules <<-EOF &&
+ [submodule "hippo"]
+ url = .
+ path = thing1
+ [submodule "hippo/hooks"]
+ url = .
+ path = thing2
+ EOF
+ git clone . thing1 &&
+ git clone . thing2 &&
+ git add .gitmodules thing1 thing2 &&
+ test_tick &&
+ git commit -m nested
+ )
+'
+
+test_expect_success 'git dirs of encoded sibling submodules must not be nested' '
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules nested clone_nested &&
+
+ verify_submodule_gitdir_path clone_nested hippo modules/hippo &&
+ git -C clone_nested config submodule.hippo.gitdir > actual &&
+ test_grep "\.git/modules/hippo$" actual &&
+
+ verify_submodule_gitdir_path clone_nested hippo/hooks modules/hippo%2fhooks &&
+ git -C clone_nested config submodule.hippo/hooks.gitdir > actual &&
+ test_grep "\.git/modules/hippo%2fhooks$" actual
+'
+
+test_expect_success 'submodule git dir nesting detection must work with parallel cloning' '
+ git clone -c extensions.submodulePathConfig=true --recurse-submodules --jobs=2 nested clone_parallel &&
+
+ verify_submodule_gitdir_path clone_parallel hippo modules/hippo &&
+ git -C clone_nested config submodule.hippo.gitdir > actual &&
+ test_grep "\.git/modules/hippo$" actual &&
+
+ verify_submodule_gitdir_path clone_parallel hippo/hooks modules/hippo%2fhooks &&
+ git -C clone_nested config submodule.hippo/hooks.gitdir > actual &&
+ test_grep "\.git/modules/hippo%2fhooks$" actual
+'
+
+test_expect_success 'disabling extensions.submodulePathConfig prevents nested submodules' '
+ (
+ cd clone_nested &&
+ # disable extension and verify failure
+ git config --replace-all extensions.submodulePathConfig false &&
+ test_must_fail git submodule add ./thing2 hippo/foobar &&
+ # re-enable extension and verify it works
+ git config --replace-all extensions.submodulePathConfig true &&
+ git submodule add ./thing2 hippo/foobar
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 09/11] submodule: fix case-folding gitdir filesystem collisions
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (7 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 08/11] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 10/11] submodule: hash the submodule name for the gitdir path Adrian Ratiu
` (2 subsequent siblings)
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Add a new check when extension.submodulePathConfig is enabled, to
detect and prevent case-folding filesystem colisions. When this
new check is triggered, a stricter casefolding aware URI encoding
is used to percent-encode uppercase characters.
By using this check/retry mechanism the uppercase encoding is
only applied when necessary, so case-sensitive filesystems are
not affected.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 26 ++++++++++-
submodule.c | 53 +++++++++++++++++++++-
t/t7425-submodule-gitdir-path-extension.sh | 35 ++++++++++++++
url.c | 7 +++
url.h | 7 +++
5 files changed, 126 insertions(+), 2 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 81ba95d11c..d601306882 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -473,7 +473,7 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
- /* Case 2: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
+ /* Case 2.1: Try URI-safe (RFC3986) encoding first, this fixes nested gitdirs */
strbuf_reset(&gitdir_path);
repo_git_path_append(the_repository, &gitdir_path, "modules/");
strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
@@ -482,6 +482,30 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
+ /* Case 2.2: Try extended uppercase URI (RFC3986) encoding, to fix case-folding */
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+
+ /* Case 2.3: Try some derived gitdir names, see if one sticks */
+ for (char c = '0'; c <= '9'; c++) {
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_rfc3986_unreserved);
+ strbuf_addch(&gitdir_path, c);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/");
+ strbuf_addstr_urlencode(&gitdir_path, submodule_name, is_casefolding_rfc3986_unreserved);
+ strbuf_addch(&gitdir_path, c);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name))
+ return;
+ }
+
/* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
diff --git a/submodule.c b/submodule.c
index cf32872ec7..834c794b7d 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2247,15 +2247,58 @@ int submodule_move_head(const char *path, const char *super_prefix,
return ret;
}
+static int check_casefolding_conflict(const char *git_dir,
+ const char *submodule_name,
+ const bool suffixes_match)
+{
+ char *p, *modules_dir = xstrdup(git_dir);
+ struct dirent *de;
+ DIR *dir = NULL;
+ int ret = 0;
+
+ if ((p = find_last_dir_sep(modules_dir)))
+ *p = '\0';
+
+ /* No conflict is possible if modules_dir doesn't exist (first clone) */
+ if (!is_directory(modules_dir))
+ goto cleanup;
+
+ dir = opendir(modules_dir);
+ if (!dir) {
+ ret = -1;
+ goto cleanup;
+ }
+
+ /* Check for another directory under .git/modules that differs only in case. */
+ while ((de = readdir(dir))) {
+ if (!strcmp(de->d_name, ".") || !strcmp(de->d_name, ".."))
+ continue;
+
+ if ((suffixes_match || is_git_directory(git_dir)) &&
+ !strcasecmp(de->d_name, submodule_name) &&
+ strcmp(de->d_name, submodule_name)) {
+ ret = -1; /* collision found */
+ break;
+ }
+ }
+
+cleanup:
+ if (dir)
+ closedir(dir);
+ free(modules_dir);
+ return ret;
+}
+
/*
* Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
* This does not print errors like the non-encoded version, because encoding is supposed
* to mitigate / fix all these.
*/
-static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name UNUSED)
+static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodule_name)
{
const char *modules_marker = "/modules/";
char *p = git_dir, *last_submodule_name = NULL;
+ int config_ignorecase = 0;
if (!the_repository->repository_format_submodule_path_cfg)
BUG("validate_submodule_encoded_git_dir() must be called with "
@@ -2271,6 +2314,14 @@ static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodu
if (!last_submodule_name || strchr(last_submodule_name, '/'))
return -1;
+ /* Prevent conflicts on case-folding filesystems */
+ repo_config_get_bool(the_repository, "core.ignorecase", &config_ignorecase);
+ if (ignore_case || config_ignorecase) {
+ bool suffixes_match = !strcmp(last_submodule_name, submodule_name);
+ return check_casefolding_conflict(git_dir, submodule_name,
+ suffixes_match);
+ }
+
return 0;
}
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index dbe18f2925..eb9c80787c 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -384,4 +384,39 @@ test_expect_success 'disabling extensions.submodulePathConfig prevents nested su
)
'
+test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are correctly encoded' '
+ git clone -c extensions.submodulePathConfig=true main cloned-folding &&
+ (
+ cd cloned-folding &&
+
+ # conflict: the "folding" gitdir will already be taken
+ git submodule add ../new-sub "folding" &&
+ test_commit lowercase &&
+ git submodule add ../new-sub "FoldinG" &&
+ test_commit uppercase &&
+
+ # conflict: the "foo" gitdir will already be taken
+ git submodule add ../new-sub "FOO" &&
+ test_commit uppercase-foo &&
+ git submodule add ../new-sub "foo" &&
+ test_commit lowercase-foo &&
+
+ # create a multi conflict between foobar, fooBar and foo%42ar
+ # the "foo" gitdir will already be taken
+ git submodule add ../new-sub "foobar" &&
+ test_commit lowercase-foobar &&
+ git submodule add ../new-sub "foo%42ar" &&
+ test_commit encoded-foo%42ar &&
+ git submodule add ../new-sub "fooBar" &&
+ test_commit mixed-fooBar
+ ) &&
+ verify_submodule_gitdir_path cloned-folding "folding" "modules/folding" &&
+ verify_submodule_gitdir_path cloned-folding "FoldinG" "modules/%46oldin%47" &&
+ verify_submodule_gitdir_path cloned-folding "FOO" "modules/FOO" &&
+ verify_submodule_gitdir_path cloned-folding "foo" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-folding "foobar" "modules/foobar" &&
+ verify_submodule_gitdir_path cloned-folding "foo%42ar" "modules/foo%42ar" &&
+ verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
+'
+
test_done
diff --git a/url.c b/url.c
index adc289229c..3ca5987e90 100644
--- a/url.c
+++ b/url.c
@@ -9,6 +9,13 @@ int is_rfc3986_unreserved(char ch)
ch == '-' || ch == '_' || ch == '.' || ch == '~';
}
+int is_casefolding_rfc3986_unreserved(char c)
+{
+ return (c >= 'a' && c <= 'z') ||
+ (c >= '0' && c <= '9') ||
+ c == '-' || c == '.' || c == '_' || c == '~';
+}
+
int is_urlschemechar(int first_flag, int ch)
{
/*
diff --git a/url.h b/url.h
index e644c3c809..cd9140e994 100644
--- a/url.h
+++ b/url.h
@@ -28,4 +28,11 @@ void str_end_url_with_slash(const char *url, char **dest);
*/
int is_rfc3986_unreserved(char ch);
+/*
+ * This is a variant of is_rfc3986_unreserved() that treats uppercase
+ * letters as "reserved". This forces them to be percent-encoded, allowing
+ * 'Foo' (%46oo) and 'foo' (foo) to be distinct on case-folding filesystems.
+ */
+int is_casefolding_rfc3986_unreserved(char c);
+
#endif /* URL_H */
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 10/11] submodule: hash the submodule name for the gitdir path
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (8 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 09/11] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 11/11] submodule: detect conflicts with existing gitdir configs Adrian Ratiu
2025-12-21 2:39 ` [PATCH v7 00/11] Add submodulePathConfig extension and gitdir encoding Junio C Hamano
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
If none of the previous plain-text / encoding / derivation steps work
and case 2.4 is reached, then try a hash of the submodule name to see
if that can be a valid gitdir before giving up and throwing an error.
This is a "last resort" type of measure to avoid conflicts since it
loses the human readability of the gitdir path. This logic will be
reached in rare cases, as can be seen in the test we added.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
builtin/submodule--helper.c | 19 +++++++
t/t7425-submodule-gitdir-path-extension.sh | 59 ++++++++++++++++++++++
2 files changed, 78 insertions(+)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d601306882..0c5563dc3f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -465,6 +465,10 @@ static int validate_and_set_submodule_gitdir(struct strbuf *gitdir_path,
static void create_default_gitdir_config(const char *submodule_name)
{
struct strbuf gitdir_path = STRBUF_INIT;
+ struct git_hash_ctx ctx;
+ char hex_name_hash[GIT_MAX_HEXSZ + 1], header[128];
+ unsigned char raw_name_hash[GIT_MAX_RAWSZ];
+ int header_len;
/* Case 1: try the plain module name */
repo_git_path_append(the_repository, &gitdir_path, "modules/%s", submodule_name);
@@ -506,6 +510,21 @@ static void create_default_gitdir_config(const char *submodule_name)
return;
}
+ /* Case 2.4: If all the above failed, try a hash of the name as a last resort */
+ header_len = snprintf(header, sizeof(header), "blob %zu", strlen(submodule_name));
+ the_hash_algo->init_fn(&ctx);
+ the_hash_algo->update_fn(&ctx, header, header_len);
+ the_hash_algo->update_fn(&ctx, "\0", 1);
+ the_hash_algo->update_fn(&ctx, submodule_name, strlen(submodule_name));
+ the_hash_algo->final_fn(raw_name_hash, &ctx);
+ hash_to_hex_algop_r(hex_name_hash, raw_name_hash, the_hash_algo);
+ strbuf_reset(&gitdir_path);
+ repo_git_path_append(the_repository, &gitdir_path, "modules/%s", hex_name_hash);
+ if (!validate_and_set_submodule_gitdir(&gitdir_path, submodule_name)) {
+ strbuf_release(&gitdir_path);
+ return;
+ }
+
/* Case 3: nothing worked, error out */
die(_("failed to set a valid default config for 'submodule.%s.gitdir'. "
"Please ensure it is set, for example by running something like: "
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index eb9c80787c..5fcfc29363 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -419,4 +419,63 @@ test_expect_success CASE_INSENSITIVE_FS 'verify case-folding conflicts are corre
verify_submodule_gitdir_path cloned-folding "fooBar" "modules/fooBar0"
'
+test_expect_success CASE_INSENSITIVE_FS 'verify hashing conflict resolution as a last resort' '
+ git clone -c extensions.submodulePathConfig=true main cloned-hash &&
+ (
+ cd cloned-hash &&
+
+ # conflict: add all submodule conflicting variants until we reach the
+ # final hashing conflict resolution for submodule "foo"
+ git submodule add ../new-sub "foo" &&
+ git submodule add ../new-sub "foo0" &&
+ git submodule add ../new-sub "foo1" &&
+ git submodule add ../new-sub "foo2" &&
+ git submodule add ../new-sub "foo3" &&
+ git submodule add ../new-sub "foo4" &&
+ git submodule add ../new-sub "foo5" &&
+ git submodule add ../new-sub "foo6" &&
+ git submodule add ../new-sub "foo7" &&
+ git submodule add ../new-sub "foo8" &&
+ git submodule add ../new-sub "foo9" &&
+ git submodule add ../new-sub "%46oo" &&
+ git submodule add ../new-sub "%46oo0" &&
+ git submodule add ../new-sub "%46oo1" &&
+ git submodule add ../new-sub "%46oo2" &&
+ git submodule add ../new-sub "%46oo3" &&
+ git submodule add ../new-sub "%46oo4" &&
+ git submodule add ../new-sub "%46oo5" &&
+ git submodule add ../new-sub "%46oo6" &&
+ git submodule add ../new-sub "%46oo7" &&
+ git submodule add ../new-sub "%46oo8" &&
+ git submodule add ../new-sub "%46oo9" &&
+ test_commit add-foo-variants &&
+ git submodule add ../new-sub "Foo" &&
+ test_commit add-uppercase-foo
+ ) &&
+ verify_submodule_gitdir_path cloned-hash "foo" "modules/foo" &&
+ verify_submodule_gitdir_path cloned-hash "foo0" "modules/foo0" &&
+ verify_submodule_gitdir_path cloned-hash "foo1" "modules/foo1" &&
+ verify_submodule_gitdir_path cloned-hash "foo2" "modules/foo2" &&
+ verify_submodule_gitdir_path cloned-hash "foo3" "modules/foo3" &&
+ verify_submodule_gitdir_path cloned-hash "foo4" "modules/foo4" &&
+ verify_submodule_gitdir_path cloned-hash "foo5" "modules/foo5" &&
+ verify_submodule_gitdir_path cloned-hash "foo6" "modules/foo6" &&
+ verify_submodule_gitdir_path cloned-hash "foo7" "modules/foo7" &&
+ verify_submodule_gitdir_path cloned-hash "foo8" "modules/foo8" &&
+ verify_submodule_gitdir_path cloned-hash "foo9" "modules/foo9" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo" "modules/%46oo" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo0" "modules/%46oo0" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo1" "modules/%46oo1" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo2" "modules/%46oo2" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo3" "modules/%46oo3" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo4" "modules/%46oo4" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo5" "modules/%46oo5" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo6" "modules/%46oo6" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo7" "modules/%46oo7" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo8" "modules/%46oo8" &&
+ verify_submodule_gitdir_path cloned-hash "%46oo9" "modules/%46oo9" &&
+ hash=$(printf "Foo" | git hash-object --stdin) &&
+ verify_submodule_gitdir_path cloned-hash "Foo" "modules/${hash}"
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* [PATCH v7 11/11] submodule: detect conflicts with existing gitdir configs
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (9 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 10/11] submodule: hash the submodule name for the gitdir path Adrian Ratiu
@ 2025-12-20 10:15 ` Adrian Ratiu
2025-12-21 2:39 ` [PATCH v7 00/11] Add submodulePathConfig extension and gitdir encoding Junio C Hamano
11 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-20 10:15 UTC (permalink / raw)
To: git
Cc: Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Junio C Hamano, Aaron Schrab, Jonathan Nieder, Patrick Steinhardt,
Josh Steadmon, Ben Knoble, Phillip Wood, Adrian Ratiu
Credit goes to Emily and Josh for testing and noticing a corner-case
which caused conflicts with existing gitdir configs to silently pass
validation, then fail later in add_submodule() with a cryptic error:
fatal: A git directory for 'nested%2fsub' is found locally with remote(s):
origin /.../trash directory.t7425-submodule-gitdir-path-extension/sub
This change ensures the validation step checks existing gitdirs for
conflicts. We only have to do this for submodules having gitdirs,
because those without submodule.%s.gitdir need to be migrated and
will throw an error earlier in the submodule codepath.
Quoting Josh:
My testing setup has been as follows:
* Using our locally-built Git with our downstream patch of [1] included:
* create a repo "sub"
* create a repo "super"
* In "super":
* mkdir nested
* git submodule add ../sub nested/sub
* Verify that the submodule's gitdir is .git/modules/nested%2fsub
* Using a build of git from upstream `next` plus this series:
* git config set --global extensions.submodulepathconfig true
* git clone --recurse-submodules super super2
* create a repo "nested%2fsub"
* In "super2":
* git submodule add ../nested%2fsub
At this point I'd expect the collision detection / encoding to take
effect, but instead I get the error listed above.
End quote
Suggested-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
---
submodule.c | 60 ++++++++++++++++++++++
t/t7425-submodule-gitdir-path-extension.sh | 27 ++++++++++
2 files changed, 87 insertions(+)
diff --git a/submodule.c b/submodule.c
index 834c794b7d..d778e8eca7 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2289,6 +2289,61 @@ static int check_casefolding_conflict(const char *git_dir,
return ret;
}
+struct submodule_from_gitdir_cb {
+ const char *gitdir;
+ const char *submodule_name;
+ bool conflict_found;
+};
+
+static int find_conflict_by_gitdir_cb(const char *var, const char *value,
+ const struct config_context *ctx UNUSED, void *data)
+{
+ struct submodule_from_gitdir_cb *cb = data;
+ const char *submodule_name_start;
+ size_t submodule_name_len;
+ const char *suffix = ".gitdir";
+ size_t suffix_len = strlen(suffix);
+
+ if (!skip_prefix(var, "submodule.", &submodule_name_start))
+ return 0;
+
+ /* Check if submodule_name_start ends with ".gitdir" */
+ submodule_name_len = strlen(submodule_name_start);
+ if (submodule_name_len < suffix_len ||
+ strcmp(submodule_name_start + submodule_name_len - suffix_len, suffix) != 0)
+ return 0; /* Does not end with ".gitdir" */
+
+ submodule_name_len -= suffix_len;
+
+ /*
+ * A conflict happens if:
+ * 1. The submodule names are different and
+ * 2. The gitdir paths resolve to the same absolute path
+ */
+ if (value && strncmp(cb->submodule_name, submodule_name_start, submodule_name_len)) {
+ char *abs_path_cb = absolute_pathdup(cb->gitdir);
+ char *abs_path_value = absolute_pathdup(value);
+
+ cb->conflict_found = !strcmp(abs_path_cb, abs_path_value);
+
+ free(abs_path_cb);
+ free(abs_path_value);
+ }
+
+ return cb->conflict_found;
+}
+static bool submodule_conflicts_with_existing(const char *gitdir, const char *submodule_name)
+{
+ struct submodule_from_gitdir_cb cb = { 0 };
+ cb.submodule_name = submodule_name;
+ cb.gitdir = gitdir;
+
+ /* Find conflicts with existing repo gitdir configs */
+ repo_config(the_repository, find_conflict_by_gitdir_cb, &cb);
+
+ return cb.conflict_found;
+}
+
/*
* Encoded gitdir validation, only used when extensions.submodulePathConfig is enabled.
* This does not print errors like the non-encoded version, because encoding is supposed
@@ -2314,6 +2369,11 @@ static int validate_submodule_encoded_git_dir(char *git_dir, const char *submodu
if (!last_submodule_name || strchr(last_submodule_name, '/'))
return -1;
+ /* Prevent conflicts with existing submodule gitdirs */
+ if (is_git_directory(git_dir) &&
+ submodule_conflicts_with_existing(git_dir, submodule_name))
+ return -1;
+
/* Prevent conflicts on case-folding filesystems */
repo_config_get_bool(the_repository, "core.ignorecase", &config_ignorecase);
if (ignore_case || config_ignorecase) {
diff --git a/t/t7425-submodule-gitdir-path-extension.sh b/t/t7425-submodule-gitdir-path-extension.sh
index 5fcfc29363..5f221c507a 100755
--- a/t/t7425-submodule-gitdir-path-extension.sh
+++ b/t/t7425-submodule-gitdir-path-extension.sh
@@ -478,4 +478,31 @@ test_expect_success CASE_INSENSITIVE_FS 'verify hashing conflict resolution as a
verify_submodule_gitdir_path cloned-hash "Foo" "modules/${hash}"
'
+test_expect_success 'submodule gitdir conflicts with previously encoded name (local config)' '
+ git init -b main super_with_encoded &&
+ (
+ cd super_with_encoded &&
+
+ git config extensions.submodulePathConfig true &&
+
+ # Add a submodule with a nested path
+ git submodule add --name "nested/sub" ../sub nested/sub &&
+ test_commit add-encoded-gitdir &&
+
+ verify_submodule_gitdir_path . "nested/sub" "modules/nested%2fsub" &&
+ test_path_is_dir ".git/modules/nested%2fsub"
+ ) &&
+
+ # create a submodule that will conflict with the encoded gitdir name:
+ # the existing gitdir is ".git/modules/nested%2fsub", which is used
+ # by "nested/sub", so the new submod will get another (non-conflicting)
+ # name: "nested%252fsub".
+ (
+ cd super_with_encoded &&
+ git submodule add ../sub "nested%2fsub" &&
+ verify_submodule_gitdir_path . "nested%2fsub" "modules/nested%252fsub" &&
+ test_path_is_dir ".git/modules/nested%252fsub"
+ )
+'
+
test_done
--
2.51.2
^ permalink raw reply related [flat|nested] 179+ messages in thread
* Re: [PATCH v7 00/11] Add submodulePathConfig extension and gitdir encoding
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
` (10 preceding siblings ...)
2025-12-20 10:15 ` [PATCH v7 11/11] submodule: detect conflicts with existing gitdir configs Adrian Ratiu
@ 2025-12-21 2:39 ` Junio C Hamano
11 siblings, 0 replies; 179+ messages in thread
From: Junio C Hamano @ 2025-12-21 2:39 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
> As always, this is based on the latest master branch, I've checkd
> for conflicts with next/seen, pushed to Github [1] and succesfully
> ran the CI [2].
OK. I've queued it on a bit older tip of master (namely, where all
the previous iterations have bene queued on top---this will make it
much easier to compare the iterative changes) and they seem to apply
cleanly.
Will replace and queue, but due to family reasons, I may not
immediately have time to look at this or any other topics much of
this week.
Thanks, and happy holidays.
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig
2025-12-20 10:15 ` [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
@ 2025-12-21 3:27 ` Junio C Hamano
2025-12-23 13:35 ` Adrian Ratiu
0 siblings, 1 reply; 179+ messages in thread
From: Junio C Hamano @ 2025-12-21 3:27 UTC (permalink / raw)
To: Adrian Ratiu
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
Adrian Ratiu <adrian.ratiu@collabora.com> writes:
This comment is not about the primary contents of this series, but
I notice that the whitespace immediately after "errors like" is not
a SP but a HT here.
> +* Encounter errors like `refusing to create ... in another submodule's git dir`
123456701234567012345670
I have already alluded to it as a #leftoverbit in a different topic,
but we probably want to have a new whitespace error class to detect
a HT in the middle of a sentence that should have been a SP.
Perhaps the rule would be something like a HT that is at the column
that is at (tab-width - 1) modulo tab-width (default 8, but the
usual attribute applies), that is surrounded by non-whitespace
characters on both sides.
I may be counting off-by-one, though ;-) The quoted problematic line
has, labeling the leftmost column as 1, the HT at the 24th column.
cf. https://lore.kernel.org/git/xmqq5xa76z0o.fsf@gitster.g/
^ permalink raw reply [flat|nested] 179+ messages in thread
* Re: [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig
2025-12-21 3:27 ` Junio C Hamano
@ 2025-12-23 13:35 ` Adrian Ratiu
0 siblings, 0 replies; 179+ messages in thread
From: Adrian Ratiu @ 2025-12-23 13:35 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Emily Shaffer, Rodrigo Damazio Bovendorp, Jeff King,
Aaron Schrab, Jonathan Nieder, Patrick Steinhardt, Josh Steadmon,
Ben Knoble, Phillip Wood
On Sun, 21 Dec 2025, Junio C Hamano <gitster@pobox.com> wrote:
> Adrian Ratiu <adrian.ratiu@collabora.com> writes:
>
> This comment is not about the primary contents of this series, but
> I notice that the whitespace immediately after "errors like" is not
> a SP but a HT here.
>
>> +* Encounter errors like `refusing to create ... in another submodule's git dir`
> 123456701234567012345670
>
> I have already alluded to it as a #leftoverbit in a different topic,
> but we probably want to have a new whitespace error class to detect
> a HT in the middle of a sentence that should have been a SP.
>
> Perhaps the rule would be something like a HT that is at the column
> that is at (tab-width - 1) modulo tab-width (default 8, but the
> usual attribute applies), that is surrounded by non-whitespace
> characters on both sides.
>
> I may be counting off-by-one, though ;-) The quoted problematic line
> has, labeling the leftmost column as 1, the HT at the 24th column.
>
> cf. https://lore.kernel.org/git/xmqq5xa76z0o.fsf@gitster.g/
I think the algorithm you pointed out is sound, seems to work, so I sent
a separate patch for it:
https://public-inbox.org/git/20251223132756.604036-1-adrian.ratiu@collabora.com/T/#u
Will fix the whitespace in this series on the next re-roll.
Thanks!
^ permalink raw reply [flat|nested] 179+ messages in thread
end of thread, other threads:[~2025-12-23 13:35 UTC | newest]
Thread overview: 179+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-16 21:36 [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-08-16 21:36 ` [PATCH 1/9] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-08-20 19:04 ` Josh Steadmon
2025-08-21 11:26 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 2/9] submodule: create new gitdirs under submodules path Adrian Ratiu
2025-09-08 14:24 ` Phillip Wood
2025-09-08 15:46 ` Adrian Ratiu
2025-09-09 8:53 ` Phillip Wood
2025-09-09 10:57 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 3/9] submodule: add gitdir path config override Adrian Ratiu
2025-08-20 19:37 ` Josh Steadmon
2025-08-21 12:18 ` Adrian Ratiu
2025-08-20 21:38 ` Josh Steadmon
2025-08-21 13:04 ` Adrian Ratiu
2025-08-20 21:50 ` Josh Steadmon
2025-08-21 13:05 ` Adrian Ratiu
2025-09-08 14:23 ` Phillip Wood
2025-09-09 12:02 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 4/9] t: submodules: add basic mixed gitdir path tests Adrian Ratiu
2025-08-20 22:07 ` Josh Steadmon
2025-09-02 23:02 ` Junio C Hamano
2025-08-16 21:36 ` [PATCH 5/9] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
2025-08-16 21:56 ` Ben Knoble
2025-08-21 13:08 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 6/9] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
2025-08-20 19:29 ` Jeff King
2025-08-21 13:14 ` Adrian Ratiu
2025-08-16 21:36 ` [PATCH 7/9] submodule: remove validate_submodule_git_dir() Adrian Ratiu
2025-09-08 14:23 ` Phillip Wood
2025-08-16 21:36 ` [PATCH 8/9] t: move nested gitdir tests to proper location Adrian Ratiu
2025-08-16 21:36 ` [PATCH 9/9] t: add gitdir encoding tests Adrian Ratiu
2025-08-18 22:06 ` Junio C Hamano
2025-08-21 13:17 ` Adrian Ratiu
2025-08-17 13:01 ` [PATCH 0/9] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 00/10] " Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-09-30 13:37 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 02/10] submodule: create new gitdirs under submodules path Adrian Ratiu
2025-09-09 7:40 ` Patrick Steinhardt
2025-09-09 16:17 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 03/10] submodule: add gitdir path config override Adrian Ratiu
2025-09-09 7:40 ` Patrick Steinhardt
2025-09-09 17:46 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 04/10] t7425: add basic mixed submodule gitdir path tests Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 05/10] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 06/10] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
2025-09-10 18:15 ` SZEDER Gábor
2025-09-10 19:30 ` Adrian Ratiu
2025-09-10 20:18 ` Kristoffer Haugsbakk
2025-09-30 13:36 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 07/10] submodule: error out if gitdir name is too long Adrian Ratiu
2025-09-08 15:51 ` Jeff King
2025-09-08 17:15 ` Adrian Ratiu
2025-09-30 13:35 ` Kristoffer Haugsbakk
2025-09-08 14:01 ` [PATCH v2 08/10] submodule: remove validate_submodule_git_dir() Adrian Ratiu
2025-09-30 13:35 ` Kristoffer Haugsbakk
2025-10-03 7:56 ` Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 09/10] t7450: move nested gitdir tests to t7425 Adrian Ratiu
2025-09-08 14:01 ` [PATCH v2 10/10] t7425: add gitdir encoding tests Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 1/5] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-10-06 16:37 ` Junio C Hamano
2025-10-07 9:23 ` Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 2/5] submodule: add gitdir path config override Adrian Ratiu
2025-10-06 16:47 ` Junio C Hamano
2025-10-07 15:41 ` Junio C Hamano
2025-10-21 8:06 ` Patrick Steinhardt
2025-10-21 11:50 ` Adrian Ratiu
2025-10-21 8:05 ` Patrick Steinhardt
2025-10-21 11:57 ` Adrian Ratiu
2025-10-06 11:25 ` [PATCH v3 3/5] strbuf: bring back is_rfc3986_unreserved Adrian Ratiu
2025-10-06 16:51 ` Junio C Hamano
2025-10-06 17:47 ` Junio C Hamano
2025-10-07 9:43 ` Adrian Ratiu
2025-10-21 8:06 ` Patrick Steinhardt
2025-10-06 11:25 ` [PATCH v3 4/5] submodule: encode gitdir paths to avoid conflicts Adrian Ratiu
2025-10-06 16:57 ` Junio C Hamano
2025-10-07 14:10 ` Adrian Ratiu
2025-10-07 17:20 ` Junio C Hamano
2025-10-07 17:41 ` Adrian Ratiu
2025-10-07 19:55 ` Junio C Hamano
2025-10-06 11:25 ` [PATCH v3 5/5] submodule: error out if gitdir name is too long Adrian Ratiu
2025-10-06 17:06 ` Junio C Hamano
2025-10-07 10:17 ` Adrian Ratiu
2025-10-07 15:58 ` Junio C Hamano
2025-10-21 8:06 ` Patrick Steinhardt
2025-10-21 13:13 ` Adrian Ratiu
2025-10-06 16:21 ` [PATCH v3 0/5] Encode submodule gitdir names to avoid conflicts Junio C Hamano
2025-10-07 11:13 ` Adrian Ratiu
2025-10-07 15:36 ` Junio C Hamano
2025-10-07 16:58 ` Adrian Ratiu
2025-10-07 17:27 ` Junio C Hamano
2025-10-07 16:21 ` Junio C Hamano
2025-10-07 17:21 ` Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 0/4] " Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 1/4] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 2/4] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 3/4] submodule: add extension to encode gitdir paths Adrian Ratiu
2025-11-07 15:05 ` [PATCH v4 4/4] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
2025-11-08 18:20 ` Aaron Schrab
2025-11-10 17:11 ` Adrian Ratiu
2025-11-10 17:31 ` Aaron Schrab
2025-11-10 18:27 ` Adrian Ratiu
2025-11-10 19:10 ` Junio C Hamano
2025-11-10 23:01 ` Adrian Ratiu
2025-11-10 23:17 ` Junio C Hamano
2025-11-11 12:41 ` Adrian Ratiu
2025-11-12 15:28 ` Adrian Ratiu
2025-11-14 23:03 ` [PATCH v4 0/4] Encode submodule gitdir names to avoid conflicts Josh Steadmon
2025-11-17 15:22 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 0/7] " Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 1/7] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 2/7] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
2025-12-05 12:16 ` Patrick Steinhardt
2025-12-05 17:25 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 3/7] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
2025-12-05 12:17 ` Patrick Steinhardt
2025-12-05 18:17 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 4/7] submodule: add extension to encode gitdir paths Adrian Ratiu
2025-12-05 12:19 ` Patrick Steinhardt
2025-12-05 19:30 ` Adrian Ratiu
2025-12-05 22:47 ` Junio C Hamano
2025-12-06 11:59 ` Patrick Steinhardt
2025-12-06 16:38 ` Junio C Hamano
2025-12-08 9:01 ` Adrian Ratiu
2025-12-08 11:46 ` Patrick Steinhardt
2025-12-08 15:48 ` Adrian Ratiu
2025-12-08 9:10 ` Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 5/7] submodule: fix case-folding gitdir filesystem colisions Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 6/7] submodule: use hashed name for gitdir Adrian Ratiu
2025-11-19 21:10 ` [PATCH v5 7/7] meson/Makefile: allow setting submodule encoding at build time Adrian Ratiu
2025-12-05 12:19 ` Patrick Steinhardt
2025-12-05 19:42 ` Adrian Ratiu
2025-12-05 22:52 ` Junio C Hamano
2025-12-06 12:02 ` Patrick Steinhardt
2025-12-06 16:48 ` Junio C Hamano
2025-12-08 9:23 ` Adrian Ratiu
2025-12-08 9:42 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 01/10] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 02/10] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-13 8:08 ` [PATCH v6 03/10] builtin/submodule--helper: add gitdir command Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 04/10] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 9:45 ` Adrian Ratiu
2025-12-16 23:22 ` Josh Steadmon
2025-12-17 7:30 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 05/10] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 10:01 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 06/10] submodule--helper: add gitdir migration command Adrian Ratiu
2025-12-16 9:09 ` Patrick Steinhardt
2025-12-16 10:17 ` Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 07/10] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 08/10] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 09/10] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
2025-12-13 8:08 ` [PATCH v6 10/10] submodule: hash the submodule name for the gitdir path Adrian Ratiu
2025-12-13 14:03 ` [PATCH v6 00/10] Add submodulePathConfig extension and gitdir encoding Ben Knoble
2025-12-15 16:28 ` Adrian Ratiu
2025-12-16 0:53 ` Junio C Hamano
2025-12-18 3:43 ` Ben Knoble
2025-12-16 23:20 ` Josh Steadmon
2025-12-17 8:17 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 00/11] " Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 01/11] submodule--helper: use submodule_name_to_gitdir in add_submodule Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 02/11] submodule: always validate gitdirs inside submodule_name_to_gitdir Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 03/11] builtin/submodule--helper: add gitdir command Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 04/11] submodule: introduce extensions.submodulePathConfig Adrian Ratiu
2025-12-21 3:27 ` Junio C Hamano
2025-12-23 13:35 ` Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 05/11] submodule: allow runtime enabling extensions.submodulePathConfig Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 06/11] submodule--helper: add gitdir migration command Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 07/11] builtin/credential-store: move is_rfc3986_unreserved to url.[ch] Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 08/11] submodule--helper: fix filesystem collisions by encoding gitdir paths Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 09/11] submodule: fix case-folding gitdir filesystem collisions Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 10/11] submodule: hash the submodule name for the gitdir path Adrian Ratiu
2025-12-20 10:15 ` [PATCH v7 11/11] submodule: detect conflicts with existing gitdir configs Adrian Ratiu
2025-12-21 2:39 ` [PATCH v7 00/11] Add submodulePathConfig extension and gitdir encoding Junio C Hamano
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).