[RFC] worktree: add --recurse-submodules support to git worktree add

[RFC] worktree: add --recurse-submodules support to git worktree add

[JAM]

[-- Attachment #1.1: Type: text/plain, Size: 2526 bytes --]

Back in 2022, Glen Choo noted [1] that git worktree add leaves submodules
unhandled and suggested a --recurse-submodules flag to fix that. The thread
went quiet. The 2015–2016 discussions by Duy and Beller [2][3] had flagged a
deeper design concern: submodule existence (which worktrees have which
submodules checked out) is tangled up with submodule configuration (URL,
path),
making per-worktree submodule support tricky to reason about.

This proposal sidesteps that concern entirely. Rather than touching
submodule
configuration at all, the idea is to reuse the git object data that's
already
present in the main repository — the same thing git worktree add already
does
for the top-level object store, extended down to the submodule layer.

The use case has also grown more pressing: multi-agent development workflows
(where several autonomous coding agents work concurrently on different
branches
of the same repository) rely heavily on worktrees for isolation, and fall
apart
on projects with submodules.

Concretely, git worktree add --recurse-submodules would:

1. Hardlink $GIT_COMMON_DIR/modules/ into the new worktree's entry.
Independent directory trees, shared inodes — no extra disk, no network.
2. Rewrite core.worktree in the hardlinked config and config.worktree
files to point at the new worktree's working directory instead of the main
repo's.
3. Run git submodule update inside the new worktree to write the .git
pointer files into each submodule directory. Entirely local since the
modules directory is already there.
4. Populate working trees with git read-tree HEAD && git checkout -- . per
submodule, since the hardlinked index files start empty.

A shell script implementing this as a prototype is attached.

The worktreeConfig extension case (step 2) is the one place that needs care,
since core.worktree may live in either config or config.worktree
depending on the submodule. The prototype handles both. The other open
question
is policy for submodules not yet initialized in the main repo — skip
silently,
warn, or error out.

Would there be interest in a proper patch series for this?

[1]
https://lore.kernel.org/git/kl6lwnimyxbq.fsf@chooglen-macbookpro.roam.corp.google.com/
[2]
https://lore.kernel.org/git/CACsJy8D8Ur4W348t-WFUPrb7SQxmff5MJ4aRp+w+ZiQ7VVvipg@mail.gmail.com/
[3]
https://lore.kernel.org/git/CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com/

Signed-off-by: Jimmy Aguilar kratsbinovish@gmail.com

[-- Attachment #1.2: Type: text/html, Size: 3014 bytes --]

[-- Attachment #2: create-worktree.sh --]
[-- Type: application/x-shellscript, Size: 4936 bytes --]

Re: [RFC] worktree: add --recurse-submodules support to git worktree add

[Phillip Wood]

On 15/04/2026 01:14, JAM wrote:
> Back in 2022, Glen Choo noted [1] that git worktree add leaves submodules
> unhandled and suggested a --recurse-submodules flag to fix that. The thread
> went quiet. The 2015–2016 discussions by Duy and Beller [2][3] had flagged a
> deeper design concern: submodule existence (which worktrees have which
> submodules checked out) is tangled up with submodule configuration (URL, 
> path),
> making per-worktree submodule support tricky to reason about.
> 
> This proposal sidesteps that concern entirely. Rather than touching 
> submodule
> configuration at all, the idea is to reuse the git object data that's 
> already
> present in the main repository — the same thing git worktree add already 
> does
> for the top-level object store, extended down to the submodule layer.

I'm not really a submodule user so maybe I'm missing something. 
Worktrees separate out per-worktree data such as refs like HEAD which 
are stored under the worktree's gitdir in 
"$GIT_COMMON_DIR/worktrees/$id" and per-repository data such as the 
object store which is stored under "$GIT_COMMON_DIR". The proposal below 
appears to use the same modules directory for each worktree so if I have 
two worktrees A and B each with a submodule "sub" how can I checkout 
commit C1 in "A/sub" and commit C2 in "B/sub" when they share the same 
gitdir? It makes sense for "A/sub" and "B/sub" to share the same object 
database, but they don't want to share the same HEAD.

Thanks

Phillip


> 
> The use case has also grown more pressing: multi-agent development workflows
> (where several autonomous coding agents work concurrently on different 
> branches
> of the same repository) rely heavily on worktrees for isolation, and 
> fall apart
> on projects with submodules.
> 
> Concretely, git worktree add --recurse-submodules would:
> 
> 1. Hardlink $GIT_COMMON_DIR/modules/ into the new worktree's entry.
> Independent directory trees, shared inodes — no extra disk, no network.
> 2. Rewrite core.worktree in the hardlinked config and config.worktree
> files to point at the new worktree's working directory instead of the main
> repo's.
> 3. Run git submodule update inside the new worktree to write the .git
> pointer files into each submodule directory. Entirely local since the
> modules directory is already there.
> 4. Populate working trees with git read-tree HEAD && git checkout -- . per
> submodule, since the hardlinked index files start empty.
> 
> A shell script implementing this as a prototype is attached.
> 
> The worktreeConfig extension case (step 2) is the one place that needs care,
> since core.worktree may live in either config or config.worktree
> depending on the submodule. The prototype handles both. The other open 
> question
> is policy for submodules not yet initialized in the main repo — skip 
> silently,
> warn, or error out.
> 
> Would there be interest in a proper patch series for this?
> 
> [1] https://lore.kernel.org/git/kl6lwnimyxbq.fsf@chooglen- 
> macbookpro.roam.corp.google.com/ <https://lore.kernel.org/git/ 
> kl6lwnimyxbq.fsf@chooglen-macbookpro.roam.corp.google.com/>
> [2] https://lore.kernel.org/git/CACsJy8D8Ur4W348t- 
> WFUPrb7SQxmff5MJ4aRp+w+ZiQ7VVvipg@mail.gmail.com/ <https:// 
> lore.kernel.org/git/CACsJy8D8Ur4W348t- 
> WFUPrb7SQxmff5MJ4aRp+w+ZiQ7VVvipg@mail.gmail.com/>
> [3] https://lore.kernel.org/git/CAGZ79kZB8U+ERNeYpZ- 
> i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com/ <https:// 
> lore.kernel.org/git/CAGZ79kZB8U+ERNeYpZ- 
> i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com/>
> 
> Signed-off-by: Jimmy Aguilar kratsbinovish@gmail.com 
> <mailto:kratsbinovish@gmail.com>

Re: [RFC] worktree: add --recurse-submodules support to git worktree add

[Jimmy Aguilar Mena]

Phillip,

Good catch — let me clarify what the script actually places on disk.

The hardlink copy does not go into $GIT_COMMON_DIR/modules/ (which
both worktrees would then share). It goes into each worktree's own
gitdir entry:

cp -al $GIT_COMMON_DIR/modules \
        $GIT_COMMON_DIR/worktrees/$id/modules

So worktree A uses $GIT_COMMON_DIR/modules/sub/ and worktree B uses
$GIT_COMMON_DIR/worktrees/B/modules/sub/ — these are physically
separate directory trees with separate paths.

Within those trees the files start out as hardlinks (same inode), but
that only affects disk space accounting. It does not mean the data is
shared. Git writes files atomically by writing to a lock file and then
calling rename(2), which gives the path a new inode. So the first time
git touches HEAD, index, or any ref in one worktree's submodule
gitdir, the rename breaks that hardlink and the file becomes
independent. The other worktree's copy is untouched.

The only files that stay hardlinked forever are pack files and loose
object files — precisely because git objects are immutable by
design. Two pack files with the same content should share the same
disk blocks; there is no correctness issue.

The net effect is the same separation git worktree already provides
for the outer repo (HEAD and index under
$GIT_COMMON_DIR/worktrees/$id/, objects shared under
$GIT_COMMON_DIR/objects/), but achieved through the filesystem rather
than explicit gitdir configuration. A native --recurse-submodules
implementation should do the same thing explicitly — probably by
running the equivalent of git -C <sub-gitdir> worktree add for each
submodule, which would give each submodule its own proper worktrees/
structure rather than relying on the hardlink + rename behaviour.


Signed-off-by: Jimmy Aguilar Mena kratsbinovish@gmail.com

Re: [RFC] worktree: add --recurse-submodules support to git worktree add

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> On 15/04/2026 01:14, JAM wrote:
>> Back in 2022, Glen Choo noted [1] that git worktree add leaves submodules
>> unhandled and suggested a --recurse-submodules flag to fix that. The thread
>> went quiet. The 2015–2016 discussions by Duy and Beller [2][3] had flagged a
>> deeper design concern: submodule existence (which worktrees have which
>> submodules checked out) is tangled up with submodule configuration (URL, 
>> path),
>> making per-worktree submodule support tricky to reason about.
>> 
>> This proposal sidesteps that concern entirely. Rather than touching 
>> submodule
>> configuration at all, the idea is to reuse the git object data that's 
>> already
>> present in the main repository — the same thing git worktree add already 
>> does
>> for the top-level object store, extended down to the submodule layer.
>
> I'm not really a submodule user so maybe I'm missing something. 
> Worktrees separate out per-worktree data such as refs like HEAD which 
> are stored under the worktree's gitdir in 
> "$GIT_COMMON_DIR/worktrees/$id" and per-repository data such as the 
> object store which is stored under "$GIT_COMMON_DIR". The proposal below 
> appears to use the same modules directory for each worktree so if I have 
> two worktrees A and B each with a submodule "sub" how can I checkout 
> commit C1 in "A/sub" and commit C2 in "B/sub" when they share the same 
> gitdir? It makes sense for "A/sub" and "B/sub" to share the same object 
> database, but they don't want to share the same HEAD.

True.  I am not a submodule user and do not have a strong interest
in submodules, but I agree that once the superproject wants to use
multiple worktrees, the submodules bound to it inevitably need to
use different worktrees that correspond to the worktrees used by the
superproject exactly because of the point you raise here.  They need
to be able to independently check out a suitable commit in the
context of the superproject's worktree that is checked out.