Adding nested repository with slash adds files instead of gitlink

Adding nested repository with slash adds files instead of gitlink

[Heiko Voigt]

Hi,

I just discovered that when you have a slash at the end of a nested
repository, the files contained in the repository get added instead of
the gitlink.

I found this when I was adding a submodule and wanted to commit a small
change before that. You get the slash by using tab autocompletion.

Here is a recipe to reproduce:

mkdir test
cd test; git init
touch a; git add a; git commit -m a
mkdir ../test.git; (cd ../test.git; git init --bare)
git remote add origin ../test.git
git push origin master
git submodule add ../test.git submodule
git reset
git add submodule/

Now instead of just submodule gitlink there is an entry for submodule/a
in the index.

I just thought I put this out there. Will have a look if I find the time
to cook up a proper testcase and investigate.

Cheers Heiko

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Mon, Jun 18, 2018 at 1:23 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Hi,
>
> I just discovered that when you have a slash at the end of a nested
> repository, the files contained in the repository get added instead of
> the gitlink.
>
> I found this when I was adding a submodule and wanted to commit a small
> change before that. You get the slash by using tab autocompletion.
>
> Here is a recipe to reproduce:
>
> mkdir test
> cd test; git init
> touch a; git add a; git commit -m a
> mkdir ../test.git; (cd ../test.git; git init --bare)
> git remote add origin ../test.git
> git push origin master
> git submodule add ../test.git submodule
> git reset
> git add submodule/
>
> Now instead of just submodule gitlink there is an entry for submodule/a
> in the index.
>
> I just thought I put this out there. Will have a look if I find the time
> to cook up a proper testcase and investigate.

This sounds like the submodule specific code in pathspec.c, which has
been replaced with something else in bw/pathspec-sans-the-index. If
you have time, try a version without those changes (e.g. v2.13 or
before) to see if it's a possible culprit.

> Cheers Heiko



-- 
Duy

Re: Adding nested repository with slash adds files instead of gitlink

[Kevin Daudt]

On Mon, Jun 18, 2018 at 01:19:19PM +0200, Heiko Voigt wrote:

Now with cc to the mailing list.

> Hi,
> 
> I just discovered that when you have a slash at the end of a nested
> repository, the files contained in the repository get added instead of
> the gitlink.
> 
> I found this when I was adding a submodule and wanted to commit a small
> change before that. You get the slash by using tab autocompletion.
> 
> Here is a recipe to reproduce:
> 
> mkdir test
> cd test; git init
> touch a; git add a; git commit -m a
> mkdir ../test.git; (cd ../test.git; git init --bare)
> git remote add origin ../test.git
> git push origin master
> git submodule add ../test.git submodule
> git reset
> git add submodule/
> 
> Now instead of just submodule gitlink there is an entry for submodule/a
> in the index.
> 
> I just thought I put this out there. Will have a look if I find the time
> to cook up a proper testcase and investigate.
> 
> Cheers Heiko

This has been the case as far as I can remember, and is basically lore
in the #git irc channel).

This can also be reproduced by just cloning a repo inside another repo
and running `git add path/`.

Re: Adding nested repository with slash adds files instead of gitlink

[Brandon Williams]

On 06/18, Duy Nguyen wrote:
> On Mon, Jun 18, 2018 at 1:23 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >
> > Hi,
> >
> > I just discovered that when you have a slash at the end of a nested
> > repository, the files contained in the repository get added instead of
> > the gitlink.
> >
> > I found this when I was adding a submodule and wanted to commit a small
> > change before that. You get the slash by using tab autocompletion.
> >
> > Here is a recipe to reproduce:
> >
> > mkdir test
> > cd test; git init
> > touch a; git add a; git commit -m a
> > mkdir ../test.git; (cd ../test.git; git init --bare)
> > git remote add origin ../test.git
> > git push origin master
> > git submodule add ../test.git submodule
> > git reset
> > git add submodule/
> >
> > Now instead of just submodule gitlink there is an entry for submodule/a
> > in the index.
> >
> > I just thought I put this out there. Will have a look if I find the time
> > to cook up a proper testcase and investigate.
> 
> This sounds like the submodule specific code in pathspec.c, which has
> been replaced with something else in bw/pathspec-sans-the-index. If
> you have time, try a version without those changes (e.g. v2.13 or
> before) to see if it's a possible culprit.

I just tested this with v2.13 and saw the same issue.  I don't actually
think this ever worked in the way you want it to Heiko.  Maybe git add
needs to be taught to be more intelligent when trying to add a submodule
which doesn't exist in the index.

-- 
Brandon Williams

Re: Adding nested repository with slash adds files instead of gitlink

[Heiko Voigt]

Hi,

On Mon, Jun 18, 2018 at 05:55:44PM +0200, Kevin Daudt wrote:
> On Mon, Jun 18, 2018 at 01:19:19PM +0200, Heiko Voigt wrote:
> > I just discovered that when you have a slash at the end of a nested
> > repository, the files contained in the repository get added instead of
> > the gitlink.
[...]
> > 
> > I just thought I put this out there. Will have a look if I find the time
> > to cook up a proper testcase and investigate.
> > 
> > Cheers Heiko
> 
> This has been the case as far as I can remember, and is basically lore
> in the #git irc channel).
> 
> This can also be reproduced by just cloning a repo inside another repo
> and running `git add path/`.

Interesting and nobody complained to the mailinglist? IMO, there is no
reason 'git add path' and 'git add path/' should behave differently or
is there?

So it seems it is a very seldom operation (in my daily work it is at
least) and people just accepted it as a quirk of git.

If someone wants to look into changing this feel free, otherwise I will
have a look, but I am not sure when yet.

Cheers Heiko

Re: Adding nested repository with slash adds files instead of gitlink

[Heiko Voigt]

On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
> On 06/18, Duy Nguyen wrote:
> > This sounds like the submodule specific code in pathspec.c, which has
> > been replaced with something else in bw/pathspec-sans-the-index. If
> > you have time, try a version without those changes (e.g. v2.13 or
> > before) to see if it's a possible culprit.
> 
> I just tested this with v2.13 and saw the same issue.  I don't actually
> think this ever worked in the way you want it to Heiko.  Maybe git add
> needs to be taught to be more intelligent when trying to add a submodule
> which doesn't exist in the index.

That was also my guess, since my feeling is that this is a quite rare
use case. Adding submodules alone is not a daily thing, let alone
selecting different changes after 'git submodule add'.

I also think git could be more intelligent here.

Cheers Heiko

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
> > On 06/18, Duy Nguyen wrote:
> > > This sounds like the submodule specific code in pathspec.c, which has
> > > been replaced with something else in bw/pathspec-sans-the-index. If
> > > you have time, try a version without those changes (e.g. v2.13 or
> > > before) to see if it's a possible culprit.
> >
> > I just tested this with v2.13 and saw the same issue.  I don't actually
> > think this ever worked in the way you want it to Heiko.  Maybe git add
> > needs to be taught to be more intelligent when trying to add a submodule
> > which doesn't exist in the index.
>
> That was also my guess, since my feeling is that this is a quite rare
> use case. Adding submodules alone is not a daily thing, let alone
> selecting different changes after 'git submodule add'.
>
> I also think git could be more intelligent here.

Ah.. the "submodule not registered in index" case. I think I remember
this (because I remember complaining about it once or two times).
Definitely agreed that git-add should do the right thing here.

Brandon already moved the submodule check outside pathspec code
(wonderful!) so adding more checks based on worktree state should not
be a big work. I think the only concern here is catching submodule
locations so we don't check submodule at the same location multiple
times.

No actually, we could do better. Let me see if I can come up with a
patch or something...
-- 
Duy

Re: Adding nested repository with slash adds files instead of gitlink

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>>
>> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
>> > On 06/18, Duy Nguyen wrote:
>> > > This sounds like the submodule specific code in pathspec.c, which has
>> > > been replaced with something else in bw/pathspec-sans-the-index. If
>> > > you have time, try a version without those changes (e.g. v2.13 or
>> > > before) to see if it's a possible culprit.
>> >
>> > I just tested this with v2.13 and saw the same issue.  I don't actually
>> > think this ever worked in the way you want it to Heiko.  Maybe git add
>> > needs to be taught to be more intelligent when trying to add a submodule
>> > which doesn't exist in the index.
>>
>> That was also my guess, since my feeling is that this is a quite rare
>> use case. Adding submodules alone is not a daily thing, let alone
>> selecting different changes after 'git submodule add'.
>>
>> I also think git could be more intelligent here.
>
> Ah.. the "submodule not registered in index" case. I think I remember
> this (because I remember complaining about it once or two times).
> Definitely agreed that git-add should do the right thing here.

I am not sure if this even needs to be implemented as "look for the
submodule in the index".  Even before submodule was added, we knew
that "git add foo/bar" should reject the request if we find foo is a
symbolic link, and we should do the same when foo/ is a directory
that is the top of a working tree under control of another
repository, no?

Hmm, what happens when we do this?

	git init
	ln -s /tmp foo
	>foo/bar
	git add foo/

I think we should say either "let's add foo symlink" or
"foo/. (directory) is beyond symlink" (the latter is preferrable,
but the former is acceptable as long as foo is pointing at a
directory; but foo could be a dangling symlink whose pointee's type
may not be discernable by "git add").

Shouldn't we be reacting pretty much the same when we see this?

	git init
	git init foo
	>foo/bar
	git add foo/

That is, either drop '/' and add 'foo' as a submodule, or say
"foo/. (directory) belongs to another repository, cannot add here"
(again, the latter is preferrable for consistency with the symlink
behaviour above).

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Tue, Jun 19, 2018 at 05:16:17PM +0200, Duy Nguyen wrote:
> No actually, we could do better. Let me see if I can come up with a
> patch or something...

OK. What we currently do is, when we search for potential untracked
paths for adding to the index, we unconditionally ignore anything
inside ".git". For example, if "foo" is a submodule, "git add ." will
visit "foo/.git" then ignore its content completely.

We could do something very similar: when we visit "foo", if "foo/.git"
exists, we ignore it as well. In other words, we extend from "ignore
anything inside a git repository" to "ignore anything inside any other
git worktree".

The following patch basically does that. If you specify "git add
foo/bar". It will still visit "foo" first, realize that it's a
submodule and drop it. At the end, it will not report foo/bar as an
untracked (i.e. add-able) entry, so you can't add it.

I didn't test it extensively to see if it breaks anything though. And
I might need to check how it affects untracked cache...

-- 8< --
diff --git a/dir.c b/dir.c
index fe9bf58e4c..8a1a5d8dd5 100644
--- a/dir.c
+++ b/dir.c
@@ -1672,6 +1672,17 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	if (dtype != DT_DIR && has_path_in_index)
 		return path_none;
 
+	if (dtype == DT_DIR) {
+		int path_len = path->len;
+		int is_submodule;
+
+		strbuf_addstr(path, "/.git");
+		is_submodule = is_directory(path->buf);
+		strbuf_setlen(path, path_len);
+		if (is_submodule)
+			return path_none;
+	}
+
 	/*
 	 * When we are looking at a directory P in the working tree,
 	 * there are three cases:
-- 8< --

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Tue, Jun 19, 2018 at 5:56 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Duy Nguyen <pclouds@gmail.com> writes:
>
> > On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >>
> >> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
> >> > On 06/18, Duy Nguyen wrote:
> >> > > This sounds like the submodule specific code in pathspec.c, which has
> >> > > been replaced with something else in bw/pathspec-sans-the-index. If
> >> > > you have time, try a version without those changes (e.g. v2.13 or
> >> > > before) to see if it's a possible culprit.
> >> >
> >> > I just tested this with v2.13 and saw the same issue.  I don't actually
> >> > think this ever worked in the way you want it to Heiko.  Maybe git add
> >> > needs to be taught to be more intelligent when trying to add a submodule
> >> > which doesn't exist in the index.
> >>
> >> That was also my guess, since my feeling is that this is a quite rare
> >> use case. Adding submodules alone is not a daily thing, let alone
> >> selecting different changes after 'git submodule add'.
> >>
> >> I also think git could be more intelligent here.
> >
> > Ah.. the "submodule not registered in index" case. I think I remember
> > this (because I remember complaining about it once or two times).
> > Definitely agreed that git-add should do the right thing here.
>
> I am not sure if this even needs to be implemented as "look for the
> submodule in the index".  Even before submodule was added, we knew
> that "git add foo/bar" should reject the request if we find foo is a
> symbolic link, and we should do the same when foo/ is a directory
> that is the top of a working tree under control of another
> repository, no?

Exactly. I started with the intention to do something related to the
index only to slowly realize that it was not the right place. We
traverse directories and stop looking inside a symlink, we can do the
same if we realize it's a submodule.
-- 
Duy

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Tue, Jun 19, 2018 at 6:09 PM Duy Nguyen <pclouds@gmail.com> wrote:
> On Tue, Jun 19, 2018 at 05:16:17PM +0200, Duy Nguyen wrote:
> > No actually, we could do better. Let me see if I can come up with a
> > patch or something...
>
> OK. What we currently do is, when we search for potential untracked
> paths for adding to the index, we unconditionally ignore anything
> inside ".git". For example, if "foo" is a submodule, "git add ." will
> visit "foo/.git" then ignore its content completely.
>
> We could do something very similar: when we visit "foo", if "foo/.git"
> exists, we ignore it as well. In other words, we extend from "ignore
> anything inside a git repository" to "ignore anything inside any other
> git worktree".
>
> The following patch basically does that. If you specify "git add
> foo/bar". It will still visit "foo" first, realize that it's a
> submodule and drop it. At the end, it will not report foo/bar as an
> untracked (i.e. add-able) entry, so you can't add it.

Another note (which I added, then thought otherwise and dropped). I
believe this approach also solves the problem that
die_path_inside_submodule() tries to work around.

When you feed a path inside a submodule, read_directory() code does
not realize it and walk through like it's part of the current worktree
(wrong!). But if read_directory() does the right thing from the
beginning, you don't need this trick. We don't even need this trick if
a submodule is not real on worktree (no ".git" directory there) but
registered in the index as a git link because the d/f check should
catch that and complain loudly anyway when you add a new entry.
-- 
Duy

Re: Adding nested repository with slash adds files instead of gitlink

[Rafael Ascensão]

On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Interesting and nobody complained to the mailinglist?
>

For reference this was sometimes called "Fake Submodules" online.

Re: Adding nested repository with slash adds files instead of gitlink

[Kevin Daudt]

On Tue, Jun 19, 2018 at 11:29:02PM +0100, Rafael Ascensão wrote:
> On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >
> > Interesting and nobody complained to the mailinglist?
> >
> 
> For reference this was sometimes called "Fake Submodules" online.

Do you refer to cloning a repository in another repository, and running
`git add path` (no trailing /) so that a gitlink is created without a
.gitmodules file? Because that's not what Heiko is referring to.

What this is about that when doing `git add path/` (with trailing /),
git will add the files of the cloned repository to the parent repo, and
basically ignore that it's another repository.

And I guess no one complained because it looked like a feature rather
than a bug.

Kevin

Re: Adding nested repository with slash adds files instead of gitlink

[Rafael Ascensão]

On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
>
> What this is about that when doing `git add path/` (with trailing /),
>

This is what I was referring to. If you search for 'Fake Submodules',
you'll see that some people were/are intentionally using this instead of
subtrees or submodules. Unfortunately the original article [1] seems to
be dead, but searching url in the mailing list archives leads to some
additional discussion on the subject [2,3].


[1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
[2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/
[3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/

Re: Adding nested repository with slash adds files instead of gitlink

[Duy Nguyen]

On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
>
> On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> >
> > What this is about that when doing `git add path/` (with trailing /),
> >
>
> This is what I was referring to. If you search for 'Fake Submodules',
> you'll see that some people were/are intentionally using this instead of
> subtrees or submodules. Unfortunately the original article [1] seems to
> be dead, but searching url in the mailing list archives leads to some
> additional discussion on the subject [2,3].

Abusing a long standing bug does not make it a feature. I'm not
opposed to having a new option to keep that behavior, but it should
not be the default. If you use it that way, you're on your own.

> [1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
> [2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/
> [3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/
-- 
Duy

Re: Adding nested repository with slash adds files instead of gitlink

[Rafael Ascensão]

On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Interesting and nobody complained to the mailinglist?
>

On Wed, Jun 20, 2018 at 3:57 PM Duy Nguyen <pclouds@gmail.com> wrote:
>
> Abusing a long standing bug does not make it a feature.
>

To make things clear, I wasn't defending if this should be considered a
bug or a feature. Was just clarifying that this isn't a newly discovered
thing and was reported/discussed in the past.

--
Cheers,
Rafael

Re: Adding nested repository with slash adds files instead of gitlink

[Phil Sainty]

> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> > > What this is about that when doing `git add path/` (with trailing /),
> >
> > This is what I was referring to. If you search for 'Fake Submodules',
> > you'll see that some people were/are intentionally using this instead of
> > subtrees or submodules. Unfortunately the original article [1] seems to
> > be dead, but searching url in the mailing list archives leads to some
> > additional discussion on the subject [2,3].
>
> Abusing a long standing bug does not make it a feature. I'm not
> opposed to having a new option to keep that behavior, but it should
> not be the default. If you use it that way, you're on your own.

Was such an option ever worked on?

(I.e. a way to git-add some sub-directory 'foo' which contains another
repository, and have git-add act as if foo/.git didn't exist -- simply
adding the (other) contents of foo to the containing repo's index.)

I haven't spotted anything in the git-add man page for v2.34, nor in the
release notes for subsequent git versions.

The old behaviour was a genuinely useful ability which I've used a great
in the past, and if there's any convenient way to achieve the same thing
nowadays, I've failed to find it.  (One can temporarily move the nested
.git directory elsewhere, run git-add, and then move the .git directory
back again; but that's frustratingly cumbersome by comparison).

If I haven't missed some existing solution, could a new git-add option
for restoring this ability be implemented?


-Phil


> > [1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
> > [2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/
> > [3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/

Re: Adding nested repository with slash adds files instead of gitlink

[Junio C Hamano]

Phil Sainty <phil@catalyst.net.nz> writes:

>> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
>> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
>> > > What this is about that when doing `git add path/` (with trailing /),
>> >
>> > This is what I was referring to. If you search for 'Fake Submodules',
>> > you'll see that some people were/are intentionally using this instead of
>> > subtrees or submodules. Unfortunately the original article [1] seems to
>> > be dead, but searching url in the mailing list archives leads to some
>> > additional discussion on the subject [2,3].
>>
>> Abusing a long standing bug does not make it a feature. I'm not
>> opposed to having a new option to keep that behavior, but it should
>> not be the default. If you use it that way, you're on your own.
>
> Was such an option ever worked on?

No.  

I do not recall hearing anybody who have been active in the
development community saying anything good about such an option.
For the past 6 or so years, nobody who actively works on git thought
it was an interesting and/or useful thing to work on.

I cannot quite say that they thought that it is actively a bad idea
to offer such an option, though.

Re: Adding nested repository with slash adds files instead of gitlink

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3149 bytes --]

Hi,

On Thu, 8 Aug 2024, Junio C Hamano wrote:

> Phil Sainty <phil@catalyst.net.nz> writes:
>
> >> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
> >> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> >> > > What this is about that when doing `git add path/` (with trailing /),
> >> >
> >> > This is what I was referring to. If you search for 'Fake Submodules',
> >> > you'll see that some people were/are intentionally using this instead of
> >> > subtrees or submodules. Unfortunately the original article [1] seems to
> >> > be dead, but searching url in the mailing list archives leads to some
> >> > additional discussion on the subject [2,3].
> >>
> >> Abusing a long standing bug does not make it a feature. I'm not
> >> opposed to having a new option to keep that behavior, but it should
> >> not be the default. If you use it that way, you're on your own.
> >
> > Was such an option ever worked on?
>
> No.
>
> I do not recall hearing anybody who have been active in the
> development community saying anything good about such an option.
> For the past 6 or so years, nobody who actively works on git thought
> it was an interesting and/or useful thing to work on.
>
> I cannot quite say that they thought that it is actively a bad idea
> to offer such an option, though.

I have encountered the wish quite frequently to be able to make changes in
a subdirectory and have them reflected as a commit that is both part of
that subdirectory's revision history as well as part of the containing
project's.

The benefits for monorepos, and for reproducible builds, are probably
obvious.

Sadly, I cannot think of an elegant technical design, and this is not for
lack of trying.

There are lots of projects I worked on that would have benefitted from
being able to track a subdirectory both independently as well as within
the context of a containing project, i.e. offering to view (and
fetch/push) the changesets in both contexts as equal first citizens.

Even the Git and libxdiff projects, as a concrete example, could
potentially have benefitted from such a feature: Ideally, it should be
possible to push commits made in the `xdiff/` directory not only to the
git/git repository but also separately from the rest of Git, say, to
xdiff/libxdiff. This way, the `libxdiff` project would still be able today
to thrive as an independent project.

The only way to implement a feature like this that I _can_ think of is to
generate duplicate commit objects, though, with a reference to the "inner"
commit stored in the "outer" commit object. And that strategy breaks down
really quickly, no matter whether I think about deep dependency trees or
about integrating commit histories that have been made in the "inner"
project separately and now need to be merged into the "outer" project, and
there are many more processes for which this strategy strikes me as
inadequate.

So yes, I think that the idea per se has a lot of merit in the real world
out there. It's the implementation details that are an obstacle.

Ciao,
Johannes

Re: Adding nested repository with slash adds files instead of gitlink

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> >> Abusing a long standing bug does not make it a feature. I'm not
>> >> opposed to having a new option to keep that behavior, but it should
>> >> not be the default. If you use it that way, you're on your own.
>> >
>> > Was such an option ever worked on?
>>
>> No.
>>
>> I do not recall hearing anybody who have been active in the
>> development community saying anything good about such an option.
>> For the past 6 or so years, nobody who actively works on git thought
>> it was an interesting and/or useful thing to work on.
>>
>> I cannot quite say that they thought that it is actively a bad idea
>> to offer such an option, though.
>
> I have encountered the wish quite frequently to be able to make changes in
> a subdirectory and have them reflected as a commit that is both part of
> that subdirectory's revision history as well as part of the containing
> project's.

Let me make sure I understand the above.  You create a commit to
contain the change in the submodule and at the same time create a
new commit to bind the updated submodule commit to superproject
tree.  If that is what you are talking about, of course, that would
be a nice thing to have.  Sort of "commit --recursive".  It is still
debatable what to do with the log messages, though (you want to
leave in the submodule an explanation that is understandable in the
scope of the submodule, while you want to record in the superproject
a bigger picture reasoning).

But I did not get the impression that it is what the original poster
wants.  My reading of the original thread (this is a resurrection of
an antient thread dating back to 2018) was that you have a submodule
at path S, you muck with a file in S/file, and you want to commit in
the context of the superproject, having the superproject track S/file
in its history (not just S gitlink).

And that is quite different from what you said.

Re: Adding nested repository with slash adds files instead of gitlink

[Phil Sainty]

Junio C Hamano <gitster@pobox.com> writes:
> Let me make sure I understand the above.  You create a commit to
> contain the change in the submodule and at the same time create a
> new commit to bind the updated submodule commit to superproject tree.

I can imagine this ability also being useful, but it would be an
independent feature from the one initially requested here...


> But I did not get the impression that it is what the original poster
> wants.  My reading of the original thread (this is a resurrection of
> an antient thread dating back to 2018) was that you have a submodule
> at path S, you muck with a file in S/file, and you want to commit in
> the context of the superproject, having the superproject track S/file
> in its history (not just S gitlink).

That's correct.

My common usage was that I would add the entire contents of S, along
with some associated configuration outside of S, and then make a commit
of all of that in the superproject.

The two repos (superproject and submodule S) are then tracking the files
in S independently; so if I was to pull new changes to the submodule
from its own upstream, git commands run from the S directory would not
show any changes vs the state of the submodule repo, whereas commands
run from the superproject would see new changes.

Cloning the superproject repo would produce its version of S, and
without the S/.git directory.


-Phil

Re: Adding nested repository with slash adds files instead of gitlink

[Johannes Schindelin]

Hi Phil & Junio,

On Wed, 14 Aug 2024, Phil Sainty wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> > Let me make sure I understand the above.  You create a commit to
> > contain the change in the submodule and at the same time create a
> > new commit to bind the updated submodule commit to superproject tree.
>
> I can imagine this ability also being useful, but it would be an
> independent feature from the one initially requested here...

FWIW I did not intend to suggest anything to do with submodules.

There are too many issues with submodules and I don't think they will ever
be resolved, as they simply don't integrate into the original, elegant
design of having all Git objects in the same repository.

What I had in mind (and which I described only very loosely) was to create
a super-project commit that references the modified subdirectory as part
of the same project, much like `xdiff/` files are tracked inside git/git.

Yes, I papered over the need to create at least one new tree object that
is derived from the super-project's root tree object and references the
root tree object of the "inner" commit. I did not want to go into too many
details about the technical implementation, and chose to use the `xdiff/`
example as a way to transport my idea, but apparently I was wrong to
believe that it would do a good enough job.

In essence, what I would love to have is more along the lines of how
Subversion allows you to check out just a subdirectory of the project, and
restrict the revision history to that directory in that checkout.

This is different from submodules, and also from subtrees, as it would
allow to treat `xdiff/` as a stand-alone repository, but also allow to
treat it as a tracked subdirectory of the super-project at the same time,
both equally-well integrated into the standard Git processes like `git log
-G`, `git bisect`, `git rebase`, etc.

> > But I did not get the impression that it is what the original poster
> > wants.  My reading of the original thread (this is a resurrection of
> > an antient thread dating back to 2018) was that you have a submodule
> > at path S, you muck with a file in S/file, and you want to commit in
> > the context of the superproject, having the superproject track S/file
> > in its history (not just S gitlink).
>
> That's correct.
>
> My common usage was that I would add the entire contents of S, along
> with some associated configuration outside of S, and then make a commit
> of all of that in the superproject.

This could be achieved in a very hacky way by moving the `.git` directory
of S out of the way, committing, then moving it back, right?

> The two repos (superproject and submodule S) are then tracking the files
> in S independently; so if I was to pull new changes to the submodule
> from its own upstream, git commands run from the S directory would not
> show any changes vs the state of the submodule repo, whereas commands
> run from the superproject would see new changes.

I did indeed have a more integrated thing in mind, where the commits of
the super-project that touch S and the commits of S were more connected.

Ciao,
Johannes

> Cloning the superproject repo would produce its version of S, and
> without the S/.git directory.