[PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Henri GEIST]

Adding a note in the submodule documentation signaling that the
automatically cloned missing submodules are cloned with a separate
gitdir. And where it is put.

Signed-off-by: Henri GEIST <geist.henri@laposte.net>
---
 Documentation/git-submodule.txt |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 21cb59a..ea837fd 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -64,6 +64,11 @@ using the 'status' subcommand and get a detailed overview of the
 difference between the index and checkouts using the 'summary'
 subcommand.
 
+*NOTE*: when submodule add or submodule update commands clone a missing
+submodule, the option --separate-git-dir is passed to the clone command
+and the gitdir of the submodule is placed outside of its working
+directory in the .git/module of the current repository.
+
 
 COMMANDS
 --------
-- 
1.7.9.3.369.gd715.dirty

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Andrew Keller]

On Mar 7, 2014, at 2:53 AM, Henri GEIST <geist.henri@laposte.net> wrote:

> Adding a note in the submodule documentation signaling that the
> automatically cloned missing submodules are cloned with a separate
> gitdir. And where it is put.
> 
> Signed-off-by: Henri GEIST <geist.henri@laposte.net>
> ---
> Documentation/git-submodule.txt |    5 +++++
> 1 file changed, 5 insertions(+)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index 21cb59a..ea837fd 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -64,6 +64,11 @@ using the 'status' subcommand and get a detailed overview of the
> difference between the index and checkouts using the 'summary'
> subcommand.
> 
> +*NOTE*: when submodule add or submodule update commands clone a missing
> +submodule, the option --separate-git-dir is passed to the clone command
> +and the gitdir of the submodule is placed outside of its working
> +directory in the .git/module of the current repository.
> +

The modules directory is 'modules'.  And, the '.git' folder is not always called '.git' -- in a submodule, for example, the directory name is the name of the module.

Also, this file contains mostly high-level documentation, and this addition feels technical in nature.  Is there a location for more technical documentation?  Or, perhaps it can be reworded to sound less technical?

> 
> COMMANDS
> --------
> -- 
> 1.7.9.3.369.gd715.dirty
> 
> 
> --

 - Andrew Keller

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Junio C Hamano]

Andrew Keller <andrew@kellerfarm.com> writes:

>> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
>> index 21cb59a..ea837fd 100644
>> --- a/Documentation/git-submodule.txt
>> +++ b/Documentation/git-submodule.txt
> ...
> Also, this file contains mostly high-level documentation, and this
> addition feels technical in nature.  Is there a location for more
> technical documentation?  Or, perhaps it can be reworded to sound
> less technical?

I tend to agree that the new paragraph looked somewhat out of place
and goes into a too low-level detail of the implementation.

The repository-layout documentation may be a better place for
readers to learn what lives where inside $GIT_DIR.

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Henri GEIST]

[-- Attachment #1: Type: text/plain, Size: 2444 bytes --]

Le vendredi 07 mars 2014 à 16:42 -0500, Andrew Keller a écrit :
> On Mar 7, 2014, at 2:53 AM, Henri GEIST <geist.henri@laposte.net> wrote:
> 
> > Adding a note in the submodule documentation signaling that the
> > automatically cloned missing submodules are cloned with a separate
> > gitdir. And where it is put.
> > 
> > Signed-off-by: Henri GEIST <geist.henri@laposte.net>
> > ---
> > Documentation/git-submodule.txt |    5 +++++
> > 1 file changed, 5 insertions(+)
> > 
> > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> > index 21cb59a..ea837fd 100644
> > --- a/Documentation/git-submodule.txt
> > +++ b/Documentation/git-submodule.txt
> > @@ -64,6 +64,11 @@ using the 'status' subcommand and get a detailed overview of the
> > difference between the index and checkouts using the 'summary'
> > subcommand.
> > 
> > +*NOTE*: when submodule add or submodule update commands clone a missing
> > +submodule, the option --separate-git-dir is passed to the clone command
> > +and the gitdir of the submodule is placed outside of its working
> > +directory in the .git/module of the current repository.
> > +
> 
> The modules directory is 'modules'.  And, the '.git' folder is not always called '.git' -- in a submodule, for example, the directory name is the name of the module.
> 
> Also, this file contains mostly high-level documentation, and this addition feels technical in nature.  Is there a location for more technical documentation?  Or, perhaps it can be reworded to sound less technical?

This information is technical in nature but has some importance for general users.
As this kind of clone have a separate gitdir, you will have a surprise if you
copy past the worktree as the gitdir will not come together.
I have done it and as the doc say nothing about that it take me some time to
understand what happened. And why this repository behave differently than others.

May be I can rephrase like this :

*NOTE*: when 'submodule add' or 'submodule update' commands clone a missing
submodule, the $GIT_DIR of the submodule containing the metadata is placed
outside of its working directory in the $GIT_DIR/modules of the current
repository. Those submodule can not directly be copied or move.
You need to use 'git clone'.


> 
> > 
> > COMMANDS
> > --------
> > -- 
> > 1.7.9.3.369.gd715.dirty
> > 
> > 
> > --
> 
>  - Andrew Keller
> 



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Junio C Hamano]

Henri GEIST <geist.henri@laposte.net> writes:

> This information is technical in nature but has some importance for general users.
> As this kind of clone have a separate gitdir, you will have a surprise if you
> copy past the worktree as the gitdir will not come together.

I am not sure if I understand exactly what you are trying to say.
Are you saying that you had a submodule at "sub/dir" in your working
tree, and then "mkdir ../another && cp -R sub/dir ../another" did
not result in a usable Git working tree in ../another directory?

It is almost like complaining that "mkdir ../newone && cp -R * ../newone/"
did not result in a usable git repository in ../newone directory and
honestly speaking, that sounds borderline insane, I'd have to say.

Yes, if a user knows what she is doing, she should be able to make
something like that work, without running "git clone" (which is
probably the way most users would do it).  And yes, it would be good
to let the user learn from the documentation enough so that she
"knows what she is doing".  But no, I do not think end-user facing
documentation for "git-submodule" subcommand is the way to do that.

That is why I suggested repository-layout as potentially a better
alternative location.

But perhaps I am mis-reading your rationale.

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Henri GEIST]

[-- Attachment #1: Type: text/plain, Size: 2301 bytes --]

Le vendredi 07 mars 2014 à 15:37 -0800, Junio C Hamano a écrit :
> Henri GEIST <geist.henri@laposte.net> writes:
> 
> > This information is technical in nature but has some importance for general users.
> > As this kind of clone have a separate gitdir, you will have a surprise if you
> > copy past the worktree as the gitdir will not come together.
> 
> I am not sure if I understand exactly what you are trying to say.
> Are you saying that you had a submodule at "sub/dir" in your working
> tree, and then "mkdir ../another && cp -R sub/dir ../another" did
> not result in a usable Git working tree in ../another directory?
> 
> It is almost like complaining that "mkdir ../newone && cp -R * ../newone/"
> did not result in a usable git repository in ../newone directory and
> honestly speaking, that sounds borderline insane, I'd have to say.
> 
> Yes, if a user knows what she is doing, she should be able to make
> something like that work, without running "git clone" (which is
> probably the way most users would do it).  And yes, it would be good
> to let the user learn from the documentation enough so that she
> "knows what she is doing".  But no, I do not think end-user facing
> documentation for "git-submodule" subcommand is the way to do that.
> 
> That is why I suggested repository-layout as potentially a better
> alternative location.
> 
> But perhaps I am mis-reading your rationale.
> 
> 

Let me rephrase my example :

To give one of my project to someone else I have copied it on a USB key.
By a simple drag and drop with the mouse.
And I am quite sure I am not alone doing this way.

I have done those kind of things lot of time without any problem.
But that day 'the_project' happened to be a submodule cloned by
'git submodule update' then on the USB key the $GIT_DIR of 'the_project'
was missing.

If 'man git-submodule' have made me aware of the particularities of submodules
clone I had write in a terminal:

git clone the_project /media/usb/the_project

Or at least I had understand what happened quicker.

I have nothing against also adding something in repository-layout but I am
pretty sure normal users never read repository-layout as it is not a command
they use. And it is not mentioned in most tutorials.



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Andrew Keller]

On Mar 7, 2014, at 7:50 PM, Henri GEIST wrote:
> Le vendredi 07 mars 2014 à 15:37 -0800, Junio C Hamano a écrit :
>> Henri GEIST <geist.henri@laposte.net> writes:
>> 
>>> This information is technical in nature but has some importance for general users.
>>> As this kind of clone have a separate gitdir, you will have a surprise if you
>>> copy past the worktree as the gitdir will not come together.
>> 
>> I am not sure if I understand exactly what you are trying to say.
>> Are you saying that you had a submodule at "sub/dir" in your working
>> tree, and then "mkdir ../another && cp -R sub/dir ../another" did
>> not result in a usable Git working tree in ../another directory?
>> 
>> It is almost like complaining that "mkdir ../newone && cp -R * ../newone/"
>> did not result in a usable git repository in ../newone directory and
>> honestly speaking, that sounds borderline insane, I'd have to say.
>> 
>> Yes, if a user knows what she is doing, she should be able to make
>> something like that work, without running "git clone" (which is
>> probably the way most users would do it).  And yes, it would be good
>> to let the user learn from the documentation enough so that she
>> "knows what she is doing".  But no, I do not think end-user facing
>> documentation for "git-submodule" subcommand is the way to do that.
>> 
>> That is why I suggested repository-layout as potentially a better
>> alternative location.
>> 
>> But perhaps I am mis-reading your rationale.
>> 
>> 
> 
> Let me rephrase my example :
> 
> To give one of my project to someone else I have copied it on a USB key.
> By a simple drag and drop with the mouse.
> And I am quite sure I am not alone doing this way.
> 
> I have done those kind of things lot of time without any problem.
> But that day 'the_project' happened to be a submodule cloned by
> 'git submodule update' then on the USB key the $GIT_DIR of 'the_project'
> was missing.
> 
> If 'man git-submodule' have made me aware of the particularities of submodules
> clone I had write in a terminal:
> 
> git clone the_project /media/usb/the_project
> 
> Or at least I had understand what happened quicker.
> 
> I have nothing against also adding something in repository-layout but I am
> pretty sure normal users never read repository-layout as it is not a command
> they use. And it is not mentioned in most tutorials.

How about something like this:

"The git directory of a submodule lives inside the git directory of the parent repository instead of within the working directory."

I'm not sure where to put it, though.

 - Andrew Keller

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Henri GEIST]

[-- Attachment #1: Type: text/plain, Size: 2918 bytes --]

Le dimanche 09 mars 2014 à 19:24 -0400, Andrew Keller a écrit :
> On Mar 7, 2014, at 7:50 PM, Henri GEIST wrote:
> > Le vendredi 07 mars 2014 à 15:37 -0800, Junio C Hamano a écrit :
> >> Henri GEIST <geist.henri@laposte.net> writes:
> >> 
> >>> This information is technical in nature but has some importance for general users.
> >>> As this kind of clone have a separate gitdir, you will have a surprise if you
> >>> copy past the worktree as the gitdir will not come together.
> >> 
> >> I am not sure if I understand exactly what you are trying to say.
> >> Are you saying that you had a submodule at "sub/dir" in your working
> >> tree, and then "mkdir ../another && cp -R sub/dir ../another" did
> >> not result in a usable Git working tree in ../another directory?
> >> 
> >> It is almost like complaining that "mkdir ../newone && cp -R * ../newone/"
> >> did not result in a usable git repository in ../newone directory and
> >> honestly speaking, that sounds borderline insane, I'd have to say.
> >> 
> >> Yes, if a user knows what she is doing, she should be able to make
> >> something like that work, without running "git clone" (which is
> >> probably the way most users would do it).  And yes, it would be good
> >> to let the user learn from the documentation enough so that she
> >> "knows what she is doing".  But no, I do not think end-user facing
> >> documentation for "git-submodule" subcommand is the way to do that.
> >> 
> >> That is why I suggested repository-layout as potentially a better
> >> alternative location.
> >> 
> >> But perhaps I am mis-reading your rationale.
> >> 
> >> 
> > 
> > Let me rephrase my example :
> > 
> > To give one of my project to someone else I have copied it on a USB key.
> > By a simple drag and drop with the mouse.
> > And I am quite sure I am not alone doing this way.
> > 
> > I have done those kind of things lot of time without any problem.
> > But that day 'the_project' happened to be a submodule cloned by
> > 'git submodule update' then on the USB key the $GIT_DIR of 'the_project'
> > was missing.
> > 
> > If 'man git-submodule' have made me aware of the particularities of submodules
> > clone I had write in a terminal:
> > 
> > git clone the_project /media/usb/the_project
> > 
> > Or at least I had understand what happened quicker.
> > 
> > I have nothing against also adding something in repository-layout but I am
> > pretty sure normal users never read repository-layout as it is not a command
> > they use. And it is not mentioned in most tutorials.
> 
> How about something like this:
> 
> "The git directory of a submodule lives inside the git directory of the parent repository instead of within the working directory."
> 
> I'm not sure where to put it, though.
> 
>  - Andrew Keller
> 

'git directory' seems ambiguous to me. Maybe we could use 'git metadata'.



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Junio C Hamano]

Andrew Keller <andrew@kellerfarm.com> writes:

> On Mar 7, 2014, at 7:50 PM, Henri GEIST wrote:
> ...
>> To give one of my project to someone else I have copied it on a USB key.
>> By a simple drag and drop with the mouse.
>> And I am quite sure I am not alone doing this way.
>> 
>> I have done those kind of things lot of time without any problem.
>> But that day 'the_project' happened to be a submodule cloned by
>> 'git submodule update' then on the USB key the $GIT_DIR of 'the_project'
>> was missing.
>> 
>> If 'man git-submodule' have made me aware of the particularities of submodules
>> clone I had write in a terminal:
>> 
>> git clone the_project /media/usb/the_project
>> 
>> Or at least I had understand what happened quicker.
>> 
>> I have nothing against also adding something in repository-layout but I am
>> pretty sure normal users never read repository-layout as it is not a command
>> they use. And it is not mentioned in most tutorials.
>
> How about something like this:
>
> "The git directory of a submodule lives inside the git directory of the parent repository instead of within the working directory."
>
> I'm not sure where to put it, though.

This is not limited to submodules.  There are multiple lower-level
mechanisms for a $path/.git to borrow the repository data from
elsewhere outside of $path and a cloned submodule uses only one of
them.  For any such $path, "cp -R $path $otherplace" will result in
an "$otherplace" that does not work as a Git repository in exactly
the same way, whether it happens to be a submodule checkout or not.

That is why I suggested to enhance description on a more general
part of the documentation that covers what a Git repository is.

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Henri GEIST]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

Le lundi 10 mars 2014 à 08:31 -0700, Junio C Hamano a écrit :
> Andrew Keller <andrew@kellerfarm.com> writes:
> 
> > On Mar 7, 2014, at 7:50 PM, Henri GEIST wrote:
> > ...
> >> To give one of my project to someone else I have copied it on a USB key.
> >> By a simple drag and drop with the mouse.
> >> And I am quite sure I am not alone doing this way.
> >> 
> >> I have done those kind of things lot of time without any problem.
> >> But that day 'the_project' happened to be a submodule cloned by
> >> 'git submodule update' then on the USB key the $GIT_DIR of 'the_project'
> >> was missing.
> >> 
> >> If 'man git-submodule' have made me aware of the particularities of submodules
> >> clone I had write in a terminal:
> >> 
> >> git clone the_project /media/usb/the_project
> >> 
> >> Or at least I had understand what happened quicker.
> >> 
> >> I have nothing against also adding something in repository-layout but I am
> >> pretty sure normal users never read repository-layout as it is not a command
> >> they use. And it is not mentioned in most tutorials.
> >
> > How about something like this:
> >
> > "The git directory of a submodule lives inside the git directory of the parent repository instead of within the working directory."
> >
> > I'm not sure where to put it, though.
> 
> This is not limited to submodules.  There are multiple lower-level
> mechanisms for a $path/.git to borrow the repository data from
> elsewhere outside of $path and a cloned submodule uses only one of
> them.  For any such $path, "cp -R $path $otherplace" will result in
> an "$otherplace" that does not work as a Git repository in exactly
> the same way, whether it happens to be a submodule checkout or not.
> 
> That is why I suggested to enhance description on a more general
> part of the documentation that covers what a Git repository is.

You are entirely right.
My view of the situation was biased by the fact that submodule cloning
is the only case I am aware of except when the user manually type
'--separate-git-dir' where this problem could occur.

If there is some other situation where this can occur as a side effect
of a git command it can be good to have the user aware of the list or
at least inform them in general case a git repository cannot be copied
in a place every body will see.
Or place a note in the manpage of every git command which can have this
side effect.



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [PATCH/RFC] Documentation: Say that submodule clones use a separate gitdirs.

[Junio C Hamano]

Henri GEIST <geist.henri@laposte.net> writes:

> Le lundi 10 mars 2014 à 08:31 -0700, Junio C Hamano a écrit :
> ...
>> This is not limited to submodules.  There are multiple lower-level
>> mechanisms for a $path/.git to borrow the repository data from
>> elsewhere outside of $path and a cloned submodule uses only one of
>> them.  For any such $path, "cp -R $path $otherplace" will result in
>> an "$otherplace" that does not work as a Git repository in exactly
>> the same way, whether it happens to be a submodule checkout or not.
>> 
>> That is why I suggested to enhance description on a more general
>> part of the documentation that covers what a Git repository is.
> ...
> If there is some other situation where this can occur as a side effect
> of a git command it can be good to have the user aware of the list or
> at least inform them in general case a git repository cannot be copied
> in a place every body will see.
> Or place a note in the manpage of every git command which can have this
> side effect.

I think we should do two things:

 - In the repository format document, state that there are two
   lower-level mechanisms and a half that lets a repository borrow
   from somewhere else, and "cp -R" of the former will not result in
   a complete, usable repository, and who employs these mechanisms.

   * Redirecting the entire .git via the textual gitfile mechanism,
     which is used by "clone --separate-git-dir" and "submodule";

   * Borrowing .git/objects read-only from elsewhere, overlaying our
     own, via .git/objects/info/alternates, which is used by "clone
     --reference";

   * Redirecting some paths in .git to another, via "git workdir";
     soon to be replaced with .git/commondir mechansim.

 - In each of the documentation page on an end-user facing command
   that will be mentioned above, add "See also" reference to the
   above description in the repository format document.

   We could elaborate the "See also" as something like "If you use
   this feature, do not think you can "cp -R" the repository to
   elsewhere and expect the copy to function; see also ...", if we
   wanted to.