[RFC] What to you think about a loose status for submodules?

[RFC] What to you think about a loose status for submodules?

[Heiko Voigt]

Hi,

for some time now I have been thinking about submodules which are
not checked out by default. So for example if you have a project which
consist of the submodules:

project/core
       /help
       /app
...

Consider the situation that 'help' is really big because it is a user
friendly application ;)

So you usually do not need the help folder to develop the application
but it still is tied to a certain revision.

For such a workflow I would like to implement what I call 'loose'
submodules. Where a

  git clone project.git
  cd project
  git submodule init && git submodule update

would omit the 'help' folder. But in case I specify it directly like

  git submodule init help

it would update to the recorded revision.

Of course the relation would be configurable. E.g.:

  git config submodule."name".relation loose

and the opposite as

  git config submodule."name".relation tight

Initially the implementation would only deal with the initialization
case. As a second step I would like to deal with the situation whether a
submodule should be shown as changed or not. What do you think about such
an extension?

cheers Heiko

Re: [RFC] What to you think about a loose status for submodules?

[Junio C Hamano]

Heiko Voigt <hvoigt@hvoigt.net> writes:

> For such a workflow I would like to implement what I call 'loose'
> submodules. Where a
>
>   git clone project.git
>   cd project
>   git submodule init && git submodule update
>
> would omit the 'help' folder. But in case I specify it directly like

I thought a blanket "submodule init/update" wasn't even a recommended
practice for this exact reason.  We tried to keep the default not to
gratuitously populate and checkout all submodule repositories, but
probably what you are trying to do was made more difficult by mistake
because people who wanted the other behaviour pushed too hard?

Defaulting to "do not populate and checkout unless explicitly asked"
sounds like the right thing to do, and if we broke it, it should be
corrected, I think.  Shouldn't it be a simple matter of teaching "--all"
option to "submodule init" (and "update") to let them blindly affect all
submodules, while making the default not to do anything?

>   git submodule init help
>
> it would update to the recorded revision.
>
> Of course the relation would be configurable. E.g.:
>
>   git config submodule."name".relation loose
>
> and the opposite as
>
>   git config submodule."name".relation tight

I do not think this should be a project-wide configuration that is
recorded in .gitmodules; if you are "help documentation" participant to
the project you would want "help" submodule, and other people will want
different submodules.

It would probably make more sense to introduce the notion of "module
groups", similar to the way "remote update <group>" can name a group of
remotes to affect.  Then documentation people can say

    submodule init doc && submodule update

if .gitmodules file records the mapping from "doc" to one or more
submodules (e.g. "help" and "doc").  If we are going to take this route,
it would still make sense to teach "--all" to "submodule init" and perhaps
default to init the "default" group if one exists, instead of making the
parameterless "init" a no-op as I suggested earlier.

But it is quite a long time since I looked at git-submodule.sh so please
take the above with a healthy dose of salt.

Re: Re: [RFC] What to you think about a loose status for submodules?

[Heiko Voigt]

On Wed, Oct 21, 2009 at 01:23:19PM -0700, Junio C Hamano wrote:
> Heiko Voigt <hvoigt@hvoigt.net> writes:
> 
> > For such a workflow I would like to implement what I call 'loose'
> > submodules. Where a
> >
> >   git clone project.git
> >   cd project
> >   git submodule init && git submodule update
> >
> > would omit the 'help' folder. But in case I specify it directly like
> 
> I thought a blanket "submodule init/update" wasn't even a recommended
> practice for this exact reason.  We tried to keep the default not to
> gratuitously populate and checkout all submodule repositories, but
> probably what you are trying to do was made more difficult by mistake
> because people who wanted the other behaviour pushed too hard?

Currently if a project has submodule its either use all submodules or
none. There is nothing in between.

> Defaulting to "do not populate and checkout unless explicitly asked"
> sounds like the right thing to do, and if we broke it, it should be
> corrected, I think.  Shouldn't it be a simple matter of teaching "--all"
> option to "submodule init" (and "update") to let them blindly affect all
> submodules, while making the default not to do anything?

Not completely. As said above I want something 'in between' because some
small submodule might be necessary to compile the project. And they are
submodules because they are shared code, so more than one project is
using them. And then there are optional submodules like pugins or
documentation which are not needed for compilation.

> >   git config submodule."name".relation tight
> 
> I do not think this should be a project-wide configuration that is
> recorded in .gitmodules; if you are "help documentation" participant to
> the project you would want "help" submodule, and other people will want
> different submodules.
> 
> It would probably make more sense to introduce the notion of "module
> groups", similar to the way "remote update <group>" can name a group of
> remotes to affect.  Then documentation people can say
> 
>     submodule init doc && submodule update

That makes sense. If I understand it correctly the group definitions
would then be stored in .gitmodules ?

Thinking further we could also name the group definition 'view'. In the
notion of different kind of views for a project. That way a doc
developer would get a set of options which make more sense for him
(thinking about extending this to sparse checkout). But first lets
consider submodules.

> if .gitmodules file records the mapping from "doc" to one or more
> submodules (e.g. "help" and "doc").  If we are going to take this route,
> it would still make sense to teach "--all" to "submodule init" and perhaps
> default to init the "default" group if one exists, instead of making the
> parameterless "init" a no-op as I suggested earlier.

I like this idea.

> But it is quite a long time since I looked at git-submodule.sh so please
> take the above with a healthy dose of salt.

I am not sure what is the best way to create such a group mapping using
our config files though. I would like to allow multiple groups/views to
have the submodules like:

[submodule "doc"]
	path = Documentation
	views = doc

[submodule "help"]
	path = help
	views = doc

[submodule "core"]
	path = core
	views = doc default

[submodule "app"]
	path = app
	views = default

What do you think?

cheers Heiko

Re: [RFC] What to you think about a loose status for submodules?

[Junio C Hamano]

Heiko Voigt <hvoigt@hvoigt.net> writes:

> I am not sure what is the best way to create such a group mapping using
> our config files though. I would like to allow multiple groups/views to
> have the submodules like:
>
> [submodule "doc"]
> 	path = Documentation
> 	views = doc
>
> [submodule "help"]
> 	path = help
> 	views = doc
>
> [submodule "core"]
> 	path = core
> 	views = doc default
>
> [submodule "app"]
> 	path = app
> 	views = default
>
> What do you think?

That, or alternatively:

    [submodule "doc"]
        path = Documentation

    [submodule "help"]
        path = help

    [submodule "core"]
        path = core

    [submodule "app"]
        path = app

    [view "default"]
        modules = core app

    [view "doc"]
        modules = core doc help

Or even (mimicking the way how "git remote" defines remote groups):

    [submodule "doc"]
        path = Documentation

    [submodule "help"]
        path = help

    [submodule "core"]
        path = core

    [submodule "app"]
        path = app

    [submodule]
        default = core app
        doc = core doc help

Re: Re: [RFC] What to you think about a loose status for submodules?

[Heiko Voigt]

On Thu, Oct 22, 2009 at 12:58:04PM -0700, Junio C Hamano wrote:
> Heiko Voigt <hvoigt@hvoigt.net> writes:
> 
> > I am not sure what is the best way to create such a group mapping using
> > our config files though. I would like to allow multiple groups/views to
> > have the submodules like:
> > What do you think?
> 
> That, or alternatively:
> 
>     [submodule "doc"]
>         path = Documentation
> 
>     [submodule "help"]
>         path = help
> 
>     [submodule "core"]
>         path = core
> 
>     [submodule "app"]
>         path = app
> 
>     [view "default"]
>         modules = core app
> 
>     [view "doc"]
>         modules = core doc help

Even though there has not been much more work on this item the idea has
become much clearer for me since the last discussion. Now that sparse
checkout is in master I would like to generalize this idea a little
further.

How about adding this to .gitattributes so it can be used for any
file/directory? E.g.:

.gitattributes:
/core           view=default, doc
/app            view=default
/Documentation  view=doc
*.loc           view=doc, default

which could then  be utilized by

  git checkout --view=doc

for example. If no view is given we default to everything.

Does the current sparse implementation work with submodule entries as
well? Then it could be even more straightforward to implement than the
previous idea.

cheers Heiko

Re: [RFC] What to you think about a loose status for submodules?

[Junio C Hamano]

Heiko Voigt <hvoigt@hvoigt.net> writes:

> Even though there has not been much more work on this item the idea has
> become much clearer for me since the last discussion. Now that sparse
> checkout is in master I would like to generalize this idea a little
> further.

Selective init/update of submodules is a fundamental design element and
the presense of "sparse checkout" does not affect it.

In the remainder of your message

> How about ...

you are talking about a change that would also apply to a project that
does not have submodules at all, and you would want to involve Nguyễn in
the discussion.

I have no strong opinion if attributes is the right vehicle to carry this
information, except for this part:

> Does the current sparse implementation work with submodule entries as
> well? Then it could be even more straightforward to implement than the
> previous idea.

What the project ships as grouping suggestions to cloners is merely a
small part of the information on submodules (others such as URL and name
are already found in .gitmodules, and .gitmodules are designed to be
extensible by allowing other "config"-ish items in it).  Separating only
the grouping info into a different file (I don't particularly care if you
use .gitattributes or something else; as long as it is not .gitmodules,
the argument is the same) is not such a good idea.