Git Submodules ref setting

Git Submodules ref setting

[Вадим Цветков]

Hello,

I've started a small project which involves several git repositories, which depends on each other.
For dealing with these repos I wanted to use git submodules as a simple package manager.
However, it seems impossible to lock a submodule to particular ref, only to a branch.
I would like to ask if this is deliberate design choice?
And if it's not, may I contribute this feature?

Re: Git Submodules ref setting

[Philip Oakley]

On 12/05/2022 21:01, Вадим Цветков wrote:
> Hello,
>
> I've started a small project which involves several git repositories, which depends on each other.
> For dealing with these repos I wanted to use git submodules as a simple package manager.
> However, it seems impossible to lock a submodule to particular ref, only to a branch.
> I would like to ask if this is deliberate design choice?

This may be a confusion about how some large projects use sub-modules 
and are now able to follow the developments within a sub-module branch.

The original sub-module mechanism is still available where the 
sub-module is locked at a specific commit object id's hash i.e. a 
specific 'library' version of that sub-module (in that context assumed 
to be rarely changing).
> And if it's not, may I contribute this feature?
--
Philip

Re: Git Submodules ref setting

[Philippe Blain]

Hi, 

Le 2022-05-12 à 16:01, Вадим Цветков a écrit :
> Hello,
> 
> I've started a small project which involves several git repositories, which depends on each other.
> For dealing with these repos I wanted to use git submodules as a simple package manager.
> However, it seems impossible to lock a submodule to particular ref, only to a branch.

Submodules are recorded in the superproject *at a specific commit*.
That's the data model. There is no other options.

There is a 'branch' setting that you can put in '.gitmodules'.
This *only* affects the command 'git submodule update --remote', nothing else.

If you mean that you would like to have a non-branch ref recorded in '.gitmodules'
i.e. something like refs/other-refs/example instead of 'some-branch' that corresponds
to 'refs/heads/some-branch', then no this is not possible either.

> I would like to ask if this is deliberate design choice?

Yes, it was a deliberate design choice to have a deterministic state
of a repository using submodules upon 'git clone'. Recording a submodule
at a specific branch instead of at a specific commit would make this design
choice impossible to achieve.

> And if it's not, may I contribute this feature?
> 

It depends on what you mean by 'lock submodule to a specific ref'.

I encourage you to read the "Git Submodules" chapter of the Pro Git book [1]
for an in-depth overview of how submodules work. And after that, for reference
the Git documentation:

- git-submodule(1): https://git-scm.com/docs/git-submodule
- gitsubmodules(7): https://git-scm.com/docs/gitsubmodules
- gitmodules(5): https://git-scm.com/docs/gitmodules

I hope that helps,

Philippe.

[1] https://git-scm.com/book/en/v2/Git-Tools-Submodules

Re: Git Submodules ref setting

[Philip Oakley]

On 12/05/2022 23:42, Philippe Blain wrote:
> Hi,
>
> Le 2022-05-12 à 16:01, Вадим Цветков a écrit :
>> Hello,
>>
>> I've started a small project which involves several git repositories, which depends on each other.
>> For dealing with these repos I wanted to use git submodules as a simple package manager.
>> However, it seems impossible to lock a submodule to particular ref, only to a branch.
> Submodules are recorded in the superproject *at a specific commit*.
> That's the data model. There is no other options.
>
> There is a 'branch' setting that you can put in '.gitmodules'.
> This *only* affects the command 'git submodule update --remote', nothing else.
>
> If you mean that you would like to have a non-branch ref recorded in '.gitmodules'
> i.e. something like refs/other-refs/example instead of 'some-branch' that corresponds
> to 'refs/heads/some-branch', then no this is not possible either.
>
>> I would like to ask if this is deliberate design choice?
> Yes, it was a deliberate design choice to have a deterministic state
> of a repository using submodules upon 'git clone'. Recording a submodule
> at a specific branch instead of at a specific commit would make this design
> choice impossible to achieve.
>
>> And if it's not, may I contribute this feature?
>>
> It depends on what you mean by 'lock submodule to a specific ref'.
>
> I encourage you to read the "Git Submodules" chapter of the Pro Git book [1]
> for an in-depth overview of how submodules work. And after that, for reference
> the Git documentation:

I didn't find the man page, or the scm chapter that great in explaining 
the overall way that sub-modules are conceived and organised.

There is an implicit assumption in the man pages that folks, in general, 
already know the background and concepts, and are just looking for the 
particular command option details. I.e that it is a reference manual. 
There has been a lot of work done over the years on sub-modules but, to 
me, the concentration has been on _implementation_ (which is obviously 
important), rather than _explanation_ of how sub-modules work and the 
newer capabilities mesh into the existing ones.

As seen here, the existing usage as a place holder for libraries at 
exact versions has been 'lost' in the flurry of newer capabilities. It 
maybe something to add to the wider UX discussions [A].
>
> - git-submodule(1): https://git-scm.com/docs/git-submodule
> - gitsubmodules(7): https://git-scm.com/docs/gitsubmodules
> - gitmodules(5): https://git-scm.com/docs/gitmodules
>
> I hope that helps,
>
> Philippe.
>
> [1] https://git-scm.com/book/en/v2/Git-Tools-Submodules
Philip

[A] 
https://lore.kernel.org/git/CA+Yb-VSaeKy-g_ywkZzQuEX=k3EXM+Ky-rHOb2az0SHGVbdaVw@mail.gmail.com/

Re: Git Submodules ref setting

[Junio C Hamano]

Philippe Blain <levraiphilippeblain@gmail.com> writes:

> Yes, it was a deliberate design choice to have a deterministic state
> of a repository using submodules upon 'git clone'. Recording a submodule
> at a specific branch instead of at a specific commit would make this design
> choice impossible to achieve.

The above sentences are all correct, but it answers a slightly
different question, I think.

I suspect that Vadim wanted to see if it is desirable to point at a
non-branch ref with "submodule.<name>.branch" entry (or invent a new
"submodule.<name>.ref" entry) in the ".gitmodules" file.

While the answer to that question is "probably not", to understand
why, your answer helps ;-).

The "submodule.<name>.branch" entry hints the name of the branch to
be used in the named submodule while working on submodule changes
meant to go with the change that will be made to the superproject.

It is expected that the workflow goes like this:

 * A contributor checks out branch X of the superproject, which
   binds another project whose name is N as its submodule at path P.
   In the tree of the commit at the tip of branch X in the
   superproject, a concrete commit C in the submodule is recorded
   at that path.

 * The .gitmodules file found in that superproject checkout may
   record submodule.N.branch with value Y.  The commit C should be
   checked out in the submodule.

 * The contributor works on something that involves both the
   superproject and the submodule.  Perhaps the new feature in the
   superproject requires an enhanced version of the submodule, so
   there is an dependency.  New commits are added on top of C in the
   submodule, and branch Y is advanced.  The commit at the tip of
   that branch is recorded in the superproject's tree, and together
   with the changes in the superproject that use the modified
   submodule advance the branch X of the superproject.

 * The contributor shares the work with other participants by
   pushing branch X of the superproject and branch Y of the
   submodule.  There may be other branches that may be pushed out
   "while at it", but at least these two need to be pushed out.

For this workflow to work, it is necessary, as you said, that an
exact commit is recorded in the tree of the superproject.  We can
guarantee a correct build of the superproject at commit at the
updated tip of branch X, because we know which submodule commit
needs to go with it.

But it is not sufficient.  The history in the submodule needs to be
manageable---the contributors who cloned the superproject and the
submodule need to check out a commit to work on and extend their
histories, and while it is possible to work always on detached HEAD,
at the place where contributors meet with their work, i.e. the
repository their changes were pushed in the last step in the above
sequence, there needs _some_ ref.  It is possible to use a
non-branch ref for that, but because the contributors have to check
that ref out after cloning and fetching the submodule, it makes
little sense not to require that ref to be a proper branch that can
be checked out (instead of merely giving a starting point of an
unnamed branch, aka detached HEAD).

And that is why "remote.<name>.branch" is there and it is not
"remote.<name>.ref".