issue with git submodules and a clone.defaultRemoteName different than origin?

issue with git submodules and a clone.defaultRemoteName different than origin?

[Jacob Keller]

Hi,

I recently ran into an issue with git submodules because my
clone.defaultRemoteName is not "origin":

> fatal: 'origin' does not appear to be a git repository
> fatal: Could not read from remote repository.
> 
> Please make sure you have the correct access rights
> and the repository exists.
> fatal: Fetched in submodule path 'submodule', but it did not contain <redacted>. Direct fetching of that commit failed.


This appears to be due to the builtin/submodule--helper hard coding
"origin" in "repo_get_default_remote".

I am unsure what the best way to fix this is. I could have the function
read the clone.defaultRemoteName, or I could have it check if there is
only one remote then use that.. or maybe add a new submodule remote name
option?

Thoughts on what the best solution is here?

I'm thinking the following:

1) check if there is only one remote, then use that
2) check clone.defaultRemoteName and use that otherwise
3) fall back to origin otherwise?

Perhaps we could insert a step 0 where we add a config option which will
have submodule clones use the given remote name + use that as the
default when in detached head state?

Thanks,
Jake

Re: issue with git submodules and a clone.defaultRemoteName different than origin?

[Junio C Hamano]

Jacob Keller <jacob.e.keller@intel.com> writes:

>> fatal: 'origin' does not appear to be a git repository
>> fatal: Could not read from remote repository.
>>  ...
> This appears to be due to the builtin/submodule--helper hard coding
> "origin" in "repo_get_default_remote".
>
> I am unsure what the best way to fix this is. I could have the function
> read the clone.defaultRemoteName, or I could have it check if there is
> only one remote then use that.. or maybe add a new submodule remote name
> option?

Is it is the nickname of the default remote that corresponds to
'origin' in other/most peoples' set-up you are using for your
submodules that is causing you?  And because you have
clone.defaultRemoteName configured either in the superproject's
.git/config or your personal ~/.gitconfig, when you activate
submodules, the "git clone" used to populate your submodule
directory from elsewhere would use that custom name?

What I am trying to figure out by thinking aloud is if there a place
where that custom name name is recorded anywhere in the submodule
repository (or superproject, but I somehow doubt it).

The clone.defaultRemoteName configuration variable can be overriden
from the command line of "git clone", and even if the process to
activate a submodule does not let you pass the "--origin", you could
have updated your clone.defaultRemoteName, so the current value of
the configuration variable is pretty much useless.  

Your "if there is only a single remote" would probably be a better
way.  The only reason you are having trouble, if I am reading the
repo_get_default_remote(), is because you are on a detached HEAD.
Do we know where that HEAD was detached from, perhaps from the
reflog?  If we are on a concrete branch, there already is a logic to
figure out what branch from what remote it integrates with, and that
would give you the most reliable answer.

> Thoughts on what the best solution is here?
>
> I'm thinking the following:
>
> 1) check if there is only one remote, then use that
> 2) check clone.defaultRemoteName and use that otherwise
> 3) fall back to origin otherwise?

I'd insert 1.5 to figure out where your detached HEAD came from and
use that as if you are on that branch.  That is the source of the
problem, right?

> Perhaps we could insert a step 0 where we add a config option which will
> have submodule clones use the given remote name + use that as the
> default when in detached head state?

Going forward, it would of course be the most reliable if we wrote a
distinct configuration variable in submodule repository when we
activate it, i.e. when "git submodule update --init" clones, it can
record the nickname that was used, so that none of the 1-3) above
methods have to be used to guess what the name is.

Or, perhaps we can update "git submodule update --init" so that when
it clones, it ignores clone.defaultRemoteName configuration, so that
this codepath always can rely on the name being 'origin'.  If you
are always accessing the submodule through the toplevel superproject,
the name used in the submodule does not make a difference, no?

Re: issue with git submodules and a clone.defaultRemoteName different than origin?

[Jacob Keller]

On 6/3/2025 4:42 PM, Junio C Hamano wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
> 
>>> fatal: 'origin' does not appear to be a git repository
>>> fatal: Could not read from remote repository.
>>>  ...
>> This appears to be due to the builtin/submodule--helper hard coding
>> "origin" in "repo_get_default_remote".
>>
>> I am unsure what the best way to fix this is. I could have the function
>> read the clone.defaultRemoteName, or I could have it check if there is
>> only one remote then use that.. or maybe add a new submodule remote name
>> option?
> 
> Is it is the nickname of the default remote that corresponds to
> 'origin' in other/most peoples' set-up you are using for your
> submodules that is causing you?  And because you have
> clone.defaultRemoteName configured either in the superproject's
> .git/config or your personal ~/.gitconfig, when you activate
> submodules, the "git clone" used to populate your submodule
> directory from elsewhere would use that custom name?
> 

In short: I set clone.defaultRemoteName in my .gitconfig to "upstream"
because that fits more naturally for most of my repositories at work.

Because of this, when a submodule is initialized, it is cloned with a
remote named "upstream" instead of "origin". (Either its cloned this
way, or I later renamed it to this as in some cases)

By default "git submodule update" checks out submodules to a detached
HEAD state.

When git is trying to find the default remote for a submodule, it will
be in this detached HEAD state, so the fallback to "origin" fails
because origin isn't a valid remote on my setup.

> What I am trying to figure out by thinking aloud is if there a place
> where that custom name name is recorded anywhere in the submodule
> repository (or superproject, but I somehow doubt it).
> 

There isn't, other than the fact that there is a remote with that name
but not one named "origin".

We could possibly add a config option which we could have the
superproject set when cloning to specify which remote is the one marked
in the parents .gitmodules?

Alternatively, we could have the parent record the remote name in its
.gitmodules.. but that seems a bit odd since a user could later rename
that remote.

> The clone.defaultRemoteName configuration variable can be overriden
> from the command line of "git clone", and even if the process to
> activate a submodule does not let you pass the "--origin", you could
> have updated your clone.defaultRemoteName, so the current value of
> the configuration variable is pretty much useless.  
> 

My idea was that its better to fall back to this as a default because
its an indication that the default is not "origin".

> Your "if there is only a single remote" would probably be a better
> way.  The only reason you are having trouble, if I am reading the
> repo_get_default_remote(), is because you are on a detached HEAD.
> Do we know where that HEAD was detached from, perhaps from the
> reflog?  If we are on a concrete branch, there already is a logic to
> figure out what branch from what remote it integrates with, and that
> would give you the most reliable answer.


It comes from the way git submodule update checks out the requested
commit in a detached HEAD state.

> 
>> Thoughts on what the best solution is here?
>>
>> I'm thinking the following:
>>
>> 1) check if there is only one remote, then use that
>> 2) check clone.defaultRemoteName and use that otherwise
>> 3) fall back to origin otherwise?
> 
> I'd insert 1.5 to figure out where your detached HEAD came from and
> use that as if you are on that branch.  That is the source of the
> problem, right?
> 

The issue is that the detached HEAD comes from the submodule update, and
may not actually be on any local branch. The parent project likely knows
based on its git modules, and we could actually maybe just look up the
remote based on its URL actually...

>> Perhaps we could insert a step 0 where we add a config option which will
>> have submodule clones use the given remote name + use that as the
>> default when in detached head state?
> 
> Going forward, it would of course be the most reliable if we wrote a
> distinct configuration variable in submodule repository when we
> activate it, i.e. when "git submodule update --init" clones, it can
> record the nickname that was used, so that none of the 1-3) above
> methods have to be used to guess what the name is.
> 

Yea, this is a good idea.

> Or, perhaps we can update "git submodule update --init" so that when
> it clones, it ignores clone.defaultRemoteName configuration, so that
> this codepath always can rely on the name being 'origin'.  If you
> are always accessing the submodule through the toplevel superproject,
> the name used in the submodule does not make a difference, no?
> 

True, but I often need to make PRs for projects from a submodule, and in
that case the github CLI likes to rename origin as "upstream" and make
"origin" be my fork. (Though I personally use "upstream" for the
upstream project and "fork" for the fork.) In both case, origin no
longer exists.

However, I had another thought while reading this:

the parent project already has a URL in its config. What if we updated
the logic to just directly use that URL instead of using a remote name?
Or at the very least, we try to pick a remote based on the URL.

Re: issue with git submodules and a clone.defaultRemoteName different than origin?

[Jacob Keller]

On 6/4/2025 10:18 AM, Jacob Keller wrote:
> 
> the parent project already has a URL in its config. What if we updated
> the logic to just directly use that URL instead of using a remote name?
> Or at the very least, we try to pick a remote based on the URL.
> 

I'm looking into this approach still, but I ran into some interesting
issues with the remote.c logic which claims to work with any "struct
repository *", but ends up calling paths to that hard code the_repository.

It looks like those end up in the "read_remotes_file" and
"read_branches_file" functions which are deprecated, and planned to be
removed in 3.0...

Would patches to modify those to take a repository pointer in order to
allow callers of read_config() to work properly with a submodule
repository be acceptable?

I also saw that many functions take a remote_state structure, but if I
wanted to make them submodule repo compatible I would need to either add
or switch to passing a repo pointer. I'm not sure what the best method
for that is. Add parameters? We can get from repo to remote_state easily
but we can't go back, so it seems like just switching them to take repo
instead of remote_state would be sufficient.

I wanted to use the remote.c helpers rather than re-inventing the logic
for reading the config to figure out remotes, both a) to find the
correct remote by its URL and b) to find out if there is exactly 1 remote.

Re: issue with git submodules and a clone.defaultRemoteName different than origin?

[Junio C Hamano]

Jacob Keller <jacob.e.keller@intel.com> writes:

> It looks like those end up in the "read_remotes_file" and
> "read_branches_file" functions which are deprecated, and planned to be
> removed in 3.0...
>
> Would patches to modify those to take a repository pointer in order to
> allow callers of read_config() to work properly with a submodule
> repository be acceptable?

The recent trend has been that it generally is very much welcomed to
fix such a code path that takes "repo" at an entry point but ends up
deep in its callchain to ignore it and use the_repository instead,
even without a potential use case to benefit from such a change.

You have a concrete use case, so I would suspect it is even better
than "acceptable".  Patrick, what do you think?


Thanks.

Re: issue with git submodules and a clone.defaultRemoteName different than origin?

[Patrick Steinhardt]

On Fri, Jun 06, 2025 at 04:32:21PM -0700, Junio C Hamano wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
> 
> > It looks like those end up in the "read_remotes_file" and
> > "read_branches_file" functions which are deprecated, and planned to be
> > removed in 3.0...
> >
> > Would patches to modify those to take a repository pointer in order to
> > allow callers of read_config() to work properly with a submodule
> > repository be acceptable?
> 
> The recent trend has been that it generally is very much welcomed to
> fix such a code path that takes "repo" at an entry point but ends up
> deep in its callchain to ignore it and use the_repository instead,
> even without a potential use case to benefit from such a change.
> 
> You have a concrete use case, so I would suspect it is even better
> than "acceptable".  Patrick, what do you think?

I definitely agree, yes. It is a long road to get rid of
`the_repository`. Taking many small steps on this road that have a
proper motivation is way easier than doing large steps that get rid of
the variable for the sake of getting rid of it.

Patrick