Relative submodule URLs, and forks that haven't forked the submodule

Relative submodule URLs, and forks that haven't forked the submodule

[Charles Brossollet]

Hi,

I'm banging my head on this problem: I have a central repo cloned by SSH, and a fork on the same server. The central remote is origin, and the fork is chbrosso-wip.

$ git remote -v | grep origin
origin  chbrosso@lltech:/git/lightct.git (fetch)
origin  chbrosso@lltech:/git/lightct.git (push)

$ git remote -v | grep chbrosso-wip
chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (fetch)
chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (push)

On a local working copy, fetched my fork and checked out a remote branch out of it. Its remote-tracking branch is on the fork.

$ git branch -vv | grep \*
* actor                         d98ec24 [chbrosso-wip/actor] (commit msg)

Now, submodules for this repo have relative URLs. And this is where the problem begins, because the submodule isn't forked, but resides only in origin.

But this shouldn't cause any problem, right? The docs says that if relative URL are used, they resolve using the origin URL. First issue, it's not the case:

$ cat .gitmodules
[submodule "motors"]
        path = motors
        url = ../motors.git
        branch = master
$ git submodule init motors
Submodule 'motors' (chbrosso@lltech:~/prog/git/motors.git) registered for path 'motors'

Here the submodule is registered on my fork, which doesn't exist, and it's wrong with what the documentation says.

Fine, I'll edit the .git/config entry to make it point to origin:

$ git config submodule.motors.url chbrosso@lltech:/git/motors.git

$ git config submodule.motors.url
chbrosso@lltech:/git/motors.git

$ ssh chbrosso@lltech "if  [ -d /git/motors.git ]; then echo 'ok'; fi"
Password:
ok

So the submodule's url is changed, and points to a correct path, let's update so that I can work

$ git submodule update motors
Password:
fatal: '~/prog/git/motors.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.
Unable to fetch in submodule path 'motors'

That's right, it is still the old url, and I can't have my submodule!
Can someone explain what's going on? And how can I get my submodule in the working copy?

git version 1.9.2.msysgit.0 on Windows 7 SP1 64 bit

Thanks, 
—- 
Charles

Re: Relative submodule URLs, and forks that haven't forked the submodule

[Fredrik Gustafsson]

So let me see if I understand you correctly.


On Wed, Jun 11, 2014 at 12:15:39PM +0200, Charles Brossollet wrote:
> Hi,
> 
> I'm banging my head on this problem: I have a central repo cloned by SSH, and a fork on the same server. The central remote is origin, and the fork is chbrosso-wip.
> 
> $ git remote -v | grep origin
> origin  chbrosso@lltech:/git/lightct.git (fetch)
> origin  chbrosso@lltech:/git/lightct.git (push)
> 
> $ git remote -v | grep chbrosso-wip
> chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (fetch)
> chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (push)
> 
> On a local working copy, fetched my fork and checked out a remote branch out of it. Its remote-tracking branch is on the fork.
> 
> $ git branch -vv | grep \*
> * actor                         d98ec24 [chbrosso-wip/actor] (commit msg)
> 
> Now, submodules for this repo have relative URLs. And this is where the problem begins, because the submodule isn't forked, but resides only in origin.

Fork is not a git thing. It's not a git command and it's not supported
by git. You can of course easily do a "fork" of a git project, but git
will be unaware of it beeing a fork.

What you're saying is that you've one repository:

lightct.git and one other repository which is a submodule to lightct.git
at motors.git. Then you've made a copy of lightct.git to an other place
for example: /some/other/path/lightct.git and the naturally the
submodule path that's relative will point to /some/other/path/motors.git
that doesn't exists, since you haven't copied motors.git

> 
> But this shouldn't cause any problem, right? The docs says that if relative URL are used, they resolve using the origin URL. First issue, it's not the case:

Orgin refers to the repository you cloned from. That is if you did
git clone lightct.git my_working_copy

the origin for my_working_copy would be lightct.git. However if you did
git clone /some/other/path/lightct.git my_working_copy

the origin for my_working_copy would be /some/other/path/lightct.git

So to me it seems to be correct.

> 
> $ cat .gitmodules
> [submodule "motors"]
>         path = motors
>         url = ../motors.git
>         branch = master
> $ git submodule init motors
> Submodule 'motors' (chbrosso@lltech:~/prog/git/motors.git) registered for path 'motors'
> 
> Here the submodule is registered on my fork, which doesn't exist, and it's wrong with what the documentation says.
> 
> Fine, I'll edit the .git/config entry to make it point to origin:
> 
> $ git config submodule.motors.url chbrosso@lltech:/git/motors.git
> 
> $ git config submodule.motors.url
> chbrosso@lltech:/git/motors.git
> 
> $ ssh chbrosso@lltech "if  [ -d /git/motors.git ]; then echo 'ok'; fi"
> Password:
> ok
> 
> So the submodule's url is changed, and points to a correct path, let's update so that I can work
> 
> $ git submodule update motors
> Password:
> fatal: '~/prog/git/motors.git' does not appear to be a git repository
> fatal: Could not read from remote repository.
> 
> Please make sure you have the correct access rights and the repository exists.
> Unable to fetch in submodule path 'motors'
> 
> That's right, it is still the old url, and I can't have my submodule!

Here you change the path to the submodule at
/some/other/path/lightct.git and then it isn't changed in my_working_copy. How could it? They don't communicate if you don't tell them to.

> Can someone explain what's going on? And how can I get my submodule in the working copy?

Either created a copy of the submodule just as you did with lightct.git
or use non-relative paths.

-- 
Med vänlig hälsning
Fredrik Gustafsson

tel: 0733-608274
e-post: iveqy@iveqy.com

Re: Relative submodule URLs, and forks that haven't forked the submodule

[Charles Brossollet]

Thanks for taking time to understand, let me make it more clear

Le 12 juin 2014 à 17:25, Fredrik Gustafsson <iveqy@iveqy.com> a écrit :

> So let me see if I understand you correctly.
> 
> 
> On Wed, Jun 11, 2014 at 12:15:39PM +0200, Charles Brossollet wrote:
>> Hi,
>> 
>> I'm banging my head on this problem: I have a central repo cloned by SSH, and a fork on the same server. The central remote is origin, and the fork is chbrosso-wip.
>> 
>> $ git remote -v | grep origin
>> origin  chbrosso@lltech:/git/lightct.git (fetch)
>> origin  chbrosso@lltech:/git/lightct.git (push)
>> 
>> $ git remote -v | grep chbrosso-wip
>> chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (fetch)
>> chbrosso-wip    chbrosso@lltech:~/prog/git/lightct.git (push)
>> 
>> On a local working copy, fetched my fork and checked out a remote branch out of it. Its remote-tracking branch is on the fork.
>> 
>> $ git branch -vv | grep \*
>> * actor                         d98ec24 [chbrosso-wip/actor] (commit msg)
>> 
>> Now, submodules for this repo have relative URLs. And this is where the problem begins, because the submodule isn't forked, but resides only in origin.
> 
> Fork is not a git thing. It's not a git command and it's not supported
> by git. You can of course easily do a "fork" of a git project, but git
> will be unaware of it beeing a fork.

OK, you get it, what I mean by fork here is an independent copy of a repository, at another remote place. 

> What you're saying is that you've one repository:
> 
> lightct.git and one other repository which is a submodule to lightct.git
> at motors.git. Then you've made a copy of lightct.git to an other place
> for example: /some/other/path/lightct.git and the naturally the
> submodule path that's relative will point to /some/other/path/motors.git
> that doesn't exists, since you haven't copied motors.git

That's right. Origin is the repository that were original cloned to the working copy, and I have a copy of it, that is in /some/other/path, without motors.git having been copied.

I haven't copied motors.git because I won't modify it, so I still want to refer it… 

>> But this shouldn't cause any problem, right? The docs says that if relative URL are used, they resolve using the origin URL. First issue, it's not the case:
> 
> Orgin refers to the repository you cloned from. That is if you did
> git clone lightct.git my_working_copy
> 
> the origin for my_working_copy would be lightct.git. However if you did
> git clone /some/other/path/lightct.git my_working_copy
> 
> the origin for my_working_copy would be /some/other/path/lightct.git
> 
> So to me it seems to be correct.

No, in the working copy, origin's location isn't changed, it is still the repository I originally (!) cloned from. I added the other remote afterward, and named it chbrosso-wip, not origin. Then, the working copy has two remotes, origin and chbrosso-wip. So if we follow the docs the URL for the submodule shouldn't be set to chbrosso-wip's URL, but this is what is happening.

>>  <snip>
>> That's right, it is still the old url, and I can't have my submodule!
> 
> Here you change the path to the submodule at
> /some/other/path/lightct.git and then it isn't changed in my_working_copy. How could it? They don't communicate if you don't tell them to.

No, you missed my point, let me explain it a more synthesized way:

There are 3 repos main, fork, and sub, having the following URLs:

/central/main
/central/sub
/user/main

sub is a submodule of main, and referred with a relative URL in .gitmodules.

In a working copy, cloned from /central/main, thus referred by git as origin, and added /user/main as another remote repository. Fetched from it.

Initially the submodule isn't cloned in the working copy.

The two problems I'm pointing are:

1. After checkout of a branch that tracks /user/main repo, call git init submodule motors. Git registers it in .git/config with URL /user/sub, while it should be /central/sub according to documentation because origin's URL is at /central.

2. For an obscure reason, changing the url in .git/config to /central/sub and call git submodule update still make git want to clone from /user/sub, and fails. There seems to be no way to tell git the right URL for this submodule, while it should be possible according to the submodule documentation.

> 
>> Can someone explain what's going on? And how can I get my submodule in the working copy?
> 
> Either created a copy of the submodule just as you did with lightct.git
> or use non-relative paths.
> 
> -- 
> Med vänlig hälsning
> Fredrik Gustafsson
> 
> tel: 0733-608274
> e-post: iveqy@iveqy.com

Re: Re: Relative submodule URLs, and forks that haven't forked the submodule

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

On Thu, Jun 12, 2014 at 06:05:10PM +0200, Charles Brossollet wrote:
> The two problems I'm pointing are:
> 
> 1. After checkout of a branch that tracks /user/main repo, call git
>    init submodule motors. Git registers it in .git/config with URL
>    /user/sub, while it should be /central/sub according to
>    documentation because origin's URL is at /central.

The logic for this is in resolve_relative_url, defined in
git-submodule.sh.  The remote it uses is calculated using
get_default_remote, defined in git-parse-remote.sh:

  get_default_remote () {
    curr_branch=$(git symbolic-ref -q HEAD)
    curr_branch="${curr_branch#refs/heads/}"
    origin=$(git config --get "branch.$curr_branch.remote")
    echo ${origin:-origin}
  }

> 2. For an obscure reason, changing the url in .git/config to
> /central/sub and call git submodule update still make git want to
> clone from /user/sub, and fails. There seems to be no way to tell
> git the right URL for this submodule, while it should be possible
> according to the submodule documentation.

This is very surprising to me.  With Git v1.9.1:

* Clone just the superproject, without it's sibling submodule projects:

  $ git clone git://github.com/wking/pygrader.git pg-1

* Clone the isolated superproject, so we'll have broken relative URLs:

  $ git clone pg-1 pg-2

* Initialize a submodule:

  $ git submodule init dep/src/pyassuan

* Fix the broken, expanded-from-relative URL to point back to the
  original:

  $ git config submodule.dep/src/pyassuan.url git://github.com/wking/pyassuan.git

* Initial, cloning update:

  $ git submodule update

That all works as expected for me.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]