Submodule using different repository URLs

Submodule using different repository URLs

[Peter Voss]

Hi,

I want to use the git submodule feature to move part of my code to a  
different repository at github.

The issue is that developers should use different repository URLs for  
the submodule depending on whether they have commit rights or not.

At the beginning I was using the public URL to set-up the submodule:
git submodule add git://github.com/x/mymodule.git mymodule

The issue is that some developers are working behind a firewall that  
blocks the git protocol. These could only use the git@github.com:x/ 
mymodule.git URL to get access.
But other developers can only go through the public URL git:// 
github.com/x/mymodule.git. So whatever I use it won't work for  
everybody.

What's the best way to deal with that? Could I set-up different  
repository URLs for one and the same submodule and use which one is  
appropriate?

Thanks,
--Peter

Re: Submodule using different repository URLs

[Santi Béjar]

2009/7/13 Peter Voss <info@petervoss.org>:
> Hi,
>
> I want to use the git submodule feature to move part of my code to a
> different repository at github.
>
> The issue is that developers should use different repository URLs for the
> submodule depending on whether they have commit rights or not.
>
> At the beginning I was using the public URL to set-up the submodule:
> git submodule add git://github.com/x/mymodule.git mymodule
>
> The issue is that some developers are working behind a firewall that blocks
> the git protocol. These could only use the git@github.com:x/mymodule.git URL
> to get access.
> But other developers can only go through the public URL
> git://github.com/x/mymodule.git. So whatever I use it won't work for
> everybody.
>
> What's the best way to deal with that? Could I set-up different repository
> URLs for one and the same submodule and use which one is appropriate?

After the "git submodule init" you can customize the url. From "man
git-submodule":

init::
        Initialize the submodules, i.e. register each submodule name
        and url found in .gitmodules into .git/config.
        The key used in .git/config is `submodule.$name.url`.
        This command does not alter existing information in .git/config.
        You can then customize the submodule clone URLs in .git/config
        for your local setup and proceed to 'git submodule update';
        you can also just use 'git submodule update --init' without
        the explicit 'init' step if you do not intend to customize
        any submodule locations.

You can also use the config url."<actual url base>".insteadOf = <other
url base>.
See the git-pull manpage for examples.

HTH,
Santi

Re: Submodule using different repository URLs

[Peter Voss]

Hi Santi,

On 13.07.2009, at 12:20, Santi Béjar wrote:

> 2009/7/13 Peter Voss <info@petervoss.org>:
>> Hi,
>>
>> I want to use the git submodule feature to move part of my code to a
>> different repository at github.
>>
>> The issue is that developers should use different repository URLs  
>> for the
>> submodule depending on whether they have commit rights or not.
>>
>> At the beginning I was using the public URL to set-up the submodule:
>> git submodule add git://github.com/x/mymodule.git mymodule
>>
>> The issue is that some developers are working behind a firewall  
>> that blocks
>> the git protocol. These could only use the git@github.com:x/ 
>> mymodule.git URL
>> to get access.
>> But other developers can only go through the public URL
>> git://github.com/x/mymodule.git. So whatever I use it won't work for
>> everybody.
>>
>> What's the best way to deal with that? Could I set-up different  
>> repository
>> URLs for one and the same submodule and use which one is appropriate?
>
> After the "git submodule init" you can customize the url. From "man
> git-submodule":
>
> init::
>        Initialize the submodules, i.e. register each submodule name
>        and url found in .gitmodules into .git/config.
>        The key used in .git/config is `submodule.$name.url`.
>        This command does not alter existing information in .git/ 
> config.
>        You can then customize the submodule clone URLs in .git/config
>        for your local setup and proceed to 'git submodule update';
>        you can also just use 'git submodule update --init' without
>        the explicit 'init' step if you do not intend to customize
>        any submodule locations.
>
> You can also use the config url."<actual url base>".insteadOf = <other
> url base>.
> See the git-pull manpage for examples.

I didn't get the insteadOf config running (maybe I have to update to a  
more recent version of git). But modifying the URL in .git/config  
should be a doable approach for us. In order to add the insteadOf  
section we would have to edit the .git/config file as well, so that  
doesn't seem to be much better.

Thanks for your help,
--Peter

> HTH,
> Santi

Re: Submodule using different repository URLs

[Johan Herland]

On Monday 13 July 2009, Peter Voss wrote:
> Hi,
>
> I want to use the git submodule feature to move part of my code to a
> different repository at github.
>
> The issue is that developers should use different repository URLs for
> the submodule depending on whether they have commit rights or not.
>
> At the beginning I was using the public URL to set-up the submodule:
> git submodule add git://github.com/x/mymodule.git mymodule
>
> The issue is that some developers are working behind a firewall that
> blocks the git protocol. These could only use the git@github.com:x/
> mymodule.git URL to get access.
> But other developers can only go through the public URL git://
> github.com/x/mymodule.git. So whatever I use it won't work for
> everybody.
>
> What's the best way to deal with that? Could I set-up different
> repository URLs for one and the same submodule and use which one is
> appropriate?

You might be able to pull this off using relative submodule URLs. If the 
submodule URLs in .gitmodules are relative (i.e. ../foo.git or 
similar), they will be resolved to absolute URLs using the origin URL 
of the super-repo. I.e. if you cloned the super-repo from 
git://github.com/x/mymodule.git, the ../foo.git submodule will be 
cloned from git://github.com/x/foo.git, and if you cloned from 
git@github.com:x/mymodule.git, the submodule will be cloned from 
git@github.com:x/foo.git.

Hope this helps,


Have fun! :)

...Johan


-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Submodule using different repository URLs

[Peter Voss]

Hi Johan,

On 13.07.2009, at 12:59, Johan Herland wrote:

> On Monday 13 July 2009, Peter Voss wrote:
>> Hi,
>>
>> I want to use the git submodule feature to move part of my code to a
>> different repository at github.
>>
>> The issue is that developers should use different repository URLs for
>> the submodule depending on whether they have commit rights or not.
>>
>> At the beginning I was using the public URL to set-up the submodule:
>> git submodule add git://github.com/x/mymodule.git mymodule
>>
>> The issue is that some developers are working behind a firewall that
>> blocks the git protocol. These could only use the git@github.com:x/
>> mymodule.git URL to get access.
>> But other developers can only go through the public URL git://
>> github.com/x/mymodule.git. So whatever I use it won't work for
>> everybody.
>>
>> What's the best way to deal with that? Could I set-up different
>> repository URLs for one and the same submodule and use which one is
>> appropriate?
>
> You might be able to pull this off using relative submodule URLs. If  
> the
> submodule URLs in .gitmodules are relative (i.e. ../foo.git or
> similar), they will be resolved to absolute URLs using the origin URL
> of the super-repo. I.e. if you cloned the super-repo from
> git://github.com/x/mymodule.git, the ../foo.git submodule will be
> cloned from git://github.com/x/foo.git, and if you cloned from
> git@github.com:x/mymodule.git, the submodule will be cloned from
> git@github.com:x/foo.git.

That's a good hint. Unfortunately I can't use this to go up 2  
directories. I.e. I can't get from
git@github.com:xxx/mymodule.git
to
git@github.com:yyy/foo.git

Using the relative URL ../../yyy/foo.git leads to the result:
Clone of 'git@github.com:xxx/yyy/foo.git' into submodule path 'foo'  
failed

So I basically cannot replace the xxx part.

Anyway, thanks for the hint.
--Peter

> Hope this helps,
>
>
> Have fun! :)
>
> ...Johan
>
>
> -- 
> Johan Herland, <johan@herland.net>
> www.herland.net

Re: Submodule using different repository URLs

[Johan Herland]

On Monday 13 July 2009, Peter Voss wrote:
> On 13.07.2009, at 12:59, Johan Herland wrote:
> > You might be able to pull this off using relative submodule URLs.
>
> That's a good hint. Unfortunately I can't use this to go up 2
> directories. I.e. I can't get from
> git@github.com:xxx/mymodule.git
> to
> git@github.com:yyy/foo.git
>
> Using the relative URL ../../yyy/foo.git leads to the result:
> Clone of 'git@github.com:xxx/yyy/foo.git' into submodule path 'foo'
> failed
>
> So I basically cannot replace the xxx part.

This is due to a small bug in git-submodule.sh. In the 
resolve_relative_url() function, when repeatedly unwrapping '../'s from 
$url in the while loop, the line remoteurl="${remoteurl%/*}" removes 
everything _after_ the last slash, which does not work for the 
git@github.com:xxx part of your URL.

This should be relatively easy to fix, although maybe not as 
straightforwardly trivial as it might seem at first sight.

I don't have the time to look into this now, so feel free to take a stab 
at it.


Have fun! :)

...Johan


-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Submodule using different repository URLs

[Johan Herland]

On Monday 13 July 2009, Johan Herland wrote:
> On Monday 13 July 2009, Peter Voss wrote:
> > On 13.07.2009, at 12:59, Johan Herland wrote:
> > > You might be able to pull this off using relative submodule URLs.
> >
> > That's a good hint. Unfortunately I can't use this to go up 2
> > directories. I.e. I can't get from
> > git@github.com:xxx/mymodule.git
> > to
> > git@github.com:yyy/foo.git
> >
> > Using the relative URL ../../yyy/foo.git leads to the result:
> > Clone of 'git@github.com:xxx/yyy/foo.git' into submodule path 'foo'
> > failed
> >
> > So I basically cannot replace the xxx part.
>
> This is due to a small bug in git-submodule.sh. In the
> resolve_relative_url() function, when repeatedly unwrapping '../'s
> from $url in the while loop, the line remoteurl="${remoteurl%/*}"
> removes everything _after_ the last slash, which does not work for
> the git@github.com:xxx part of your URL.
>
> This should be relatively easy to fix, although maybe not as
> straightforwardly trivial as it might seem at first sight.
>
> I don't have the time to look into this now, so feel free to take a
> stab at it.

Here's a first stab at it (TOTALLY UNTESTED). Obviously it needs a
testcase (typically in t7400-submodule-basic.sh) as well.


Have fun! :)

...Johan


diff --git a/git-submodule.sh b/git-submodule.sh
index ebed711..7d8f7a7 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -27,12 +27,19 @@ resolve_relative_url ()
 		die "remote ($remote) does not have a url defined in .git/config"
 	url="$1"
 	remoteurl=${remoteurl%/}
+	sep="/"
 	while test -n "$url"
 	do
 		case "$url" in
 		../*)
 			url="${url#../}"
-			remoteurl="${remoteurl%/*}"
+			newremoteurl="${remoteurl%/*}"
+			if test "$newremoteurl" = "$remoteurl" -a "$sep" = "/"
+			then
+				newremoteurl="${remoteurl%:*}"
+				sep=":"
+			fi
+			remoteurl="$newremoteurl"
 			;;
 		./*)
 			url="${url#./}"
@@ -41,7 +48,7 @@ resolve_relative_url ()
 			break;;
 		esac
 	done
-	echo "$remoteurl/${url%/}"
+	echo "$remoteurl$sep${url%/}"
 }
 
 #


-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Submodule using different repository URLs

[Peter Voss]

Hi Johan,

On 13.07.2009, at 14:43, Johan Herland wrote:

> On Monday 13 July 2009, Johan Herland wrote:
>> On Monday 13 July 2009, Peter Voss wrote:
>>> On 13.07.2009, at 12:59, Johan Herland wrote:
>>>> You might be able to pull this off using relative submodule URLs.
>>>
>>> That's a good hint. Unfortunately I can't use this to go up 2
>>> directories. I.e. I can't get from
>>> git@github.com:xxx/mymodule.git
>>> to
>>> git@github.com:yyy/foo.git
>>>
>>> Using the relative URL ../../yyy/foo.git leads to the result:
>>> Clone of 'git@github.com:xxx/yyy/foo.git' into submodule path 'foo'
>>> failed
>>>
>>> So I basically cannot replace the xxx part.
>>
>> This is due to a small bug in git-submodule.sh. In the
>> resolve_relative_url() function, when repeatedly unwrapping '../'s
>> from $url in the while loop, the line remoteurl="${remoteurl%/*}"
>> removes everything _after_ the last slash, which does not work for
>> the git@github.com:xxx part of your URL.
>>
>> This should be relatively easy to fix, although maybe not as
>> straightforwardly trivial as it might seem at first sight.
>>
>> I don't have the time to look into this now, so feel free to take a
>> stab at it.
>
> Here's a first stab at it (TOTALLY UNTESTED). Obviously it needs a
> testcase (typically in t7400-submodule-basic.sh) as well.

And I thought you didn't have time. ;-) Thanks for this patch. I will  
test this out tomorrow.

Thanks,
--Peter

>
>
> Have fun! :)
>
> ...Johan
>
>
> diff --git a/git-submodule.sh b/git-submodule.sh
> index ebed711..7d8f7a7 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -27,12 +27,19 @@ resolve_relative_url ()
> 		die "remote ($remote) does not have a url defined in .git/config"
> 	url="$1"
> 	remoteurl=${remoteurl%/}
> +	sep="/"
> 	while test -n "$url"
> 	do
> 		case "$url" in
> 		../*)
> 			url="${url#../}"
> -			remoteurl="${remoteurl%/*}"
> +			newremoteurl="${remoteurl%/*}"
> +			if test "$newremoteurl" = "$remoteurl" -a "$sep" = "/"
> +			then
> +				newremoteurl="${remoteurl%:*}"
> +				sep=":"
> +			fi
> +			remoteurl="$newremoteurl"
> 			;;
> 		./*)
> 			url="${url#./}"
> @@ -41,7 +48,7 @@ resolve_relative_url ()
> 			break;;
> 		esac
> 	done
> -	echo "$remoteurl/${url%/}"
> +	echo "$remoteurl$sep${url%/}"
> }
>
> #
>
>
> -- 
> Johan Herland, <johan@herland.net>
> www.herland.net