.gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

.gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Jonathan Leonard]

The title pretty much says it all.  Lack of this feature (or presence
of this bug [depending on your perspective]) is a major PITA.

--Jonathan

Re: .gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Jens Lehmann]

Am 29.05.2014 04:07, schrieb Jonathan Leonard:
> The title pretty much says it all.

But you do not give much information about your special use
case. I assume you have submodule repositories for which some
developers have a valid ssh key and others don't (maybe
because they should only have read access via https)?

If that is the case you might want to look into access control
tools like gitolite.

>  Lack of this feature (or presence
> of this bug [depending on your perspective]) is a major PITA.

But why is https special? Why not fall back to the git
protocol? Or http? (And no: I'm not serious here ;-)

After the first failed clone of the submodule at via SSH the
developer could also just do a

   git config submodule.<name>.url https://host/repo

and override the URL from .gitmodules.

But maybe I misunderstood why you want to have a fall back?

Re: .gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Am 29.05.2014 04:07, schrieb Jonathan Leonard:
>> The title pretty much says it all.
>
> But you do not give much information about your special use
> case.

Perhaps "git grep insteadOf Documentation/" is all that is needed?

Re: .gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Jonathan Leonard]

> But you do not give much information about your special use
> case. I assume you have submodule repositories for which some
> developers have a valid ssh key and others don't (maybe
> because they should only have read access via https)?
>

Precisely. Specifically this is for a collection (17 or more) of
GitHub-hosted projects which are maintained by only a couple of people
(who have the ability to directly push via git:// or ssh://).
Everyone else (including deployments and ordinary users) who clones
the repo should be able to just grab the code via HTTPS and have it
work.

> If that is the case you might want to look into access control
> tools like gitolite.
>

We are using GitHub.

>>  Lack of this feature (or presence
>> of this bug [depending on your perspective]) is a major PITA.
>
> But why is https special? Why not fall back to the git
> protocol? Or http? (And no: I'm not serious here ;-)
>

HTTPS isn't special except in that it is the least privileged
transport type (and thus should be the last resort). Whether to
fallback to git:// from ssh:// or vice versa is inconsequential to
this request.

> After the first failed clone of the submodule at via SSH the
> developer could also just do a
>
>    git config submodule.<name>.url https://host/repo
>
> and override the URL from .gitmodules.
>

Yes, this would work. But it would be a painful manual step which we
would not want to force on ordinary users (and would not want to
experience ourselves either).

It should be noted that this is only really a problem as the other
options GitHub gives us are also equally (or more) painful:
a) - a unique deploy key per machine and project. (which at current
would be 17 * 3 keys all manually maintained via clicking through a
GUI [unless we wanted to automate via GitHub API (which is also a
non-trivial amount of work)]).
- or -
b) - a fake 'team' with read-only access with a single fake GitHub
account as member thereof.

I imagine this feature would be convenient for non-GitHub scenarios as
well though.

--Jonathan

Re: .gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Jeff King]

On Thu, May 29, 2014 at 04:12:38PM -0700, Jonathan Leonard wrote:

> We are using GitHub.
> [...]
> > But why is https special? Why not fall back to the git
> > protocol? Or http? (And no: I'm not serious here ;-)
> >
> 
> HTTPS isn't special except in that it is the least privileged
> transport type (and thus should be the last resort). Whether to
> fallback to git:// from ssh:// or vice versa is inconsequential to
> this request.

That's not quite true. git:// is the least privileged transport, as it
always anonymous and read-only (there ways to allow insecure pushes over
it, but GitHub does not enable them). Https is actually the most
flexible protocol, in that the same URL works of the box for both
logged-in and anonymous users (the latter assuming the repo is publicly
available). The server only prompts for credentials if necessary.

For that reason, it's a good choice for things like submodule URLs baked
into .gitmodules files.

The reasons not to are:

  1. It isn't _quite_ as efficient or robust as the regular git
     protocol, though in practice it's generally not a big deal.

  2. Pushers may prefer to authenticate with ssh keys (e.g., because
     they run ssh agent). I hope with modern credential helpers that
     logging in via http should not be a pain, either, though.

> > After the first failed clone of the submodule at via SSH the
> > developer could also just do a
> >
> >    git config submodule.<name>.url https://host/repo
> >
> > and override the URL from .gitmodules.
>
> Yes, this would work. But it would be a painful manual step which we
> would not want to force on ordinary users (and would not want to
> experience ourselves either).

Using insteadOf of in your global ~/.gitconfig would make this a
one-liner per-user. So one option would be to reverse things. Put
"https" URLs into the .gitmodules file, so most people have to do
nothing, and then developers who really want to do git-over-ssh can do a
one-time:

  git config --global url.git@github.com:.insteadOf https://github.com/

-Peff

Re: .gitmodules containing SSH clone URLs should fall back to HTTPS when SSH key is not valid/existent

[Chris Packham]

On Fri, May 30, 2014 at 11:12 AM, Jonathan Leonard <johanatan@gmail.com> wrote:
>> But you do not give much information about your special use
>> case. I assume you have submodule repositories for which some
>> developers have a valid ssh key and others don't (maybe
>> because they should only have read access via https)?
>>
>
> Precisely. Specifically this is for a collection (17 or more) of
> GitHub-hosted projects which are maintained by only a couple of people
> (who have the ability to directly push via git:// or ssh://).
> Everyone else (including deployments and ordinary users) who clones
> the repo should be able to just grab the code via HTTPS and have it
> work.
>
>> If that is the case you might want to look into access control
>> tools like gitolite.
>>
>
> We are using GitHub.
>
>>>  Lack of this feature (or presence
>>> of this bug [depending on your perspective]) is a major PITA.
>>
>> But why is https special? Why not fall back to the git
>> protocol? Or http? (And no: I'm not serious here ;-)
>>
>
> HTTPS isn't special except in that it is the least privileged
> transport type (and thus should be the last resort). Whether to
> fallback to git:// from ssh:// or vice versa is inconsequential to
> this request.
>

The problem is that a ssh:// url can't necessarily be transformed into
a correct https:// or git:// with a simple sed 's/ssh/https/' chances
are other parts of the URL will need changing. Which quickly becomes
non-trivial.

One solution that we use at work is to use relative paths (e.g.
../code/mod1.git) in .gitmodules (assuming the submodules are all part
of the same project). That way if you clone the superproject over
https:// all the submodules use that too. This works well for us using
local mirrors across multiple sites _and_ different protocols.

Another option would be to have a policy of storing the most
permissive transport in .gitmodules which makes it easy for users and
puts the special config requirements on the maintainers.

Both of these are obviously solutions you need to convince the
maintainer(s) of the superproject to implement.

Perhaps what git could do is allow multiple urls for a submodule and
at git submodule init time try them in order until the fetch is
successful. Or provide a mechanism to map transports, arguably this is
the url.<foo>.insteadOf mechanisim that has already been suggested.