Clone to an SSH destination

Clone to an SSH destination

[Mark Hills]

How do I clone a repo _to_ a new repo over SSH? I tried:

  cd xx
  git clone --bare . gitserver:/scm/xx.git
  git clone --bare . ssh://gitserver/scm/xx.git

This does not have the expected result, and instead a local path of the 
given name is created (eg. a 'gitserver:' directory)

This seems to be a FAQ, but the only answer I can find (Google) is to 
login to the server and create the repo, setup a remote and push to it.
 
This is quite cumbersome; we have a large team of devs who use a simple 
'git clone' to an NFS directory, but we wish to retire NFS access.

Is there a technical limiation preventing clone-to-ssh, or just something 
waiting to be implemented?

(Please keep me CC'd; I am currently reading the web archives)

Many thanks

-- 
Mark

Re: Clone to an SSH destination

[Ævar Arnfjörð Bjarmason]

On Mon, Sep 3, 2012 at 12:21 PM, Mark Hills <Mark.Hills@framestore.com> wrote:
> How do I clone a repo _to_ a new repo over SSH? I tried:
>
>   cd xx
>   git clone --bare . gitserver:/scm/xx.git
>   git clone --bare . ssh://gitserver/scm/xx.git
>
> This does not have the expected result, and instead a local path of the
> given name is created (eg. a 'gitserver:' directory)
>
> This seems to be a FAQ, but the only answer I can find (Google) is to
> login to the server and create the repo, setup a remote and push to it.

Basically Git doesn't support this yet, mainly because it could only
be supported with the ssh or local transports.

With anything else it would break, so push can only assume that
something on the other end can receive data, can update branch
pointers etc. Not create a brand new repository.

You could of course supply your devs with an alias that ssh's to that
server, does an init if needed, and then does a push.

> This is quite cumbersome; we have a large team of devs who use a simple
> 'git clone' to an NFS directory, but we wish to retire NFS access.
>
> Is there a technical limiation preventing clone-to-ssh, or just something
> waiting to be implemented?

But I'm actually more curious about why you need this in the first
place, there's a bunch of devs where I work as well, but they never
have the need to create new repos on some NFS drive in this manner.

What are your devs doing when they do clone their current working
directory to some NFS location, maybe there's a better way to do it.

Re: Clone to an SSH destination

[Mark Hills]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2529 bytes --]

On Mon, 3 Sep 2012, Ævar Arnfjörð Bjarmason wrote:

> On Mon, Sep 3, 2012 at 12:21 PM, Mark Hills <Mark.Hills@framestore.com> wrote:
> > How do I clone a repo _to_ a new repo over SSH? I tried:
> >
> >   cd xx
> >   git clone --bare . gitserver:/scm/xx.git
> >   git clone --bare . ssh://gitserver/scm/xx.git
> >
> > This does not have the expected result, and instead a local path of the
> > given name is created (eg. a 'gitserver:' directory)
> >
> > This seems to be a FAQ, but the only answer I can find (Google) is to
> > login to the server and create the repo, setup a remote and push to it.
> 
> Basically Git doesn't support this yet, mainly because it could only
> be supported with the ssh or local transports.
> 
> With anything else it would break, so push can only assume that
> something on the other end can receive data, can update branch
> pointers etc. Not create a brand new repository.
> 
> You could of course supply your devs with an alias that ssh's to that
> server, does an init if needed, and then does a push.

Yes, a last resort, but do-able of course.

> > This is quite cumbersome; we have a large team of devs who use a simple
> > 'git clone' to an NFS directory, but we wish to retire NFS access.
> >
> > Is there a technical limiation preventing clone-to-ssh, or just something
> > waiting to be implemented?
> 
> But I'm actually more curious about why you need this in the first
> place, there's a bunch of devs where I work as well, but they never
> have the need to create new repos on some NFS drive in this manner.

Without a command-line onto the filesystem (either local or NFS), how do 
you create a new repository for a new project?

We have a fairly large team on a diverse set of projects. Projects come 
and go, so it's a burden if the administrator is needed just to create 
repos.

Likewise, it's a step backwards for the developer to need to login 
themselves over SSH -- whereas 'git clone' is so easy to NFS.

> What are your devs doing when they do clone their current working 
> directory to some NFS location, maybe there's a better way to do it.

Most projects start as a small test at some point; eg.

  mkdir xx
  cd xx
  git init
  <write some code>
  git commit
  ...

When a project becomes more official, the developer clones to a central 
location; eg.

  git clone --bare . /net/git/xx.git

This is the step that is inconvenient if only SSH access is available.

Thanks for your reply,

-- 
Mark

Re: Clone to an SSH destination

[Konstantin Khomoutov]

On Mon, 3 Sep 2012 14:07:48 +0100 (BST)
Mark Hills <Mark.Hills@framestore.com> wrote:

[...]
> > But I'm actually more curious about why you need this in the first
> > place, there's a bunch of devs where I work as well, but they never
> > have the need to create new repos on some NFS drive in this manner.
> 
> Without a command-line onto the filesystem (either local or NFS), how
> do you create a new repository for a new project?
> 
> We have a fairly large team on a diverse set of projects. Projects
> come and go, so it's a burden if the administrator is needed just to
> create repos.
> 
> Likewise, it's a step backwards for the developer to need to login 
> themselves over SSH -- whereas 'git clone' is so easy to NFS.
> 
> > What are your devs doing when they do clone their current working 
> > directory to some NFS location, maybe there's a better way to do it.
> 
> Most projects start as a small test at some point; eg.
> 
>   mkdir xx
>   cd xx
>   git init
>   <write some code>
>   git commit
>   ...
> 
> When a project becomes more official, the developer clones to a
> central location; eg.
> 
>   git clone --bare . /net/git/xx.git
> 
> This is the step that is inconvenient if only SSH access is available.

Well, then it looks you want something like github.
In this case look at some more integrated solution such as Gitlab [1]
-- I did not try it, but it looks like you import your users there and
then they can log in, add their SSH keys and create their projects.

I also think gitolite has some way to actually use regular SSH users
(or even users coming from a web server which is a front-end for Smart
HTTP Git transport, doing its own authentication).  This is explained
in [2], and I hope Sitaram could provide more insight on setting things
up this way, if needed  (I did not use this feature).

1. http://gitlabhq.com/
2. http://sitaramc.github.com/gitolite/g2/auth.html

Re: Clone to an SSH destination

[Sitaram Chamarty]

On Mon, Sep 3, 2012 at 7:38 PM, Konstantin Khomoutov
<flatworm@users.sourceforge.net> wrote:
> On Mon, 3 Sep 2012 14:07:48 +0100 (BST)
> Mark Hills <Mark.Hills@framestore.com> wrote:
>
> [...]
>> > But I'm actually more curious about why you need this in the first
>> > place, there's a bunch of devs where I work as well, but they never
>> > have the need to create new repos on some NFS drive in this manner.
>>
>> Without a command-line onto the filesystem (either local or NFS), how
>> do you create a new repository for a new project?
>>
>> We have a fairly large team on a diverse set of projects. Projects
>> come and go, so it's a burden if the administrator is needed just to
>> create repos.
>>
>> Likewise, it's a step backwards for the developer to need to login
>> themselves over SSH -- whereas 'git clone' is so easy to NFS.
>>
>> > What are your devs doing when they do clone their current working
>> > directory to some NFS location, maybe there's a better way to do it.
>>
>> Most projects start as a small test at some point; eg.
>>
>>   mkdir xx
>>   cd xx
>>   git init
>>   <write some code>
>>   git commit
>>   ...
>>
>> When a project becomes more official, the developer clones to a
>> central location; eg.
>>
>>   git clone --bare . /net/git/xx.git
>>
>> This is the step that is inconvenient if only SSH access is available.
>
> Well, then it looks you want something like github.
> In this case look at some more integrated solution such as Gitlab [1]
> -- I did not try it, but it looks like you import your users there and
> then they can log in, add their SSH keys and create their projects.

Anything web based would be even more overhead than a simple:

    ssh server git init --bare foo/bar.git && git push --mirror
ssh://git/~/foo/bar.git

Gitolite of course is even closer, as we discussed earlier.

> I also think gitolite has some way to actually use regular SSH users
> (or even users coming from a web server which is a front-end for Smart
> HTTP Git transport, doing its own authentication).  This is explained
> in [2], and I hope Sitaram could provide more insight on setting things
> up this way, if needed  (I did not use this feature).

As I said earlier, regardless of how he does it, authentication will
change, since he is no longer using a local (well, locally mounted)
file system as the "server".  That may be "get everyone to send us a
pub key" or "give everyone an http password and use smart http".

In addition, if they choose smart http, they *have to* use gitolite.
Unlike ssh, where that two command sequence above would do it all for
them, there is no eqvt if your git server is behind http.

>
> 1. http://gitlabhq.com/
> 2. http://sitaramc.github.com/gitolite/g2/auth.html
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Sitaram

Re: Clone to an SSH destination

[Enrico Weigelt]

Hi,

> Without a command-line onto the filesystem (either local or NFS), how
> do you create a new repository for a new project?

These things are out of the scope of git itself, they belong to another
layer - you're probably looking for some repo hosting solution.

One already was mentioned: gitolite. Another one is gitosis.

> We have a fairly large team on a diverse set of projects. Projects
> come and go, so it's a burden if the administrator is needed just to
> create repos.

Maybe you'd also like to have some project management / ticketing
tool. We've got an redmine and gitolite based solution for that,
that's automatically creating repos and managing access control
on per-project basis. Will be publically released in a few days,
just drop a note if you're interested in it.

> Most projects start as a small test at some point; eg.
> 
>   mkdir xx
>   cd xx
>   git init
>   <write some code>
>   git commit
>   ...
> 
> When a project becomes more official, the developer clones to a
> central
> location; eg.
> 
>   git clone --bare . /net/git/xx.git

Do you have any kind of access control in place, or can everybody
write just everywhere ?


cu
-- 
Mit freundlichen Grüßen / Kind regards 

Enrico Weigelt 
VNC - Virtual Network Consult GmbH 
Head Of Development 

Pariser Platz 4a, D-10117 Berlin
Tel.: +49 (30) 3464615-20
Fax: +49 (30) 3464615-59

enrico.weigelt@vnc.biz; www.vnc.de 

Re: Clone to an SSH destination

[Konstantin Khomoutov]

On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
Mark Hills <Mark.Hills@framestore.com> wrote:

> How do I clone a repo _to_ a new repo over SSH? I tried:
> 
>   cd xx
>   git clone --bare . gitserver:/scm/xx.git
>   git clone --bare . ssh://gitserver/scm/xx.git
>
> This does not have the expected result, and instead a local path of
> the given name is created (eg. a 'gitserver:' directory)

No, `git clone` is intended to work with your local machine.
And I can't really imagine how it would work in a way you want.

> This seems to be a FAQ, but the only answer I can find (Google) is to 
> login to the server and create the repo, setup a remote and push to
> it. 
> This is quite cumbersome; we have a large team of devs who use a
> simple 'git clone' to an NFS directory, but we wish to retire NFS
> access.
> 
> Is there a technical limiation preventing clone-to-ssh, or just
> something waiting to be implemented?

I think this is more a conceptual issue: such a "remote cloning" would
mean *creation* of a repository as part of the process.  I'm not aware
of any [D]VCS system which would routinely allow such a thing: usually
the creation of a repository is an "administrative" act, which is
distinct from regular (push/fetch) access to that repository.

gitolite kind of implements this ("wild repos") [1], you could look if
it suits your needs.

Now back to your problem.

If you have NFS access, you can probably do proper cloning using NFS:
1) Export via NFS the directory in which you want your resulting
   bare repository located.
2) Do simple *local* clone (using paths accessed by NFS):
   git clone --bare /path/to/source/repo /path/to/destination/repo
3) Do certain fixups in the destination repo, if needed, then
   access it normally via SSH (or whatever protocol you prefer).

If you don't want to do this, then revert to a normal two-step
procedure:
1) Log into the server, create a repo by hand:
   git init --bare /path/to/repo
2) Push everything relevant from a source repo there:
   cd /path/to/local/repo
   git remote add origin ssh://gitserver/scm/xx.git
   git push origin refs/*:refs/*

Step (1) may depends on what server-side solution your need; for
instance, in gitolite you would be configuring a new repo in the
configuration file in your local clone of the gitolite admin repo and
pushing your changes to the server.

1. http://sitaramc.github.com/gitolite/wild.html

Re: Clone to an SSH destination

[Sitaram Chamarty]

On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov
<flatworm@users.sourceforge.net> wrote:
> On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
> Mark Hills <Mark.Hills@framestore.com> wrote:

[snip]

>> This is quite cumbersome; we have a large team of devs who use a
>> simple 'git clone' to an NFS directory, but we wish to retire NFS
>> access.

[snip]

> gitolite kind of implements this ("wild repos") [1], you could look if
> it suits your needs.

The simplest conf to do what you want in gitolite is something like this:

    repo [a-zA-Z0-9]..*
        C   =   @all
        RW+ =   @all

But of course your *user* authentication will probably change quite a
bit, since gitolite runs as one Unix user and merely simulates many
"gitolite users", while in the NFS method each of your devs probably
has a full login to the server.

Re: Clone to an SSH destination

[Mark Hills]

On Mon, 3 Sep 2012, Sitaram Chamarty wrote:

> On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov
> <flatworm@users.sourceforge.net> wrote:
> > On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
> > Mark Hills <Mark.Hills@framestore.com> wrote:
> 
> [snip]
> 
> >> This is quite cumbersome; we have a large team of devs who use a
> >> simple 'git clone' to an NFS directory, but we wish to retire NFS
> >> access.
> 
> [snip]
> 
> > gitolite kind of implements this ("wild repos") [1], you could look if
> > it suits your needs.
> 
> The simplest conf to do what you want in gitolite is something like this:
> 
>     repo [a-zA-Z0-9]..*
>         C   =   @all
>         RW+ =   @all
> 
> But of course your *user* authentication will probably change quite a
> bit, since gitolite runs as one Unix user and merely simulates many
> "gitolite users", while in the NFS method each of your devs probably
> has a full login to the server.

I'll check out gitolite, thanks.

We use unix users extensively (groups, permissions etc.) with YP, and this 
works well; a separate permissions scheme is not very desireable.

The ssh method works very well right now, and nicely transparent. It's 
only the initial clone/creation that is harder than it was over NFS. And 
it prevents the use of git-shell too.

Thanks

-- 
Mark

Re: Clone to an SSH destination

[Sitaram Chamarty]

On Mon, Sep 3, 2012 at 6:45 PM, Mark Hills <Mark.Hills@framestore.com> wrote:
> On Mon, 3 Sep 2012, Sitaram Chamarty wrote:
>
>> On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov
>> <flatworm@users.sourceforge.net> wrote:
>> > On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
>> > Mark Hills <Mark.Hills@framestore.com> wrote:
>>
>> [snip]
>>
>> >> This is quite cumbersome; we have a large team of devs who use a
>> >> simple 'git clone' to an NFS directory, but we wish to retire NFS
>> >> access.
>>
>> [snip]
>>
>> > gitolite kind of implements this ("wild repos") [1], you could look if
>> > it suits your needs.
>>
>> The simplest conf to do what you want in gitolite is something like this:
>>
>>     repo [a-zA-Z0-9]..*
>>         C   =   @all
>>         RW+ =   @all
>>
>> But of course your *user* authentication will probably change quite a
>> bit, since gitolite runs as one Unix user and merely simulates many
>> "gitolite users", while in the NFS method each of your devs probably
>> has a full login to the server.
>
> I'll check out gitolite, thanks.
>
> We use unix users extensively (groups, permissions etc.) with YP, and this
> works well; a separate permissions scheme is not very desireable.
> The ssh method works very well right now, and nicely transparent. It's
> only the initial clone/creation that is harder than it was over NFS. And
> it prevents the use of git-shell too.

If I had to do this, and didn't want to use gitolite or something like
it, I'd just make a script that will create the repo using an ssh call
then do a 'git push --mirror' to it.

Call it "git-new" or something and train people to use that instead of
"clone" when the repo doesn't even exist yet.

Bound to be easier than the administrative hassle you spoke of in your
other email...