GIT and SSH

GIT and SSH

[Reza Mostafid]

I am starting to use GIT and I would be grateful for a simple answer to a
specific situation to help me find the right ball-park.


a.) Does the communication that takes place between a GIT `client` and a remote
    GIT `repository` involve 'ssh' traffic?


Our connections here are heavily censored and ssh traffic is suppressed most of
the time which is why I need to figure out why a simple  

    $ git clone git://<URL> 

command chokes to a halt and freezes most of the time ( the same command when
executed remotely on our V.P.S. in Europe works flawlessly ).
 

b.) Are there means to make the `git` client on my machine circumvent this?


Y/N answers or brief hints to my questions suffice, I'll work out the rest. 

Basically I would like to know whether there is a point at all trying to make
git work from where I am, given the limitations mentioned.

Regards

Reza

Re: GIT and SSH

[Dov Grobgeld]

Git supports multiple transport protocols. Among them are git, ssh,
and https. (You can also use direct file system access, but it is
questionable whether to call that a protocol). Each of the protocols
have their advantages and drawbacks. The git protocol is only used for
reading, and not for writing, but is supposed to be very fast. The
common firewall filtering of the git protocol port 9418 is another
problem. ssh is the prefered protocol for writing to a remote
protocol. But if ssh is filtered, then http/https may be used, which
is very slow for large repositories, but it has the advantage that it
is the least blocked protocol.

For more info see:

    http://progit.org/book/ch4-1.html

Regards,
Dov


On Wed, Dec 28, 2011 at 10:43, Reza Mostafid <m.r.mostafid@gmail.com> wrote:
> I am starting to use GIT and I would be grateful for a simple answer to a
> specific situation to help me find the right ball-park.
>
>
> a.) Does the communication that takes place between a GIT `client` and a remote
>    GIT `repository` involve 'ssh' traffic?
>
>
> Our connections here are heavily censored and ssh traffic is suppressed most of
> the time which is why I need to figure out why a simple
>
>    $ git clone git://<URL>
>
> command chokes to a halt and freezes most of the time ( the same command when
> executed remotely on our V.P.S. in Europe works flawlessly ).
>
>
> b.) Are there means to make the `git` client on my machine circumvent this?
>
>
> Y/N answers or brief hints to my questions suffice, I'll work out the rest.
>
> Basically I would like to know whether there is a point at all trying to make
> git work from where I am, given the limitations mentioned.
>
> Regards
>
> Reza
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GIT and SSH

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 2104 bytes --]

On Wed, Dec 28, 2011 at 11:55:24AM +0200, Dov Grobgeld wrote:
> Git supports multiple transport protocols. Among them are git, ssh,
> and https. (You can also use direct file system access, but it is
> questionable whether to call that a protocol). Each of the protocols
> have their advantages and drawbacks. The git protocol is only used for
> reading, and not for writing, but is supposed to be very fast. The
> common firewall filtering of the git protocol port 9418 is another
> problem. ssh is the prefered protocol for writing to a remote
> protocol. But if ssh is filtered, then http/https may be used, which
> is very slow for large repositories, but it has the advantage that it
> is the least blocked protocol.

Slow for large repositories? Are you thinking of the dumb HTTP
transport? That one shouldn't be used for doing any serious work. The
Smart HTTP transport is just the git smart protocol (the same one
git:// and ssh:// URLs speak) wrapped inside HTTP communication. The
negotiation phase is a more expensive than with either git or ssh, as
HTTP is stateless and you need to remind the remote what you want and
what you've already agreed on, but the actual transfer of data is done
the same way than with the others so overall it shouldn't be that
noticeable.

Now, to the OP's concerns: yes, the ssh transport does generate ssh
transport, as that's the whole point. You authenticate against the
remote machine's ssh daemon and run git-upload-pack or
git-receive-pack as needed (for fetching or pushing). If corporate
policy doesn't allow ssh you should either fix the policy or use the
smart HTTP protocol, though this involves messing with passwords and
their associated problems. I'm not saying ssh keys don't have their
complications, but I much prefer them.

We can't help you diagnose why your clone is stalling without more
information. It could be that the connection between the computers is
flaky, git trying to take too much memory on the remote or local
machines or any number of things. See if setting GIT_TRACE=1 in the
environment helps to see what's going on.

   cmn

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: GIT and SSH

[Jakub Narebski]

Carlos Martín Nieto <cmn@elego.de> writes:
> On Wed, Dec 28, 2011 at 11:55:24AM +0200, Dov Grobgeld wrote:

> > Git supports multiple transport protocols. Among them are git, ssh,
> > and https. (You can also use direct file system access, but it is
> > questionable whether to call that a protocol). Each of the protocols
> > have their advantages and drawbacks. The git protocol is only used for
> > reading, and not for writing, but is supposed to be very fast. The

FYI git:// protocol can theoretically be used for pushing, but it is
not recommended because git:// protocol is not authenthicated - that
is why you need to enable pushing via git:// explicitly.

Just git trivia.

> > common firewall filtering of the git protocol port 9418 is another
> > problem. ssh is the prefered protocol for writing to a remote
> > protocol. But if ssh is filtered, then http/https may be used, which
> > is very slow for large repositories, but it has the advantage that it
> > is the least blocked protocol.
> 
> Slow for large repositories? Are you thinking of the dumb HTTP
> transport? That one shouldn't be used for doing any serious work. The
> Smart HTTP transport is just the git smart protocol (the same one
> git:// and ssh:// URLs speak) wrapped inside HTTP communication. The
> negotiation phase is a more expensive than with either git or ssh, as
> HTTP is stateless and you need to remind the remote what you want and
> what you've already agreed on, but the actual transfer of data is done
> the same way than with the others so overall it shouldn't be that
> noticeable.

Note that for "smart" transports ("smart" HTTP, SSH) you need to have
git installed on server, or at least have git-upload-pack and
git-receive-pack somewhere (you can configure client where it is to be
found on server).
 
Note that git uses curl for both smart and dumb HTTP transports, so
things like 'http_proxy' and 'HTTPS_PROXY' environment variables
should work.

> Now, to the OP's concerns: yes, the ssh transport does generate ssh
> transport, as that's the whole point. You authenticate against the
> remote machine's ssh daemon and run git-upload-pack or
> git-receive-pack as needed (for fetching or pushing). If corporate
> policy doesn't allow ssh you should either fix the policy or use the
> smart HTTP protocol, though this involves messing with passwords and
> their associated problems. I'm not saying ssh keys don't have their
> complications, but I much prefer them.

Note that if the problem is giving shell accounts with SSH access, the
solution is to use one of git repository management tools, like
Gitosis (Python + setuptools, no longer developed) or Gitolite (Perl).
From what I remember both use _single_ *restricted* account and
public-key authenthication.
 
If I am not mistaken Gitolite can help with "smart" HTTP transport
access too.

> We can't help you diagnose why your clone is stalling without more
> information. It could be that the connection between the computers is
> flaky, git trying to take too much memory on the remote or local
> machines or any number of things. See if setting GIT_TRACE=1 in the
> environment helps to see what's going on.

Or even undocumented (!) GIT_TRACE_PACKET.

-- 
Jakub Narebski

Re: GIT and SSH

[Reza Mostafid]

I forgot to mention, I am an embedded developer located in Iran.

The filtering I am talking about is by the government. All ISP's get their
bandwith from centrally allocated trunks. This allows control.

I know that ssh packets get dropped for extended periods frequently.
We have an Ubuntu virtual server outside of Iran which we use as a proxy and
connect to it via 'ssh' sox provxy ( -D 8090 ).

Many times some sort of script or intelligence is operation which severely
throttles the connection as soon as the data rate exceeds certain "benign" 
levels. All this has been confirmed by the network `gurus` and admin people I
work with. This is the best we can tell from what we observe.

As mentioned when we execute the simple GIT clone command from our VPS 
( located outside Iran ) the command works flawlessly.

What I would be interested in is to somehow make git avoid using transport
over ssh. The government censors are interested only in blocking people
accessing illicit sites via S.O.X-5 or VPN. 

To them anything over SSH is suspicious. If we could somehow update using a
plain transport method, they couldn't care less about source code being sent to 
us.

Regards

Reza

Re: GIT and SSH

[Thomas Hochstein]

Reza Mostafid schrieb:

> a.) Does the communication that takes place between a GIT `client` and a remote
>     GIT `repository` involve 'ssh' traffic?

Depends on how you do it (as already described by others).

> Our connections here are heavily censored and ssh traffic is suppressed most of
> the time which is why I need to figure out why a simple  
>
>     $ git clone git://<URL> 
>
> command chokes to a halt and freezes most of the time ( the same command when
> executed remotely on our V.P.S. in Europe works flawlessly ).

"git://" uses the git protocol which does not involve SSH.

    $ git clone ssh://<URL>

would use SSH.

-thh

Re: GIT and SSH

[Reza Mostafid]

Thanks for all the feedback so far.....helps me find out where to look.

Regards

Reza