How to limit bandwidth used by git over SSH ?

How to limit bandwidth used by git over SSH ?

[Ralph Seichter]

Hello list,

I'm not really sure if this is a git or SSH issue, but I haven't figured
out a solution yet. To transfer my data between two machines, I use

  git-push ssh://machineB.tld/foo/proj.git master

from machineA. Unfortunately, the machines are connected by a very low-
bandwidth connection, which is completely choked by git-push. From what
I've read so far, I guess this is the result of SSH setting TOS to
"minimize delay", which prevents almost all other traffic during the
push operation. Not good.

I haven't yet found a way of telling git (or SSH) to use no more than a
given maximum bandwidth, like I could do with "scp -l {limit}". Did I
miss something in the documentation?

Your help is appreciated.

-Ralph

Re: How to limit bandwidth used by git over SSH ?

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 1652 bytes --]

On Sun, 2011-07-17 at 12:22 +0200, Ralph Seichter wrote:
> Hello list,
> 
> I'm not really sure if this is a git or SSH issue, but I haven't figured
> out a solution yet. To transfer my data between two machines, I use
> 
>   git-push ssh://machineB.tld/foo/proj.git master
> 
> from machineA. Unfortunately, the machines are connected by a very low-
> bandwidth connection, which is completely choked by git-push. From what
> I've read so far, I guess this is the result of SSH setting TOS to
> "minimize delay", which prevents almost all other traffic during the
> push operation. Not good.

A bit OT, but the situation where a link is chocked by a slow uplink
getting filled is most likely due to buffer bloat on your modem/router
which confuses TCP's congestion detection by buffering your packets too
aggressively. But that's neither here nor there, I just wanted to point
out it's not all ssh or git's fault.

> 
> I haven't yet found a way of telling git (or SSH) to use no more than a
> given maximum bandwidth, like I could do with "scp -l {limit}". Did I
> miss something in the documentation?

But coming back to SSH, scp knows it's going to do a file transfer, but
git uses the ssh client to do a "normal" connection to a remote machine,
to ssh, it's no different from a user typing really fast, which is
probably why it's not supported out of the box.

There is a tool called trickle which takes over the network functions
and can be used to limit upload and download speeds, so using it like

    trickle -u 20 git push ssh://machineB.tld/foo/proj.git master

should do the trick.

Cheers,
   cmn


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: How to limit bandwidth used by git over SSH ?

[Ralph Seichter]

On 19.07.11 10:42, Carlos Martín Nieto wrote:

> There is a tool called trickle which takes over the network functions
> and can be used to limit upload and download speeds [...]

Thanks for the hint, Carlos. I tried to compile trickle on OS X 10.6.8,
but without success. AFAIK, trickle has not been tested with OS X, and I
am not even sure it trickle could work at all. Members of the MacPorts
mailing list seem to confirm my suspicion. I've tried to contact the
author of trickle; hopefully he will reply.

The general idea to use a tool besides git or SSH to shape the bandwidth
usage seems valid, though.

-Ralph

Re: How to limit bandwidth used by git over SSH ?

[Andreas Krey]

On Wed, 20 Jul 2011 21:18:56 +0000, Ralph Seichter wrote:
...
> The general idea to use a tool besides git or SSH to shape the bandwidth
> usage seems valid, though.

There are ways to integrate such a tool easier than the way trickle
does; by using the 'ProxyCommand' configuration option of ssh, like
'ProxyCommand netcat $yourhost 22' or similar. Unfortunately, I don't
know a suitable program offhand, netcat does not seem to have a bandwidth
limiting option. But then such a beast is relatively easy to write.

Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800