Simulating network throttling

Simulating network throttling

[Bagas Sanjaya]

Hi,

I would like to test my Git repository in case the network is throttled 
(that is the network speed is reduced from its full speed). For example, 
I would like to test git clone under maximum download speed of 9.6 KB/s 
(GPRS speed).

I know how to test for throttling in browser, but since Git is 
command-line application, is there any way to simulate network throttling?

-- 
An old man doll... just what I always wanted! - Clara

Re: Simulating network throttling

[Bryan Turner]

On Tue, Jun 15, 2021 at 9:56 PM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>
> Hi,
>
> I would like to test my Git repository in case the network is throttled
> (that is the network speed is reduced from its full speed). For example,
> I would like to test git clone under maximum download speed of 9.6 KB/s
> (GPRS speed).
>
> I know how to test for throttling in browser, but since Git is
> command-line application, is there any way to simulate network throttling?

I've had some luck using toxiproxy[1] to MITM the connection, at least
over plain HTTP. It's worth noting a warning brian m. carlson has
given others in the past about this sort of thing, though, which is
that many MITM solutions are not fully transparent, which can result
in protocol errors and other abnormal behaviors from Git. Your mileage
may vary.

[1] https://github.com/shopify/toxiproxy

Re: Simulating network throttling

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1934 bytes --]

On 2021-06-16 at 04:59:00, Bryan Turner wrote:
> On Tue, Jun 15, 2021 at 9:56 PM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> >
> > Hi,
> >
> > I would like to test my Git repository in case the network is throttled
> > (that is the network speed is reduced from its full speed). For example,
> > I would like to test git clone under maximum download speed of 9.6 KB/s
> > (GPRS speed).
> >
> > I know how to test for throttling in browser, but since Git is
> > command-line application, is there any way to simulate network throttling?
> 
> I've had some luck using toxiproxy[1] to MITM the connection, at least
> over plain HTTP. It's worth noting a warning brian m. carlson has
> given others in the past about this sort of thing, though, which is
> that many MITM solutions are not fully transparent, which can result
> in protocol errors and other abnormal behaviors from Git. Your mileage
> may vary.

That is a warning I frequently give, both here and on Stack Overflow,
but I think if you're using a controlled environment and proxying data
just for yourself, the only person you're hurting is you, so I'm not
going to complain about it very much.  It may nevertheless result in
false failures that you'd otherwise want to avoid, though.

On Linux, you can also use the tc command to do traffic shaping and
policing to allow only a certain amount of bandwidth, and you can use it
in conjunction with iptables or nftables to do it only on certain ports
or IPs.  It's very powerful and doesn't suffer from the limitations of
proxies, but it also is rather complicated to set up, so you may want to
try a proxy first to see if it meets your needs with less work.

OpenBSD's (and FreeBSD's, Darwin's, etc.) pf supports the same
functionality but with a much nicer and easier to use interface (and I
say this as a Linux user).
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Simulating network throttling

[Jeff King]

On Wed, Jun 16, 2021 at 10:57:26PM +0000, brian m. carlson wrote:

> On Linux, you can also use the tc command to do traffic shaping and
> policing to allow only a certain amount of bandwidth, and you can use it
> in conjunction with iptables or nftables to do it only on certain ports
> or IPs.  It's very powerful and doesn't suffer from the limitations of
> proxies, but it also is rather complicated to set up, so you may want to
> try a proxy first to see if it meets your needs with less work.
> 
> OpenBSD's (and FreeBSD's, Darwin's, etc.) pf supports the same
> functionality but with a much nicer and easier to use interface (and I
> say this as a Linux user).

Yeah, these are both good suggestions that work transparently with any
protocol. I second the notion that tc is complicated to set up. ;)

If you really just want a pipe that will do rate-limiting in one
direction, many tools (e.g., "pv", which is available in debian) will do
it. That's hard to use with http, but you can stick it into an ssh
pipeline:

  $ git clone git@github.com:torvalds/linux.git
  Cloning into 'linux'...
  remote: Enumerating objects: 8129587, done.
  Receiving objects:   2% (163965/8129587), 70.89 MiB | 20.22 MiB/s
  ^C

  $ GIT_SSH_COMMAND='f() { ssh "$@" | pv -qL 512k; }; f' \
    git clone git@github.com:torvalds/linux.git
  Cloning into 'linux'...
  remote: Enumerating objects: 8129587, done.
  Receiving objects:   0% (15263/8129587), 7.57 MiB | 512.00 KiB/s
  ^C

-Peff