limit speed of file transfer

limit speed of file transfer

[Micah Wedemeyer]

Hi,

My system consistently locks up on file transfers across the LAN.  When
transferring a file to it using scp, it dies after less than 5 seconds.
However, if I use it to download a file from the Internet (using wget),
it does just fine.

I have heard that my motherboard (a Via Epia MII) has DMA problems and
that high Ethernet traffic is known to lock them up.  My theory is that
the Internet connection is slow enough to not hit whatever magic
threshold causes the crash.  So, I was hoping to find a way to limit the
speed of transfers with scp and ftp so that they also don't lock up the
machine.

I asked on a few message boards, and they all mentioned iptables.  I've
never used it before, and am a total noob when it comes to networking
stuff.  So, I was hoping that the people on this list could point me in
the right direction of where to start.

To sum up:  I need a way to limit the speed of incoming file transfers
using ftp or scp.

Any help would be greatly appreciated,
Micah

Re: limit speed of file transfer

[Jason Opperisano]

On Fri, 2005-01-21 at 20:08, Micah Wedemeyer wrote:
> Hi,
> 
> My system consistently locks up on file transfers across the LAN.  When
> transferring a file to it using scp, it dies after less than 5 seconds.
> However, if I use it to download a file from the Internet (using wget),
> it does just fine.
> 
> I have heard that my motherboard (a Via Epia MII) has DMA problems and
> that high Ethernet traffic is known to lock them up.  My theory is that
> the Internet connection is slow enough to not hit whatever magic
> threshold causes the crash.  So, I was hoping to find a way to limit the
> speed of transfers with scp and ftp so that they also don't lock up the
> machine.
> 
> I asked on a few message boards, and they all mentioned iptables.  I've
> never used it before, and am a total noob when it comes to networking
> stuff.  So, I was hoping that the people on this list could point me in
> the right direction of where to start.
> 
> To sum up:  I need a way to limit the speed of incoming file transfers
> using ftp or scp.
> 
> Any help would be greatly appreciated,
> Micah

brute force solution:  force your NIC and the switch port you plug into
to 10-Mbit Half-Duplex:

  mii-tool --force=10baseT-HD eth0
-OR-
  ethtool -s eth0 speed 10 duplex half autoneg off

-j

--
"What's the point of going out, we're just going to end up back
 here anyway?"
	--The Simpsons

Re: limit speed of file transfer

[R. DuFresne]

On Fri, 21 Jan 2005, Micah Wedemeyer wrote:

> Hi,
> 
> My system consistently locks up on file transfers across the LAN.  When
> transferring a file to it using scp, it dies after less than 5 seconds.
> However, if I use it to download a file from the Internet (using wget),
> it does just fine.


Are you saying that only scp/ftp transfers on the local net are hanging?
they work fine across your 'internet' nic?

> 
> I have heard that my motherboard (a Via Epia MII) has DMA problems and
> that high Ethernet traffic is known to lock them up.  My theory is that
> the Internet connection is slow enough to not hit whatever magic
> threshold causes the crash.  So, I was hoping to find a way to limit the
> speed of transfers with scp and ftp so that they also don't lock up the
> machine.
> 
> I asked on a few message boards, and they all mentioned iptables.  I've
> never used it before, and am a total noob when it comes to networking
> stuff.  So, I was hoping that the people on this list could point me in
> the right direction of where to start.
> 
> To sum up:  I need a way to limit the speed of incoming file transfers
> using ftp or scp.
> 

Perhaps you do, perhaps you do not.  I've seen issues with file transfers
due to a number of reasons, tcp-keepalives was not set properly for the
timeouts on firewalls that limit inactivity <yes these firewalls, most
often pixen, do not know that a ftp or scp is active>, ssh2/scp2 being
used inconjucntion with an older ssh1/ssh2/scp1/scp2 deamon/code <dropping
to ssh1/scp1 with really old deamon/code seems to fix the probs, a 10/100
nic is not auto-negociating it's connection speed properly, and this can
often be due to a router in the connection stream being set half/full
opposed to the nic's setting on this <hard coding the duplex and spped of
the connection stream in the nic and/or the router so they match resolves
this issue.



Thanks,

Ron DuFresne
-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        admin & senior security consultant:  sysinfo.com
                        http://sysinfo.com

...Love is the ultimate outlaw.  It just won't adhere to rules.
The most any of us can do is sign on as it's accomplice.  Instead
of vowing to honor and obey, maybe we should swear to aid and abet.
That would mean that security is out of the question.  The words
"make" and "stay" become inappropriate.  My love for you has no
strings attached.  I love you for free...
                        -Tom Robins <Still Life With Woodpecker>

Re: limit speed of file transfer

[Jose Maria Lopez]

El sáb, 22 de 01 de 2005 a las 02:08, Micah Wedemeyer escribió:
> Hi,
> 
> My system consistently locks up on file transfers across the LAN.  When
> transferring a file to it using scp, it dies after less than 5 seconds.
> However, if I use it to download a file from the Internet (using wget),
> it does just fine.
> 
> I have heard that my motherboard (a Via Epia MII) has DMA problems and
> that high Ethernet traffic is known to lock them up.  My theory is that
> the Internet connection is slow enough to not hit whatever magic
> threshold causes the crash.  So, I was hoping to find a way to limit the
> speed of transfers with scp and ftp so that they also don't lock up the
> machine.
> 
> I asked on a few message boards, and they all mentioned iptables.  I've
> never used it before, and am a total noob when it comes to networking
> stuff.  So, I was hoping that the people on this list could point me in
> the right direction of where to start.
> 
> To sum up:  I need a way to limit the speed of incoming file transfers
> using ftp or scp.
> 
> Any help would be greatly appreciated,
> Micah

Maybe iproute2 can help you. It does QoS and can limit the rate
of a connection. You can find more information on the LARTC HOWTO
at the site: http://lartc.org

Regards.

-- 
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jkerouac@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA

The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
                -- Jack Kerouac, "On the Road"

Re: limit speed of file transfer

[Rudi Starcevic]

Hi,

An option may be to use SSH with RSYNC instead of SCP.
The Rsync command has a bandwidth limit option.

I use it like this:

rsync -avz --bwlimit 10 -e ssh remoteuser@remotehost:/remote/dir /this/dir/

Check the 'man' pages for more info on the 'bwlimit' option.

HTH
Regards
Rudi.

Micah Wedemeyer wrote:

>Hi,
>
>My system consistently locks up on file transfers across the LAN.  When
>transferring a file to it using scp, it dies after less than 5 seconds.
>However, if I use it to download a file from the Internet (using wget),
>it does just fine.
>
>I have heard that my motherboard (a Via Epia MII) has DMA problems and
>that high Ethernet traffic is known to lock them up.  My theory is that
>the Internet connection is slow enough to not hit whatever magic
>threshold causes the crash.  So, I was hoping to find a way to limit the
>speed of transfers with scp and ftp so that they also don't lock up the
>machine.
>
>I asked on a few message boards, and they all mentioned iptables.  I've
>never used it before, and am a total noob when it comes to networking
>stuff.  So, I was hoping that the people on this list could point me in
>the right direction of where to start.
>
>To sum up:  I need a way to limit the speed of incoming file transfers
>using ftp or scp.
>
>Any help would be greatly appreciated,
>Micah
>
>
>
>
>  
>

Re: limit speed of file transfer

[R. DuFresne]

On Mon, 24 Jan 2005, Rudi Starcevic wrote:

> Hi,
> 
> An option may be to use SSH with RSYNC instead of SCP.
> The Rsync command has a bandwidth limit option.
> 
> I use it like this:
> 
> rsync -avz --bwlimit 10 -e ssh remoteuser@remotehost:/remote/dir /this/dir/
> 
> Check the 'man' pages for more info on the 'bwlimit' option.


The main caveat here is that it requires that rsync be installed on both
ends.  Rsync is a great tool under ssh, but, still is not all that common.
Especially with main vendor *nix's

Thanks,

Ron DuFresne
-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        admin & senior security consultant:  sysinfo.com
                        http://sysinfo.com

...Love is the ultimate outlaw.  It just won't adhere to rules.
The most any of us can do is sign on as it's accomplice.  Instead
of vowing to honor and obey, maybe we should swear to aid and abet.
That would mean that security is out of the question.  The words
"make" and "stay" become inappropriate.  My love for you has no
strings attached.  I love you for free...
                        -Tom Robins <Still Life With Woodpecker>