Re: many outgoing tcp sockets are slower than a few

[Baruch Even <baruch@ev-en.org>]

Christian Schmid wrote:
> bert hubert wrote:
> 
>> On Mon, Feb 21, 2005 at 01:35:33AM +0100, Christian Schmid wrote:
> Outgoing data. I am using sendfile() to send the file on a non-blocking 
> socket but the call blocks for 100 ms per socket if I get over 3000 
> sockets. Thats causing the massive slowdown in sum. I first thought its 
> a disk-issue but I tried with pure-cache data as well and it still blocks.
> 
>>> 3000 sockets = no slowdown at all (500 MBit in use)
>>> 3300 sockets = 10% slowdown
>>> 3600 sockets = 30% slowdown
>>> 4000 sockets = 60% slowdown (i aborted here, as it only uses 200 MBit 
>>> for sending... catastrophy!)
>>>
[snip]
>> I'm a bit confused, it is a download service so you are probably 
>> *sending*
>> data? 
> 
> Only sending. Receiving ACKs of course.

I've been doing some work to improve ACK performance, you can find some 
patches for Linux 2.6.6 at http://hamilton.ie/net/

I've only tested these patches for a single (or few) very fast 
connections, but I'd expect the problem might manifest itself for a very 
large number of connections as well. Though then you might hit other 
bottlenecks (memory access for different structures).

Baruch