Re: PROBLEM: Linux kernel 2.6.31 IPv4 TCP fails to open huge amount of outgoing connections (unable to bind ... )

[Ben Greear <greearb@candelatech.com>]

On 04/20/2010 04:20 PM, Gaspar Chilingarov wrote:
> Opening that amount of incoming connections in not a problem at all.
>
> I'm speaking of outgoing connections.
>
> What kind of hack do you have on your kernel?

send-to-self logic and similar..but nothing that should be
better at doing lots of sockets than the default kernel.

I have 40,000 out-going connections to myself..basically I'm
acting as both client and server.  So, I'd expect you to be
able to do around 80,000 connections to some external system on
similar hardware and efficient client software.  At least with
my software, I'd need more RAM to scale much beyond that, but I'm
trying to generate traffic on these sockets (which requires user-space
buffers per socket in my implementation), and I have a decent bit
of overhead gathering stats and such.  A simpler application with
less overhead should do more sockets.

Be sure to use lots of processes so you don't end up with
a poll loop with 80,000 fd entries.  I found that 10k sockets
per process was a good upper limit..but it's not a hard
upper limit.

And be sure to do the bind logic correctly as Eric pointed out in an
earlier email.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com