Socket Server breaks after 1000 client requests

Socket Server breaks after 1000 client requests

[Dragoslav Zaric]

Hi,

I wrote C++ Socket class. This class wraps methods both for client and server,
so you can create server or client depending what methods you call.

Anyway, all in all, problem is:

I have 3 small programs. One is server that listens in loop on port 30000.
Second is client program that sends one message to the server.
The third one on is small program (called spawn) that forks and
executes client program in limited for loop. So , idea is to simulate
a lot of connections to the server and see how many he can handle and
how.

So, at first I start server, execute spawn, and server gets client
messages. First I try 10 clients, then 100, then 1000 and it worked
Then I tried 10000 clients, and then it breaks just above 1000
clients, plus minus 100 !! So then I tried to put in server code 5
threads and mutex, but the result is same around 1000 clients and
breaks. Then I put semaphore, but then i realized that in this
scenario semaphores are of no use, because you can not sync with
random client events.

I also tried to change value in listen() method, but then I read
somewhere that maximum is 5 connections, which is default.

Does this problem requires some change in kernel code, or tcp/ip
stack, or the solution is simple, like reset server when it has first
break. I tried this last solution, but nothing happens.

So , does anyone have any idea?

Please help, I am working on this for days

thanks

Re: Socket Server breaks after 1000 client requests

[Ben Greear]

On 12/31/2009 11:11 AM, Dragoslav Zaric wrote:
> Hi,
>
> I wrote C++ Socket class. This class wraps methods both for client and server,
> so you can create server or client depending what methods you call.
>
> Anyway, all in all, problem is:

This is likely some bug with your code..as I have code that easily handles
10k connections.

> I have 3 small programs. One is server that listens in loop on port 30000.
> Second is client program that sends one message to the server.
> The third one on is small program (called spawn) that forks and
> executes client program in limited for loop. So , idea is to simulate
> a lot of connections to the server and see how many he can handle and
> how.

Use non-blocking IO and poll() instead of forking for each client.  10k processes
are going to take almost any machine to it's knees.

>
> So, at first I start server, execute spawn, and server gets client
> messages. First I try 10 clients, then 100, then 1000 and it worked
> Then I tried 10000 clients, and then it breaks just above 1000
> clients, plus minus 100 !! So then I tried to put in server code 5
> threads and mutex, but the result is same around 1000 clients and
> breaks. Then I put semaphore, but then i realized that in this
> scenario semaphores are of no use, because you can not sync with
> random client events.
>
> I also tried to change value in listen() method, but then I read
> somewhere that maximum is 5 connections, which is default.

No, you can easily make this bigger, and default is 128 for somaxconn

	echo -n "Increase somaxcon to 10000......................................... "
	echo 10000 > /proc/sys/net/core/somaxconn
	echo "DONE"

You should also let us know what you mean by 'it breaks'.  What exactly breaks?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Socket Server breaks after 1000 client requests

[Dragoslav Zaric]

Thanks Ben,

The error was definitely in my code. I used c++ functions for send and receive.
Now I am using write() and read() system calls to write and read
directly to sockets and
it works flawlessly ;)

When I am sending 100 000 messages in loop from client to server it goes for few
seconds.

regards,

dragoslav zaric