Beyond 64K TCP connections limit per IP-address

Beyond 64K TCP connections limit per IP-address

[Robert Iakobashvili]

If I am correct, a TCP server can make up to
64K accepts for a port at a single IP-address.

If one would like to have e.g. 1M TCP connections
to a single IP-address of a powerful server (without load balancer),
how to make such setup?

There is a workaround to arrange several secondary IP-addresses
each serving ~64K connections.
Any other options?

Server places to the outgoing packet source port the port
of the server listening socket. Actually, the ephemeral ports at the
server side are used as the hashing keys.
Could it be played, tricked somehow?

Y comments would be appreciated.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
A web testing and traffic generation tool.

Re: Beyond 64K TCP connections limit per IP-address

[Florian Weimer]

* Robert Iakobashvili:

> If I am correct, a TCP server can make up to
> 64K accepts for a port at a single IP-address.

I don't think such a limit exists.  In typical configurations, a
single client IP address can only establish a few tens of thousands of
TCP connections to one server port.  But as soon as multiple clients
are involved, there is virtually no protocol-imposed limit.

-- 
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Re: Beyond 64K TCP connections limit per IP-address

[Evgeniy Polyakov]

On Wed, Jul 04, 2007 at 09:50:31AM +0200, Robert Iakobashvili (coroberti@gmail.com) wrote:
> If I am correct, a TCP server can make up to
> 64K accepts for a port at a single IP-address.

No, it is essentially unlimited - linux uses local/remote addr/port
tuples for hash chains, so there is no per-addr limits.
If there is some kind of binds, then yes, only 64k ports per address.


-- 
	Evgeniy Polyakov

Re: Beyond 64K TCP connections limit per IP-address

[Robert Iakobashvili]

On 7/4/07, Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
> On Wed, Jul 04, 2007 at 09:50:31AM +0200, Robert Iakobashvili (coroberti@gmail.com) wrote:
> > If I am correct, a TCP server can make up to
> > 64K accepts for a port at a single IP-address.
>
> No, it is essentially unlimited - linux uses local/remote addr/port
> tuples for hash chains, so there is no per-addr limits.
> If there is some kind of binds, then yes, only 64k ports per address.

Thanks, it clarified me the issue.
Probably, I am experiencing some local problem with
the web-server I am using for tests.


-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
A web testing and traffic generation tool.

Re: Beyond 64K TCP connections limit per IP-address

[Eric Dumazet]

On Wed, 4 Jul 2007 11:40:48 +0200
"Robert Iakobashvili" <coroberti@gmail.com> wrote:

> On 7/4/07, Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
> > On Wed, Jul 04, 2007 at 09:50:31AM +0200, Robert Iakobashvili (coroberti@gmail.com) wrote:
> > > If I am correct, a TCP server can make up to
> > > 64K accepts for a port at a single IP-address.
> >
> > No, it is essentially unlimited - linux uses local/remote addr/port
> > tuples for hash chains, so there is no per-addr limits.
> > If there is some kind of binds, then yes, only 64k ports per address.
> 
> Thanks, it clarified me the issue.
> Probably, I am experiencing some local problem with
> the web-server I am using for tests.

If your setup is :

Server A with one IP address listening to port 80

'Client B' with one IP address, trying to open many sockets to A (port 80)

Then yes you have a 64k limit for this particular client B. Just add 15 more clients (or 16 IP addresses on B) if you really want to stress A ;)

Re: Beyond 64K TCP connections limit per IP-address

[Robert Iakobashvili]

On 7/4/07, Eric Dumazet <dada1@cosmosbay.com> wrote:
> On Wed, 4 Jul 2007 11:40:48 +0200
> "Robert Iakobashvili" <coroberti@gmail.com> wrote:
>
> > On 7/4/07, Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
> > > On Wed, Jul 04, 2007 at 09:50:31AM +0200, Robert Iakobashvili (coroberti@gmail.com) wrote:
> > > > If I am correct, a TCP server can make up to
> > > > 64K accepts for a port at a single IP-address.
> > >
> > > No, it is essentially unlimited - linux uses local/remote addr/port
> > > tuples for hash chains, so there is no per-addr limits.
> > > If there is some kind of binds, then yes, only 64k ports per address.
> >
> > Thanks, it clarified me the issue.
> > Probably, I am experiencing some local problem with
> > the web-server I am using for tests.
>
> If your setup is :
> Server A with one IP address listening to port 80
> 'Client B' with one IP address, trying to open many sockets to A (port 80)
> Then yes you have a 64k limit for this particular client B. Just add 15 more clients (or 16 IP addresses on B) if you really want to stress A ;)

Actually, each client has its own secondary IP, and
the problem was in nginx server configuration (errare humanum est).
Thank you, Eric.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
A web testing and traffic generation tool.