Re: auto recycling of TIME_WAIT connections

["Pádraig Brady" <P@draigBrady.com>]

Rick Jones wrote:
>> The first issue, requires a large timeout, and
>> the TIME_WAIT timeout is currently 60 seconds on linux.
>> That timeout effectively limits the connection rate between
>> local TCP clients and a server to 32k/60s or around 500
>> connections/second.
> 
> Actually, it would be more like 60k/60s if the application were making
> explicit calls to bind() as arguably it should if it is going to be
> churning through so many connections.


> This was an issue over a decade ago with SPECweb96 benchmarking.  The
> initial solution was to make the explicit bind() calls and not rely on
> the anonymous/ephemeral port space.  After that, one starts adding
> additional IP's into the mix (at least where possible).  And if that
> fails, one has to go back to the beginning and ask oneself exactly why a
> client is trying to churn through so many connections per second in the
> first place.

right. This is for benchmarking mainly.
Sane applications would use persistent connections,
or a different form of IPC.

> 
> If we were slavishly conformant to the RFC's :) that 60 seconds would be
> 240 seconds...
> 
>> But that issue can't really happen when the client
>> and server are on the same machine can it, and
>> even if it could, the timeouts involved would be shorter.
>>
>> Now linux does have an (undocumented)
>> /proc/sys/net/ipv4/tcp_tw_recycle flag
>> to enable recycling of TIME_WAIT connections. This is global however
>> and could cause
>> problems in general for external connections.
> 
> Rampant speculation begins...
> 
> If the client can be convinced to just call shutdown(SHUT_RDWR) rather
> than close(), and be the first to do so, ahead of the server, I think it
> will retain a link to the TCP endpoint in TIME_WAIT.  It could then, in
> TCP theory, call connect() again, and go through a path that allows
> transition from TIME_WAIT to ESTABLISHED if all the right things wrt
> Initial Sequence Number selection happen.  Whether randomization of the
> ISN allows that today is questionable.

Sounds good, unfortunately connect() returns EISCONN
unless you do a close().

> 
>> So how about auto enabling recycling for local connections?
> 
> I think the standard response is that one can never _really_ know what
> is local and what not, particularly in the presence of netfilter and the
> rewriting of headers behind one's back.

Hmm, I was afraid someone would say that :)

thanks for the feedback,
Pádraig.