[PATCH net-next] tcp: more documentation for tcp_tw_reuse and tcp_tw_recycle

[PATCH net-next] tcp: more documentation for tcp_tw_reuse and tcp_tw_recycle

[Vincent Bernat]

Hi!

I think this is not the first tentative to try to complete the
documentation of tcp_tw_reuse and tcp_tw_recycle. Last one was 4 years
ago I think. Here is another one which tries to stay short, warns
about the dangers of recycling and which kind of connections are
affected. The general warning is kept.

[PATCH] tcp: more documentation for tcp_tw_reuse and tcp_tw_recycle

[Vincent Bernat]

The documentation is not very helpful about what those settings
affect. We find numerous tuning guides advising to set both these
settings to 1 to reduce the number of entries in the TIME-WAIT
state. However, enabling tcp_tw_recycle will cause massive problems when
working with NAT.

The documentation is completed a bit to explain quickly what kind of
connections both those settings will affect and to encourage the use of
tcp_tw_reuse instead of tcp_tw_recycle for outgoing connections.
---
 Documentation/networking/ip-sysctl.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt
index ab42c95f9985..d771aa064af8 100644
--- a/Documentation/networking/ip-sysctl.txt
+++ b/Documentation/networking/ip-sysctl.txt
@@ -532,14 +532,18 @@ tcp_tso_win_divisor - INTEGER
 	Default: 3
 
 tcp_tw_recycle - BOOLEAN
-	Enable fast recycling TIME-WAIT sockets. Default value is 0.
-	It should not be changed without advice/request of technical
+	Enable fast recycling TIME-WAIT sockets. Default value is
+	0. Enabling this option will cause problems when incoming
+	connections originate from NAT devices. For outgoing
+	connections, prefer the use of tcp_tw_reuse which is safer. It
+	should not be changed without advice/request of technical
 	experts.
 
 tcp_tw_reuse - BOOLEAN
-	Allow to reuse TIME-WAIT sockets for new connections when it is
-	safe from protocol viewpoint. Default value is 0.
-	It should not be changed without advice/request of technical
+	Allow to reuse TIME-WAIT sockets for new outgoing connections
+	when it is safe from protocol viewpoint. Default value is
+	0. This parameter has no effect on incoming connections.  It
+	should not be changed without advice/request of technical
 	experts.
 
 tcp_window_scaling - BOOLEAN
-- 
2.0.0.rc0

Re: [PATCH] tcp: more documentation for tcp_tw_reuse and tcp_tw_recycle

[David Miller]

From: Vincent Bernat <vincent@bernat.im>
Date: Sun,  4 May 2014 11:41:39 +0200

> The documentation is not very helpful about what those settings
> affect. We find numerous tuning guides advising to set both these
> settings to 1 to reduce the number of entries in the TIME-WAIT
> state. However, enabling tcp_tw_recycle will cause massive problems when
> working with NAT.
> 
> The documentation is completed a bit to explain quickly what kind of
> connections both those settings will affect and to encourage the use of
> tcp_tw_reuse instead of tcp_tw_recycle for outgoing connections.

First of all your change locks a proper signoff.

Second of all, both options can cause problems in the presence of NAT
because both optimizations assume unique IP addresses identify unique
physical hosts.

Re: [PATCH] tcp: more documentation for tcp_tw_reuse and tcp_tw_recycle

[Vincent Bernat]

 ❦  7 mai 2014 15:16 -0400, David Miller <davem@davemloft.net> :

>> The documentation is not very helpful about what those settings
>> affect. We find numerous tuning guides advising to set both these
>> settings to 1 to reduce the number of entries in the TIME-WAIT
>> state. However, enabling tcp_tw_recycle will cause massive problems when
>> working with NAT.
>> 
>> The documentation is completed a bit to explain quickly what kind of
>> connections both those settings will affect and to encourage the use of
>> tcp_tw_reuse instead of tcp_tw_recycle for outgoing connections.
>
> First of all your change locks a proper signoff.

Sorry for that. I'll resend once the other problem is fixed.

> Second of all, both options can cause problems in the presence of NAT
> because both optimizations assume unique IP addresses identify unique
> physical hosts.

If NAT is done at the remote end (the outgoing connection is to some
load-balanced VIP) and if a TW state is reused for another
host than the original host, this can be one of those cases:

 1. There was no prior connection to the other host, so no problem.
 
 2. There was a prior connection to this host and the TW state has
    properly expired (60-second regular timeout), no problem.
 
 3. There was a prior connection to this host and the TW state has been
    reused previously, so we are already in the right condition
    (timestamps) to do the same thing.

I don't see a scenario where NAT can be a problem with tcp_tw_reuse.

If the NAT is done on the local end (we are behind a NAT device), as the
TW is on our side, I don't see what problem could have the remote end
which has properly closed the connection. Even without tcp_tw_reuse, the
remote side could get a legit connection from another local host.

We could be on the safe side and say that both settings may interact
badly with NAT gateways (or any altering gateways), but in this case,
both settings will look the same and the documentation will be as
unhelpful as it is now to someone which insists on using those settings.
-- 
 /* After several hours of tedious analysis, the following hash
  * function won.  Do not mess with it... -DaveM
  */
	2.2.16 /usr/src/linux/fs/buffer.c