default value of nf_conntrack_tcp_timeout_close_wait

default value of nf_conntrack_tcp_timeout_close_wait

[Joakim Axelsson]

Hi,

I've notices that the default value of
/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait (atleast in
kernel 2.6.20.4) is only set to 60 seconds. CLOSE_WAIT is the state where
one side has sent a FIN but not the other. Meaning we can still send data in
one direction. This is a state which can live a long time. Much longer than
just 60 seconds.

I did some googling on the issue to find previous discussions on netfilter
on the subject, and i can see that an old patch was sent in 2003 to kernel
to fix this problem. I can only guess that the newer conntrack code brought
the bug back:
http://www.linuxarkivet.se/mlists/netfilter-devel/0310/msg00016.html

It says there that the timeout is raised to 3 days. Sound more correct to
me.

To easy reproduce this you can use the command line tools "dict" where the
server closes the conntion up on answer directly but the client (you?) only
sends the FIN when closing the PAGER of dict (normally more or less).

Any thoughts on this?

/Joakim Axelsson

Re: default value of nf_conntrack_tcp_timeout_close_wait

[Henrik Nordstrom]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

ons 2007-04-04 klockan 14:23 +0200 skrev Joakim Axelsson:

> I've notices that the default value of
> /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait (atleast in
> kernel 2.6.20.4) is only set to 60 seconds. CLOSE_WAIT is the state where
> one side has sent a FIN but not the other. Meaning we can still send data in
> one direction. This is a state which can live a long time. Much longer than
> just 60 seconds.

I remember this.. was small due to being afraid that connections where
one of the endpoints have gone away may otherwise accumulate.

The best solution I can think of is to not look for FIN to detect such
connections but assymetric traffic where sent data (SYN, data or FIN) is
sent in one direction but no ACKs flowing in the other direction within
a reasonable timeframe (several minutes). Not perfect, but it's hard to
get perfect on this one..

Another idea to close the gaps further would be to drop the "assured"
bit on the connection when a FIN gets ACKed.

Regards
Henrik

[-- Attachment #2: Detta är en digitalt signerad meddelandedel --]
[-- Type: application/pgp-signature, Size: 307 bytes --]

Re: default value of nf_conntrack_tcp_timeout_close_wait

[Patrick McHardy]

Henrik Nordstrom wrote:
> ons 2007-04-04 klockan 14:23 +0200 skrev Joakim Axelsson:
> 
> 
>>I've notices that the default value of
>>/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait (atleast in
>>kernel 2.6.20.4) is only set to 60 seconds. CLOSE_WAIT is the state where
>>one side has sent a FIN but not the other. Meaning we can still send data in
>>one direction. This is a state which can live a long time. Much longer than
>>just 60 seconds.
> 
> 
> I remember this.. was small due to being afraid that connections where
> one of the endpoints have gone away may otherwise accumulate.
>
> The best solution I can think of is to not look for FIN to detect such
> connections but assymetric traffic where sent data (SYN, data or FIN) is
> sent in one direction but no ACKs flowing in the other direction within
> a reasonable timeframe (several minutes). Not perfect, but it's hard to
> get perfect on this one..


The idea is not bad IMO, but I think it would require an additional
timer, which pretty much kills it since that would grow struct
ip_conntrack by quite a lot.


> Another idea to close the gaps further would be to drop the "assured"
> bit on the connection when a FIN gets ACKed.


That sounds better, but I have to think about how this affects
the TCP conntrack internal use of the ASSURED bit.

Re: default value of nf_conntrack_tcp_timeout_close_wait

[Henrik Nordstrom]

[-- Attachment #1: Type: text/plain, Size: 581 bytes --]

ons 2007-04-04 klockan 17:42 +0200 skrev Patrick McHardy:

> The idea is not bad IMO, but I think it would require an additional
> timer, which pretty much kills it since that would grow struct
> ip_conntrack by quite a lot.

Should suffice with one bit I think, keeping track of which direction
last saw data and a new timeout value for "ACK_WAIT".

Packet with data (incl FIN) -> ACK_WAIT timeout
Bare ACK in opposite direction -> Actual timeout for current state

Requires "ACK_WAIT" timeout to be sufficiently large to cover retransmit
delays.

Regards
Henrik

[-- Attachment #2: Detta är en digitalt signerad meddelandedel --]
[-- Type: application/pgp-signature, Size: 307 bytes --]

sendto failed

[Manish Jain]

Hello,

We have some rate limiting rules on OUTPUT side on our box, so if a daemon
tries to send message beyond a limit, sendto() starts failing. Can we find
out from return value or errno that we had failed because of firewall.

Best Regards,
Manish Jain

Re: sendto failed

[Remi Denis-Courmont]

On Thu, 5 Apr 2007 17:09:12 +0530, "Manish Jain" <manish.jain@globallogic.com> wrote:
> We have some rate limiting rules on OUTPUT side on our box, so if a daemon
> tries to send message beyond a limit, sendto() starts failing. Can we find
> out from return value or errno that we had failed because of firewall.

IIRC, the DROP target will return EACCESS (or maybe it was EPERM?) if it's hit from a
local OUTPUT rule - you can check by yourself anyway. Also IIRC, grsecurity has some
patch to make the OUTPUT filtering silently fail.

Regards,

--
Rémi Denis-Courmont
http://www.remlab.net/

Re: default value of nf_conntrack_tcp_timeout_close_wait

[Jozsef Kadlecsik]

Hi,

On Wed, 4 Apr 2007, Henrik Nordstrom wrote:

> ons 2007-04-04 klockan 14:23 +0200 skrev Joakim Axelsson:
>
>> I've notices that the default value of
>> /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait (atleast in
>> kernel 2.6.20.4) is only set to 60 seconds. CLOSE_WAIT is the state where
>> one side has sent a FIN but not the other. Meaning we can still send data in
>> one direction. This is a state which can live a long time. Much longer than
>> just 60 seconds.
>
> I remember this.. was small due to being afraid that connections where
> one of the endpoints have gone away may otherwise accumulate.

Yes, this is a potential source to fill up the conntrack table by stale 
connections (or which looks like that).

> The best solution I can think of is to not look for FIN to detect such
> connections but assymetric traffic where sent data (SYN, data or FIN) is
> sent in one direction but no ACKs flowing in the other direction within
> a reasonable timeframe (several minutes). Not perfect, but it's hard to
> get perfect on this one..

As the timeout parameter controls how much time may pass without detecting 
any valid traffic in either directions, I'd say that raising the limit 
from 60s to 3d were too much - but that's a feeling only, not backed by 
real data.

> Another idea to close the gaps further would be to drop the "assured"
> bit on the connection when a FIN gets ACKed.

That could really help if we'd have a better (i.e. not much slower) 
algorithm to find potential connections to evict when the table is full. 
Currently we haven't got much chance to find a non-assured connection as 
it must be in the same hash chain as in which we want to insert a new 
connection.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
           H-1525 Budapest 114, POB. 49, Hungary

Re: default value of nf_conntrack_tcp_timeout_close_wait

[Patrick McHardy]

Joakim Axelsson wrote:
> Hi,
> 
> I've notices that the default value of
> /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait (atleast in
> kernel 2.6.20.4) is only set to 60 seconds. CLOSE_WAIT is the state where
> one side has sent a FIN but not the other. Meaning we can still send data in
> one direction. This is a state which can live a long time. Much longer than
> just 60 seconds.
> 
> I did some googling on the issue to find previous discussions on netfilter
> on the subject, and i can see that an old patch was sent in 2003 to kernel
> to fix this problem. I can only guess that the newer conntrack code brought
> the bug back:
> http://www.linuxarkivet.se/mlists/netfilter-devel/0310/msg00016.html


No, it was changed back without much explanation by this commit (from
the history.git tree):

[NETFILTER]: Sanitize ip_ct_tcp_timeout_close_wait value, from 2.4.x

---
commit b3cf20c77584ce8268be77032d305fe46da09ac6
tree 7c9737e62b90eb5b8c2378779f4ccb2d211dfdce
parent 474669a2743d375e7db3f0e4670c184335a38aee
author Harald Welte <laforge@netfilter.org> Tue, 02 Dec 2003 19:57:05 -0800
committer Linus Torvalds <torvalds@home.osdl.org> Tue, 02 Dec 2003
19:57:05 -0800

-unsigned long ip_ct_tcp_timeout_close_wait =    3 DAYS;
+unsigned long ip_ct_tcp_timeout_close_wait =   60 SECS;