/proc/net/ip_conntrack trange behavior

/proc/net/ip_conntrack trange behavior

[Dong_Wei]

Hi, all
	When I checked /proc/net/ip_conntrack in my Linux server, I found some 
strange tracks like the following:

[normal case]
tcp      6 387113 ESTABLISHED src=219.135.189.40 dst=58.60.9.41
sport=2391 dport=443 src=58.60.9.41 dst=219.135.189.40 sport=443
dport=2391 [ASSURED] use=1

[strange track]
tcp      6 377231 ESTABLISHED src=222.173.17.207 dst=219.135.189.11
sport=19691 dport=3815 [UNREPLIED] src=219.135.189.11 dst=222.173.17.207
sport=3815 dport=19691 use=1

As we kown after SYN->SYN,ACK->ACK. TCP connection is in ESTABLISHED
state. and the ip_conntrack "see" the REPLY direction packet, I think
the conntrak should be ASSURED. not UNREPLIED.

So, pls help me. My kernel version is 2.4.22

Thanks in advance  :-)

Re: /proc/net/ip_conntrack trange behavior

[Yasuyuki KOZAKAI]

From: Dong_Wei <Dong_Wei@nj.cpsecure.com>
Date: Fri, 24 Aug 2007 15:43:01 +0800

> Hi, all
> 	When I checked /proc/net/ip_conntrack in my Linux server, I found some 
> strange tracks like the following:
> 
> [normal case]
> tcp      6 387113 ESTABLISHED src=219.135.189.40 dst=58.60.9.41
> sport=2391 dport=443 src=58.60.9.41 dst=219.135.189.40 sport=443
> dport=2391 [ASSURED] use=1
> 
> [strange track]
> tcp      6 377231 ESTABLISHED src=222.173.17.207 dst=219.135.189.11
> sport=19691 dport=3815 [UNREPLIED] src=219.135.189.11 dst=222.173.17.207
> sport=3815 dport=19691 use=1
> 
> As we kown after SYN->SYN,ACK->ACK. TCP connection is in ESTABLISHED
> state. and the ip_conntrack "see" the REPLY direction packet, I think
> the conntrak should be ASSURED. not UNREPLIED.
> 
> So, pls help me. My kernel version is 2.4.22
> 
> Thanks in advance  :-)
> 

When only SYN and SYN/ACK are detected by ip_conntrack, the state of the
connection is ESTABLISHED. Please note that 'ESTABLISHED' is not TCP state,
but generic connection state.

-- Yasuyuki Kozakai

Re: /proc/net/ip_conntrack trange behavior

[Patrick McHardy]

Yasuyuki KOZAKAI wrote:
> From: Dong_Wei <Dong_Wei@nj.cpsecure.com>
> Date: Fri, 24 Aug 2007 15:43:01 +0800
> 
>> Hi, all
>> 	When I checked /proc/net/ip_conntrack in my Linux server, I found some 
>> strange tracks like the following:
>>
>> [normal case]
>> tcp      6 387113 ESTABLISHED src=219.135.189.40 dst=58.60.9.41
>> sport=2391 dport=443 src=58.60.9.41 dst=219.135.189.40 sport=443
>> dport=2391 [ASSURED] use=1
>>
>> [strange track]
>> tcp      6 377231 ESTABLISHED src=222.173.17.207 dst=219.135.189.11
>> sport=19691 dport=3815 [UNREPLIED] src=219.135.189.11 dst=222.173.17.207
>> sport=3815 dport=19691 use=1
>>
>> As we kown after SYN->SYN,ACK->ACK. TCP connection is in ESTABLISHED
>> state. and the ip_conntrack "see" the REPLY direction packet, I think
>> the conntrak should be ASSURED. not UNREPLIED.
>>
>> So, pls help me. My kernel version is 2.4.22
>>
>> Thanks in advance  :-)
>>
> 
> When only SYN and SYN/ACK are detected by ip_conntrack, the state of the
> connection is ESTABLISHED. Please note that 'ESTABLISHED' is not TCP state,
> but generic connection state.


Well, its TCP connection state. Whats probably happening is that you
have connection pickup enabled, in which case a single ACK can lead
to TCP ESTABLISHED state. It won't match "-m state --state ESTABLISHED"
though.

Re: /proc/net/ip_conntrack trange behavior

[Dong_Wei]

Thanks a lot. Yasuyuki and kaber

I have looked the 2.6.20 kernel and found that the reason.

When ip_conntrack try to pick up an ESTABLISHED TCP connection, this 
issue will happen.

2.6 set an 'sysctl' value to deal with this situation

static int tcp_new()
{
	...snip...
          if (new_state == TCP_CONNTRACK_SYN_SENT) {
		do_something();
          } else if (ip_ct_tcp_loose == 0) {
                  /* Don't try to pick up connections. */
                  return 0;
          } else {
		do_something();
          }
}

And we can't avoid this issue on 2.4, because no code deal with the 
special case.

Thanks!

> Yasuyuki KOZAKAI wrote:
>> From: Dong_Wei <Dong_Wei@nj.cpsecure.com>
>> Date: Fri, 24 Aug 2007 15:43:01 +0800
>>
>>> Hi, all
>>>     When I checked /proc/net/ip_conntrack in my Linux server, I found 
>>> some strange tracks like the following:
>>>
>>> [normal case]
>>> tcp      6 387113 ESTABLISHED src=219.135.189.40 dst=58.60.9.41
>>> sport=2391 dport=443 src=58.60.9.41 dst=219.135.189.40 sport=443
>>> dport=2391 [ASSURED] use=1
>>>
>>> [strange track]
>>> tcp      6 377231 ESTABLISHED src=222.173.17.207 dst=219.135.189.11
>>> sport=19691 dport=3815 [UNREPLIED] src=219.135.189.11 dst=222.173.17.207
>>> sport=3815 dport=19691 use=1
>>>
>>> As we kown after SYN->SYN,ACK->ACK. TCP connection is in ESTABLISHED
>>> state. and the ip_conntrack "see" the REPLY direction packet, I think
>>> the conntrak should be ASSURED. not UNREPLIED.
>>>
>>> So, pls help me. My kernel version is 2.4.22
>>>
>>> Thanks in advance  :-)
>>>
>>
>> When only SYN and SYN/ACK are detected by ip_conntrack, the state of the
>> connection is ESTABLISHED. Please note that 'ESTABLISHED' is not TCP 
>> state,
>> but generic connection state.
> 
> 
> Well, its TCP connection state. Whats probably happening is that you
> have connection pickup enabled, in which case a single ACK can lead
> to TCP ESTABLISHED state. It won't match "-m state --state ESTABLISHED"
> though.
> 

Re: /proc/net/ip_conntrack trange behavior

[Patrick McHardy]

Dong_Wei wrote:
> Thanks a lot. Yasuyuki and kaber
> 
> I have looked the 2.6.20 kernel and found that the reason.
> 
> When ip_conntrack try to pick up an ESTABLISHED TCP connection, this 
> issue will happen.


Exactly.

> 2.6 set an 'sysctl' value to deal with this situation
> 
> [..]
> 
> And we can't avoid this issue on 2.4, because no code deal with the 
> special case.


Thats correct. Is there a problem with this behaviour?

Re: /proc/net/ip_conntrack trange behavior

[Dong_Wei]

Hi Yasuyuki san

> Dong_Wei wrote:
>> Thanks a lot. Yasuyuki and kaber
>>
>> I have looked the 2.6.20 kernel and found that the reason.
>>
>> When ip_conntrack try to pick up an ESTABLISHED TCP connection, this 
>> issue will happen.
> 
> 
> Exactly.
> 
>> 2.6 set an 'sysctl' value to deal with this situation
>>
>> [..]
>>
>> And we can't avoid this issue on 2.4, because no code deal with the 
>> special case.
> 
> 
> Thats correct. Is there a problem with this behaviour?

As we know, ip_conntrack has a hash_size to control the ip_conntrack 
record size. and if tcp in ESTABLISH, and ip_conntrack will keep it for 
5 DAYs.

For exsample, a NAT server can handle 8000 connections, and if sometime, 
NAT server need reboot(6000 conntrack in ESTBLISHED state now). Also we 
set ip_conntrack allowing pickup ESTABLISHED tcp connection, when NAT 
server available again, ip_conntrack will take 6000 conntracks for the 
original connections. but actully these connections are INVALID for the 
web server, because the source ip is the private IPs,such as 
192.168.0.1. and web server will not answer the client's request. But 
still the NAT server will keep the 6000 conntrack for 5 DAYs. And just 
2000 conntracks can be used for the clients. If more than 2000 client 
want connect to some websites, some ip_conntrack will be droped.

BR. DongWei

Re: /proc/net/ip_conntrack trange behavior

[Patrick McHardy]

On Thu, 30 Aug 2007, Dong_Wei wrote:

>>> And we can't avoid this issue on 2.4, because no code deal with the 
>>> special case.
>> 
>> 
>> Thats correct. Is there a problem with this behaviour?
>
> As we know, ip_conntrack has a hash_size to control the ip_conntrack record 
> size. and if tcp in ESTABLISH, and ip_conntrack will keep it for 5 DAYs.
>
> For exsample, a NAT server can handle 8000 connections, and if sometime, NAT 
> server need reboot(6000 conntrack in ESTBLISHED state now). Also we set 
> ip_conntrack allowing pickup ESTABLISHED tcp connection, when NAT server 
> available again, ip_conntrack will take 6000 conntracks for the original 
> connections. but actully these connections are INVALID for the web server, 
> because the source ip is the private IPs,such as 192.168.0.1. and web server 
> will not answer the client's request. But still the NAT server will keep the 
> 6000 conntrack for 5 DAYs. And just 2000 conntracks can be used for the 
> clients. If more than 2000 client want connect to some websites, some 
> ip_conntrack will be droped.


You can drop ACK packets in state NEW to avoid connection pickup.

Re: /proc/net/ip_conntrack trange behavior

[Dong_Wei]

Hi Patrick

> On Thu, 30 Aug 2007, Dong_Wei wrote:
> 
>>>> And we can't avoid this issue on 2.4, because no code deal with the 
>>>> special case.
>>>
>>>
>>> Thats correct. Is there a problem with this behaviour?
>>
>> As we know, ip_conntrack has a hash_size to control the ip_conntrack 
>> record size. and if tcp in ESTABLISH, and ip_conntrack will keep it 
>> for 5 DAYs.
>>
>> For exsample, a NAT server can handle 8000 connections, and if 
>> sometime, NAT server need reboot(6000 conntrack in ESTBLISHED state 
>> now). Also we set ip_conntrack allowing pickup ESTABLISHED tcp 
>> connection, when NAT server available again, ip_conntrack will take 
>> 6000 conntracks for the original connections. but actully these 
>> connections are INVALID for the web server, because the source ip is 
>> the private IPs,such as 192.168.0.1. and web server will not answer 
>> the client's request. But still the NAT server will keep the 6000 
>> conntrack for 5 DAYs. And just 2000 conntracks can be used for the 
>> clients. If more than 2000 client want connect to some websites, some 
>> ip_conntrack will be droped.
> 
> 
> You can drop ACK packets in state NEW to avoid connection pickup.
That's a good idea, I will make a patch for my own 2.4 Linux NAT server, 
thanks a lot :-)

Re: /proc/net/ip_conntrack trange behavior

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1492 bytes --]



On Fri, 31 Aug 2007, Dong_Wei wrote:

> Hi Patrick
>
>> On Thu, 30 Aug 2007, Dong_Wei wrote:
>> 
>>>>> And we can't avoid this issue on 2.4, because no code deal with the 
>>>>> special case.
>>>> 
>>>> 
>>>> Thats correct. Is there a problem with this behaviour?
>>> 
>>> As we know, ip_conntrack has a hash_size to control the ip_conntrack 
>>> record size. and if tcp in ESTABLISH, and ip_conntrack will keep it for 5 
>>> DAYs.
>>> 
>>> For exsample, a NAT server can handle 8000 connections, and if sometime, 
>>> NAT server need reboot(6000 conntrack in ESTBLISHED state now). Also we 
>>> set ip_conntrack allowing pickup ESTABLISHED tcp connection, when NAT 
>>> server available again, ip_conntrack will take 6000 conntracks for the 
>>> original connections. but actully these connections are INVALID for the 
>>> web server, because the source ip is the private IPs,such as 192.168.0.1. 
>>> and web server will not answer the client's request. But still the NAT 
>>> server will keep the 6000 conntrack for 5 DAYs. And just 2000 conntracks 
>>> can be used for the clients. If more than 2000 client want connect to some 
>>> websites, some ip_conntrack will be droped.
>> 
>> 
>> You can drop ACK packets in state NEW to avoid connection pickup.
> That's a good idea, I will make a patch for my own 2.4 Linux NAT server, 
> thanks a lot :-)

No need to, as all you should do is a proper iptables rule.

Best regards,

 				Krzysztof Olędzki

Re: /proc/net/ip_conntrack trange behavior

[Dong_Wei]

Hi


> On Fri, 31 Aug 2007, Dong_Wei wrote:
> 
>> Hi Patrick
>>
>>> On Thu, 30 Aug 2007, Dong_Wei wrote:
>>>
>>>>>> And we can't avoid this issue on 2.4, because no code deal with 
>>>>>> the special case.
>>>>>
>>>>>
>>>>> Thats correct. Is there a problem with this behaviour?
>>>>
>>>> As we know, ip_conntrack has a hash_size to control the ip_conntrack 
>>>> record size. and if tcp in ESTABLISH, and ip_conntrack will keep it 
>>>> for 5 DAYs.
>>>>
>>>> For exsample, a NAT server can handle 8000 connections, and if 
>>>> sometime, NAT server need reboot(6000 conntrack in ESTBLISHED state 
>>>> now). Also we set ip_conntrack allowing pickup ESTABLISHED tcp 
>>>> connection, when NAT server available again, ip_conntrack will take 
>>>> 6000 conntracks for the original connections. but actully these 
>>>> connections are INVALID for the web server, because the source ip is 
>>>> the private IPs,such as 192.168.0.1. and web server will not answer 
>>>> the client's request. But still the NAT server will keep the 6000 
>>>> conntrack for 5 DAYs. And just 2000 conntracks can be used for the 
>>>> clients. If more than 2000 client want connect to some websites, 
>>>> some ip_conntrack will be droped.
>>>
>>>
>>> You can drop ACK packets in state NEW to avoid connection pickup.
>> That's a good idea, I will make a patch for my own 2.4 Linux NAT 
>> server, thanks a lot :-)
> 
> No need to, as all you should do is a proper iptables rule.
what kind of rules should I take? It's seemed that there is no rule to 
deal with this special case on 2.4 :(

> Best regards,
> 
>                 Krzysztof Olędzki

Re: /proc/net/ip_conntrack trange behavior

[Philip Craig]

Dong_Wei wrote:
> what kind of rules should I take? It's seemed that there is no rule to 
> deal with this special case on 2.4 :(

$IPTABLES -A INPUT -p tcp ! --syn -m state --state NEW -j DROP

(From http://iptables-tutorial.frozentux.net/iptables-tutorial.html#NEWNOTSYN)

Re: /proc/net/ip_conntrack trange behavior

[Dong_Wei]

> Dong_Wei wrote:
>> what kind of rules should I take? It's seemed that there is no rule to 
>> deal with this special case on 2.4 :(
> 
> $IPTABLES -A INPUT -p tcp ! --syn -m state --state NEW -j DROP
Thanks a lot Philip :)