ip_conntrack table is full with razor requests. Something isn't timing out.

ip_conntrack table is full with razor requests. Something isn't timing out.

[Matthew Schumacher]

List,

Since upgrading to 2.6.10 I have been having problems with my 
ip_conntrack table filling up.  It appears it is full of razor 
(http://razor.sf.net) requests from my internal mail server.

I raised the ip_conntrack_max to 8192 and there are only a few hosts 
behind nat so I am certain something isn't getting flushed out.

How do I go about diagnosing this.  What specifically does ip_conntrack 
need to see in the tcp session to mark the session as expired in the table?

Thanks,

schu

tcp      6 424864 ESTABLISHED src=192.168.98.2 dst=66.151.150.12 
sport=51075 dport=2703 packets=9 bytes=421 src=66.151.150.12 
dst=64.181.100.18 sport=2703 dport=51075 packets=7 bytes=577 [ASSURED] 
mark=0 use=1
tcp      6 401401 ESTABLISHED src=192.168.98.2 dst=66.151.150.35 
sport=50791 dport=2703 packets=7 bytes=317 src=66.151.150.35 
dst=64.181.100.18 sport=2703 dport=50791 packets=5 bytes=370 [ASSURED] 
mark=0 use=1
tcp      6 393358 ESTABLISHED src=192.168.98.2 dst=66.151.150.12 
sport=50627 dport=2703 packets=8 bytes=365 src=66.151.150.12 
dst=64.181.100.18 sport=2703 dport=50627 packets=5 bytes=370 [ASSURED] 
mark=0 use=1
tcp      6 376557 ESTABLISHED src=192.168.98.2 dst=66.151.150.35 
sport=49950 dport=2703 packets=9 bytes=421 src=66.151.150.35 
dst=64.181.100.18 sport=2703 dport=49950 packets=7 bytes=577 [ASSURED] 
mark=0 use=1
tcp      6 369805 ESTABLISHED src=192.168.98.2 dst=66.151.150.12 
sport=48990 dport=2703 packets=7 bytes=317 src=66.151.150.12 
dst=64.181.100.18 sport=2703 dport=48990 packets=5 bytes=370 [ASSURED] 
mark=0 use=1
tcp      6 368538 ESTABLISHED src=192.168.98.2 dst=66.151.150.35 
sport=48738 dport=2703 packets=7 bytes=317 src=66.151.150.35 
dst=64.181.100.18 sport=2703 dport=48738 packets=5 bytes=370 [ASSURED] 
mark=0 use=1
tcp      6 365641 ESTABLISHED src=192.168.98.2 dst=66.151.150.12 
sport=47914 dport=2703 packets=9 bytes=421 src=66.151.150.12 
dst=64.181.100.18 sport=2703 dport=47914 packets=7 bytes=577 [ASSURED] 
mark=0 use=1

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[Michael Tautschnig]

Hello!

>
> Since upgrading to 2.6.10 I have been having problems with my ip_conntrack 
> table filling up.  It appears it is full of razor (http://razor.sf.net) 
> requests from my internal mail server.
>
So I'm not the only one with these problems - I can confirm this 
completely. It's razor and kernel 2.6.10 here too.

[...]

It would be great, if somebody could help us - or tell us, what 
information one should supply...

Thanks in advance,
Michael

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[Matthew Schumacher]

Michael Tautschnig wrote:
> Hello!
> 
>>
>> Since upgrading to 2.6.10 I have been having problems with my 
>> ip_conntrack table filling up.  It appears it is full of razor 
>> (http://razor.sf.net) requests from my internal mail server.
>>
> So I'm not the only one with these problems - I can confirm this 
> completely. It's razor and kernel 2.6.10 here too.
> 
> [...]
> 
> It would be great, if somebody could help us - or tell us, what 
> information one should supply...
> 
> Thanks in advance,
> Michael

Michael,

Thanks for your note on this... looks like a netfilter bug to me.  Did 
you have any problems running 2.6.9?  I am looking at the diffs and it 
looks like there are a number of changes to the ip_conntrack code in 
2.6.10.  When reading though the changelog for 2.6.11-rc5 it doesn't 
appear that any of these issues are resolved, so I may have to go back 
to 2.6.9 if it is known to work.

schu

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[Jozsef Kadlecsik]

Hi,

On Tue, 1 Mar 2005, Matthew Schumacher wrote:

> Since upgrading to 2.6.10 I have been having problems with my
> ip_conntrack table filling up.  It appears it is full of razor
> (http://razor.sf.net) requests from my internal mail server.
>
> I raised the ip_conntrack_max to 8192 and there are only a few hosts
> behind nat so I am certain something isn't getting flushed out.
>
> How do I go about diagnosing this.  What specifically does ip_conntrack
> need to see in the tcp session to mark the session as expired in the table?

Run tcpdump and record at least one full session of the razor traffic.
Best is if you capture the traffic on both side of the firewall in order
to make sure nothing got lost. Collect anything relevant from the
kernel log file and attach the /proc/net/ip_conntrack lines referring to
the session. Post the collected data and then we can start to hunt down
the reason of the problem.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[KOVACS Krisztian]

  Hi,

2005-03-01, k keltezéssel 14.47-kor Matthew Schumacher ezt írta:
> >> Since upgrading to 2.6.10 I have been having problems with my 
> >> ip_conntrack table filling up.  It appears it is full of razor 
> >> (http://razor.sf.net) requests from my internal mail server.
> >>
> > So I'm not the only one with these problems - I can confirm this 
> > completely. It's razor and kernel 2.6.10 here too.
> > 
> > [...]
> > 
> > It would be great, if somebody could help us - or tell us, what 
> > information one should supply...
> > 
> > Thanks in advance,
> > Michael
> 
> Michael,
> 
> Thanks for your note on this... looks like a netfilter bug to me.  Did 
> you have any problems running 2.6.9?  I am looking at the diffs and it 
> looks like there are a number of changes to the ip_conntrack code in 
> 2.6.10.  When reading though the changelog for 2.6.11-rc5 it doesn't 
> appear that any of these issues are resolved, so I may have to go back 
> to 2.6.9 if it is known to work.

  Could you try whether or not the following patch fixes your problem?

https://lists.netfilter.org/pipermail/netfilter-devel/2004-December/017908.html

-- 
KOVACS Krisztian <hidden@sch.bme.hu>

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[Michael Tautschnig]

>>
>> Thanks for your note on this... looks like a netfilter bug to me.  Did
>> you have any problems running 2.6.9?  I am looking at the diffs and it
>> looks like there are a number of changes to the ip_conntrack code in
>> 2.6.10.  When reading though the changelog for 2.6.11-rc5 it doesn't
>> appear that any of these issues are resolved, so I may have to go back
>> to 2.6.9 if it is known to work.
>
>  Could you try whether or not the following patch fixes your problem?
>
> https://lists.netfilter.org/pipermail/netfilter-devel/2004-December/017908.html
>
Thanks - it seems as if it did the trick! Connections are set up properly 
and marked as CLOSE afterwards, and cleaned out properly. Great!

Thank you very much,
Michael

Re: ip_conntrack table is full with razor requests. Something isn't timing out.

[Matthew Schumacher]

Jozsef Kadlecsik wrote:
> Hi,
> 
> On Tue, 1 Mar 2005, Matthew Schumacher wrote:
> 
> 
>>Since upgrading to 2.6.10 I have been having problems with my
>>ip_conntrack table filling up.  It appears it is full of razor
>>(http://razor.sf.net) requests from my internal mail server.
>>
>>I raised the ip_conntrack_max to 8192 and there are only a few hosts
>>behind nat so I am certain something isn't getting flushed out.
>>
>>How do I go about diagnosing this.  What specifically does ip_conntrack
>>need to see in the tcp session to mark the session as expired in the table?
> 
> 
> Run tcpdump and record at least one full session of the razor traffic.
> Best is if you capture the traffic on both side of the firewall in order
> to make sure nothing got lost. Collect anything relevant from the
> kernel log file and attach the /proc/net/ip_conntrack lines referring to
> the session. Post the collected data and then we can start to hunt down
> the reason of the problem.
> 

Jozsef,

Here is the relevant part of the dump:


10:15:34.144668 IP 66.151.150.24.2703 > 64.4.232.33.54120: P 37:47(10) 
ack 78 win 5840
10:15:34.147076 IP 64.4.232.33.54120 > 66.151.150.24.2703: . ack 47 win 
49640
10:15:34.151703 IP 64.4.232.33.54120 > 66.151.150.24.2703: P 78:83(5) 
ack 47 win 49640
10:15:34.153156 IP 64.4.232.33.54120 > 66.151.150.24.2703: F 83:83(0) 
ack 47 win 49640
10:15:34.217685 IP 66.151.150.24.2703 > 64.4.232.33.54120: . ack 83 win 5840
10:15:34.256491 IP 66.151.150.24.2703 > 64.4.232.33.54120: . ack 84 win 5840
10:15:34.311607 IP 66.151.150.24.2703 > 64.4.232.33.54120: R 47:47(0) 
ack 84 win 5840

Since there is a ACK before the RST it is safe to assume that the bug 
documented at 
https://lists.netfilter.org/pipermail/netfilter-devel/2004-December/017908.html 
was the problem.

Since kernel 2.6.11 came out this morning, and the bug fix appears in 
the changelog, I went ahead and upgraded and the problems are resolved.

If you would like more information then let me know, but at this point, 
I think it's safe to say that this bug is squashed.

schu