ipt_MASQUERADE weirdness (consuming CPU cycles while not used)

ipt_MASQUERADE weirdness (consuming CPU cycles while not used)

[Denys Fedoryschenko]

I have loaded pppoe (1700 users). I test one rule for short time with -j 
MASQUERADE, then removed it and reset conntrack (conntrack -F). But still i 
can see it is consuming CPU even when it is not used in any rule. Even i 
reboot server and just load rules that dont have MASQUERADE, and just load 
module - it will start consuming CPU immediately.

Here is details:

Chain PREROUTING (policy ACCEPT 8232K packets, 473M bytes)
 pkts bytes target     prot opt in     out     source               
destination
 299K   32M ACCEPT     all  --  *      *       0.0.0.0/0            
192.168.0.0/16
 161K   14M ACCEPT     all  --  *      *       0.0.0.0/0            
194.146.152.0/22
 8396  811K ACCEPT     all  --  *      *       0.0.0.0/0            10.0.0.0/8
 445K  126M ACCEPT     all  --  *      *       0.0.0.0/0            
172.16.0.0/16
 268K   17M ACCEPT     all  --  *      *       0.0.0.0/0            2.0.0.0/8
 914K   47M DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           
tcp dpt:80 to:192.168.254.2-192.168.254.7:8080
40206 2579K DNAT       udp  --  *      *       0.0.0.0/0           !1.2.3.4             
udp dpt:53 to:1.2.3.4:53

Chain POSTROUTING (policy ACCEPT 534K packets, 159M bytes)
 pkts bytes target     prot opt in     out     source               
destination
68011 7381K ACCEPT     all  --  *      *       0.0.0.0/0            
194.146.XXX.XXX/22
 268K   18M ACCEPT     all  --  *      *       0.0.0.0/0            2.2.2.0/24
1845K  119M SNAT       all  --  *      eth0    172.16.0.0/16        0.0.0.0/0           
to:194.146.XXX.XXX

Chain OUTPUT (policy ACCEPT 97383 packets, 37M bytes)
 pkts bytes target     prot opt in     out     source               
destination


defaulthost ~ # lsmod|grep MASQ
ipt_MASQUERADE          2348  0

Oprofile:

samples  %        image name               app name                 symbol 
name
163475    9.5182  libc-2.9.so              
libc-2.9.so              /lib/libc-2.9.so
65095     3.7901  vmlinux                  vmlinux                  
memory_open
64811     3.7735  ipt_MASQUERADE           ipt_MASQUERADE           device_cmp
57111     3.3252  e1000                    e1000                    e1000_intr
55725     3.2445  e1000                    e1000                    
e1000_clean

Re: ipt_MASQUERADE weirdness (consuming CPU cycles while not used)

[Pablo Neira Ayuso]

Denys Fedoryschenko wrote:
> I have loaded pppoe (1700 users). I test one rule for short time with -j 
> MASQUERADE, then removed it and reset conntrack (conntrack -F). But still i 
> can see it is consuming CPU even when it is not used in any rule. Even i 
> reboot server and just load rules that dont have MASQUERADE, and just load 
> module - it will start consuming CPU immediately.

Are you using 2.6.29 with any conntrack helper loaded? In that case this
fix is not in -stable yet.

http://kerneltrap.org/mailarchive/linux-netdev/2009/4/8/5440564

> 64811     3.7735  ipt_MASQUERADE           ipt_MASQUERADE           device_cmp

device_cmp() by nf_ct_iterate_cleanup() when NETDEV_DOWN event is
received. Weird, is your device going down quite often? Another
possibility is that there's some entry stuck in the conntrack table that
we cannot delete, perhaps we're leaking refcounts somewhere.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: ipt_MASQUERADE weirdness (consuming CPU cycles while not used)

[Denys Fedoryschenko]

On Thursday 21 May 2009 21:00:52 Pablo Neira Ayuso wrote:
> Denys Fedoryschenko wrote:
> > I have loaded pppoe (1700 users). I test one rule for short time with -j
> > MASQUERADE, then removed it and reset conntrack (conntrack -F). But still
> > i can see it is consuming CPU even when it is not used in any rule. Even
> > i reboot server and just load rules that dont have MASQUERADE, and just
> > load module - it will start consuming CPU immediately.
>
> Are you using 2.6.29 with any conntrack helper loaded? In that case this
> fix is not in -stable yet.
>
> http://kerneltrap.org/mailarchive/linux-netdev/2009/4/8/5440564
>
> > 64811     3.7735  ipt_MASQUERADE           ipt_MASQUERADE          
> > device_cmp
>
> device_cmp() by nf_ct_iterate_cleanup() when NETDEV_DOWN event is
> received. Weird, is your device going down quite often? Another
> possibility is that there's some entry stuck in the conntrack table that
> we cannot delete, perhaps we're leaking refcounts somewhere.
It is loaded pppoe server (2k interfaces), sure they are 
appearing-disappearing non-stop. Thats maybe case, but weird that it is 
consuming CPU time while module not used at all anywhere (no rules with 
MASQUERADE).

Re: ipt_MASQUERADE weirdness (consuming CPU cycles while not used)

[Patrick McHardy]

Denys Fedoryschenko wrote:
> On Thursday 21 May 2009 21:00:52 Pablo Neira Ayuso wrote:
>> Denys Fedoryschenko wrote:
>>> I have loaded pppoe (1700 users). I test one rule for short time with -j
>>> MASQUERADE, then removed it and reset conntrack (conntrack -F). But still
>>> i can see it is consuming CPU even when it is not used in any rule. Even
>>> i reboot server and just load rules that dont have MASQUERADE, and just
>>> load module - it will start consuming CPU immediately.
>> Are you using 2.6.29 with any conntrack helper loaded? In that case this
>> fix is not in -stable yet.
>>
>> http://kerneltrap.org/mailarchive/linux-netdev/2009/4/8/5440564
>>
>>> 64811     3.7735  ipt_MASQUERADE           ipt_MASQUERADE          
>>> device_cmp
>> device_cmp() by nf_ct_iterate_cleanup() when NETDEV_DOWN event is
>> received. Weird, is your device going down quite often? Another
>> possibility is that there's some entry stuck in the conntrack table that
>> we cannot delete, perhaps we're leaking refcounts somewhere.
 >
> It is loaded pppoe server (2k interfaces), sure they are 
> appearing-disappearing non-stop. Thats maybe case, but weird that it is 
> consuming CPU time while module not used at all anywhere (no rules with 
> MASQUERADE).

It doesn't know that until it has iterated over the conntrack table
and looked at all the entries. We could add a module parameter to
disable the "autoclean" feature, but it seems easier to just not
load it if you don't actually need it.