re:re: Is this nf_conn "use" counter bug???

re:re: Is this nf_conn "use" counter bug???

[kendo]

----- Original Message ----- 
From: "kendo" <kendo999@sohu.com>
To: "Patrick McHardy" <kaber@trash.net>
Sent: Monday, November 23, 2009 7:12 PM
Subject: Re: Is this nf_conn "use" counter bug???


> 
>> This code looks like from the 2.4 era.
> ----I'm sorry, I copy from 2.6.12,not 2.6.30.9.Because I also opened up them.
> 
>> Most likely there's an skb leak somewhere and the conntrack references
>> are not released.
> ----Yes, I think so.But why is that so??kfree_skb function has not been properly invoked when skb leave?
> 
>>> Environment:
>>> -----internet--cisco switch (two port bonding)-----linux box(two bridge)----cisco switch(two port bonging)---server
>>> 
>>> Cisco switch link aggregation，use two port.Linux box have two Independent network bridge.
>> 
>                                           (ge0/1) ---------------(eth0) (eth1)------------(ge1/0/1)
> ---interface-----cisco switch 1                           linux box                           cisco switch 2--------servers
>                                           (ge0/2)----------------(eth2) (eth3)------------(ge1/0/1)
> 
> switch 1:
> port 0/1 and port 0/2 link aggregation
> 
> switch 2:
> port 1/0/1 and port 1/0/2 link aggregation
> 
> linux config:
>    eth0, eth1 => bridge0 up, no ip, stp off.
>    eth2, eth3 => bridge1 up, no ip, stp off.
>    2.6.30.9
>    xeon 5420 *8
>    intel 82575 * 8(version 2.0.6 or linux 2.6.30.9 inside)
> iptables rule:
>    no any thing
> 
> no other config.
> 
> thanks
> 
> kendo

Re: Is this nf_conn "use" counter bug???

[Patrick McHardy]

kendo wrote:
> ----- Original Message ----- 
> From: "kendo" <kendo999@sohu.com>
> To: "Patrick McHardy" <kaber@trash.net>
> Sent: Monday, November 23, 2009 7:12 PM
> Subject: Re: Is this nf_conn "use" counter bug???
> 
> 
>>> This code looks like from the 2.4 era.
>> ----I'm sorry, I copy from 2.6.12,not 2.6.30.9.Because I also opened up them.
>>
>>> Most likely there's an skb leak somewhere and the conntrack references
>>> are not released.
>> ----Yes, I think so.But why is that so??kfree_skb function has not been properly invoked when skb leave?

Possibly. You can check the skbuff caches in /proc/slabinfo to see
whether this is the case.

>>>> Environment:
>>>> -----internet--cisco switch (two port bonding)-----linux box(two bridge)----cisco switch(two port bonging)---server
>>>>
>>>> Cisco switch link aggregation，use two port.Linux box have two Independent network bridge.
>>                                           (ge0/1) ---------------(eth0) (eth1)------------(ge1/0/1)
>> ---interface-----cisco switch 1                           linux box                           cisco switch 2--------servers
>>                                           (ge0/2)----------------(eth2) (eth3)------------(ge1/0/1)
>>
>> switch 1:
>> port 0/1 and port 0/2 link aggregation
>>
>> switch 2:
>> port 1/0/1 and port 1/0/2 link aggregation

So you're bridging two ports which are connected to aggregated links?
What is the purpose of doing this?

>> linux config:
>>    eth0, eth1 => bridge0 up, no ip, stp off.
>>    eth2, eth3 => bridge1 up, no ip, stp off.
>>    2.6.30.9
>>    xeon 5420 *8
>>    intel 82575 * 8(version 2.0.6 or linux 2.6.30.9 inside)
>> iptables rule:
>>    no any thing
>>
>> no other config.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is this nf_conn "use" counter bug???

[Patrick McHardy]

Patrick McHardy wrote:
> kendo wrote:
>> ----- Original Message ----- 
>> From: "kendo" <kendo999@sohu.com>
>> To: "Patrick McHardy" <kaber@trash.net>
>> Sent: Monday, November 23, 2009 7:12 PM
>> Subject: Re: Is this nf_conn "use" counter bug???
>>
>>
>>>> This code looks like from the 2.4 era.
>>> ----I'm sorry, I copy from 2.6.12,not 2.6.30.9.Because I also opened up them.
>>>
>>>> Most likely there's an skb leak somewhere and the conntrack references
>>>> are not released.
>>> ----Yes, I think so.But why is that so??kfree_skb function has not been properly invoked when skb leave?
> 
> Possibly. You can check the skbuff caches in /proc/slabinfo to see
> whether this is the case.
> 
>>>>> Environment:
>>>>> -----internet--cisco switch (two port bonding)-----linux box(two bridge)----cisco switch(two port bonging)---server
>>>>>
>>>>> Cisco switch link aggregation，use two port.Linux box have two Independent network bridge.
>>>                                           (ge0/1) ---------------(eth0) (eth1)------------(ge1/0/1)
>>> ---interface-----cisco switch 1                           linux box                           cisco switch 2--------servers
>>>                                           (ge0/2)----------------(eth2) (eth3)------------(ge1/0/1)
>>>
>>> switch 1:
>>> port 0/1 and port 0/2 link aggregation
>>>
>>> switch 2:
>>> port 1/0/1 and port 1/0/2 link aggregation
> 
> So you're bridging two ports which are connected to aggregated links?
> What is the purpose of doing this?

Ah I see this was a misunderstanding.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Is this nf_conn "use" counter bug???

[kendo]

Hi.

 I'm sorry,before an E-mail encoding format error, repeat it!
 
 In my linux 2.6.30.9,I found a strange question about netfilter:
  
[root@localhost ~]# conntrack -L | grep flow
conntrack v0.9.13 (conntrack-tools): 321 flow entries have been shown.

[root@localhost ~]# conntrack -C
14360

ct.count(14360) is far greater than the conntrack hash table number(321).Final,table full...game over.

Further analysis of statistics counters:

[root@localhost ~]# cat /proc/net/stat/nf_conntrack
entries  searched found new invalid ignore delete delete_list insert insert_failed drop early_drop icmp_error  expect_new expect_create expect_delete
0000392c  00000078 00036049 000042b9 000005f7 0000037f 00000468 000039af 00004291 0000002a 00000000 00000000 0000015c  00000000 00000000 00000000 
0000392c  00000003 0000873e 00000012 00000061 00000414 00000053 000007a5 00000012 00000000 00000000 00000000 00000002  00000000 00000000 00000000 
0000392c  00000000 000017c4 00000002 00000012 00000eb8 00000103 00000009 00000002 00000000 00000000 00000000 00000000  00000000 00000000 00000000 
0000392c  00000004 00002a4d 00000004 00000018 000005a3 000000f6 00000011 00000003 00000000 00000000 00000000 00000003  00000000 00000000 00000000 
0000392c  00000006 000034d6 0000007d 0000004f 00000197 000000a5 00000031 00000030 00000051 00000000 00000000 0000000c  00000000 00000000 00000000 
0000392c  00000004 000038a8 00000004 00000055 00000383 000000cf 00000016 00000004 00000000 00000000 00000000 00000002  00000000 00000000 00000000 
0000392c  00000004 00002e82 00000004 0000000d 0000179c 0000010e 00000014 00000004 00000000 00000000 00000000 00000000  00000000 00000000 00000000 
0000392c  00000001 00003911 00000005 0000006d 000002fd 000000f9 00000011 00000005 00000000 00000000 00000000 00000002  00000000 00000000 00000000

I found "delete_list"(16868) countres is far greater than the "delete"(2607) counters.Is cant't be release when a conntrack be delete from conntrack hash table???

static void death_by_timeout(unsigned long ul_conntrack)
{
 struct ip_conntrack *ct = (void *)ul_conntrack;

 WRITE_LOCK(&ip_conntrack_lock);
 /* Inside lock so preempt is disabled on module removal path.
  * Otherwise we can get spurious warnings. */
 CONNTRACK_STAT_INC(delete_list);
 clean_from_lists(ct);
 WRITE_UNLOCK(&ip_conntrack_lock);
 ip_conntrack_put(ct);
}

static inline void nf_conntrack_put(struct nf_conntrack *nfct)
{
 if (nfct && atomic_dec_and_test(&nfct->use))
  nf_conntrack_destroy(nfct);
}

What is wrong with the nfct->use counter may be the problem,Lead to nf_conntrack_destroy functions can not properly be called? From a statistical counter to the results of,clean_from_lists have been implemented,but the nf_conntrack_destroy does not seem to be running.

This is only my guess,I didn't fond any thing from the source code.However,if it is not so,What causes my connection to track the total number of non-stop to rise，table full and memory has been eaten?

Thanks,Hope to get your help!

Environment:
-----internet--cisco switch (two port bonding)-----linux box(two bridge)----cisco switch(two port bonging)---server

Cisco switch link aggregation，use two port.Linux box have two Independent network bridge.

kendo

kendo999@sohu.com

Re: Is this nf_conn "use" counter bug???

[Patrick McHardy]

kendo wrote:
>  In my linux 2.6.30.9,I found a strange question about netfilter:
>   
> [root@localhost ~]# conntrack -L | grep flow
> conntrack v0.9.13 (conntrack-tools): 321 flow entries have been shown.
> 
> [root@localhost ~]# conntrack -C
> 14360
> 
> ct.count(14360) is far greater than the conntrack hash table number(321).Final,table full...game over.
> 
> Further analysis of statistics counters:
> 
> [root@localhost ~]# cat /proc/net/stat/nf_conntrack
> entries  searched found new invalid ignore delete delete_list insert insert_failed drop early_drop icmp_error  expect_new expect_create expect_delete
> 0000392c  00000078 00036049 000042b9 000005f7 0000037f 00000468 000039af 00004291 0000002a 00000000 00000000 0000015c  00000000 00000000 00000000 
> 0000392c  00000003 0000873e 00000012 00000061 00000414 00000053 000007a5 00000012 00000000 00000000 00000000 00000002  00000000 00000000 00000000 
> 0000392c  00000000 000017c4 00000002 00000012 00000eb8 00000103 00000009 00000002 00000000 00000000 00000000 00000000  00000000 00000000 00000000 
> 0000392c  00000004 00002a4d 00000004 00000018 000005a3 000000f6 00000011 00000003 00000000 00000000 00000000 00000003  00000000 00000000 00000000 
> 0000392c  00000006 000034d6 0000007d 0000004f 00000197 000000a5 00000031 00000030 00000051 00000000 00000000 0000000c  00000000 00000000 00000000 
> 0000392c  00000004 000038a8 00000004 00000055 00000383 000000cf 00000016 00000004 00000000 00000000 00000000 00000002  00000000 00000000 00000000 
> 0000392c  00000004 00002e82 00000004 0000000d 0000179c 0000010e 00000014 00000004 00000000 00000000 00000000 00000000  00000000 00000000 00000000 
> 0000392c  00000001 00003911 00000005 0000006d 000002fd 000000f9 00000011 00000005 00000000 00000000 00000000 00000002  00000000 00000000 00000000
> 
> I found "delete_list"(16868) countres is far greater than the "delete"(2607) counters.Is cant't be release when a conntrack be delete from conntrack hash table???
> 
> static void death_by_timeout(unsigned long ul_conntrack)
> {
>  struct ip_conntrack *ct = (void *)ul_conntrack;
> 
>  WRITE_LOCK(&ip_conntrack_lock);
>  /* Inside lock so preempt is disabled on module removal path.
>   * Otherwise we can get spurious warnings. */
>  CONNTRACK_STAT_INC(delete_list);
>  clean_from_lists(ct);
>  WRITE_UNLOCK(&ip_conntrack_lock);
>  ip_conntrack_put(ct);
> }

This code looks like from the 2.4 era.

> static inline void nf_conntrack_put(struct nf_conntrack *nfct)
> {
>  if (nfct && atomic_dec_and_test(&nfct->use))
>   nf_conntrack_destroy(nfct);
> }
> 
> What is wrong with the nfct->use counter may be the problem,Lead to nf_conntrack_destroy functions can not properly be called? From a statistical counter to the results of,clean_from_lists have been implemented,but the nf_conntrack_destroy does not seem to be running.
> 
> This is only my guess,I didn't fond any thing from the source code.However,if it is not so,What causes my connection to track the total number of non-stop to rise，table full and memory has been eaten?

Most likely there's an skb leak somewhere and the conntrack references
are not released.

> Environment:
> -----internet--cisco switch (two port bonding)-----linux box(two bridge)----cisco switch(two port bonging)---server
> 
> Cisco switch link aggregation，use two port.Linux box have two Independent network bridge.

Please post your full network settings and netfilter rules.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html