netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Dumazet <dada1@cosmosbay.com>
To: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
Cc: Patrick McHardy <kaber@trash.net>,
	avorontsov@ru.mvista.com, netdev@vger.kernel.org,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: ucc_geth: nf_conntrack: table full, dropping packet.
Date: Tue, 24 Mar 2009 10:12:53 +0100	[thread overview]
Message-ID: <49C8A415.1090606@cosmosbay.com> (raw)
In-Reply-To: <OFBD3D31D8.7AD81126-ONC1257583.002C5C2B-C1257583.002DFF13@transmode.se>

Joakim Tjernlund a écrit :
> Patrick McHardy <kaber@trash.net> wrote on 23/03/2009 18:49:15:
>> Joakim Tjernlund wrote:
>>> Patrick McHardy <kaber@trash.net> wrote on 23/03/2009 13:29:33:
>>>
>>>
>>>>> There is no /proc/net/netfilter/nf_conntrack. There is a
>>>>> /proc/net/nf_conntrack though and it is empty. If I telnet
>>>>> to the board I see:
>>>>>
>>>> That means that something is leaking conntrack references, most 
> likely
>>>> by leaking skbs. Since I haven't seen any other reports, my guess 
> would
>>>> be the ucc_geth driver.
>>>>
>>> Mucking around with the ucc_geth driver I found that if I:
>>>  - Move TX from IRQ to NAPI context
>>>  - double the weight.
>>>  - after booting up, wait a few mins until the JFFS2 GC kernel thread 
> has 
>>> stopped
>>>    scanning the FS 
>>>
>>> Then the "nf_conntrack: table full, dropping packet." msgs stops.
>>> Does this seem right to you guys?
>> No. As I said, something seems to be leaking packets. You should be
>> able to confirm that by checking the sk_buff slabs in /proc/slabinfo.
>> If that *doesn't* show any signs of a leak, please run "conntrack -E"
>> to capture the conntrack events before the "table full" message
>> appears and post the output.
> 
> skbuff does not differ much, but others do
> 
> Before ping:
>   skbuff_fclone_cache    0      0    352   11    1 : tunables   54   27 0 
> : slabdata      0      0      0
>   skbuff_head_cache     20     20    192   20    1 : tunables  120   60 0 
> : slabdata      1      1      0
>   size-64              731    767     64   59    1 : tunables  120   60 0 
> : slabdata     13     13      0
>   nf_conntrack          10     19    208   19    1 : tunables  120   60 0 
> : slabdata      1      1      0
> 
> During ping: 
>   skbuff_fclone_cache    0      0    352   11    1 : tunables   54   27 0 
> : slabdata      0      0      0
>   skbuff_head_cache     40     40    192   20    1 : tunables  120   60 0 
> : slabdata      2      2      0
>   size-64             8909   8909     64   59    1 : tunables  120   60 0 
> : slabdata    151    151      0
>   nf_conntrack        5111   5111    208   19    1 : tunables  120   60 0 
> : slabdata    269    269      0
> 
> This feels more like the freeing of conntrack objects are delayed and 
> builds up when ping flooding.
> 
> Don't have "conntrack -E" for my embedded board so that will have to wait 
> a bit longer.

I dont understand how your ping can use so many conntrack entries...

Then, as I said yesterday, I believe you have a RCU delay, because of
a misbehaving driver or something...

grep RCU .config
grep CONFIG_SMP .config

You could change qhimark from 10000 to 1000 in kernel/rcuclassic.c (line 80)
as a workaround. It should force a quiescent state after 1000 freed conntracks.



  reply	other threads:[~2009-03-24  9:13 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-03-23 10:42 ucc_geth: nf_conntrack: table full, dropping packet Joakim Tjernlund
2009-03-23 12:15 ` Patrick McHardy
2009-03-23 12:25   ` Joakim Tjernlund
2009-03-23 12:29     ` Patrick McHardy
2009-03-23 12:59       ` Joakim Tjernlund
     [not found]       ` <OF387EC803.F810F72A-ONC1257582.00468C6E-C1257582.00475783@LocalDomain>
2009-03-23 13:09         ` Joakim Tjernlund
2009-03-23 17:42       ` Joakim Tjernlund
2009-03-23 17:49         ` Patrick McHardy
2009-03-24  8:22           ` Joakim Tjernlund
2009-03-24  9:12             ` Eric Dumazet [this message]
2009-03-24 10:55               ` Joakim Tjernlund
2009-03-24 12:07                 ` [PATCH] conntrack: Reduce conntrack count in nf_conntrack_free() Eric Dumazet
2009-03-24 12:25                   ` Eric Dumazet
2009-03-24 12:43                     ` Patrick McHardy
2009-03-24 13:32                       ` Eric Dumazet
2009-03-24 13:38                         ` Patrick McHardy
2009-03-24 13:47                           ` Eric Dumazet
     [not found]                             ` <49C8F871.9070600@cosmosbay.com>
     [not found]                               ` <49C8F8E0.9050502@trash.net>
2009-03-25  3:53                                 ` Eric Dumazet
2009-03-25 13:39                                   ` Patrick McHardy
2009-03-25 13:44                                     ` Eric Dumazet
2009-03-24 13:20                   ` Joakim Tjernlund
2009-03-24 13:28                     ` Patrick McHardy
2009-03-24 13:29                     ` Eric Dumazet
2009-03-24 13:41                       ` Joakim Tjernlund
2009-03-24 15:17                   ` Maxime Bizon
2009-03-24 15:21                     ` Patrick McHardy
2009-03-24 15:27                     ` Eric Dumazet
2009-03-24 19:54                       ` [PATCH] netfilter: Use hlist_add_head_rcu() in nf_conntrack_set_hashsize() Eric Dumazet
2009-03-25 16:26                         ` Patrick McHardy
2009-03-25 17:53                       ` [PATCH] conntrack: use SLAB_DESTROY_BY_RCU for nf_conn structs Eric Dumazet
2009-03-25 18:05                         ` Patrick McHardy
2009-03-25 18:06                           ` Patrick McHardy
2009-03-25 18:15                           ` Eric Dumazet
2009-03-25 18:24                             ` Patrick McHardy
2009-03-25 18:53                               ` Eric Dumazet
2009-03-25 19:00                                 ` Patrick McHardy
2009-03-25 19:17                                   ` Eric Dumazet
2009-03-25 19:41                                     ` Patrick McHardy
2009-03-25 19:58                                       ` Eric Dumazet
2009-03-25 20:10                                         ` Patrick McHardy
2009-03-24 18:29                     ` [PATCH] conntrack: Reduce conntrack count in nf_conntrack_free() Joakim Tjernlund
2009-03-23 17:49         ` ucc_geth: nf_conntrack: table full, dropping packet Eric Dumazet
2009-03-23 18:04           ` Joakim Tjernlund
2009-03-23 18:08             ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=49C8A415.1090606@cosmosbay.com \
    --to=dada1@cosmosbay.com \
    --cc=Joakim.Tjernlund@transmode.se \
    --cc=avorontsov@ru.mvista.com \
    --cc=kaber@trash.net \
    --cc=netdev@vger.kernel.org \
    --cc=paulmck@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).