From: Ben Greear <greearb@candelatech.com>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
netdev <netdev@vger.kernel.org>
Subject: Re: kmemleak reports in kernel 3.9.5+
Date: Tue, 11 Jun 2013 12:52:41 -0700 [thread overview]
Message-ID: <51B78009.2060808@candelatech.com> (raw)
In-Reply-To: <CAHkRjk5SDCbqr0Br24YQuHr=j7DmnpYk9CShrz1qiuYff7+qAw@mail.gmail.com>
On 06/10/2013 03:32 PM, Catalin Marinas wrote:
> On 10 June 2013 19:22, Ben Greear <greearb@candelatech.com> wrote:
>> We had a system go OOM while doing lots of wireless
>> stations. (System had 8GB of RAM, so I suspect a leak).
>>
>> I enabled kmemleak in a 3.9.5 (plus some local patches) and
>> I see the entries below. Any idea if these are real or not?
>>
>> unreferenced object 0xffff880212281c80 (size 128):
>> comm "systemd", pid 1, jiffies 4294682684 (age 1159.517s)
>> hex dump (first 32 bytes):
>> 60 39 27 12 02 88 ff ff 00 02 20 00 00 00 ad de `9'....... .....
>> 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
>> [<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
>> [<ffffffff8118d605>] __kmalloc+0xf9/0x122
>> [<ffffffff8154946d>] kzalloc.clone.0+0xe/0x10
>> [<ffffffff81549494>] fib_default_rule_add+0x25/0x7a
>> [<ffffffffa014f5a9>] ip6mr_net_init+0x7e/0x118 [ipv6]
>> [<ffffffff8152c992>] ops_init+0xd6/0xf7
>> [<ffffffff8152cb51>] register_pernet_operations+0xc2/0x16b
>> [<ffffffff8152cc87>] register_pernet_subsys+0x2e/0x47
>> [<ffffffffa016db69>] 0xffffffffa016db69
>> [<ffffffffa016d109>] 0xffffffffa016d109
>> [<ffffffff8100207f>] do_one_initcall+0x7f/0x13e
>> [<ffffffff810f3985>] do_init_module+0x44/0x18f
>> [<ffffffff810f5da7>] load_module+0x14d1/0x168e
>> [<ffffffff810f6114>] sys_init_module+0xfd/0x101
>> [<ffffffff815f6599>] system_call_fastpath+0x16/0x1b
>
> No idea yet. You can try:
>
> echo clear > /sys/kernel/debug/kmemleak
>
> and see if there are more appearing after. All seem to have a common
> allocation path via debug_object_activate -> ... ->
> rcuhead_fixup_activate -> ... -> __debug_object_init.
I tried the command below, and it printed out quite a few things.
I'll try building a kernel without the extra SLUB debugging
to see if that helps.
Also, I read the kmemleak.txt documentation, but a question remains:
If I enable kmemleak at compile time, but disable it at boot
time using kmemleak=off, is there any significant runtime overhead?
[root@LEC2220-1 ~]# echo clear > /debug/kmemleak;sleep 60;echo scan > /debug/kmemleak; cat /debug/kmemleak
unreferenced object 0xffff88021867e450 (size 40):
comm "chrony-helper", pid 1138, jiffies 4294699268 (age 91773.781s)
hex dump (first 32 bytes):
d0 cf e5 18 02 88 ff ff 80 0b be 19 02 88 ff ff ................
01 00 00 00 00 00 00 00 f0 34 a3 12 02 88 ff ff .........4......
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c62>] debug_object_init+0x14/0x16
[<ffffffff810b4e0a>] rcuhead_fixup_activate+0x2b/0xba
[<ffffffff81315f12>] debug_object_fixup+0x15/0x1d
[<ffffffff81316557>] debug_object_activate+0x126/0x139
[<ffffffff81118e4a>] __call_rcu.clone.1+0x58/0x22a
[<ffffffff81119065>] call_rcu+0x17/0x19
[<ffffffff811971f6>] put_object+0x46/0x4a
[<ffffffff811974e3>] delete_object_full+0x2d/0x32
[<ffffffff815de663>] kmemleak_free+0x59/0x7a
[<ffffffff8118bc0a>] slab_free_hook+0x21/0x87
[<ffffffff8118e888>] kmem_cache_free+0xbe/0x15d
[<ffffffff811acb71>] __d_free+0x56/0x5b
unreferenced object 0xffff880211893b50 (size 40):
comm "nmcli", pid 1178, jiffies 4294699660 (age 91773.390s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 f0 ac bd 19 02 88 ff ff ................
01 00 00 00 00 00 00 00 e0 6c 6e 01 02 88 ff ff .........ln.....
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c62>] debug_object_init+0x14/0x16
[<ffffffff810b4e0a>] rcuhead_fixup_activate+0x2b/0xba
[<ffffffff81315f12>] debug_object_fixup+0x15/0x1d
[<ffffffff81316557>] debug_object_activate+0x126/0x139
[<ffffffff81118e4a>] __call_rcu.clone.1+0x58/0x22a
[<ffffffff81119065>] call_rcu+0x17/0x19
[<ffffffff8119be2b>] file_free+0x31/0x35
[<ffffffff8119c05a>] __fput+0x1bb/0x1db
[<ffffffff8119c0ca>] ____fput+0xe/0x10
[<ffffffff810b48d5>] task_work_run+0x85/0xb0
[<ffffffff8109ccc1>] do_exit+0x3c9/0x978
[<ffffffff8109d2f3>] do_group_exit+0x83/0xae
....
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
next prev parent reply other threads:[~2013-06-11 19:52 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-10 18:22 kmemleak reports in kernel 3.9.5+ Ben Greear
2013-06-10 22:32 ` Catalin Marinas
2013-06-11 18:36 ` Ben Greear
2013-06-11 19:52 ` Ben Greear [this message]
2013-06-12 0:28 ` Ben Greear
2013-06-13 15:50 ` Catalin Marinas
2013-06-17 22:45 ` Ben Greear
2013-06-13 15:48 ` Catalin Marinas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51B78009.2060808@candelatech.com \
--to=greearb@candelatech.com \
--cc=catalin.marinas@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).