From: Fernando Lopez-Lezcano <nando@ccrma.Stanford.EDU>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Fernando Lopez-Lezcano <nando@ccrma.Stanford.EDU>,
Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>,
linux-rt-users <linux-rt-users@vger.kernel.org>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Jason Wessel <jason.wessel@windriver.com>,
Pekka Enberg <penberg@kernel.org>
Subject: Re: [ANNOUNCE] 3.0-rt4
Date: Thu, 28 Jul 2011 17:17:59 -0700 [thread overview]
Message-ID: <4E31FC37.7080803@localhost> (raw)
In-Reply-To: <1311888176.2617.379.camel@laptop>
On 07/28/2011 02:22 PM, Peter Zijlstra wrote:
> On Thu, 2011-07-28 at 12:16 -0700, Fernando Lopez-Lezcano wrote:
>> [ 0.000000] =============================================
>> [ 0.000000] [ INFO: possible recursive locking detected ]
>> [ 0.000000] 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1
>> [ 0.000000] ---------------------------------------------
>> [ 0.000000] swapper/0 is trying to acquire lock:
>> [ 0.000000] (&parent->list_lock){+.+...}, at: [<c04fb406>] __cache_free+0x43/0xc3
>> [ 0.000000]
>> [ 0.000000] but task is already holding lock:
>> [ 0.000000] (&parent->list_lock){+.+...}, at: [<c04fc538>] do_tune_cpucache+0xf2/0x2bb
>> [ 0.000000]
>> [ 0.000000] other info that might help us debug this:
>> [ 0.000000] Possible unsafe locking scenario:
>> [ 0.000000]
>> [ 0.000000] CPU0
>> [ 0.000000] ----
>> [ 0.000000] lock(&parent->list_lock);
>> [ 0.000000] lock(&parent->list_lock);
>> [ 0.000000]
>> [ 0.000000] *** DEADLOCK ***
>> [ 0.000000]
>> [ 0.000000] May be due to missing lock nesting notation
>> [ 0.000000]
>> [ 0.000000] 3 locks held by swapper/0:
>> [ 0.000000] #0: (cache_chain_mutex){+.+...}, at: [<c0bd9d2b>] kmem_cache_init_late+0xe/0x61
>> [ 0.000000] #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [<c04faa65>] __local_lock_irq+0x1e/0x5b
>> [ 0.000000] #2: (&parent->list_lock){+.+...}, at: [<c04fc538>] do_tune_cpucache+0xf2/0x2bb
>> [ 0.000000]
>> [ 0.000000] stack backtrace:
>> [ 0.000000] Pid: 0, comm: swapper Not tainted 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1
>> [ 0.000000] Call Trace:
>> [ 0.000000] [<c0856355>] ? printk+0x2d/0x2f
>> [ 0.000000] [<c0474a4b>] __lock_acquire+0x805/0xb57
>> [ 0.000000] [<c0472604>] ? lock_release_holdtime.part.10+0x4b/0x51
>> [ 0.000000] [<c085ecb4>] ? _raw_spin_unlock+0x31/0x3d
>> [ 0.000000] [<c085dbc5>] ? rt_spin_lock_slowlock+0x75/0x190
>> [ 0.000000] [<c04720c3>] ? trace_hardirqs_off+0xb/0xd
>> [ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
>> [ 0.000000] [<c0475215>] lock_acquire+0xde/0x11d
>> [ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
>> [ 0.000000] [<c085e24f>] rt_spin_lock+0x50/0x56
>> [ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
>> [ 0.000000] [<c04fb406>] __cache_free+0x43/0xc3
>> [ 0.000000] [<c043646d>] ? test_ti_thread_flag+0x8/0x10
>> [ 0.000000] [<c04fb23f>] kmem_cache_free+0x6c/0xdc
>> [ 0.000000] [<c04fb2fe>] slab_destroy+0x4f/0x53
>> [ 0.000000] [<c04fb396>] free_block+0x94/0xc1
>> [ 0.000000] [<c04fc551>] do_tune_cpucache+0x10b/0x2bb
>> [ 0.000000] [<c04fc8dc>] enable_cpucache+0x7b/0xa7
>> [ 0.000000] [<c0bd9d3c>] kmem_cache_init_late+0x1f/0x61
>> [ 0.000000] [<c0bba687>] start_kernel+0x24c/0x363
>> [ 0.000000] [<c0bba1c4>] ? loglevel+0x18/0x18
>> [ 0.000000] [<c0bba0ba>] i386_start_kernel+0xa9/0xaf
>
> Ooh, fun.. one does wonder why mainline doesn't show this..
>
> This is the normal OFF_SLAB recursion, and the reason this shows up is
> because this is ran before we do the lockdep fixup.
>
> Fernando, does something like the below (not actually against -rt, but
> it shouldn't matter much) fix things?
Strange. Looks like it only happens on a cold start (I would have to
test more to be sure), so far two warm starts and it is not there, one
cold start and it is there - sorry, I have run out of time now for more
tests... (I checked my build logs and the patch applied cleanly, no
complains there). Come to think of it I think this was not always
happening before the patch (ie: not getting the error on all boots).
-- Fernando
[ 0.000000] =============================================
[ 0.000000] [ INFO: possible recursive locking detected ]
[ 0.000000] 3.0.0-1.rt6.1.fc15.ccrma.i686.rtPAE #1
[ 0.000000] ---------------------------------------------
[ 0.000000] swapper/0 is trying to acquire lock:
[ 0.000000] (&parent->list_lock){+.+...}, at: [<c04fb406>]
__cache_free+0x43/0xc3
[ 0.000000]
[ 0.000000] but task is already holding lock:
[ 0.000000] (&parent->list_lock){+.+...}, at: [<c04fc538>]
do_tune_cpucache+0xf2/0x2bb
[ 0.000000]
[ 0.000000] other info that might help us debug this:
[ 0.000000] Possible unsafe locking scenario:
[ 0.000000]
[ 0.000000] CPU0
[ 0.000000] ----
[ 0.000000] lock(&parent->list_lock);
[ 0.000000] lock(&parent->list_lock);
[ 0.000000]
[ 0.000000] *** DEADLOCK ***
[ 0.000000]
[ 0.000000] May be due to missing lock nesting notation
[ 0.000000]
[ 0.000000] 3 locks held by swapper/0:
[ 0.000000] #0: (cache_chain_mutex){+.+...}, at: [<c0bd9d3b>]
kmem_cache_init_late+0x1e/0x6a
[ 0.000000] #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at:
[<c04faa65>] __local_lock_irq+0x1e/0x5b
[ 0.000000] #2: (&parent->list_lock){+.+...}, at: [<c04fc538>]
do_tune_cpucache+0xf2/0x2bb
[ 0.000000]
[ 0.000000] stack backtrace:
[ 0.000000] Pid: 0, comm: swapper Not tainted
3.0.0-1.rt6.1.fc15.ccrma.i686.rtPAE #1
[ 0.000000] Call Trace:
[ 0.000000] [<c0856355>] ? printk+0x2d/0x2f
[ 0.000000] [<c0474a5b>] __lock_acquire+0x805/0xb57
[ 0.000000] [<c0472614>] ? lock_release_holdtime.part.10+0x4b/0x51
[ 0.000000] [<c085ecb4>] ? _raw_spin_unlock+0x31/0x3d
[ 0.000000] [<c085dbc5>] ? rt_spin_lock_slowlock+0x75/0x190
[ 0.000000] [<c04720d3>] ? trace_hardirqs_off+0xb/0xd
[ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
[ 0.000000] [<c0475225>] lock_acquire+0xde/0x11d
[ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
[ 0.000000] [<c085e24f>] rt_spin_lock+0x50/0x56
[ 0.000000] [<c04fb406>] ? __cache_free+0x43/0xc3
[ 0.000000] [<c04fb406>] __cache_free+0x43/0xc3
[ 0.000000] [<c043646d>] ? test_ti_thread_flag+0x8/0x10
[ 0.000000] [<c04fb23f>] kmem_cache_free+0x6c/0xdc
[ 0.000000] [<c04fb2fe>] slab_destroy+0x4f/0x53
[ 0.000000] [<c04fb396>] free_block+0x94/0xc1
[ 0.000000] [<c04fc551>] do_tune_cpucache+0x10b/0x2bb
[ 0.000000] [<c04fc8dc>] enable_cpucache+0x7b/0xa7
[ 0.000000] [<c0bd9d4c>] kmem_cache_init_late+0x2f/0x6a
[ 0.000000] [<c0bba687>] start_kernel+0x24c/0x363
[ 0.000000] [<c0bba1c4>] ? loglevel+0x18/0x18
[ 0.000000] [<c0bba0ba>] i386_start_kernel+0xa9/0xaf
[ 0.000000] Console: colour VGA+ 80x25
next prev parent reply other threads:[~2011-07-29 0:18 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-07-27 21:37 [ANNOUNCE] 3.0-rt4 Thomas Gleixner
2011-07-27 21:57 ` Frank Rowand
2011-07-27 22:05 ` Thomas Gleixner
2011-07-27 22:27 ` Frank Rowand
2011-07-28 7:33 ` Nikita V. Youshchenko
2011-07-28 8:06 ` Thomas Gleixner
2011-07-28 8:24 ` Nikita V. Youshchenko
2011-07-28 8:44 ` Peter Zijlstra
2011-07-28 8:51 ` Nikita V. Youshchenko
2011-07-28 9:21 ` Anca Emanuel
2011-07-28 16:10 ` Thomas Gleixner
2011-07-28 9:37 ` Thomas Gleixner
2011-07-28 8:43 ` Peter Zijlstra
2011-07-28 9:41 ` Yong Zhang
2011-07-28 9:49 ` Thomas Gleixner
2011-07-28 11:22 ` N, Mugunthan V
2011-07-28 11:36 ` Thomas Gleixner
2011-07-28 15:59 ` hermann
2011-07-28 17:41 ` Paul E. McKenney
2011-07-28 17:43 ` Thomas Gleixner
2011-07-28 18:32 ` Paul E. McKenney
2011-07-28 19:05 ` Thomas Gleixner
2011-07-28 19:34 ` Paul E. McKenney
2011-07-28 20:29 ` Paul E. McKenney
2011-07-28 17:42 ` Jason Wessel
2011-07-28 17:49 ` Thomas Gleixner
2011-07-28 19:16 ` Fernando Lopez-Lezcano
2011-07-28 19:36 ` Thomas Gleixner
2011-07-29 0:13 ` Fernando Lopez-Lezcano
2011-07-28 21:22 ` Peter Zijlstra
2011-07-28 22:55 ` Pekka Enberg
2011-07-29 0:17 ` Fernando Lopez-Lezcano [this message]
2011-08-04 8:36 ` [tip:core/urgent] slab, lockdep: Annotate the locks before using them tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4E31FC37.7080803@localhost \
--to=nando@ccrma.stanford.edu \
--cc=jason.wessel@windriver.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=penberg@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox