From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Zijlstra Subject: Re: [ANNOUNCE] 3.0-rt4 Date: Thu, 28 Jul 2011 23:22:56 +0200 Message-ID: <1311888176.2617.379.camel@laptop> References: <4E31B59F.1000607@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Thomas Gleixner , LKML , linux-rt-users , "Paul E. McKenney" , Steven Rostedt , Jason Wessel , Pekka Enberg To: Fernando Lopez-Lezcano Return-path: Received: from casper.infradead.org ([85.118.1.10]:39892 "EHLO casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755080Ab1G1VSV (ORCPT ); Thu, 28 Jul 2011 17:18:21 -0400 Received: from j77219.upc-j.chello.nl ([24.132.77.219] helo=dyad.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.76 #1 (Red Hat Linux)) id 1QmXyG-0004AK-Gb for linux-rt-users@vger.kernel.org; Thu, 28 Jul 2011 21:18:20 +0000 In-Reply-To: <4E31B59F.1000607@localhost> Sender: linux-rt-users-owner@vger.kernel.org List-ID: On Thu, 2011-07-28 at 12:16 -0700, Fernando Lopez-Lezcano wrote: > [ 0.000000] ============================================= > [ 0.000000] [ INFO: possible recursive locking detected ] > [ 0.000000] 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1 > [ 0.000000] --------------------------------------------- > [ 0.000000] swapper/0 is trying to acquire lock: > [ 0.000000] (&parent->list_lock){+.+...}, at: [] __cache_free+0x43/0xc3 > [ 0.000000] > [ 0.000000] but task is already holding lock: > [ 0.000000] (&parent->list_lock){+.+...}, at: [] do_tune_cpucache+0xf2/0x2bb > [ 0.000000] > [ 0.000000] other info that might help us debug this: > [ 0.000000] Possible unsafe locking scenario: > [ 0.000000] > [ 0.000000] CPU0 > [ 0.000000] ---- > [ 0.000000] lock(&parent->list_lock); > [ 0.000000] lock(&parent->list_lock); > [ 0.000000] > [ 0.000000] *** DEADLOCK *** > [ 0.000000] > [ 0.000000] May be due to missing lock nesting notation > [ 0.000000] > [ 0.000000] 3 locks held by swapper/0: > [ 0.000000] #0: (cache_chain_mutex){+.+...}, at: [] kmem_cache_init_late+0xe/0x61 > [ 0.000000] #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [] __local_lock_irq+0x1e/0x5b > [ 0.000000] #2: (&parent->list_lock){+.+...}, at: [] do_tune_cpucache+0xf2/0x2bb > [ 0.000000] > [ 0.000000] stack backtrace: > [ 0.000000] Pid: 0, comm: swapper Not tainted 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1 > [ 0.000000] Call Trace: > [ 0.000000] [] ? printk+0x2d/0x2f > [ 0.000000] [] __lock_acquire+0x805/0xb57 > [ 0.000000] [] ? lock_release_holdtime.part.10+0x4b/0x51 > [ 0.000000] [] ? _raw_spin_unlock+0x31/0x3d > [ 0.000000] [] ? rt_spin_lock_slowlock+0x75/0x190 > [ 0.000000] [] ? trace_hardirqs_off+0xb/0xd > [ 0.000000] [] ? __cache_free+0x43/0xc3 > [ 0.000000] [] lock_acquire+0xde/0x11d > [ 0.000000] [] ? __cache_free+0x43/0xc3 > [ 0.000000] [] rt_spin_lock+0x50/0x56 > [ 0.000000] [] ? __cache_free+0x43/0xc3 > [ 0.000000] [] __cache_free+0x43/0xc3 > [ 0.000000] [] ? test_ti_thread_flag+0x8/0x10 > [ 0.000000] [] kmem_cache_free+0x6c/0xdc > [ 0.000000] [] slab_destroy+0x4f/0x53 > [ 0.000000] [] free_block+0x94/0xc1 > [ 0.000000] [] do_tune_cpucache+0x10b/0x2bb > [ 0.000000] [] enable_cpucache+0x7b/0xa7 > [ 0.000000] [] kmem_cache_init_late+0x1f/0x61 > [ 0.000000] [] start_kernel+0x24c/0x363 > [ 0.000000] [] ? loglevel+0x18/0x18 > [ 0.000000] [] i386_start_kernel+0xa9/0xaf Ooh, fun.. one does wonder why mainline doesn't show this.. This is the normal OFF_SLAB recursion, and the reason this shows up is because this is ran before we do the lockdep fixup. Fernando, does something like the below (not actually against -rt, but it shouldn't matter much) fix things? --- Subject: slab, lockdep: Annotate the locks before using Fernando found we hit the regular OFF_SLAB 'recursion' before we annotate the locks, cure this. Reported-by: Fernando Lopez-Lezcano Signed-off-by: Peter Zijlstra --- Index: linux-2.6/mm/slab.c =================================================================== --- linux-2.6.orig/mm/slab.c +++ linux-2.6/mm/slab.c @@ -1665,6 +1665,9 @@ void __init kmem_cache_init_late(void) { struct kmem_cache *cachep; + /* Annotate slab for lockdep -- annotate the malloc caches */ + init_lock_keys(); + /* 6) resize the head arrays to their final sizes */ mutex_lock(&cache_chain_mutex); list_for_each_entry(cachep, &cache_chain, next) @@ -1675,9 +1678,6 @@ void __init kmem_cache_init_late(void) /* Done! */ g_cpucache_up = FULL; - /* Annotate slab for lockdep -- annotate the malloc caches */ - init_lock_keys();