[PATCH -next] lockdep: skip debugobjects for lock_classes[] init

[PATCH -next] lockdep: skip debugobjects for lock_classes[] init

[Qian Cai]

The linux-next commit a0b0fd53e1e6 ("locking/lockdep: Free lock classes
that are no longer in use") let init_data_structures_once() call
init_rcu_head(). As the result, smp_setup_processor_id() calls
debug_object_init() before debug_objects_early_init() triggering errors
below on arm64 and ppc64le. Since debug_objects_early_init() depends on
smp_setup_processor_id(), so it is not straightforward to move the
former earlier in start_kernel(). Hence, just skip all debugobjects
creations in init_data_structures_once() as a good trade-off by losing
track of some RCU list heads debugging.

BUG: spinlock bad magic on CPU#0, swapper/0
 lock: obj_hash+0xc3048/0x100000, .magic: 00000000, .owner: <none>/-1,
.owner_cpu: 0
CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc8-next-20190301+ #1
Call trace:
 dump_backtrace+0x0/0x450
 show_stack+0x20/0x2c
 dump_stack+0xe0/0x16c
 spin_bug+0x200/0x224
 do_raw_spin_lock+0x1bc/0x23c
 _raw_spin_lock_irqsave+0xac/0xd0
 __debug_object_init+0xf0/0x19dc
 debug_object_init+0x54/0x6c
 init_rcu_head+0x24/0x30
 init_data_structures_once+0xb0/0x41c
 register_lock_class+0x70c/0x1454
 __lock_acquire+0xc4/0x235c
 lock_acquire+0x508/0x668
 _raw_spin_lock+0x88/0xac
 vprintk_emit+0x33c/0x53c
 vprintk_default+0x54/0x7c
 vprintk_func+0x320/0x40c
 printk+0x64/0x8c
 smp_setup_processor_id+0x6c/0x84
 start_kernel+0x38/0x8f8
ODEBUG: Out of memory. ODEBUG disabled

Signed-off-by: Qian Cai <cai@lca.pw>
---
 kernel/locking/lockdep.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index 21cb81fe6359..1309a6fe88ee 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -988,7 +988,6 @@ static void init_data_structures_once(void)
 
 	initialization_happened = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
 	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
 	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);
 
-- 
2.17.2 (Apple Git-113)

Re: [PATCH -next] lockdep: skip debugobjects for lock_classes[] init

[Bart Van Assche]

On 3/2/19 8:02 PM, Qian Cai wrote:
> The linux-next commit a0b0fd53e1e6 ("locking/lockdep: Free lock classes
> that are no longer in use") let init_data_structures_once() call
> init_rcu_head(). As the result, smp_setup_processor_id() calls
> debug_object_init() before debug_objects_early_init() triggering errors
> below on arm64 and ppc64le. Since debug_objects_early_init() depends on
> smp_setup_processor_id(), so it is not straightforward to move the
> former earlier in start_kernel(). Hence, just skip all debugobjects
> creations in init_data_structures_once() as a good trade-off by losing
> track of some RCU list heads debugging.
> 
> BUG: spinlock bad magic on CPU#0, swapper/0
>   lock: obj_hash+0xc3048/0x100000, .magic: 00000000, .owner: <none>/-1,
> .owner_cpu: 0
> CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc8-next-20190301+ #1
> Call trace:
>   dump_backtrace+0x0/0x450
>   show_stack+0x20/0x2c
>   dump_stack+0xe0/0x16c
>   spin_bug+0x200/0x224
>   do_raw_spin_lock+0x1bc/0x23c
>   _raw_spin_lock_irqsave+0xac/0xd0
>   __debug_object_init+0xf0/0x19dc
>   debug_object_init+0x54/0x6c
>   init_rcu_head+0x24/0x30
>   init_data_structures_once+0xb0/0x41c
>   register_lock_class+0x70c/0x1454
>   __lock_acquire+0xc4/0x235c
>   lock_acquire+0x508/0x668
>   _raw_spin_lock+0x88/0xac
>   vprintk_emit+0x33c/0x53c
>   vprintk_default+0x54/0x7c
>   vprintk_func+0x320/0x40c
>   printk+0x64/0x8c
>   smp_setup_processor_id+0x6c/0x84
>   start_kernel+0x38/0x8f8
> ODEBUG: Out of memory. ODEBUG disabled
> 
> Signed-off-by: Qian Cai <cai@lca.pw>
> ---
>   kernel/locking/lockdep.c | 1 -
>   1 file changed, 1 deletion(-)
> 
> diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
> index 21cb81fe6359..1309a6fe88ee 100644
> --- a/kernel/locking/lockdep.c
> +++ b/kernel/locking/lockdep.c
> @@ -988,7 +988,6 @@ static void init_data_structures_once(void)
>   
>   	initialization_happened = true;
>   
> -	init_rcu_head(&delayed_free.rcu_head);
>   	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
>   	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);

Hi Qian,

Thanks for having reported this issue. Had you considered to follow an 
approach similar to the patch below?

Thanks,

Bart.


[PATCH] locking/lockdep: Only call init_rcu_head() after RCU has been
  initialized

init_data_structures_once() is called for the first time before RCU has
been initialized. Make sure that init_rcu_head() is called before the
RCU head is used and after RCU has been initialized.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
  kernel/locking/lockdep.c | 15 +++++++++++----
  1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index 21cb81fe6359..c8c83e1e2664 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -980,15 +980,22 @@ static inline void check_data_structures(void) { }
   */
  static void init_data_structures_once(void)
  {
-	static bool initialization_happened;
+	static bool ds_initialized, rcu_head_initialized;
  	int i;

-	if (likely(initialization_happened))
+	if (likely(rcu_head_initialized))
  		return;

-	initialization_happened = true;
+	if (system_state >= SYSTEM_SCHEDULING) {
+		init_rcu_head(&delayed_free.rcu_head);
+		rcu_head_initialized = true;
+	}
+
+	if (ds_initialized)
+		return;
+
+	ds_initialized = true;

-	init_rcu_head(&delayed_free.rcu_head);
  	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
  	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);

-- 
2.20.1

Re: [PATCH -next] lockdep: skip debugobjects for lock_classes[] init

[Qian Cai]

On 3/3/19 1:19 PM, Bart Van Assche wrote:
> [PATCH] locking/lockdep: Only call init_rcu_head() after RCU has been
>  initialized
> 
> init_data_structures_once() is called for the first time before RCU has
> been initialized. Make sure that init_rcu_head() is called before the
> RCU head is used and after RCU has been initialized.
> 
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>

I like your approach more.

Tested-by: Qian Cai <cai@lca.pw>

[tip:locking/urgent] locking/lockdep: Only call init_rcu_head() after RCU has been initialized

[tip-bot for Bart Van Assche]

Commit-ID:  0126574fca2ce0f0d5beb9dade6efb823ff7407b
Gitweb:     https://git.kernel.org/tip/0126574fca2ce0f0d5beb9dade6efb823ff7407b
Author:     Bart Van Assche <bvanassche@acm.org>
AuthorDate: Sun, 3 Mar 2019 10:19:01 -0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Sat, 9 Mar 2019 14:15:51 +0100

locking/lockdep: Only call init_rcu_head() after RCU has been initialized

init_data_structures_once() is called for the first time before RCU has
been initialized. Make sure that init_rcu_head() is called before the
RCU head is used and after RCU has been initialized.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will.deacon@arm.com>
Cc: longman@redhat.com
Link: https://lkml.kernel.org/r/c20aa0f0-42ab-a884-d931-7d4ec2bf0cdc@acm.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/locking/lockdep.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index 35a144dfddf5..34cdcbedda49 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -982,15 +982,22 @@ static inline void check_data_structures(void) { }
  */
 static void init_data_structures_once(void)
 {
-	static bool initialization_happened;
+	static bool ds_initialized, rcu_head_initialized;
 	int i;
 
-	if (likely(initialization_happened))
+	if (likely(rcu_head_initialized))
 		return;
 
-	initialization_happened = true;
+	if (system_state >= SYSTEM_SCHEDULING) {
+		init_rcu_head(&delayed_free.rcu_head);
+		rcu_head_initialized = true;
+	}
+
+	if (ds_initialized)
+		return;
+
+	ds_initialized = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
 	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
 	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);